MemoryError

[halilozercan]

When I try to upload a file with

api.post('/path/data', file=file('filename', 'rb'))

I get a MemoryError. It is because my server has only 512 mb ram and my upload size does not fit into memory usually. I think it is possible to do a streaming upload but I don't have enough knowledge on requests library.

[grunichev]

It's definitely a bug. Even though requests library itself support uploading large files, smartfile library doesn't.

from smartfile import BasicClient

api = BasicClient(settings.SMARTFILE_API_KEY, settings.SMARTFILE_API_PASSWORD)
with open('large.file', 'rb') as file:
    api.upload('test.large', file)
... 
Traceback (most recent call last):
  File "<stdin>", line 2, in <module>
  File "/usr/local/lib/python3.6/site-packages/smartfile/__init__.py", line 146, in upload
    return self.post('/path/data/', file=arg)
  File "/usr/local/lib/python3.6/site-packages/smartfile/__init__.py", line 131, in post
    return self._request('post', endpoint, id=id, data=kwargs)
  File "/usr/local/lib/python3.6/site-packages/smartfile/__init__.py", line 110, in _request
    return self._do_request(request, url, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/smartfile/__init__.py", line 207, in _do_request
    return super(BasicClient, self)._do_request(*args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/smartfile/__init__.py", line 53, in _do_request
    response = request(url, stream=True, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/requests/api.py", line 110, in post
    return request('post', url, data=data, json=json, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/requests/api.py", line 56, in request
    return session.request(method=method, url=url, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/requests/sessions.py", line 474, in request
    prep = self.prepare_request(req)
  File "/usr/local/lib/python3.6/site-packages/requests/sessions.py", line 407, in prepare_request
    hooks=merge_hooks(request.hooks, self.hooks),
  File "/usr/local/lib/python3.6/site-packages/requests/models.py", line 305, in prepare
    self.prepare_body(data, files, json)
  File "/usr/local/lib/python3.6/site-packages/requests/models.py", line 483, in prepare_body
    (body, content_type) = self._encode_files(files, data)
  File "/usr/local/lib/python3.6/site-packages/requests/models.py", line 164, in _encode_files
    body, content_type = encode_multipart_formdata(new_fields)
  File "/usr/local/lib/python3.6/site-packages/requests/packages/urllib3/filepost.py", line 86, in encode_multipart_formdata
    body.write(data)
MemoryError