How to upload file with python requests?

ID : 20119

viewed : 43

Tags : pythonfilefile-uploadpython-requestspython

Top 5 Answer for How to upload file with python requests?

vote vote

100

If upload_file is meant to be the file, use:

files = {'upload_file': open('file.txt','rb')} values = {'DB': 'photcat', 'OUT': 'csv', 'SHORT': 'short'}  r = requests.post(url, files=files, data=values) 

and requests will send a multi-part form POST body with the upload_file field set to the contents of the file.txt file.

The filename will be included in the mime header for the specific field:

>>> import requests >>> open('file.txt', 'wb')  # create an empty demo file <_io.BufferedWriter name='file.txt'> >>> files = {'upload_file': open('file.txt', 'rb')} >>> print(requests.Request('POST', 'http://example.com', files=files).prepare().body.decode('ascii')) --c226ce13d09842658ffbd31e0563c6bd Content-Disposition: form-data; name="upload_file"; filename="file.txt"   --c226ce13d09842658ffbd31e0563c6bd-- 

Note the filename="file.txt" parameter.

You can use a tuple for the files mapping value, with between 2 and 4 elements, if you need more control. The first element is the filename, followed by the contents, and an optional content-type header value and an optional mapping of additional headers:

files = {'upload_file': ('foobar.txt', open('file.txt','rb'), 'text/x-spam')} 

This sets an alternative filename and content type, leaving out the optional headers.

If you are meaning the whole POST body to be taken from a file (with no other fields specified), then don't use the files parameter, just post the file directly as data. You then may want to set a Content-Type header too, as none will be set otherwise. See Python requests - POST data from a file.

vote vote

81

(2018) the new python requests library has simplified this process, we can use the 'files' variable to signal that we want to upload a multipart-encoded file

url = 'http://httpbin.org/post' files = {'file': open('report.xls', 'rb')}  r = requests.post(url, files=files) r.text 
vote vote

70

Client Upload

If you want to upload a single file with Python requests library, then requests lib supports streaming uploads, which allow you to send large files or streams without reading into memory.

with open('massive-body', 'rb') as f:     requests.post('http://some.url/streamed', data=f) 

Server Side

Then store the file on the server.py side such that save the stream into file without loading into the memory. Following is an example with using Flask file uploads.

@app.route("/upload", methods=['POST']) def upload_file():     from werkzeug.datastructures import FileStorage     FileStorage(request.stream).save(os.path.join(app.config['UPLOAD_FOLDER'], filename))     return 'OK', 200 

Or use werkzeug Form Data Parsing as mentioned in a fix for the issue of "large file uploads eating up memory" in order to avoid using memory inefficiently on large files upload (s.t. 22 GiB file in ~60 seconds. Memory usage is constant at about 13 MiB.).

@app.route("/upload", methods=['POST']) def upload_file():     def custom_stream_factory(total_content_length, filename, content_type, content_length=None):         import tempfile         tmpfile = tempfile.NamedTemporaryFile('wb+', prefix='flaskapp', suffix='.nc')         app.logger.info("start receiving file ... filename => " + str(tmpfile.name))         return tmpfile      import werkzeug, flask     stream, form, files = werkzeug.formparser.parse_form_data(flask.request.environ, stream_factory=custom_stream_factory)     for fil in files.values():         app.logger.info(" ".join(["saved form name", fil.name, "submitted as", fil.filename, "to temporary file", fil.stream.name]))         # Do whatever with stored file at `fil.stream.name`     return 'OK', 200 
vote vote

68

@martijn-pieters answer is correct, however I wanted to add a bit of context to data= and also to the other side, in the Flask server, in the case where you are trying to upload files and a JSON.

From the request side, this works as Martijn describes:

files = {'upload_file': open('file.txt','rb')} values = {'DB': 'photcat', 'OUT': 'csv', 'SHORT': 'short'}  r = requests.post(url, files=files, data=values) 

However, on the Flask side (the receiving webserver on the other side of this POST), I had to use form

@app.route("/sftp-upload", methods=["POST"]) def upload_file():     if request.method == "POST":         # the mimetype here isnt application/json         # see here: https://stackoverflow.com/questions/20001229/how-to-get-posted-json-in-flask         body = request.form         print(body)  # <- immutable dict 

body = request.get_json() will return nothing. body = request.get_data() will return a blob containing lots of things like the filename etc.

Here's the bad part: on the client side, changing data={} to json={} results in this server not being able to read the KV pairs! As in, this will result in a {} body above:

r = requests.post(url, files=files, json=values). # No! 

This is bad because the server does not have control over how the user formats the request; and json= is going to be the habbit of requests users.

vote vote

51

In Ubuntu you can apply this way,

to save file at some location (temporary) and then open and send it to API

      path = default_storage.save('static/tmp/' + f1.name, ContentFile(f1.read()))       path12 = os.path.join(os.getcwd(), "static/tmp/" + f1.name)       data={} #can be anything u want to pass along with File       file1 = open(path12, 'rb')       header = {"Content-Disposition": "attachment; filename=" + f1.name, "Authorization": "JWT " + token}        res= requests.post(url,data,header) 

Top 3 video Explaining How to upload file with python requests?

Related QUESTION?