The requests library is a great tool for making requests to the web using Python. This article will discuss using this module to send form data to a site. In particular, we will be using the requests.post(url, data=None, json=None) function.
For demonstration purposes, we will send POST requests to https://httpbin.org – an open-source site that allows one to inspect HTTPS requests before using them in production.
Example 1
Let’s start by posting a string to httpbin.org and see the result.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 |
# if requests is not installed, you can install it using pip # by running: pip install requests import requests # URL to send sends a POST request to. url = 'https://httpbin.org/post' # Send the POST request with data="value1" response = requests.post(url, data="value1") # Decode the response content. Decoding converts the byte response # into text (native Python strings) response = response.content.decode() # or simply use response.text print(response) |
Output (truncated):
{
"args": {},
"data": "value1",
"files": {},
"form": {},
…
"json": null,
"origin": "105.161.11.xxx",
"url": "https://httpbin.org/post"
}
The server response in the output shows the data value we passed into requests.post() is posted under “data”. Form data should be posted on the “form” property. It is empty at the moment. Let’s post some form data in the following example.
Example 2
To pass form-encoded data, just like you will do on HTML forms, you need to pass dictionary-formatted items to the data argument on requests.posts().
The dictionary will be automatically passed as form data when the POST request is made. Here is an example.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 |
import requests url = 'https://httpbin.org/post' # Dictionary-formatted data # This will be passed as form data when making the request data = { "comments": "this is a comment", "custname": "Karl", "custtel": "01883", "delivery": "12:00", "size": "large", "topping": "cheese" } # Making the actual request response = requests.post(url, data=data) # Deconde the byte-type response into string response = response.content.decode() # or simply use response.text print(response) |
Output (truncated):
{
"args": {},
"data": "",
"files": {},
"form": {
"comments": "this is a comment",
"custemail": "[email protected]",
"custname": "Karl",
"custtel": "01883",
"delivery": "12:00",
"size": "large",
"topping": "cheese"
},
…
"origin": "105.161.11.xxx",
"url": "https://httpbin.org/post"
}
As shown in the output, the dictionary data was passed as a form when the request was made. That is why, in this example, the “form” property of the response has the data we passed to requests.post() function. Let’s go into another interesting example.
Example 3
The requests.post() also allows the user to pass multiple values to a single dictionary key (form property). This can be done in two ways:
- Using a list of tuples. Here is an example with two values for the “items” property.
|
1 |
data = [("customer_name", "Smith"), ("delivery", "12:00"), ("items", "item1"), ("items", "item2")] |
- Using a dictionary with all possible values of a given key in a list. The example in the first method can be represented as:
|
1 2 3 4 5 |
data = { "customer_name": "Smith", "delivery": "12:00", "items": ["item1", "item2"] } |
Let’s post that data using requests and parse the response as a JSON object.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 |
import requests # Data with a key having multiple values # of course, you can use the tuples format as well # as described above. data = { "customer_name": "Smith", "delivery": "12:00", "items": ["item1", "item2"] } # Post the data response = requests.post('https://httpbin.org/post', data=data) # You can also parse the response as JSON as follows. response_json = response.json() print("Full JSON response: ",response_json) # the JSON response is a Python dictionary; therefore, # we can access the items as we will do in a native dict. # Extracting the form only in this case. print("Extracting the form property: ", response_json["form"]) |
Output (truncated):
Full JSON response: {'args': {}, 'data': '', 'files': {}, 'form': {'customer_name': 'Smith', 'delivery': '12:00', 'items': ['item1', 'item2']}, … , 'url': 'https://httpbin.org/post'}
Extracting the form property: {'customer_name': 'Smith', 'delivery': '12:00', 'items': ['item1', 'item2']}
Example 4:
Suppose you want to send data that is not form-encoded but using dictionary-formatted input. To do that, you need to use json.dumps() function to serialize the input dictionary into a string. You can read more here: https://docs.python.org/3/library/json.html#json.dumps.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 |
import requests import json url = 'https://httpbin.org/post' # data as a Python dictionary data = {"key1": "value1", "key2": None } # Serializing data into JSON string using json.dumps() data = json.dumps(data) # Posting the data response = requests.post(url, data=data) # Convert the response into a text response = response.text print(response) |
Output (truncated):
{
"args": {},
"data": "{\"key1\": \"value1\", \"key2\": null}",
"files": {},
"form": {},
…,
"json": {
"key1": "value1",
"key2": null
},
"origin": "105.161.11.227",
"url": "https://httpbin.org/post"
}
As expected, the output shows that the data was posted directly instead of being form-encoded. Additionally, the values for the “json” property on the response have also been automatically generated.
Conclusion
Sending form data to a site using requests is as simple as passing the data in dictionary format. The five examples presented in this post covered all the caveats that may be faced when posting form data using Python requests.