Send Form Data With Requests in Python

The requests library is a great tool for making requests to the web using Python. This article will discuss using this module to send form data to a site. In particular, we will be using the requests.post(url, data=None, json=None) function.

For demonstration purposes, we will send POST requests to https://httpbin.org – an open-source site that allows one to inspect HTTPS requests before using them in production.

Example 1

Let’s start by posting a string to httpbin.org and see the result.

Output (truncated):

{
  "args": {},
  "data": "value1",
  "files": {},
  "form": {},
  …
  "json": null,
  "origin": "105.161.11.xxx",
  "url": "https://httpbin.org/post"
}

The server response in the output shows the data value we passed into requests.post() is posted under “data”. Form data should be posted on the “form” property. It is empty at the moment. Let’s post some form data in the following example.

Example 2

To pass form-encoded data, just like you will do on HTML forms, you need to pass dictionary-formatted items to the data argument on requests.posts().

The dictionary will be automatically passed as form data when the POST request is made. Here is an example.

Output (truncated):

{
  "args": {},
  "data": "",
  "files": {},
  "form": {
	"comments": "this is a comment",
	"custemail": "[email protected]",
	"custname": "Karl",
	"custtel": "01883",
	"delivery": "12:00",
	"size": "large",
	"topping": "cheese"
  },
  …
  "origin": "105.161.11.xxx",
  "url": "https://httpbin.org/post"
}

As shown in the output, the dictionary data was passed as a form when the request was made. That is why, in this example, the “form” property of the response has the data we passed to requests.post() function. Let’s go into another interesting example.

Example 3

The requests.post() also allows the user to pass multiple values to a single dictionary key (form property). This can be done in two ways:

  • Using a list of tuples. Here is an example with two values for the “items” property.
  • Using a dictionary with all possible values of a given key in a list. The example in the first method can be represented as:

Let’s post that data using requests and parse the response as a JSON object.

Output (truncated):

Full JSON response:  {'args': {}, 'data': '', 'files': {}, 'form': {'customer_name': 'Smith', 'delivery': '12:00', 'items': ['item1', 'item2']}, … , 'url': 'https://httpbin.org/post'}
Extracting the form property:  {'customer_name': 'Smith', 'delivery': '12:00', 'items': ['item1', 'item2']}

Example 4:

Suppose you want to send data that is not form-encoded but using dictionary-formatted input. To do that, you need to use json.dumps() function to serialize the input dictionary into a string. You can read more here: https://docs.python.org/3/library/json.html#json.dumps.

Output (truncated):

{
  "args": {},
  "data": "{\"key1\": \"value1\", \"key2\": null}",
  "files": {},
  "form": {},
  …,
  "json": {
	"key1": "value1",
	"key2": null
  },
  "origin": "105.161.11.227",
  "url": "https://httpbin.org/post"
}

As expected, the output shows that the data was posted directly instead of being form-encoded. Additionally, the values for the “json” property on the response have also been automatically generated.

Conclusion

Sending form data to a site using requests is as simple as passing the data in dictionary format. The five examples presented in this post covered all the caveats that may be faced when posting form data using Python requests.