JSON data can exist in two forms: in a file or data saved in a variable stored in memory. This article will show how to compress and decompress JSON data in both formats.
Compressing JSON Data Stored in Python Dictionary Using Gzip
Compressing JSON data using gzip package is done in three steps:
- Convert the object passed into JSON string. The object must be JSON-serializable,
- Encode JSON string into bytes,
- Compress the result using gzip.compress() function.
Here is an example.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 |
import json import gzip def compress_JSON(data): # Convert serializable data to JSON string json_data = json.dumps(data, indent=2) # Convert JSON string to bytes encoded = json_data.encode("utf-8") # Compress compressed = gzip.compress(encoded) return compressed # A list of dictionaries - a JSON serializable object data1 = [ {"Name": "Allan", "Registration": 2709, "Address": "Chicago", "Marks": [56, 89, 72]}, {"Name": "Bob", "Registration": 2451, "Address": "Michigan", "Marks": None}, {"Name": "Allan", "AdRegistrationm": 2709, "Address": "Berlin", "Marks": [82, 88, 65]} ] # Call compress_JSON function to compress data1 compressed_json = compress_JSON(data1) print(compressed_json) |
Output (truncated):
b'\x1f\x8b\x08\x00FCvd\x02\xff\x8b…\x99)\xc2\xa3\\\xb1\x00t\xe1{8o\x01\x00\x00'
Then you can decompress the compressed data using gzip.decompress(), as shown below.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 |
import json import gzip def decompress_JSON(compressed_json): # Decompress a compressed JSON data to bytes and decode it to string data2 = gzip.decompress(compressed_json).decode("utf-8") # Convert JSON string into Python dictionary decompressed = json.loads(data2) return decompressed compressed_json = b"\x1f\x8b\x08\x00\xc8Bvd\x02\xff\x8b\xe6RP\xa8\x06b\x05\x05%\xbf\xc4\xdcT%+\x05%\xc7\x9c\x9c\xc4<%\x1d\x88`PjzfqIQbIf~\x1eP\xd2\xc8\xdc\xc0\x12*\xe3\x98\x92R\x94Z\\\x0c\xd2\xe1\x9c\x91\x99\x9c\x98\x9e\x0f\xd3\xe3\x9bX\x94\r\x12\x8f\x06s\x15\x14L\xcdt\xa0,\x0bK\x18\xcb\xdc\x08\xcc\x88\x05\x92\xb5:\x98np\xcaO\xc2\xe5\x02\x13SCL\x17\xf8f&gd\xa6#\x9c\rsB^iN\x0e\x0e+P\xbc\xe9\x98\x82lM.N\x9f:\xa5\x16\xe5d\xe6\xe1\xf2\xa8\x85\x11\xdc\xa3\x160\x96\x99)\xc2\xa3\\\xb1\x00t\xe1{8o\x01\x00\x00" decompressed = decompress_JSON(compressed_json) print(decompressed) |
Output:
[{'Name': 'Allan', 'Registration': 2709, 'Address': 'Chicago', 'Marks': [56, 89, 72]}, {'Name': 'Bob', 'Registration': 2451, 'Address': 'Michigan', 'Marks': None}, {'Name': 'Allan', 'AdRegistrationm': 2709, 'Address': 'Berlin', 'Marks': [82, 88, 65]}]
Compressing JSON Data into a Gzip File
If you have JSON data you want to send to a Gzip file, the following code example should do the job.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 |
import gzip import json data1 = [ {"Name": "Allan", "Registration": 2709, "Address": "Chicago", "Marks": [56, 89, 72]}, {"Name": "Bob", "Registration": 2451, "Address": "Michigan", "Marks": None}, {"Name": "Allan", "AdRegistrationm": 2709, "Address": "Berlin", "Marks": [82, 88, 65]} ] # Python dictionary into JSON (str) json_str = json.dumps(data1, indent=3) # JSON string into UTF-8 encoded bytes json_bytes = json_str.encode("utf-8") # Open GZIP file in write mode (w) and write JSON data into the GZIP with gzip.open("file1.json.gz", "w") as outfile: outfile.write(json_bytes) |
You can shorten the code above, as shown below.
|
1 2 3 4 5 6 7 8 9 10 11 12 |
import json, gzip data1 = [ {"Name": "Allan", "Registration": 2709, "Address": "Chicago" ,"Marks": [56, 89, 72]}, {"Name": "Bob", "Registration": 2451, "Address": "Michigan" ,"Marks": None}, {"Name": "Allan", "AdRegistrationm": 2709, "Address": "Berlin" ,"Marks": [82, 88, 65]} ] # The following with-statement: # > Opens GZIP file in write mode and writes data1 into the file. with gzip.open("file2.json.gz", "w") as outfile: outfile.write(json.dumps(data1, indent=3).encode("utf-8")) |
Then you can read JSON data from a Gzip file with the following code.
|
1 2 3 4 5 6 7 8 9 10 11 |
import gzip, json # Open gzip file and read content # The loaded contents in bytes with gzip.open("file1.json.gz", "r") as infile: json_bytes = infile.read() # Decode bytes into JSON string with UTF-8 encoding. json_str = json_bytes.decode("utf-8") # Convert JSON into Python dictionary data = json.loads(json_str) print(data) |
You can also shorten the code above as follows.
|
1 2 3 4 5 6 |
import json, gzip # Read GZIP file and convert the loaded bytes into Python dictionary with gzip.open("file2.json.gz", "r") as infile: data1 = json.loads(infile.read().decode("utf-8")) print(data1) |
Compress an Existing JSON File Using Gzip
This Section shows how to compress a JSON file into a .gz file using gzip module. The following code should serve the purpose.
|
1 2 3 4 5 6 7 8 9 |
import gzip import shutil # Open JSON file in binary read mode (rb) with open("./file2.json", "rb") as infile: # Open GZIP in binary write mode with gzip.open("./file44.json.gz", "wb") as outfile: # Copy contents of the JSON file into GZIP using shutil shutil.copyfileobj(infile, outfile) |