Skip to content

Instantly share code, notes, and snippets.

@georgf
Created May 18, 2018 10:20
Show Gist options
  • Save georgf/7bdf97d7a1ce9f6442bc75b8cd1f0c2b to your computer and use it in GitHub Desktop.
Save georgf/7bdf97d7a1ce9f6442bc75b8cd1f0c2b to your computer and use it in GitHub Desktop.
quick test of msgpack vs. json encoding
import os
import sys
import datetime
import json
import umsgpack
def time(what, fn):
start = datetime.datetime.now()
fn()
end = datetime.datetime.now()
print what + " took " + str(end - start)
paths = sys.argv[1:]
data = []
print "... loading from disk"
for path in paths:
with open(path) as f:
data.append(json.load(f))
print "... timing json encoding of " + str(len(data)) + " files"
def json_encoding():
for d in data:
json.dumps(d)
time("json encoding", json_encoding)
end = datetime.datetime.now()
print "... timing msgpack encoding of " + str(len(data)) + " files"
def msgpack_encoding():
for d in data:
umsgpack.packb(d)
time("msgpack encoding", msgpack_encoding)
@georgf
Copy link
Author

georgf commented May 18, 2018

At least the Python implementations encoding speed doesn't seem promising:

... loading from disk
... timing json encoding of 1385 files
json encoding took 0:00:02.195731
... timing msgpack encoding of 1385 files
msgpack encoding took 0:00:33.083538

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment