Skip to content

Instantly share code, notes, and snippets.

@lorneli
Last active March 18, 2017 16:12

Revisions

  1. lorneli revised this gist Mar 18, 2017. 1 changed file with 1 addition and 2 deletions.
    3 changes: 1 addition & 2 deletions benchmark-redis-store-large-value
    Original file line number Diff line number Diff line change
    @@ -47,8 +47,7 @@ length: 512M, spend: 5.3249s, memory 74.0%, swap 8.8%
    This test executes on a 2GB memory machine. After preparing buffers in script, memory usage has grown up to 59%.
    When setting a 512M value, machine runs out of memory and instead uses swap partition. The corresponding elapsed
    time, 5s, is nearly unacceptable for server.



    Before 512M, elapsed time increases linearly with the size of value. A 64M value will block others about 100ms on
    this machine.

  2. lorneli revised this gist Mar 18, 2017. 1 changed file with 2 additions and 2 deletions.
    4 changes: 2 additions & 2 deletions benchmark-redis-store-large-value
    Original file line number Diff line number Diff line change
    @@ -1,4 +1,4 @@
    Since redis uses single thread to process client requests one by one, storing large value in SET request blocks others.
    Since redis uses single thread to process client requests one by one, storing large value for a key blocks others.
    I tested how long this takes in different size value.

    1.Test environment
    @@ -9,7 +9,7 @@ I tested how long this takes in different size value.
    2.Procedure
    Disable persistence.
    Prepare buffers to be sent
    Send request and measure elapsed time and memory usage.
    Send request. Measure elapsed time and memory usage.

    3.Script

  3. lorneli created this gist Mar 18, 2017.
    54 changes: 54 additions & 0 deletions benchmark-redis-store-large-value
    Original file line number Diff line number Diff line change
    @@ -0,0 +1,54 @@
    Since redis uses single thread to process client requests one by one, storing large value in SET request blocks others.
    I tested how long this takes in different size value.

    1.Test environment
    4core 2gb
    ubuntu server 14.04
    redis-version: 3.2.8

    2.Procedure
    Disable persistence.
    Prepare buffers to be sent
    Send request and measure elapsed time and memory usage.

    3.Script

    def benchmark_large_value():
    # Prepare values
    lengths = [1, 4, 32, 64, 128, 256, 512] # M
    values = []
    with open("/dev/urandom", "rb") as f:
    for length in lengths:
    value = f.read(length * 1024 * 1024)
    values.append(value)

    r = redis.StrictRedis()
    # Send request and measure elapsed time
    for value, length in zip(values, lengths):
    key = "key-%d" % length
    start = time.time()
    r.set(key, value)
    end = time.time()
    print 'length: %3dM, spend: %.4fs, memory %.1f%%, swap %.1f%%' % \
    (length, end-start, psutil.virtual_memory().percent, psutil.swap_memory().percent)

    4.Result

    length: 1M, spend: 0.0084s, memory 59.0%, swap 0.0%
    length: 4M, spend: 0.0117s, memory 59.6%, swap 0.0%
    length: 32M, spend: 0.0626s, memory 62.6%, swap 0.0%
    length: 64M, spend: 0.1335s, memory 67.6%, swap 0.0%
    length: 128M, spend: 0.2659s, memory 77.3%, swap 0.0%
    length: 256M, spend: 0.5842s, memory 96.6%, swap 0.0%
    length: 512M, spend: 5.3249s, memory 74.0%, swap 8.8%

    5.Conclusion

    This test executes on a 2GB memory machine. After preparing buffers in script, memory usage has grown up to 59%.
    When setting a 512M value, machine runs out of memory and instead uses swap partition. The corresponding elapsed
    time, 5s, is nearly unacceptable for server.


    Before 512M, elapsed time increases linearly with the size of value. A 64M value will block others about 100ms on
    this machine.