Simple compressing base64 string in JavaScript

simple-compressing.md

Story

In needed include base64 encoded simple image into HTML page in two formats:

1st as PNG format has length 196 characters and looks like this:

iVBORw0KGgoAAAANSUhEUgAAABAAAAAQCAYAAAAf8/9hAAAAWElEQVR42mNkwA/qgbgRnwJGAgb8BwI7RkbGw5QYUAs0oGXUAPwGgKKqgYF0ANLTyAi1xhZI2WOYzsjYDJTbC2QewGHIwcERBsPcgHqgAX8pMQAcxfhyIwATTkxL+hgX2QAAAABJRU5ErkJggg==

2nd as CUR format has length 1536 characters and looks like this:

AAACAAEAEBAAAAcABwBoBAAAFgAAACgAAAAQAAAAIAAAAAEAIAAAAAAAQAQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAP///z0AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAH////98AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAD/////fAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA/////3wAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAP////98AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAD/////fAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA/////3wAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAP///z3///98////fP///3z///98////fAAAAP////+8////fP///3z///98////fP///3z///98////PQAAAH8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAAfwAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA/////3wAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAP////98AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAD/////fAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA/////3wAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAP////98AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAD/////PQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAfwAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA/38AAP5/AAD+fwAA/n8AAP5/AAD+fwAA/n8AAIAAAAAAAQAA/n8AAP5/AAD+fwAA/n8AAP5/AAD+fwAA/v8AAA==

CUR format is amost 8 times bigger than PNG. And as you can see CUR format has repeating many characters. Therefore I used next simple technique to compress that.

Compressing

let compressedImg = imgCur.split('').reduce((o, c) => {
  if (o[o.length - 2] === c && o[o.length - 1] < 35) o[o.length - 1]++;
  else o.push(c, 0);
  return o;
},[]).map(_ => typeof _ === 'number' ? _.toString(36) : _).join('');

• This code replace a every consecutive equaly chars by single char and by count of chars minus one.
• Count of chars is desired to take only one char place and javascript natively supports to convert number into string by using max. 36-base format, therefore max length of consecutive equaly chars can be max. 35 characters. Compresed string has only 556 characters (which is about 36.2% of source size) and looks like:

A2C0A1E0A0E0B0A3c0A0B0w0B0o0B0A2F0g0A2C0g0A3Q0A3I0A4E0A0I0A6Q0A0Q0AzAtP0/2z000AzAzA5H0/39080AzAzA5D0/4f0AzAzA6/430w0AzAzA5P0/39080AzAzA5D0/4f0AzAzA6/430w0AzA5P0/2z030/29080/3f0P0/230z0/29080/3f0A3P0/3+080/3f0P0/230z0/29080/3f0P0/230z0/29080/3P0Q0A2H080A2D0/0A3/0w0A2P080A2D0/0A3/0w0A2P080A2D0/0A3/0w0A2P080A2D0/0A3/0w0A2P080A2D0/0A3f0w0AzA9/430w0AzAzA5P0/39080AzAzA5D0/4f0AzAzA6/430w0AzAzA5P0/39080AzAzA5D0/4P0Q0AzAzA5f0w0AzA5/03080A1P050/0A1D0+0f0w0A1/0n080A1P050/0A1D0+0f0w0A1/0n080A1I0A6Q0A1/0n080A1P050/0A1D0+0f0w0A1/0n080A1P050/0A1D0+0f0w0A1/0v080A2=1

Decompressing

let decompressedImgCur = compressedImgCur
  .split('').map((c,i,a)=>i%2?undefined:new Array(2+parseInt(a[i+1],36)).join(c)).join('');

• As you can see, decompressing code is even simpler than compressing one.
• Is intenationaly written in compressed way because it's length (88 chars) should be included in calculation of compression rate.
• So true compression rate is (556 + 88) / 1536 which is almost 42%.

If you are curious what the image looks like...

I have a problem of sending Base64String to AWS API, there is a limitation of 10 MB and my Base64String is bigger than that, how can use your method to compress the string and how can I decompres it in backend (C#)?

This simple compression method works only if there are many repeating characters. It is probably not suitable for base64 encoded string. You can try lz-string if is suitable for you case (there is also C# implementation). However compression helps, but is not universal solution. There must exists way to split data longer than 10MB, send parts separately and concat them at AWS.

I'm afraid I need to agree with @profiprog, I ran a few tests and the compressed result was longer than the not-compressed one

You are right @Rubioli, compressed result can easily be longer than the original. The compression rate depends on the data you have. This one is suitable only for images where bites are repeating many times. PNG is also using this characteristic, so it would never be suitable for this compression method. CUR format is suitable only when PNG format is also much smaller. Unfortunately I don't known how to convert PNG to CUR on client therefore I choose this simple strategy. po 9. 3. 2020 o 23:43 Rubioli <[email protected]> napísal(a):

…

I'm afraid I need to agree with @profiprog <https://github.com/profiprog>, I ran a few tests and the compressed result was longer than the not-compressed one — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <https://gist.github.com/88327d5cb4599b1370d1252d262dd42e?email_source=notifications&email_token=AAMPSL6NLO52NZCYB33KSNLRGVWILA5CNFSM4JFQWOSKYY3PNVWWK3TUL52HS4DFVNDWS43UINXW23LFNZ2KUY3PNVWWK3TUL5UWJTQAGDVUW#gistcomment-3205963>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAMPSL5XUA7ET2U67R5N5NTRGVWILANCNFSM4JFQWOSA> .

-- Milan Skuhra (konateľ spoločnosti ProfiProg s.r.o.) http://www.profiprog.com, [email protected] mobile: +421-949-122217

Hey, so I have these strings
nY1yGcMHw4wUOuicvKVen49O6dUqtLQD5qFgBvXC+pb88oLUyJItIvzJ/pfy64KMibSKhKv37br067uWjquNhq/nn7r58Y0h1+u7tqeKh7GFvqmYj6sl0ZDp6fjR0drz88CsrLrBwc/umqGOsqHN8aH95KJUyy9pDPYfAvFeXwpeWqFMZKF2AZGCoeYhoaGhp++loSGhkaGx4Oe9yoGgm+ZBoaVBoZsD8auNYu+dpbDolLQVgqM3/C4zhaAnOdBB5Jf7z7Gj0KWh14QsoVJR4VGAJ7nA46kx5eKp5aEwZ+iF0AmTZpXSooUz6IWnqp/hoaBhqqClu4kHPkGhqaGhmCGhoaGk9Hek1HbEIfjEIQ==
and
PKYbpgEQAQpGbHV0dGVyQmF02gFAI9SqZ5VTLcAvL8V5edGNZk4tJviZrdcyZsCKj6Jzd5lzd6F9gIxaX5tyf6JSbJ1LZt+On8mAj85+j6mLWUs/LjckHMGcYaCCU5Z6TYZtR15ON41YNHVGLW5IKmI8J0UsIHERCaoZDVsLBAAc5AYEWAIMABGgAFJgIQYAgoAAww5E0ACP/gABxAA5JDASUWZxt0hhOOOE002QMhPQ8IIhFISThAEqAQUIACpYACAY3gEEEEQQA4AWAV2GWm1QLjrz8CwAIAAeAAIQAAIoogAIoowAAIwAAIoogIAAAQA=
can someone help me decode it? it's compressed data.