Base16, Base32 and Base64 in .NET

Encoding.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "1c383da1",
   "metadata": {},
   "source": [
    "# Binary to text data conversion\n",
    "\n",
    "* Encoding binary data to text string for various purposes.\n",
    "* It's encoding, not encryption! \n",
    "* Although often used in crypto contexts.\n",
    "* Commonly used methods defined in [RFC4648](https://www.rfc-editor.org/info/rfc4648)."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "20df17ee",
   "metadata": {},
   "source": [
    "## Very long number\n",
    "\n",
    "* Take any data as a (often) very long number - sequence of bits.\n",
    "* We can use different number bases to display that number, ie.:\n",
    "    * Base2 - binary,\n",
    "    * Base256 - 8-bit data,\n",
    "    * other bases, if we need special properties.\n",
    "\n",
    "Base | Value\n",
    "---- | ----------------------------------------------\n",
    "   2 | `01001000 01000101 01001100 01001100 01001111`\n",
    "  10 | `310 400 273 487`\n",
    "  16 | `48 45 4C 4C 4F`\n",
    " 256 | `H E L L O`"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "57b47d19",
   "metadata": {},
   "source": [
    "## Base64\n",
    "\n",
    "* It uses the alphabet `A...Za...z0...9+/` (64 characters), `=` is used for padding at the end.\n",
    "* Converts 8-bit to 6-bit alphabet.\n",
    "* Increases data volume by approximately one third."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "fe33aa6f",
   "metadata": {
    "language_info": {
     "name": "polyglot-notebook"
    },
    "polyglot_notebook": {
     "kernelName": "csharp"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div class=\"dni-plaintext\"><pre>[ 101, 42, 249, 34, 31, 236, 33, 93, 11, 212, 255, 132, 89, 89, 223, 138, 246, 182, 54, 176 ... (12 more) ]</pre></div><style>\r\n",
       ".dni-code-hint {\r\n",
       "    font-style: italic;\r\n",
       "    overflow: hidden;\r\n",
       "    white-space: nowrap;\r\n",
       "}\r\n",
       ".dni-treeview {\r\n",
       "    white-space: nowrap;\r\n",
       "}\r\n",
       ".dni-treeview td {\r\n",
       "    vertical-align: top;\r\n",
       "    text-align: start;\r\n",
       "}\r\n",
       "details.dni-treeview {\r\n",
       "    padding-left: 1em;\r\n",
       "}\r\n",
       "table td {\r\n",
       "    text-align: start;\r\n",
       "}\r\n",
       "table tr { \r\n",
       "    vertical-align: top; \r\n",
       "    margin: 0em 0px;\r\n",
       "}\r\n",
       "table tr td pre \r\n",
       "{ \r\n",
       "    vertical-align: top !important; \r\n",
       "    margin: 0em 0px !important;\r\n",
       "} \r\n",
       "table th {\r\n",
       "    text-align: start;\r\n",
       "}\r\n",
       "</style>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/plain": [
       "ZSr5Ih/sIV0L1P+EWVnfiva2NrCDFgyMWeq+hZModR0="
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<div class=\"dni-plaintext\"><pre>[ 101, 42, 249, 34, 31, 236, 33, 93, 11, 212, 255, 132, 89, 89, 223, 138, 246, 182, 54, 176 ... (12 more) ]</pre></div><style>\r\n",
       ".dni-code-hint {\r\n",
       "    font-style: italic;\r\n",
       "    overflow: hidden;\r\n",
       "    white-space: nowrap;\r\n",
       "}\r\n",
       ".dni-treeview {\r\n",
       "    white-space: nowrap;\r\n",
       "}\r\n",
       ".dni-treeview td {\r\n",
       "    vertical-align: top;\r\n",
       "    text-align: start;\r\n",
       "}\r\n",
       "details.dni-treeview {\r\n",
       "    padding-left: 1em;\r\n",
       "}\r\n",
       "table td {\r\n",
       "    text-align: start;\r\n",
       "}\r\n",
       "table tr { \r\n",
       "    vertical-align: top; \r\n",
       "    margin: 0em 0px;\r\n",
       "}\r\n",
       "table tr td pre \r\n",
       "{ \r\n",
       "    vertical-align: top !important; \r\n",
       "    margin: 0em 0px !important;\r\n",
       "} \r\n",
       "table th {\r\n",
       "    text-align: start;\r\n",
       "}\r\n",
       "</style>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "// Generate random data \n",
    "var randomData = new byte[32];\n",
    "Random.Shared.NextBytes(randomData);\n",
    "randomData.Display();\n",
    "\n",
    "// Convert to Base64 string\n",
    "var randomString = Convert.ToBase64String(randomData);\n",
    "randomString.Display();\n",
    "\n",
    "// Convert back to byte array\n",
    "var decodedData = Convert.FromBase64String(randomString);\n",
    "decodedData.Display();"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "36a4892b",
   "metadata": {},
   "source": [
    "The standard alphabet contains `+` and `/` characters that can cause problems in URLs. They can be replaced with `-` and `_` for URL-safe Base64. The same goes for `=` padding, which can be omitted. The `Base64Url` class is available since .NET 9."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "2a4f131e",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "ZSr5Ih_sIV0L1P-EWVnfiva2NrCDFgyMWeq-hZModR0"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<div class=\"dni-plaintext\"><pre>[ 101, 42, 249, 34, 31, 236, 33, 93, 11, 212, 255, 132, 89, 89, 223, 138, 246, 182, 54, 176 ... (12 more) ]</pre></div><style>\r\n",
       ".dni-code-hint {\r\n",
       "    font-style: italic;\r\n",
       "    overflow: hidden;\r\n",
       "    white-space: nowrap;\r\n",
       "}\r\n",
       ".dni-treeview {\r\n",
       "    white-space: nowrap;\r\n",
       "}\r\n",
       ".dni-treeview td {\r\n",
       "    vertical-align: top;\r\n",
       "    text-align: start;\r\n",
       "}\r\n",
       "details.dni-treeview {\r\n",
       "    padding-left: 1em;\r\n",
       "}\r\n",
       "table td {\r\n",
       "    text-align: start;\r\n",
       "}\r\n",
       "table tr { \r\n",
       "    vertical-align: top; \r\n",
       "    margin: 0em 0px;\r\n",
       "}\r\n",
       "table tr td pre \r\n",
       "{ \r\n",
       "    vertical-align: top !important; \r\n",
       "    margin: 0em 0px !important;\r\n",
       "} \r\n",
       "table th {\r\n",
       "    text-align: start;\r\n",
       "}\r\n",
       "</style>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "// Convert to URL-safe Base64 string\n",
    "var urlSafeString = System.Buffers.Text.Base64Url.EncodeToString(randomData);\n",
    "urlSafeString.Display();\n",
    "\n",
    "// Convert back to byte array\n",
    "var urlSafeDecodedData = System.Buffers.Text.Base64Url.DecodeFromChars(urlSafeString);\n",
    "urlSafeDecodedData.Display();"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "488d902c",
   "metadata": {},
   "source": [
    "If encoding larger amout of data, lines are often limited to 76 characters."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "cfb30a44",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "7T76od5wa4VoY0Lmf2C9LYVU8Vum751MIO3U3CinUXLkfhFgzN45VONkU+7LMMDkNxxieAByCTe5\r\n",
       "VMsCwwZkdonZd0m980oawR2nfnsN6WUtcMW2lEkgumTlTs3+Gj/nO70mmobEmPUOTW2cbViSwSJ3\r\n",
       "vX7apf1DEpuIPwwBmngtOE/jmn9fs2aY+DUj77x3lHAdtx7HnlBazEr7kpVcJUlK8dSr6dDu9TXE\r\n",
       "tFtfK21pCt9ekADqOawAYnQKbGICNnBhKyNflXbGGHOMFG/h/cmIaBaDVrnKn+Xc0lOUiMFlVlJP\r\n",
       "XZLwalNd2XSL+rSl7sTuuS/DdGdLJvtAjBk46Q=="
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "var bigRandomData = new byte[256];\n",
    "Random.Shared.NextBytes(bigRandomData);\n",
    "var bigRandomString = Convert.ToBase64String(bigRandomData, Base64FormattingOptions.InsertLineBreaks);\n",
    "bigRandomString.Display();"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1b81dab2",
   "metadata": {},
   "source": [
    "## Base16\n",
    "\n",
    "* Simply converts the binary data to their hexadecimal representations.\n",
    "* Uses `0-9A-F` or `0-9a-f` alphabet.\n",
    "* Recommended to be treated as case insensitive.\n",
    "* Doubles the amount of data.\n",
    "* Recommended for smaller amount of data, like hashes.\n",
    "* Often prefixed with `0x...`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "3d903764",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "652AF9221FEC215D0BD4FF845959DF8AF6B636B083160C8C59EABE859328751D"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/plain": [
       "652af9221fec215d0bd4ff845959df8af6b636b083160c8c59eabe859328751d"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<div class=\"dni-plaintext\"><pre>[ 101, 42, 249, 34, 31, 236, 33, 93, 11, 212, 255, 132, 89, 89, 223, 138, 246, 182, 54, 176 ... (12 more) ]</pre></div><style>\r\n",
       ".dni-code-hint {\r\n",
       "    font-style: italic;\r\n",
       "    overflow: hidden;\r\n",
       "    white-space: nowrap;\r\n",
       "}\r\n",
       ".dni-treeview {\r\n",
       "    white-space: nowrap;\r\n",
       "}\r\n",
       ".dni-treeview td {\r\n",
       "    vertical-align: top;\r\n",
       "    text-align: start;\r\n",
       "}\r\n",
       "details.dni-treeview {\r\n",
       "    padding-left: 1em;\r\n",
       "}\r\n",
       "table td {\r\n",
       "    text-align: start;\r\n",
       "}\r\n",
       "table tr { \r\n",
       "    vertical-align: top; \r\n",
       "    margin: 0em 0px;\r\n",
       "}\r\n",
       "table tr td pre \r\n",
       "{ \r\n",
       "    vertical-align: top !important; \r\n",
       "    margin: 0em 0px !important;\r\n",
       "} \r\n",
       "table th {\r\n",
       "    text-align: start;\r\n",
       "}\r\n",
       "</style>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<div class=\"dni-plaintext\"><pre>[ 101, 42, 249, 34, 31, 236, 33, 93, 11, 212, 255, 132, 89, 89, 223, 138, 246, 182, 54, 176 ... (12 more) ]</pre></div><style>\r\n",
       ".dni-code-hint {\r\n",
       "    font-style: italic;\r\n",
       "    overflow: hidden;\r\n",
       "    white-space: nowrap;\r\n",
       "}\r\n",
       ".dni-treeview {\r\n",
       "    white-space: nowrap;\r\n",
       "}\r\n",
       ".dni-treeview td {\r\n",
       "    vertical-align: top;\r\n",
       "    text-align: start;\r\n",
       "}\r\n",
       "details.dni-treeview {\r\n",
       "    padding-left: 1em;\r\n",
       "}\r\n",
       "table td {\r\n",
       "    text-align: start;\r\n",
       "}\r\n",
       "table tr { \r\n",
       "    vertical-align: top; \r\n",
       "    margin: 0em 0px;\r\n",
       "}\r\n",
       "table tr td pre \r\n",
       "{ \r\n",
       "    vertical-align: top !important; \r\n",
       "    margin: 0em 0px !important;\r\n",
       "} \r\n",
       "table th {\r\n",
       "    text-align: start;\r\n",
       "}\r\n",
       "</style>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "// Convert to Base16 (Hexadecimal) string\n",
    "var hexStringUpper = Convert.ToHexString(randomData);\n",
    "hexStringUpper.Display();\n",
    "\n",
    "var hexStringLower = Convert.ToHexString(randomData).ToLowerInvariant();\n",
    "hexStringLower.Display();\n",
    "\n",
    "// Convert back to byte array\n",
    "var hexDecodedDataUpper = Convert.FromHexString(hexStringUpper);\n",
    "hexDecodedDataUpper.Display();\n",
    "\n",
    "var hexDecodedDataLower = Convert.FromHexString(hexStringLower);\n",
    "hexDecodedDataLower.Display();"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "03169a70",
   "metadata": {},
   "source": [
    "## Base32\n",
    "\n",
    "* Middle ground between Base16 and Base32.\n",
    "* Overhead 60%\n",
    "* Various 32-characters alphabet - many competing standards.\n",
    "* Not supported natively in .NET\n",
    "* There is a package for that - [SimpleBase](https://github.com/ssg/SimpleBase), supports many different encodings."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "5de42f1b",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div><div></div><div></div><div><strong>Installed Packages</strong><ul><li><span>SimpleBase, 5.4.1</span></li></ul></div></div>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "#r \"nuget: SimpleBase, *\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "9482a32e",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "MUVPSIQ75QQV2C6U76CFSWO7RL3LMNVQQMLAZDCZ5K7ILEZIOUOQ"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<div class=\"dni-plaintext\"><pre>[ 101, 42, 249, 34, 31, 236, 33, 93, 11, 212, 255, 132, 89, 89, 223, 138, 246, 182, 54, 176 ... (12 more) ]</pre></div><style>\r\n",
       ".dni-code-hint {\r\n",
       "    font-style: italic;\r\n",
       "    overflow: hidden;\r\n",
       "    white-space: nowrap;\r\n",
       "}\r\n",
       ".dni-treeview {\r\n",
       "    white-space: nowrap;\r\n",
       "}\r\n",
       ".dni-treeview td {\r\n",
       "    vertical-align: top;\r\n",
       "    text-align: start;\r\n",
       "}\r\n",
       "details.dni-treeview {\r\n",
       "    padding-left: 1em;\r\n",
       "}\r\n",
       "table td {\r\n",
       "    text-align: start;\r\n",
       "}\r\n",
       "table tr { \r\n",
       "    vertical-align: top; \r\n",
       "    margin: 0em 0px;\r\n",
       "}\r\n",
       "table tr td pre \r\n",
       "{ \r\n",
       "    vertical-align: top !important; \r\n",
       "    margin: 0em 0px !important;\r\n",
       "} \r\n",
       "table th {\r\n",
       "    text-align: start;\r\n",
       "}\r\n",
       "</style>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/plain": [
       "CMNFJ8GZXGGNT2YMZY25JPEZHBVBCDNGGCB0S32SXAZ8B4S8EMEG"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<div class=\"dni-plaintext\"><pre>[ 101, 42, 249, 34, 31, 236, 33, 93, 11, 212, 255, 132, 89, 89, 223, 138, 246, 182, 54, 176 ... (12 more) ]</pre></div><style>\r\n",
       ".dni-code-hint {\r\n",
       "    font-style: italic;\r\n",
       "    overflow: hidden;\r\n",
       "    white-space: nowrap;\r\n",
       "}\r\n",
       ".dni-treeview {\r\n",
       "    white-space: nowrap;\r\n",
       "}\r\n",
       ".dni-treeview td {\r\n",
       "    vertical-align: top;\r\n",
       "    text-align: start;\r\n",
       "}\r\n",
       "details.dni-treeview {\r\n",
       "    padding-left: 1em;\r\n",
       "}\r\n",
       "table td {\r\n",
       "    text-align: start;\r\n",
       "}\r\n",
       "table tr { \r\n",
       "    vertical-align: top; \r\n",
       "    margin: 0em 0px;\r\n",
       "}\r\n",
       "table tr td pre \r\n",
       "{ \r\n",
       "    vertical-align: top !important; \r\n",
       "    margin: 0em 0px !important;\r\n",
       "} \r\n",
       "table th {\r\n",
       "    text-align: start;\r\n",
       "}\r\n",
       "</style>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/plain": [
       "cwix1eo97ooi4n6w96nf1sq9tm5mcpioocmy3dn37k9emr3eqwqo"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<div class=\"dni-plaintext\"><pre>[ 101, 42, 249, 34, 31, 236, 33, 93, 11, 212, 255, 132, 89, 89, 223, 138, 246, 182, 54, 176 ... (12 more) ]</pre></div><style>\r\n",
       ".dni-code-hint {\r\n",
       "    font-style: italic;\r\n",
       "    overflow: hidden;\r\n",
       "    white-space: nowrap;\r\n",
       "}\r\n",
       ".dni-treeview {\r\n",
       "    white-space: nowrap;\r\n",
       "}\r\n",
       ".dni-treeview td {\r\n",
       "    vertical-align: top;\r\n",
       "    text-align: start;\r\n",
       "}\r\n",
       "details.dni-treeview {\r\n",
       "    padding-left: 1em;\r\n",
       "}\r\n",
       "table td {\r\n",
       "    text-align: start;\r\n",
       "}\r\n",
       "table tr { \r\n",
       "    vertical-align: top; \r\n",
       "    margin: 0em 0px;\r\n",
       "}\r\n",
       "table tr td pre \r\n",
       "{ \r\n",
       "    vertical-align: top !important; \r\n",
       "    margin: 0em 0px !important;\r\n",
       "} \r\n",
       "table th {\r\n",
       "    text-align: start;\r\n",
       "}\r\n",
       "</style>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "// Convert to Base32 string using RFC4648 alphabet\n",
    "var base32Encoded = SimpleBase.Base32.Rfc4648.Encode(randomData);\n",
    "base32Encoded.Display();\n",
    "\n",
    "// Convert back to byte array\n",
    "var base32Decoded = SimpleBase.Base32.Rfc4648.Decode(base32Encoded);\n",
    "base32Decoded.Display();\n",
    "\n",
    "// Convert to Base32 string using Crockford's alphabet\n",
    "var base32CrockfordEncoded = SimpleBase.Base32.Crockford.Encode(randomData);\n",
    "base32CrockfordEncoded.Display();\n",
    "\n",
    "// Convert back to byte array\n",
    "var base32CrockfordDecoded = SimpleBase.Base32.Crockford.Decode(base32CrockfordEncoded);\n",
    "base32CrockfordDecoded.Display();\n",
    "\n",
    "// Convert to Base32 string using ZBase32 alphabet\n",
    "var base32ZBase32Encoded = SimpleBase.Base32.ZBase32.Encode(randomData);\n",
    "base32ZBase32Encoded.Display();\n",
    "// Convert back to byte array\n",
    "var base32ZBase32Decoded = SimpleBase.Base32.ZBase32.Decode(base32ZBase32Encoded);\n",
    "base32ZBase32Decoded.Display();"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": ".NET (C#)",
   "language": "C#",
   "name": ".net-csharp"
  },
  "language_info": {
   "name": "polyglot-notebook"
  },
  "polyglot_notebook": {
   "kernelInfo": {
    "defaultKernelName": "csharp",
    "items": [
     {
      "aliases": [],
      "languageName": "csharp",
      "name": "csharp"
     }
    ]
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}