Skip to content

Instantly share code, notes, and snippets.

@thmosqueiro
Created September 26, 2024 15:50
Show Gist options
  • Save thmosqueiro/ce35fe041f139d5783d1911fa8689f0f to your computer and use it in GitHub Desktop.
Save thmosqueiro/ce35fe041f139d5783d1911fa8689f0f to your computer and use it in GitHub Desktop.
Pytorch example
Display the source blob
Display the rendered blob
Raw
{
"cells": [
{
"cell_type": "markdown",
"id": "a25cbbef-5225-4dfe-b01c-c01d06b293bc",
"metadata": {},
"source": [
"# PyTorch & MNIST Intro"
]
},
{
"cell_type": "markdown",
"id": "ec6ffb13-d039-4023-b897-118a134a9fa0",
"metadata": {},
"source": [
"Let's go through a simple example of Pytorch and MNIST as a way to level set NNs and the use of notebooks."
]
},
{
"cell_type": "code",
"execution_count": 2,
"id": "b3622662-4d3f-4b42-8f1c-d1b63177488b",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"<torch._C.Generator at 0x7f9221c41570>"
]
},
"execution_count": 2,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"import torch\n",
"import torchvision\n",
"import matplotlib.pyplot as pl\n",
"random_seed = 1\n",
"torch.backends.cudnn.enabled = False\n",
"torch.manual_seed(random_seed)"
]
},
{
"cell_type": "markdown",
"id": "f094b84f-d36b-4936-9364-a6812076d007",
"metadata": {},
"source": [
"## Dataset"
]
},
{
"cell_type": "markdown",
"id": "d86ff127-447e-4854-aa6c-e794e4875867",
"metadata": {},
"source": [
"MNIST is probably the most traditionally used dataset for neural networks, as it is a relatively challenging problem in computer vision: recognizing single-digit numbers from a hand-written digital format. Classically, this dataset takes the form of samples of $28 \\times 28$ matrices."
]
},
{
"cell_type": "code",
"execution_count": 39,
"id": "9e07bfef-c64c-4926-bf5c-d4955da85817",
"metadata": {},
"outputs": [],
"source": [
"batch_size_train = 64\n",
"batch_size_test = 1000"
]
},
{
"cell_type": "code",
"execution_count": 4,
"id": "b469c45f-cc76-4308-b302-5288192a927c",
"metadata": {},
"outputs": [],
"source": [
"train_loader = torch.utils.data.DataLoader(\n",
" torchvision.datasets.MNIST('./files/', train=True, download=True,\n",
" transform=torchvision.transforms.Compose([\n",
" torchvision.transforms.ToTensor(),\n",
" torchvision.transforms.Normalize((0.1307,), (0.3081,))\n",
" ])),\n",
" batch_size=batch_size_train,\n",
" shuffle=True)\n",
"\n",
"test_loader = torch.utils.data.DataLoader(\n",
" torchvision.datasets.MNIST('./files/', train=False, download=True,\n",
" transform=torchvision.transforms.Compose([\n",
" torchvision.transforms.ToTensor(),\n",
" torchvision.transforms.Normalize(\n",
" (0.1307,), (0.3081,))\n",
" ])),\n",
" batch_size=batch_size_test,\n",
" shuffle=True)"
]
},
{
"cell_type": "code",
"execution_count": 5,
"id": "6c0c70ae-311c-41fe-9780-9cb91f98a991",
"metadata": {},
"outputs": [],
"source": [
"examples = enumerate(test_loader)\n",
"batch_idx, (example_data, example_targets) = next(examples)"
]
},
{
"cell_type": "code",
"execution_count": 6,
"id": "493a311a-0090-4780-8c76-5a0ebca43e52",
"metadata": {},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAmYAAAGkCAYAAACb5OmoAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjguNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8fJSN1AAAACXBIWXMAAA9hAAAPYQGoP6dpAAAvyklEQVR4nO3de3xU5Z3H8e8kGJNAoBCQS7jflYuvciuIIhdFQEDkKlUuJaIstYggFcg2QETUUEC2CxZ3BS9VaBQRLIgXjBYQROq6AZRlaQsFgySE5RoghMz+QUkb8xwyJzOTeSb5vF8v/sg3J+f8JjkP+eWZeebxeL1erwAAABByEaEuAAAAAFfRmAEAAFiCxgwAAMASNGYAAACWoDEDAACwBI0ZAACAJWjMAAAALEFjBgAAYAkaMwAAAEuEtDHLyMhQYmKimjVrppiYGMXExKhFixZ69NFHtXv37lCW5jePx6O5c+c6fr5nz57yeDwl/rveOXyRm5uruXPn6tNPPy32ublz58rj8ejEiRN+XeOHkpKS9OMf/1g1atRQdHS0mjZtqkceeUSHDx8O6HVgxrgqn+MqLy9PycnJatKkiaKiotSoUSPNmjVLFy5cCOh1YMa4Kp/j6p9duHBBLVu2lMfj0a9//eugXacklUJ14RUrVuixxx5Tq1at9Pjjj6tNmzbyeDz69ttvtXr1anXu3FkHDx5Us2bNQlViUC1fvlxnzpwp/Hjjxo2aP3++Vq1apdatWxfm9evX9+s6ubm5mjdvnqSrg6ssnDp1SqNHj9bNN9+suLg4ffPNN5o/f742bNigffv2KT4+vkzqqIgYV+V3XI0ePVqbNm1ScnKyOnfurB07dmj+/Pnat2+fNmzYUCY1VFSMq/I7rv7Zr371K50/f77Mr/tDIWnMtm/frsmTJ+vee+/V22+/raioqMLP9e7dWz//+c/11ltvKSYm5rrnyc3NVWxsbLDLDYpbbrmlyMf79++XJLVt21adOnVy/LpweMzLli0r8nHPnj3VpEkTDRgwQOvXr9eECRNCVFn5xrgqv+Nq586deuedd7Ro0SJNmzZNknTXXXepUqVKmj17tj766CPdfffdIa6yfGJcld9x9c927dql3/zmN3rjjTc0YsSIkNYSkqcyFyxYoMjISK1YsaLITf7PRowYoXr16hV+PH78eFWpUkV79uxR3759FRcXpz59+kiSTp48qcmTJyshIUFRUVFq2rSpkpKSdOnSpcKvP3TokDwej1555ZVi1/rhFOy1KdN9+/Zp9OjRqlatmmrXrq0JEybo9OnTRb72zJkzmjhxouLj41WlShX169dPBw4c8OO78w/X6vjqq680fPhwVa9evfAvsp49exr/ohg/frwaN25c+Jhr1aolSZo3b17hdPP48eOLfM3x48dLfJz+ulZHpUohm6Qt9xhXvgnHcbV9+3ZJ0oABA4rkAwcOlCStXbu2VOdFyRhXvgnHcXVNXl6eJkyYoJ///OfXbTTLSpn/lrxy5YrS09PVqVMn1a1b19XX5uXlafDgwXr00Uc1c+ZM5efn6+LFi+rVq5f+/Oc/a968eWrfvr22bt2qZ599Vl9//bU2btxY6lqHDRumUaNGKTExUXv27NGsWbMkSStXrpQkeb1eDRkyRJ9//nnh0wvbt29X//79S31Nk6FDh+qBBx7QpEmTXE2z1q1bV5s3b1a/fv2UmJiohx9+WNI/mqRrSnqc0tVBN2/ePKWnp/s8xZyfn6/Lly9r//79mjp1qlq2bKmhQ4f6XD98x7hyL5zGVV5eniTpxhtvLJJf+zgjI8Pn+uE7xpV74TSurklJSdH58+f19NNPKzs72+eag6XMG7MTJ07owoULatSoUbHPXblyRV6vt/DjyMhIeTyewo8vX76s5ORk/exnPyvMVqxYoYyMDKWlpRVOP959992qUqWKnnrqKb+m+BMTEzVjxgxJV582OHjwoFauXKmXX35ZHo9HH3zwgdLT07V06VJNmTKl8NpRUVFKSkoq1TVNxo0bV/i8uxs33nijOnbsKOnqc/9du3Y1HlfS45SkiIiIYj+P6/n++++L/Ef2k5/8ROnp6apSpYrrx4GSMa7cC6dxde2ppO3bt6tJkyaF+bZt2yRJOTk5rh8HSsa4ci+cxpUkff3110pNTdV7772nypUrW9GYWfV2GR07dtQNN9xQ+G/RokXFjhk2bFiRjz/55BNVrlxZw4cPL5Jfm/7csmVLqesZPHhwkY/bt2+vixcvKisrS5KUnp4uSXrwwQeLHPfTn/601Nc0+eFjDrSSHqckJScnKz8/X3feeadP56xZs6a+/PJLbdu2Tf/xH/+hkydPqlevXjp27FhAa0fJGFdm4TSu+vfvr+bNmxf+8j516pQ2b96s2bNnKzIyUhERVv1XXiEwrszCaVzl5+drwoQJGjVqlO65556g1FsaZT5jVrNmTcXExBjfOuHNN99Ubm6ujh07VuybL0mxsbGqWrVqkSwnJ0d16tQp1hnfdNNNqlSpkl9/Sf5w9eC1pw2uLU/PyclRpUqVih1Xp06dUl/TxO0UulslPc7SqFSpUuFz9d27d1e/fv3UpEkTPffcc1q6dGnpi4UR48q9cBpXUVFRev/99zVmzBj17dtXklS5cmUtWLBATz/9tBISEvwvGMUwrtwLp3H1wgsv6C9/+YvS0tJ06tQpSSpcfXrx4kWdOnVKcXFxioyM9K9ol8r8z6zIyEj17t1bu3fvLjZ7csstt6hTp05q166d8WtN05Lx8fE6fvx4kSllScrKylJ+fr5q1qwpSYqOjpakIi+wlPx7CiA+Pl75+fnFzvH999+X+pwmpscdHR1d7LFICup7vPijfv36qlevXsBeaIqiGFfuhdu4at68uXbs2KGjR48qIyNDWVlZGjFihE6cOKEePXqEurxyiXHlXjiNq7179+r06dNq0aKFqlevrurVq+vWW2+VdPWtM6pXr649e/aUeV0hmf+eNWuWrly5okmTJuny5ct+natPnz46d+6c3n333SL5a6+9Vvh5Sapdu7aio6OLvUh2/fr1pb52r169JElvvPFGkfzNN98s9Tl91bhxYx04cKDIzZ6Tk6PPP/+8yHGBmP0KhIMHD+ro0aNq3rx5SOsozxhX/guHcZWQkKB27dopNjZWCxcuVOXKlZWYmFjmdVQUjCv/2TquZs6cqfT09CL/Vq9eLUmaNGmS0tPTQ/I7KyTvXdC9e3ctW7ZMv/jFL9ShQwc98sgjatOmjSIiInTs2LHCpd8/nAY2GTt2rJYtW6Zx48bp0KFDateunbZt26YFCxZowIABuuuuuyRd7eIfeughrVy5Us2aNdOtt96qXbt2+XVT9u3bVz169NAvf/lLnT9/Xp06ddL27dv1+uuvl/qcvhozZoxWrFihhx56SBMnTlROTo5SU1OLfc/i4uLUqFEjrV+/Xn369FGNGjVUs2bNwiXKvkpJSVFKSoq2bNly3eftMzIy9MQTT2j48OFq2rSpIiIitGfPHi1ZskTx8fF68sknS/Nw4QPGlf9sHVeSlJqaqjp16qhhw4Y6fvy40tLS9O677+r111/nqcwgYlz5z9Zx1bp16yJvkCtdfdsOSWrWrFlI3uRWCuE7/0+aNEndunXT0qVLtWTJEmVmZsrj8ah+/fq67bbbtGXLFvXu3bvE80RHRys9PV1JSUlauHChsrOzlZCQoCeffFJz5swpcuy1F2empqbq3Llz6t27t/7whz+4/qFfExERoQ0bNmjatGlKTU1VXl6eunfvrk2bNhX7YQda9+7d9eqrr+q5557Tfffdp6ZNm2rOnDnatGlTse0sXn75Zc2YMUODBw/WpUuXNG7cOOP741xPQUFBsVVIJrVr11a9evW0aNEiHTt2TPn5+apfv74GDhyo2bNnq0GDBi4fKdxgXPnH1nElXX3NS0pKio4ePaqYmBh17dpVn376qe644w5X14R7jCv/2DyubOTxhmvlAAAA5QxrrAEAACxBYwYAAGAJGjMAAABL0JgBAABYgsYMAADAEjRmAAAAlvDpfcwKCgqUmZmpuLg4n3ZrB8qK1+vV2bNnVa9evbDbyJlxBVsxroDA83Vc+dSYZWZm8sagsNqRI0dUv379UJfhCuMKtmNcAYFX0rjy6U+huLi4gBUEBEM43qPhWDMqlnC8R8OxZlQsJd2jPjVmTAfDduF4j4ZjzahYwvEeDceaUbGUdI+G14sHAAAAyjEaMwAAAEvQmAEAAFiCxgwAAMASNGYAAACWoDEDAACwBI0ZAACAJWjMAAAALEFjBgAAYAkaMwAAAEvQmAEAAFiCxgwAAMASNGYAAACWoDEDAACwBI0ZAACAJWjMAAAALEFjBgAAYAkaMwAAAEvQmAEAAFiCxgwAAMASNGYAAACWoDEDAACwBI0ZAACAJWjMAAAALEFjBgAAYAkaMwAAAEvQmAEAAFiiUqgLCLWWLVsa8w4dOhjzc+fOGfMWLVr4fM327dsb87Fjx/p8juuJiDD32wUFBa7OM2rUKGP+9ttvu64JCKTPP/+8WNatWzfjsdOmTTPmS5YsCWhNABAIzJgBAABYgsYMAADAEjRmAAAAlqAxAwAAsASNGQAAgCUqzKpMp9WXGzduNOYJCQnG/MqVK8Y8JibGmHs8nmKZ1+s1HuuUu+W0+tLt+QcNGmTMWZWJsmJafSk5r8A0Wbx4sTF3uo+PHDni87kB+KZjx47GfNOmTcb8ww8/NOZjxowJWE22YsYMAADAEjRmAAAAlqAxAwAAsASNGQAAgCVozAAAACxRYVZlOu19efnyZWMeFRUVtFpycnJcXTMuLi5otUjS3r17jfkrr7wS1OsC1yxatMiYO62+NK2c7N69u/HYv/3tb66uOXLkSGMOoPQmTpxozOPj441569atg1mO1ZgxAwAAsASNGQAAgCVozAAAACxBYwYAAGAJGjMAAABLVJhVmWvWrDHm27ZtM+Zdu3YNWi1OqzJTU1ON+Y9//OOAXHfHjh3GfMiQIcbcqU4g0EaMGOHq+IYNG/p8rNN973TNBg0aGHP20ESoNWrUyJi/9tprxvyFF14w5uvWrQtUST6rVauWMTftJ329vCJgxgwAAMASNGYAAACWoDEDAACwBI0ZAACAJWjMAAAALFFhVmU6OXr0qDF/++23A3L+O++8s1g2ffp047GBWn352WefGfOFCxcac1ZfItScVkIuXrzY73M7jXEnw4cPN+ZLlizxuxbAH07jwWmf2NjYWGMeilWZTqv/vV6vq7wiYMYMAADAEjRmAAAAlqAxAwAAsASNGQAAgCVozAAAACxR4VdlBopp9aUkffrpp8WygoICV+c+e/asMf/P//xPY/7kk0+6Oj9QVp544glXxzvt9eeG2304u3XrZsxZlYmykpSUZMyHDh1qzJ1+p5w4cSJgNfkrIsI8D+RUO3tlAgAAIORozAAAACxBYwYAAGAJGjMAAABL0JgBAABYglWZLvXv39+Yr1692pibVpy43QMsJibGmEdFRbk6DxBqTisenRw5csTnY92u+HRSv379gJwHKInT6suZM2cac6cVjE6/U5555pnSFRYEbmtfu3ZtMMuxGjNmAAAAlqAxAwAAsASNGQAAgCVozAAAACxBYwYAAGAJVmW6NH78eGNepUqVoF0zMjLSmE+ePNmYT5kyJWi1AP4I5opHt3tiOnG7chQoSa1atYz5gw8+aMxjY2ONeW5urjEfO3asMd+2bZsP1ZUNt3tf5uTkBKkS+zFjBgAAYAkaMwAAAEvQmAEAAFiCxgwAAMASNGYAAACWYFWmS6tWrTLmXbp0MeZbt24tlm3YsMF47PTp0415586dfazuqh/96EfG/NSpU67OAwTajh07jLnTSsiuXbsa8507d/p8DidvvfWWMQ/U6k7gmlmzZhnzVq1aGXOn/SP3799vzNetW1e6wsqQ02Nyu3d0RcCMGQAAgCVozAAAACxBYwYAAGAJGjMAAABL8OJ/lzZv3mzMmzRp4ve5jx49aszdbqvxr//6r8b8ySefdF0TEEgvvPCCMZ82bZoxT0tLM+Zu7uXFixe7uqbT8UBp3XHHHcY8IsI8N1JQUGDMx4wZE7CagqVHjx7G3O2WTBUZM2YAAACWoDEDAACwBI0ZAACAJWjMAAAALEFjBgAAYAlWZVrEtM2MJK1du9aYDx8+3Jh36tTJmFepUsWYnzt3zofqAP8dOXLEmLvdHun3v/+9z9ds0KCBz8dez8iRI4250zh0Oh4Vz7fffmvMO3ToYMydtilasmSJMbdpS6bbb7/dmLMlk++YMQMAALAEjRkAAIAlaMwAAAAsQWMGAABgCRozAAAAS3i8PiyJOHPmjKpVq1YW9cDAaQXasGHDjLnTnmQNGzY05t99913pCrPI6dOnVbVq1VCX4QrjqmRPPPGEMQ/FfpY7duww5qNGjTLmTitQwwnjKrh2795tzFu1amXMK1eubMydfo07/S5wc3wwz329451WO9u0ArW0ShpXzJgBAABYgsYMAADAEjRmAAAAlqAxAwAAsASNGQAAgCXYK7McyszMNOZ5eXllXAngH6e9AQOxKtNpf87p06cb8/KwyhJ2cdrXeNasWcZ8/vz5xtztfpNOx2/dutXnc9x8882uzl2rVi1Xx1dkzJgBAABYgsYMAADAEjRmAAAAlqAxAwAAsASNGQAAgCWsX5VZu3ZtY3727FljnpubG8xywoLT3prZ2dllXAngnwYNGgTt3CNHjgzauQF/PPvss65ym9x///3GfO3ata7O47SKsyJgxgwAAMASNGYAAACWoDEDAACwBI0ZAACAJWjMAAAALGHNqszk5GRjPnHiRGP+ySefGPNx48YFrKZw9d5774W6BCAgpk6d6vc5duzY4X8hAHyybt06Y+60JyZ7ZRbHjBkAAIAlaMwAAAAsQWMGAABgCRozAAAAS9CYAQAAWKLMV2U+8cQTxnzOnDmuzjNw4EBj3qFDB2P+1VdfuTp/KMycOdOYDx8+3NV5Pvvss0CUA4Rct27d/D7HtGnTAlAJAH94PJ5QlxA2mDEDAACwBI0ZAACAJWjMAAAALEFjBgAAYAkaMwAAAEuU+arMAwcOGPPc3FxjHhMTY8yrVatmzD/66CNjPmnSJGN+5MgRY75z505j7lbLli2LZWPGjDEeO336dGPOXmIo7xo0aGDM3azKfOutt4x5oMYygNJjr0zfMWMGAABgCRozAAAAS9CYAQAAWILGDAAAwBI0ZgAAAJYo81WZGzduNOZTpkwx5o8//rgxb9u2rTF3Wq25Zs0aY37ixAljfvDgQWPuVt26dYtlDRs2dHUOp5Wsr7zySmlKAqwzdepUv88xcuRI/wsBEBQ5OTnGPD4+3phX5L01mTEDAACwBI0ZAACAJWjMAAAALEFjBgAAYAkaMwAAAEuU+apMJ6tWrTLmGzZsMOa9evVydf4VK1YYc6cVIU65W6aVJU57g2VlZRlzpz0033///dIXBoQxp30xAdjpnXfeMeYPP/ywMa/Ie2gyYwYAAGAJGjMAAABL0JgBAABYgsYMAADAEjRmAAAAlrBmVaYTp/213n77bVfn+ctf/mLMe/ToYcyd9rN02tPTja+//tqYDxo0yJgfO3bM72sCNvviiy9cHb948eIgVQIgGLKzs425056YL730UjDLsRozZgAAAJagMQMAALAEjRkAAIAlaMwAAAAsQWMGAABgCY/Xhw2pzpw5o2rVqpVFPUCpnD59WlWrVg11Ga4wrmA7xhUCpVGjRsb81VdfNeY9e/YMYjWhVdK4YsYMAADAEjRmAAAAlqAxAwAAsASNGQAAgCVozAAAACxh/V6ZAAAgvB0+fNiYl+fVl6XFjBkAAIAlaMwAAAAsQWMGAABgCRozAAAAS9CYAQAAWILGDAAAwBI0ZgAAAJagMQMAALAEjRkAAIAlaMwAAAAs4VNj5vV6g10H4JdwvEfDsWZULOF4j4ZjzahYSrpHfWrMzp49G5BigGAJx3s0HGtGxRKO92g41oyKpaR71OP14c+LgoICZWZmKi4uTh6PJ2DFAf7yer06e/as6tWrp4iI8HpmnnEFWzGugMDzdVz51JgBAAAg+MLrTyEAAIByjMYMAADAEjRmAAAAlqAxAwAAsASNGQAAgCVozAAAACxBYwYAAGAJGjMAAABL0JgBAABYgsYMAADAEjRmAAAAlqAxAwAAsASNGQAAgCVozAAAACxBYwYAAGAJGjMAAABL0JgBAABYgsYMAADAEjRmAAAAlqAxAwAAsASNGQAAgCVozAAAACxBYwYAAGCJkDZmGRkZSkxMVLNmzRQTE6OYmBi1aNFCjz76qHbv3h3K0vzm8Xg0d+5cx8/37NlTHo+nxH/XO4cvcnNzNXfuXH366afFPjd37lx5PB6dOHHCr2v8UF5enpKTk9WkSRNFRUWpUaNGmjVrli5cuBDQ68CMccW4QuAxrsrnuJKkjz/+WN26dVNsbKxq1qyp8ePHKysrK+DX8VWlUF14xYoVeuyxx9SqVSs9/vjjatOmjTwej7799lutXr1anTt31sGDB9WsWbNQlRhUy5cv15kzZwo/3rhxo+bPn69Vq1apdevWhXn9+vX9uk5ubq7mzZsn6ergKgujR4/Wpk2blJycrM6dO2vHjh2aP3++9u3bpw0bNpRJDRUV44pxhcBjXJXfcfXZZ5+pf//+uvfee7V+/XplZWXpqaeeUp8+fbR7927deOONZVJHEd4Q2LZtmzciIsI7aNAg76VLl4zHpKWleb/77rvrnuf8+fPBKC8gJHnnzJnj8/GrVq3ySvJ++eWX1z3O7WPOzs52rGXOnDleSd7s7GxX57yeHTt2eCV5Fy1aVCRfsGCBV5L3ww8/DNi1UBTjqjjGFfzFuCquvIwrr9fr7dy5s/eWW27xXr58uTDbvn27V5J3+fLlAb2Wr0LyVOaCBQsUGRmpFStWKCoqynjMiBEjVK9evcKPx48frypVqmjPnj3q27ev4uLi1KdPH0nSyZMnNXnyZCUkJCgqKkpNmzZVUlKSLl26VPj1hw4dksfj0SuvvFLsWj+cgr02Zbpv3z6NHj1a1apVU+3atTVhwgSdPn26yNeeOXNGEydOVHx8vKpUqaJ+/frpwIEDfnx3/uFaHV999ZWGDx+u6tWrF/5F1rNnT+NfFOPHj1fjxo0LH3OtWrUkSfPmzSucbh4/fnyRrzl+/HiJj9NX27dvlyQNGDCgSD5w4EBJ0tq1a0t1XpSMceUbxhXcYFz5JhzH1Xfffacvv/xSY8aMUaVK/3gC8bbbblPLli21bt26Up3XX2X+VOaVK1eUnp6uTp06qW7duq6+Ni8vT4MHD9ajjz6qmTNnKj8/XxcvXlSvXr305z//WfPmzVP79u21detWPfvss/r666+1cePGUtc6bNgwjRo1SomJidqzZ49mzZolSVq5cqUkyev1asiQIfr8888Ln17Yvn27+vfvX+prmgwdOlQPPPCAJk2apPPnz/v8dXXr1tXmzZvVr18/JSYm6uGHH5akwpv/mpIep3R10M2bN0/p6enXnWLOy8uTpGLTv9c+zsjI8Ll++I5x5R7jCiVhXLkXTuNq7969kqT27dsX+1z79u0L/yAqa2XemJ04cUIXLlxQo0aNin3uypUr8nq9hR9HRkbK4/EUfnz58mUlJyfrZz/7WWG2YsUKZWRkKC0tTSNGjJAk3X333apSpYqeeuopffTRR7r77rtLVWtiYqJmzJghSbrrrrt08OBBrVy5Ui+//LI8Ho8++OADpaena+nSpZoyZUrhtaOiopSUlFSqa5qMGzeu8Hl3N2688UZ17NhR0tXn/rt27Wo8rqTHKUkRERHFfh4mt9xyi6Srf+E3adKkMN+2bZskKScnx/XjQMkYV+4xrlASxpV74TSuro2bGjVqFPtcjRo1QjaurHq7jI4dO+qGG24o/Ldo0aJixwwbNqzIx5988okqV66s4cOHF8mvTX9u2bKl1PUMHjy4yMft27fXxYsXC1drpKenS5IefPDBIsf99Kc/LfU1TX74mAOtpMcpScnJycrPz9edd9553XP1799fzZs3L/xP5tSpU9q8ebNmz56tyMhIRURYdctVCIwrM8YV/MG4MguncXWNUwNXUmMXLGU+mmvWrKmYmBgdPny42OfefPNNffnll44rjGJjY1W1atUiWU5OjurUqVPsG3jTTTepUqVKfnW88fHxRT6+9rTBteXpOTk5qlSpUrHj6tSpU+prmridQnerpMfpRlRUlN5//301bNhQffv2VfXq1TV8+HDNnj1b1atXV0JCQkBqRlGMK/cYVygJ48q9cBpX185l+r6fPHnSOJNWFsq8MYuMjFTv3r21e/duHTt2rMjnbrnlFnXq1Ent2rUzfq2pe42Pj9fx48eLTClLUlZWlvLz81WzZk1JUnR0tCQVeYGl5N9TAPHx8crPzy92ju+//77U5zQxPe7o6Ohij0VSUN7jxa3mzZtrx44dOnr0qDIyMpSVlaURI0boxIkT6tGjR6jLK5cYV+4xrlASxpV74TSu2rZtK0nas2dPsc/t2bOn8PNlLSTz37NmzdKVK1c0adIkXb582a9z9enTR+fOndO7775bJH/ttdcKPy9JtWvXVnR0dLEXya5fv77U1+7Vq5ck6Y033iiSv/nmm6U+p68aN26sAwcOFLnZc3Jy9Pnnnxc5zp+/JvyVkJCgdu3aKTY2VgsXLlTlypWVmJhY5nVUFIwr/zGu8EOMK//ZOq4SEhLUpUsX/e53v9OVK1cK8507d+p//ud/NHTo0DKp44dC8gaz3bt317Jly/SLX/xCHTp00COPPKI2bdooIiJCx44dK1z6/cNpYJOxY8dq2bJlGjdunA4dOqR27dpp27ZtWrBggQYMGKC77rpL0tUu/qGHHtLKlSvVrFkz3Xrrrdq1a5dfN2Xfvn3Vo0cP/fKXv9T58+fVqVMnbd++Xa+//nqpz+mrMWPGaMWKFXrooYc0ceJE5eTkKDU1tdj3LC4uTo0aNdL69evVp08f1ahRQzVr1ixcouyrlJQUpaSkaMuWLSU+b5+amqo6deqoYcOGOn78uNLS0vTuu+/q9ddf5ymXIGJc+Y9xhR9iXPnP5nH1/PPP6+6779aIESM0efJkZWVlaebMmWrbtm2RhRtlKWTv/D9p0iR169ZNS5cu1ZIlS5SZmSmPx6P69evrtttu05YtW9S7d+8SzxMdHa309HQlJSVp4cKFys7OVkJCgp588knNmTOnyLHXXpyZmpqqc+fOqXfv3vrDH/7g+od+TUREhDZs2KBp06YpNTVVeXl56t69uzZt2lTk3ZCDoXv37nr11Vf13HPP6b777lPTpk01Z84cbdq0qdh2Fi+//LJmzJihwYMH69KlSxo3bpzx/XGup6CgoNgqJCcXL15USkqKjh49qpiYGHXt2lWffvqp7rjjDlfXhHuMK/8wrmDCuPKPzeOqZ8+ehTtqDBo0SLGxsRo4cKAWLlwYmnf9l+Tx+lI5AAAAgo411gAAAJagMQMAALAEjRkAAIAlaMwAAAAsQWMGAABgCRozAAAAS/j0PmYFBQXKzMxUXFxcyDb1BEy8Xq/Onj2revXqhd1Gzowr2IpxBQSer+PKp8YsMzNTDRo0CFhxQKAdOXJE9evXD3UZrjCuYDvGFRB4JY0rn/4UiouLC1hBQDCE4z0ajjWjYgnHezQca0bFUtI96lNjxnQwbBeO92g41oyKJRzv0XCsGRVLSfdoeL14AAAAoByjMQMAALAEjRkAAIAlaMwAAAAsQWMGAABgCRozAAAAS9CYAQAAWILGDAAAwBI0ZgAAAJagMQMAALAEjRkAAIAlaMwAAAAsQWMGAABgCRozAAAAS9CYAQAAWKJSqAsAAADhJSLCPK/TpUsXV+f53//9X2Oek5PjuqbyghkzAAAAS9CYAQAAWILGDAAAwBI0ZgAAAJagMQMAALAEqzIBAKhAnFZUejyeYpnX6zUe+/TTTxvzWbNmuarlyJEjxvzRRx815ps3b3Z1/nDEjBkAAIAlaMwAAAAsQWMGAABgCRozAAAAS9CYAQAAWIJVmQAAhLGaNWsa86VLlxrzqlWrGvMvvviiWLZ8+XLjsT169DDma9asMeaHDx825hMmTDDm77zzjjFPSkoy5kuWLDHm4YgZMwAAAEvQmAEAAFiCxgwAAMASNGYAAACWoDEDAACwBKsyQ6Bx48bGvGfPnsa8Y8eOxnz06NHG3LTfmSQNGDDAmJtW4gAA7OK0mvLf/u3fjPkDDzxgzE+ePGnMDx065POx9957rzE/c+aMMXfy9ttvG3OnVZYpKSnGvKCgwJg7rUy1GTNmAAAAlqAxAwAAsASNGQAAgCVozAAAACxBYwYAAGAJj9fr9ZZ00JkzZ1StWrWyqCds3XPPPcbctHLlwQcfNB4bqO+x06rMrKwsY37zzTcb81OnTgWknrJw+vRpxxVLtqqI48rpZzRmzBhj/tRTTxnzBg0aFMt8+K+sCKf7+/nnnzfmq1atMuZO46o8YFyFRtu2bY35hx9+aMx/9KMfGfMZM2YY85deesmYX758ueTiykirVq2M+bPPPmvMu3btaszbtWtnzHNyckpXWACUNK6YMQMAALAEjRkAAIAlaMwAAAAsQWMGAABgCRozAAAAS1T4VZlOq1mmTp1qzCdNmmTMq1evbswrVSq+HanTtzw3N9eYO62UcfqZOK3KdLpu3bp1jXl2drYxtxGrx+zSpUsXY56WlmbMGzZs6Or8R48eLZa5XZVZr149Yx4ZGWnM33rrLWM+atQoV9cNJ4yr4GratKkx/+yzz4x5QkKCMf/d735nzMeOHVu6wizm9A4IGzduNOZ/+tOfjPntt99uzMtiZSqrMgEAAMIEjRkAAIAlaMwAAAAsQWMGAABgCRozAAAASxRfMlhOtWzZ0pivWbPGmLdv3z4g1zWtrnn33XeNx3788cfGPC8vz5jv2LHDmNeoUcO34v7OaXVaOK3KhF3mzp1rzJ1WX+7bt8+YL1q0yJibVqHl5+f7VtzfTZ8+3ZjPnDnTmDv9HxITE2PML1y44KoeVDxOq/ydVl8eO3bMmE+ZMiVgNdnugw8+MOZfffWVMe/cubMxHzhwoDFft25d6QoLIGbMAAAALEFjBgAAYAkaMwAAAEvQmAEAAFiCxgwAAMAS5W6vzNGjRxvz3/zmN8bcaa9MJ5mZma6uu337dlfnN6lcubIx/6//+i9j3qxZM2Pu9KM+ceKEMV+4cKExd1opF0rs6Rcaw4YNM+a///3vjfnhw4eNudPemjk5OaUrzA+7du0y5p06dTLmTqs4U1NTA1ZTqDCuAsPpHklJSTHmly5dMuZOKwz3799fusLKkdatWxvzb775xpg7rQR3+r8okKus2SsTAAAgTNCYAQAAWILGDAAAwBI0ZgAAAJagMQMAALCE9Xtl3nTTTcZ8xowZxnzatGnG3OPxGPPTp08b89mzZxvzF1980ZgHk9NKHKfVl06P1UmtWrWM+b333mvMbVyVidBw2j8yIsL8N5/TCuBQrL4MFKcVdKh42rRpY8yTkpKMeaVK5l/BycnJxpzVl86cVnynpaUZ85EjRxrzdu3aGXOn1drBwIwZAACAJWjMAAAALEFjBgAAYAkaMwAAAEvQmAEAAFjCmlWZPXv2NObLli0z5q1atTLmTvtBOu0r6bSH2ccff2zMA8HpsTqtjnRaPeLDNqdlejxQkho1ahhzp70NnVZNB8IzzzxjzFu0aGHMd+/ebcx/+9vfBqwmhLfHH3/cmDvtd/zZZ58Z8+effz5gNVUUTntZvvrqq8Z8xIgRwSzHL8yYAQAAWILGDAAAwBI0ZgAAAJagMQMAALAEjRkAAIAlynxVptPeYE8//bQxd1p96dbQoUON+cWLF4250x6d9erVM+b33XefMZ88eXKxrGrVqsZjb7jhBmMeKrm5uaEuAZZzWjV9zz33GPMePXoY82PHjhnzQ4cOGfP09PSSi/u7Xr16GXOn/1uc9prNzs425uyViWucfs/k5+cbc6fVl6yID5z333/fmDv97r///vuNOXtlAgAAVEA0ZgAAAJagMQMAALAEjRkAAIAlyvzF/04vuO3WrVtQr/vXv/7VmAf7RZamFxLb9sLOb7/91pj/y7/8SxlXgnBz5swZYz516lRj/uKLLxrzn/zkJ8a8devWrvJg2rdvX5lfE+ElPj7emDu9AH3z5s3BLAfX4bTIx+lnWJaYMQMAALAEjRkAAIAlaMwAAAAsQWMGAABgCRozAAAAS5T5qsyTJ08a88zMTGPutAVSuFi+fHmxbP369cZjnb4H48ePN+bTpk0rdV3/7K233jLmR48eDcj5UfF8/fXXxrx///7G/I477nB1/ubNmxfLmjZtajzW6f+cX/3qV66uuXv3blfHo+LZs2ePMa9SpYoxd1pdvH///oDVBLOMjAxj3r59+zKupDhmzAAAACxBYwYAAGAJGjMAAABL0JgBAABYgsYMAADAEmW+KvPYsWPGfMCAAca8c+fOxjw6OtrVdbdu3WrM9+7d6+o8wRQXF2fMhwwZYsyd9vqKiDD326dOnTLm2dnZJdYGBILTPfjee+8F7Zpr1qxxdXxBQYExd6oduOaLL74w5omJicZ80KBBxpxVmcG3a9cuY/7YY4+VcSXFMWMGAABgCRozAAAAS9CYAQAAWILGDAAAwBI0ZgAAAJYo81WZTpxWR9q0ajLYHnjgAWPutAeg1+s15k6ryv793//dmL/44os+VAeEpxtuuMHV8V999ZUx//DDDwNRDsoxpxXATqsyU1JSjHlaWpoxP3z4cOkKq8DGjh1rzMeMGWPM3a7iDgZmzAAAACxBYwYAAGAJGjMAAABL0JgBAABYgsYMAADAEtasyqxI2rRpY8yfeeaZoF73448/Dur5gfLAab9DoCROK3r/+Mc/GvMePXoY89/+9rfGfMKECcbcaQ/qisTpXQ0WL15szKtVq2bM9+3bF7CaSosZMwAAAEvQmAEAAFiCxgwAAMASNGYAAACWoDEDAACwBKsyQ2Dq1KnGvEaNGgE5/65du4z5N998E5DzAzaKjo425m3bti3jSlBR/d///Z8xv//++415RkaGMe/Xr58xd9qvdd68ecb8woULxnzPnj3G/G9/+5sxD6Y6deoY85o1axrzpKQkYz5y5EhjHhFhnn967LHHjPlLL71kzMsSM2YAAACWoDEDAACwBI0ZAACAJWjMAAAALEFjBgAAYAlWZQbZkCFDimWjRo0K6jVfe+01Y56dnR3U6wKhdOONNxrzFi1auDrPxo0bA1EOUMhptWb79u2N+fr164357bffbszT0tJc1XP+/Hlj/uWXX7o6TyA47R190003GXOv12vM//u//9uYp6amGnOn79mVK1eMeVlixgwAAMASNGYAAACWoDEDAACwBI0ZAACAJWjMAAAALMGqzCBLSUkpllWuXDmo19y6dWtQzw/YqGfPngE5z4kTJwJyHqAkTqs1+/TpY8z79+9vzIcNG2bMY2JijHmrVq2MeXx8fLGsXbt2xmPdKigoMOb79u0z5n/961+N+YIFC4z5+++/b8wvX77sQ3V2YcYMAADAEjRmAAAAlqAxAwAAsASNGQAAgCVozAAAACzBqswAad68uTE37QPmtNeXW4sWLTLme/fuDcj5gXDSuHHjUJcABITTSsINGza4ygOhS5cuATmP0x6Uf/rTnwJy/vKEGTMAAABL0JgBAABYgsYMAADAEjRmAAAAlqAxAwAAsASrMgNk9erVQTv34cOHjfmvf/3roF0TCDcHDhwIyHmc9hHcvXt3QM4PhJNdu3aFuoQKhxkzAAAAS9CYAQAAWILGDAAAwBI0ZgAAAJagMQMAALAEqzID5OTJk36fw2l/tPnz5xvz7Oxsv68JlBd//OMfjXlWVpYxv+mmm4x5165djfkbb7xRusIAwAVmzAAAACxBYwYAAGAJGjMAAABL0JgBAABYgsYMAADAEqzKDJDRo0cb8/T09GJZdHS08diUlBRjzmowoGTnz5835ps3bzbmY8eONeYPPvigMXcahzt37vShOgDwDTNmAAAAlqAxAwAAsASNGQAAgCVozAAAACxBYwYAAGAJVmUGiNNembfeemsZVwLgn02fPt2Yt2nTxpjHxsYa81q1agWsJgBwwowZAACAJWjMAAAALEFjBgAAYAkaMwAAAEvQmAEAAFiCVZkAyrWcnBxj3rlz5zKuBABKxowZAACAJWjMAAAALEFjBgAAYAkaMwAAAEv41Jh5vd5g1wH4JRzv0XCsGRVLON6j4VgzKpaS7lGfGrOzZ88GpBggWMLxHg3HmlGxhOM9Go41o2Ip6R71eH3486KgoECZmZmKi4uTx+MJWHGAv7xer86ePat69eopIiK8nplnXMFWjCsg8HwdVz41ZgAAAAi+8PpTCAAAoByjMQMAALAEjRkAAIAlaMwAAAAsQWMGAABgCRozAAAAS9CYAQAAWOL/ATVj06ss5k2pAAAAAElFTkSuQmCC",
"text/plain": [
"<Figure size 640x480 with 6 Axes>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"fig = pl.figure()\n",
"for i in range(6):\n",
" pl.subplot(2,3,i+1)\n",
" pl.tight_layout()\n",
" pl.imshow(example_data[i][0], cmap='gray', interpolation='none')\n",
" pl.title(\"Ground Truth: {}\".format(example_targets[i]))\n",
" pl.xticks([])\n",
" pl.yticks([])\n",
"pl.show()"
]
},
{
"cell_type": "markdown",
"id": "47132e3b-113b-4a78-a004-d8c54cc0193d",
"metadata": {},
"source": [
"## Building the model"
]
},
{
"cell_type": "markdown",
"id": "44918747-e0d5-465c-b2f2-8ec065cdb0a6",
"metadata": {},
"source": [
"We need to specify the model through a Python class. Below we show how to create a Feedforward Neural Network model using Pytorch."
]
},
{
"cell_type": "code",
"execution_count": 7,
"id": "4d389c91-bc58-4a60-a9b8-9a205d5981c6",
"metadata": {},
"outputs": [],
"source": [
"import torch.nn as nn\n",
"import torch.nn.functional as F\n",
"import torch.optim as optim"
]
},
{
"cell_type": "code",
"execution_count": 8,
"id": "4a230969-61fb-4d6e-8b99-8dc12a79fff1",
"metadata": {},
"outputs": [],
"source": [
"class FeedforwardNeuralNetModel(nn.Module):\n",
" def __init__(self):\n",
" input_dim = 28*28\n",
" num_classes = 10\n",
" super(FeedforwardNeuralNetModel, self).__init__()\n",
" self.fc1 = nn.Linear(input_dim, num_classes) \n",
"\n",
" def forward(self, x):\n",
" out = self.fc1(x)\n",
" return F.log_softmax(out)"
]
},
{
"cell_type": "markdown",
"id": "2017bb8f-de8f-4133-99d7-3ab30021344e",
"metadata": {},
"source": [
"You'll need to instantiate this class as well as an optimizer, which will apply an algorithm to find the internal parameters of that model, such as matrix weights and biases. As an example, we will use the Stochastic Gradient Descent algorithm."
]
},
{
"cell_type": "code",
"execution_count": 9,
"id": "77ebfc80-28cf-4cf5-bee5-e730d14a3c73",
"metadata": {},
"outputs": [],
"source": [
"learning_rate = 0.001\n",
"momentum = 0.1\n",
"log_interval = 10"
]
},
{
"cell_type": "code",
"execution_count": 10,
"id": "ce470749-5291-40f8-9ff6-f42a161dab73",
"metadata": {},
"outputs": [],
"source": [
"network = FeedforwardNeuralNetModel()\n",
"optimizer = optim.SGD(network.parameters(), lr=learning_rate, momentum=momentum)"
]
},
{
"cell_type": "markdown",
"id": "7bcb7f9c-2102-490b-9b2e-8114251ee04e",
"metadata": {},
"source": [
"## Training"
]
},
{
"cell_type": "markdown",
"id": "0bbdbe6c-332f-480c-9d9f-f46ff253d965",
"metadata": {},
"source": [
"Next, we will define the training procedure."
]
},
{
"cell_type": "code",
"execution_count": 12,
"id": "ec6be983-1bf6-43fe-9172-481538887466",
"metadata": {},
"outputs": [],
"source": [
"n_epochs = 5\n",
"\n",
"train_losses = []\n",
"train_counter = []\n",
"test_losses = []\n",
"test_counter = [i*len(train_loader.dataset) for i in range(n_epochs + 1)]"
]
},
{
"cell_type": "code",
"execution_count": 13,
"id": "344c8817-e6ed-4af8-9487-4ec1e70ffc8b",
"metadata": {},
"outputs": [],
"source": [
"! mkdir -p results"
]
},
{
"cell_type": "code",
"execution_count": 14,
"id": "bcd61e8f-621c-4679-b96f-9b6175f3dc6e",
"metadata": {},
"outputs": [],
"source": [
"def train(epoch):\n",
" network.train()\n",
" for batch_idx, (data, target) in enumerate(train_loader):\n",
" optimizer.zero_grad() # clears gradients\n",
" output = network(data.reshape(-1, 28*28))\n",
" \n",
" loss = F.nll_loss(output, target)\n",
" loss.backward()\n",
" \n",
" optimizer.step()\n",
" \n",
" if batch_idx % log_interval == 0:\n",
" print('Train Epoch: {} [{}/{} ({:.0f}%)]\\tLoss: {:.6f}'.format(\n",
" epoch, batch_idx * len(data), len(train_loader.dataset),\n",
" 100. * batch_idx / len(train_loader), loss.item())\n",
" )\n",
" train_losses.append(loss.item())\n",
" train_counter.append((batch_idx*64) + ((epoch-1)*len(train_loader.dataset))\n",
" torch.save(network.state_dict(), f'./results/model_iteration-{epoch}.pth')\n",
" torch.save(optimizer.state_dict(), f'./results/optimizer_iteration-{epoch}.pth')"
]
},
{
"cell_type": "code",
"execution_count": 15,
"id": "887895af-acc2-47db-b39d-080ea333813e",
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"/tmp/ipykernel_2928/3418281087.py:10: UserWarning: Implicit dimension choice for log_softmax has been deprecated. Change the call to include dim=X as an argument.\n",
" return F.log_softmax(out)\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Train Epoch: 1 [0/60000 (0%)]\tLoss: 2.462653\n",
"Train Epoch: 1 [640/60000 (1%)]\tLoss: 2.314634\n",
"Train Epoch: 1 [1280/60000 (2%)]\tLoss: 2.172709\n",
"Train Epoch: 1 [1920/60000 (3%)]\tLoss: 2.038150\n",
"Train Epoch: 1 [2560/60000 (4%)]\tLoss: 2.011766\n",
"Train Epoch: 1 [3200/60000 (5%)]\tLoss: 1.793893\n",
"Train Epoch: 1 [3840/60000 (6%)]\tLoss: 1.784095\n",
"Train Epoch: 1 [4480/60000 (7%)]\tLoss: 1.759681\n",
"Train Epoch: 1 [5120/60000 (9%)]\tLoss: 1.595977\n",
"Train Epoch: 1 [5760/60000 (10%)]\tLoss: 1.612938\n",
"Train Epoch: 1 [6400/60000 (11%)]\tLoss: 1.453784\n",
"Train Epoch: 1 [7040/60000 (12%)]\tLoss: 1.386518\n",
"Train Epoch: 1 [7680/60000 (13%)]\tLoss: 1.438524\n",
"Train Epoch: 1 [8320/60000 (14%)]\tLoss: 1.395226\n",
"Train Epoch: 1 [8960/60000 (15%)]\tLoss: 1.237687\n",
"Train Epoch: 1 [9600/60000 (16%)]\tLoss: 1.228426\n",
"Train Epoch: 1 [10240/60000 (17%)]\tLoss: 1.189160\n",
"Train Epoch: 1 [10880/60000 (18%)]\tLoss: 1.303519\n",
"Train Epoch: 1 [11520/60000 (19%)]\tLoss: 1.156207\n",
"Train Epoch: 1 [12160/60000 (20%)]\tLoss: 1.144378\n",
"Train Epoch: 1 [12800/60000 (21%)]\tLoss: 1.040590\n",
"Train Epoch: 1 [13440/60000 (22%)]\tLoss: 1.126434\n",
"Train Epoch: 1 [14080/60000 (23%)]\tLoss: 1.047394\n",
"Train Epoch: 1 [14720/60000 (25%)]\tLoss: 0.997867\n",
"Train Epoch: 1 [15360/60000 (26%)]\tLoss: 0.978863\n",
"Train Epoch: 1 [16000/60000 (27%)]\tLoss: 1.071712\n",
"Train Epoch: 1 [16640/60000 (28%)]\tLoss: 1.151430\n",
"Train Epoch: 1 [17280/60000 (29%)]\tLoss: 0.980280\n",
"Train Epoch: 1 [17920/60000 (30%)]\tLoss: 0.843591\n",
"Train Epoch: 1 [18560/60000 (31%)]\tLoss: 0.994908\n",
"Train Epoch: 1 [19200/60000 (32%)]\tLoss: 0.877850\n",
"Train Epoch: 1 [19840/60000 (33%)]\tLoss: 0.772137\n",
"Train Epoch: 1 [20480/60000 (34%)]\tLoss: 0.830754\n",
"Train Epoch: 1 [21120/60000 (35%)]\tLoss: 0.874241\n",
"Train Epoch: 1 [21760/60000 (36%)]\tLoss: 0.802472\n",
"Train Epoch: 1 [22400/60000 (37%)]\tLoss: 0.952104\n",
"Train Epoch: 1 [23040/60000 (38%)]\tLoss: 0.818028\n",
"Train Epoch: 1 [23680/60000 (39%)]\tLoss: 0.768295\n",
"Train Epoch: 1 [24320/60000 (41%)]\tLoss: 0.909778\n",
"Train Epoch: 1 [24960/60000 (42%)]\tLoss: 0.703258\n",
"Train Epoch: 1 [25600/60000 (43%)]\tLoss: 0.806243\n",
"Train Epoch: 1 [26240/60000 (44%)]\tLoss: 0.710597\n",
"Train Epoch: 1 [26880/60000 (45%)]\tLoss: 0.831528\n",
"Train Epoch: 1 [27520/60000 (46%)]\tLoss: 0.806455\n",
"Train Epoch: 1 [28160/60000 (47%)]\tLoss: 0.687596\n",
"Train Epoch: 1 [28800/60000 (48%)]\tLoss: 0.783745\n",
"Train Epoch: 1 [29440/60000 (49%)]\tLoss: 0.819251\n",
"Train Epoch: 1 [30080/60000 (50%)]\tLoss: 0.681386\n",
"Train Epoch: 1 [30720/60000 (51%)]\tLoss: 0.702068\n",
"Train Epoch: 1 [31360/60000 (52%)]\tLoss: 0.868806\n",
"Train Epoch: 1 [32000/60000 (53%)]\tLoss: 0.922439\n",
"Train Epoch: 1 [32640/60000 (54%)]\tLoss: 0.597353\n",
"Train Epoch: 1 [33280/60000 (55%)]\tLoss: 0.663679\n",
"Train Epoch: 1 [33920/60000 (57%)]\tLoss: 0.822791\n",
"Train Epoch: 1 [34560/60000 (58%)]\tLoss: 1.010872\n",
"Train Epoch: 1 [35200/60000 (59%)]\tLoss: 0.782502\n",
"Train Epoch: 1 [35840/60000 (60%)]\tLoss: 0.826569\n",
"Train Epoch: 1 [36480/60000 (61%)]\tLoss: 0.677710\n",
"Train Epoch: 1 [37120/60000 (62%)]\tLoss: 0.614317\n",
"Train Epoch: 1 [37760/60000 (63%)]\tLoss: 0.618042\n",
"Train Epoch: 1 [38400/60000 (64%)]\tLoss: 0.661478\n",
"Train Epoch: 1 [39040/60000 (65%)]\tLoss: 0.580700\n",
"Train Epoch: 1 [39680/60000 (66%)]\tLoss: 0.779643\n",
"Train Epoch: 1 [40320/60000 (67%)]\tLoss: 0.542680\n",
"Train Epoch: 1 [40960/60000 (68%)]\tLoss: 0.707844\n",
"Train Epoch: 1 [41600/60000 (69%)]\tLoss: 0.802536\n",
"Train Epoch: 1 [42240/60000 (70%)]\tLoss: 0.598446\n",
"Train Epoch: 1 [42880/60000 (71%)]\tLoss: 0.606198\n",
"Train Epoch: 1 [43520/60000 (72%)]\tLoss: 0.578928\n",
"Train Epoch: 1 [44160/60000 (74%)]\tLoss: 0.535659\n",
"Train Epoch: 1 [44800/60000 (75%)]\tLoss: 0.725130\n",
"Train Epoch: 1 [45440/60000 (76%)]\tLoss: 0.754411\n",
"Train Epoch: 1 [46080/60000 (77%)]\tLoss: 0.579297\n",
"Train Epoch: 1 [46720/60000 (78%)]\tLoss: 0.573667\n",
"Train Epoch: 1 [47360/60000 (79%)]\tLoss: 0.585305\n",
"Train Epoch: 1 [48000/60000 (80%)]\tLoss: 0.679498\n",
"Train Epoch: 1 [48640/60000 (81%)]\tLoss: 0.760280\n",
"Train Epoch: 1 [49280/60000 (82%)]\tLoss: 0.498441\n",
"Train Epoch: 1 [49920/60000 (83%)]\tLoss: 0.650688\n",
"Train Epoch: 1 [50560/60000 (84%)]\tLoss: 0.604967\n",
"Train Epoch: 1 [51200/60000 (85%)]\tLoss: 0.771571\n",
"Train Epoch: 1 [51840/60000 (86%)]\tLoss: 0.607910\n",
"Train Epoch: 1 [52480/60000 (87%)]\tLoss: 0.649972\n",
"Train Epoch: 1 [53120/60000 (88%)]\tLoss: 0.742860\n",
"Train Epoch: 1 [53760/60000 (90%)]\tLoss: 0.697826\n",
"Train Epoch: 1 [54400/60000 (91%)]\tLoss: 0.586808\n",
"Train Epoch: 1 [55040/60000 (92%)]\tLoss: 0.536884\n",
"Train Epoch: 1 [55680/60000 (93%)]\tLoss: 0.623721\n",
"Train Epoch: 1 [56320/60000 (94%)]\tLoss: 0.615557\n",
"Train Epoch: 1 [56960/60000 (95%)]\tLoss: 0.621626\n",
"Train Epoch: 1 [57600/60000 (96%)]\tLoss: 0.790871\n",
"Train Epoch: 1 [58240/60000 (97%)]\tLoss: 0.586313\n",
"Train Epoch: 1 [58880/60000 (98%)]\tLoss: 0.567281\n",
"Train Epoch: 1 [59520/60000 (99%)]\tLoss: 0.501192\n"
]
}
],
"source": [
"train(1)"
]
},
{
"cell_type": "markdown",
"id": "7f182b23-fd7c-460d-a440-d38d7ec633c2",
"metadata": {},
"source": [
"<br />\n",
"Alongside trainig, we will also monitor the performance of the model on a set of samples not seen during the training."
]
},
{
"cell_type": "code",
"execution_count": 16,
"id": "76c5f60d-264a-4cb7-91bf-b62013fa8fd5",
"metadata": {},
"outputs": [],
"source": [
"def test():\n",
" network.eval()\n",
" test_loss = 0\n",
" correct = 0\n",
" with torch.no_grad():\n",
" for data, target in test_loader:\n",
" output = network(data.reshape(-1, 28*28))\n",
" test_loss += F.nll_loss(output, target, size_average=False).item()\n",
" pred = output.data.max(1, keepdim=True)[1]\n",
" correct += pred.eq(target.data.view_as(pred)).sum()\n",
" test_loss /= len(test_loader.dataset)\n",
" test_losses.append(test_loss)\n",
" print('\\nTest set: Avg. loss: {:.4f}, Accuracy: {}/{} ({:.0f}%)\\n' \\\n",
" .format(test_loss, correct, len(test_loader.dataset), 100. * correct / len(test_loader.dataset))\n",
" )"
]
},
{
"cell_type": "code",
"execution_count": 17,
"id": "7c2472ee-40e3-4b54-a37f-d46cb570e631",
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"/tmp/ipykernel_2928/3418281087.py:10: UserWarning: Implicit dimension choice for log_softmax has been deprecated. Change the call to include dim=X as an argument.\n",
" return F.log_softmax(out)\n",
"/opt/conda/lib/python3.10/site-packages/torch/nn/_reduction.py:42: UserWarning: size_average and reduce args will be deprecated, please use reduction='sum' instead.\n",
" warnings.warn(warning.format(ret))\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"\n",
"Test set: Avg. loss: 0.5596, Accuracy: 8696/10000 (87%)\n",
"\n"
]
}
],
"source": [
"test()"
]
},
{
"cell_type": "markdown",
"id": "4dcf0d6c-98c0-4e4a-acbf-786a99892815",
"metadata": {},
"source": [
"## Training and evaluating for multiple epochs"
]
},
{
"cell_type": "markdown",
"id": "651146e4-13fe-4052-8bd7-1bf4d633ae41",
"metadata": {},
"source": [
"Let's train now for all desired epochs."
]
},
{
"cell_type": "code",
"execution_count": 18,
"id": "3ea9fa2a-1d5a-4b42-8631-2ff9afb1de97",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Train Epoch: 2 [0/60000 (0%)]\tLoss: 0.666574\n",
"Train Epoch: 2 [640/60000 (1%)]\tLoss: 0.541078\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"/tmp/ipykernel_2928/3418281087.py:10: UserWarning: Implicit dimension choice for log_softmax has been deprecated. Change the call to include dim=X as an argument.\n",
" return F.log_softmax(out)\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Train Epoch: 2 [1280/60000 (2%)]\tLoss: 0.564097\n",
"Train Epoch: 2 [1920/60000 (3%)]\tLoss: 0.666229\n",
"Train Epoch: 2 [2560/60000 (4%)]\tLoss: 0.488055\n",
"Train Epoch: 2 [3200/60000 (5%)]\tLoss: 0.434858\n",
"Train Epoch: 2 [3840/60000 (6%)]\tLoss: 0.543594\n",
"Train Epoch: 2 [4480/60000 (7%)]\tLoss: 0.447622\n",
"Train Epoch: 2 [5120/60000 (9%)]\tLoss: 0.549790\n",
"Train Epoch: 2 [5760/60000 (10%)]\tLoss: 0.652385\n",
"Train Epoch: 2 [6400/60000 (11%)]\tLoss: 0.670172\n",
"Train Epoch: 2 [7040/60000 (12%)]\tLoss: 0.586397\n",
"Train Epoch: 2 [7680/60000 (13%)]\tLoss: 0.554513\n",
"Train Epoch: 2 [8320/60000 (14%)]\tLoss: 0.585009\n",
"Train Epoch: 2 [8960/60000 (15%)]\tLoss: 0.654413\n",
"Train Epoch: 2 [9600/60000 (16%)]\tLoss: 0.667827\n",
"Train Epoch: 2 [10240/60000 (17%)]\tLoss: 0.639856\n",
"Train Epoch: 2 [10880/60000 (18%)]\tLoss: 0.650934\n",
"Train Epoch: 2 [11520/60000 (19%)]\tLoss: 0.545639\n",
"Train Epoch: 2 [12160/60000 (20%)]\tLoss: 0.532475\n",
"Train Epoch: 2 [12800/60000 (21%)]\tLoss: 0.734861\n",
"Train Epoch: 2 [13440/60000 (22%)]\tLoss: 0.680402\n",
"Train Epoch: 2 [14080/60000 (23%)]\tLoss: 0.570717\n",
"Train Epoch: 2 [14720/60000 (25%)]\tLoss: 0.624158\n",
"Train Epoch: 2 [15360/60000 (26%)]\tLoss: 0.704827\n",
"Train Epoch: 2 [16000/60000 (27%)]\tLoss: 0.668154\n",
"Train Epoch: 2 [16640/60000 (28%)]\tLoss: 0.572991\n",
"Train Epoch: 2 [17280/60000 (29%)]\tLoss: 0.581778\n",
"Train Epoch: 2 [17920/60000 (30%)]\tLoss: 0.600190\n",
"Train Epoch: 2 [18560/60000 (31%)]\tLoss: 0.550936\n",
"Train Epoch: 2 [19200/60000 (32%)]\tLoss: 0.541474\n",
"Train Epoch: 2 [19840/60000 (33%)]\tLoss: 0.352336\n",
"Train Epoch: 2 [20480/60000 (34%)]\tLoss: 0.388920\n",
"Train Epoch: 2 [21120/60000 (35%)]\tLoss: 0.466117\n",
"Train Epoch: 2 [21760/60000 (36%)]\tLoss: 0.525197\n",
"Train Epoch: 2 [22400/60000 (37%)]\tLoss: 0.454179\n",
"Train Epoch: 2 [23040/60000 (38%)]\tLoss: 0.515984\n",
"Train Epoch: 2 [23680/60000 (39%)]\tLoss: 0.621901\n",
"Train Epoch: 2 [24320/60000 (41%)]\tLoss: 0.476662\n",
"Train Epoch: 2 [24960/60000 (42%)]\tLoss: 0.492097\n",
"Train Epoch: 2 [25600/60000 (43%)]\tLoss: 0.756423\n",
"Train Epoch: 2 [26240/60000 (44%)]\tLoss: 0.639166\n",
"Train Epoch: 2 [26880/60000 (45%)]\tLoss: 0.574907\n",
"Train Epoch: 2 [27520/60000 (46%)]\tLoss: 0.544177\n",
"Train Epoch: 2 [28160/60000 (47%)]\tLoss: 0.394125\n",
"Train Epoch: 2 [28800/60000 (48%)]\tLoss: 0.650866\n",
"Train Epoch: 2 [29440/60000 (49%)]\tLoss: 0.353484\n",
"Train Epoch: 2 [30080/60000 (50%)]\tLoss: 0.563469\n",
"Train Epoch: 2 [30720/60000 (51%)]\tLoss: 0.427483\n",
"Train Epoch: 2 [31360/60000 (52%)]\tLoss: 0.404265\n",
"Train Epoch: 2 [32000/60000 (53%)]\tLoss: 0.485940\n",
"Train Epoch: 2 [32640/60000 (54%)]\tLoss: 0.526042\n",
"Train Epoch: 2 [33280/60000 (55%)]\tLoss: 0.554431\n",
"Train Epoch: 2 [33920/60000 (57%)]\tLoss: 0.413831\n",
"Train Epoch: 2 [34560/60000 (58%)]\tLoss: 0.572252\n",
"Train Epoch: 2 [35200/60000 (59%)]\tLoss: 0.551418\n",
"Train Epoch: 2 [35840/60000 (60%)]\tLoss: 0.526227\n",
"Train Epoch: 2 [36480/60000 (61%)]\tLoss: 0.587115\n",
"Train Epoch: 2 [37120/60000 (62%)]\tLoss: 0.539950\n",
"Train Epoch: 2 [37760/60000 (63%)]\tLoss: 0.489990\n",
"Train Epoch: 2 [38400/60000 (64%)]\tLoss: 0.457468\n",
"Train Epoch: 2 [39040/60000 (65%)]\tLoss: 0.703489\n",
"Train Epoch: 2 [39680/60000 (66%)]\tLoss: 0.490875\n",
"Train Epoch: 2 [40320/60000 (67%)]\tLoss: 0.385928\n",
"Train Epoch: 2 [40960/60000 (68%)]\tLoss: 0.684328\n",
"Train Epoch: 2 [41600/60000 (69%)]\tLoss: 0.463061\n",
"Train Epoch: 2 [42240/60000 (70%)]\tLoss: 0.568630\n",
"Train Epoch: 2 [42880/60000 (71%)]\tLoss: 0.557115\n",
"Train Epoch: 2 [43520/60000 (72%)]\tLoss: 0.567219\n",
"Train Epoch: 2 [44160/60000 (74%)]\tLoss: 0.476868\n",
"Train Epoch: 2 [44800/60000 (75%)]\tLoss: 0.638940\n",
"Train Epoch: 2 [45440/60000 (76%)]\tLoss: 0.554818\n",
"Train Epoch: 2 [46080/60000 (77%)]\tLoss: 0.459391\n",
"Train Epoch: 2 [46720/60000 (78%)]\tLoss: 0.490846\n",
"Train Epoch: 2 [47360/60000 (79%)]\tLoss: 0.328903\n",
"Train Epoch: 2 [48000/60000 (80%)]\tLoss: 0.518024\n",
"Train Epoch: 2 [48640/60000 (81%)]\tLoss: 0.432704\n",
"Train Epoch: 2 [49280/60000 (82%)]\tLoss: 0.412061\n",
"Train Epoch: 2 [49920/60000 (83%)]\tLoss: 0.432209\n",
"Train Epoch: 2 [50560/60000 (84%)]\tLoss: 0.298506\n",
"Train Epoch: 2 [51200/60000 (85%)]\tLoss: 0.429069\n",
"Train Epoch: 2 [51840/60000 (86%)]\tLoss: 0.452532\n",
"Train Epoch: 2 [52480/60000 (87%)]\tLoss: 0.385696\n",
"Train Epoch: 2 [53120/60000 (88%)]\tLoss: 0.677558\n",
"Train Epoch: 2 [53760/60000 (90%)]\tLoss: 0.461628\n",
"Train Epoch: 2 [54400/60000 (91%)]\tLoss: 0.568202\n",
"Train Epoch: 2 [55040/60000 (92%)]\tLoss: 0.480006\n",
"Train Epoch: 2 [55680/60000 (93%)]\tLoss: 0.618786\n",
"Train Epoch: 2 [56320/60000 (94%)]\tLoss: 0.605008\n",
"Train Epoch: 2 [56960/60000 (95%)]\tLoss: 0.499009\n",
"Train Epoch: 2 [57600/60000 (96%)]\tLoss: 0.394007\n",
"Train Epoch: 2 [58240/60000 (97%)]\tLoss: 0.620327\n",
"Train Epoch: 2 [58880/60000 (98%)]\tLoss: 0.437920\n",
"Train Epoch: 2 [59520/60000 (99%)]\tLoss: 0.610073\n",
"\n",
"Test set: Avg. loss: 0.4506, Accuracy: 8851/10000 (89%)\n",
"\n",
"Train Epoch: 3 [0/60000 (0%)]\tLoss: 0.515682\n",
"Train Epoch: 3 [640/60000 (1%)]\tLoss: 0.381055\n",
"Train Epoch: 3 [1280/60000 (2%)]\tLoss: 0.478112\n",
"Train Epoch: 3 [1920/60000 (3%)]\tLoss: 0.565628\n",
"Train Epoch: 3 [2560/60000 (4%)]\tLoss: 0.559797\n",
"Train Epoch: 3 [3200/60000 (5%)]\tLoss: 0.439313\n",
"Train Epoch: 3 [3840/60000 (6%)]\tLoss: 0.475197\n",
"Train Epoch: 3 [4480/60000 (7%)]\tLoss: 0.401082\n",
"Train Epoch: 3 [5120/60000 (9%)]\tLoss: 0.358811\n",
"Train Epoch: 3 [5760/60000 (10%)]\tLoss: 0.548729\n",
"Train Epoch: 3 [6400/60000 (11%)]\tLoss: 0.526900\n",
"Train Epoch: 3 [7040/60000 (12%)]\tLoss: 0.450414\n",
"Train Epoch: 3 [7680/60000 (13%)]\tLoss: 0.407663\n",
"Train Epoch: 3 [8320/60000 (14%)]\tLoss: 0.351334\n",
"Train Epoch: 3 [8960/60000 (15%)]\tLoss: 0.423336\n",
"Train Epoch: 3 [9600/60000 (16%)]\tLoss: 0.462153\n",
"Train Epoch: 3 [10240/60000 (17%)]\tLoss: 0.492132\n",
"Train Epoch: 3 [10880/60000 (18%)]\tLoss: 0.275345\n",
"Train Epoch: 3 [11520/60000 (19%)]\tLoss: 0.453334\n",
"Train Epoch: 3 [12160/60000 (20%)]\tLoss: 0.443971\n",
"Train Epoch: 3 [12800/60000 (21%)]\tLoss: 0.430721\n",
"Train Epoch: 3 [13440/60000 (22%)]\tLoss: 0.447697\n",
"Train Epoch: 3 [14080/60000 (23%)]\tLoss: 0.355809\n",
"Train Epoch: 3 [14720/60000 (25%)]\tLoss: 0.414375\n",
"Train Epoch: 3 [15360/60000 (26%)]\tLoss: 0.591316\n",
"Train Epoch: 3 [16000/60000 (27%)]\tLoss: 0.389881\n",
"Train Epoch: 3 [16640/60000 (28%)]\tLoss: 0.578114\n",
"Train Epoch: 3 [17280/60000 (29%)]\tLoss: 0.588548\n",
"Train Epoch: 3 [17920/60000 (30%)]\tLoss: 0.391995\n",
"Train Epoch: 3 [18560/60000 (31%)]\tLoss: 0.450514\n",
"Train Epoch: 3 [19200/60000 (32%)]\tLoss: 0.639105\n",
"Train Epoch: 3 [19840/60000 (33%)]\tLoss: 0.390941\n",
"Train Epoch: 3 [20480/60000 (34%)]\tLoss: 0.392755\n",
"Train Epoch: 3 [21120/60000 (35%)]\tLoss: 0.380556\n",
"Train Epoch: 3 [21760/60000 (36%)]\tLoss: 0.480599\n",
"Train Epoch: 3 [22400/60000 (37%)]\tLoss: 0.572423\n",
"Train Epoch: 3 [23040/60000 (38%)]\tLoss: 0.412562\n",
"Train Epoch: 3 [23680/60000 (39%)]\tLoss: 0.487716\n",
"Train Epoch: 3 [24320/60000 (41%)]\tLoss: 0.510381\n",
"Train Epoch: 3 [24960/60000 (42%)]\tLoss: 0.377208\n",
"Train Epoch: 3 [25600/60000 (43%)]\tLoss: 0.300404\n",
"Train Epoch: 3 [26240/60000 (44%)]\tLoss: 0.401228\n",
"Train Epoch: 3 [26880/60000 (45%)]\tLoss: 0.594074\n",
"Train Epoch: 3 [27520/60000 (46%)]\tLoss: 0.328740\n",
"Train Epoch: 3 [28160/60000 (47%)]\tLoss: 0.382568\n",
"Train Epoch: 3 [28800/60000 (48%)]\tLoss: 0.427094\n",
"Train Epoch: 3 [29440/60000 (49%)]\tLoss: 0.327248\n",
"Train Epoch: 3 [30080/60000 (50%)]\tLoss: 0.414064\n",
"Train Epoch: 3 [30720/60000 (51%)]\tLoss: 0.424928\n",
"Train Epoch: 3 [31360/60000 (52%)]\tLoss: 0.445438\n",
"Train Epoch: 3 [32000/60000 (53%)]\tLoss: 0.382040\n",
"Train Epoch: 3 [32640/60000 (54%)]\tLoss: 0.342853\n",
"Train Epoch: 3 [33280/60000 (55%)]\tLoss: 0.451347\n",
"Train Epoch: 3 [33920/60000 (57%)]\tLoss: 0.363770\n",
"Train Epoch: 3 [34560/60000 (58%)]\tLoss: 0.412778\n",
"Train Epoch: 3 [35200/60000 (59%)]\tLoss: 0.307116\n",
"Train Epoch: 3 [35840/60000 (60%)]\tLoss: 0.614837\n",
"Train Epoch: 3 [36480/60000 (61%)]\tLoss: 0.436802\n",
"Train Epoch: 3 [37120/60000 (62%)]\tLoss: 0.436361\n",
"Train Epoch: 3 [37760/60000 (63%)]\tLoss: 0.254122\n",
"Train Epoch: 3 [38400/60000 (64%)]\tLoss: 0.576679\n",
"Train Epoch: 3 [39040/60000 (65%)]\tLoss: 0.388012\n",
"Train Epoch: 3 [39680/60000 (66%)]\tLoss: 0.545191\n",
"Train Epoch: 3 [40320/60000 (67%)]\tLoss: 0.414379\n",
"Train Epoch: 3 [40960/60000 (68%)]\tLoss: 0.278662\n",
"Train Epoch: 3 [41600/60000 (69%)]\tLoss: 0.551912\n",
"Train Epoch: 3 [42240/60000 (70%)]\tLoss: 0.466143\n",
"Train Epoch: 3 [42880/60000 (71%)]\tLoss: 0.468168\n",
"Train Epoch: 3 [43520/60000 (72%)]\tLoss: 0.276545\n",
"Train Epoch: 3 [44160/60000 (74%)]\tLoss: 0.355101\n",
"Train Epoch: 3 [44800/60000 (75%)]\tLoss: 0.527041\n",
"Train Epoch: 3 [45440/60000 (76%)]\tLoss: 0.536015\n",
"Train Epoch: 3 [46080/60000 (77%)]\tLoss: 0.342275\n",
"Train Epoch: 3 [46720/60000 (78%)]\tLoss: 0.588447\n",
"Train Epoch: 3 [47360/60000 (79%)]\tLoss: 0.269670\n",
"Train Epoch: 3 [48000/60000 (80%)]\tLoss: 0.278183\n",
"Train Epoch: 3 [48640/60000 (81%)]\tLoss: 0.388054\n",
"Train Epoch: 3 [49280/60000 (82%)]\tLoss: 0.243378\n",
"Train Epoch: 3 [49920/60000 (83%)]\tLoss: 0.693933\n",
"Train Epoch: 3 [50560/60000 (84%)]\tLoss: 0.393076\n",
"Train Epoch: 3 [51200/60000 (85%)]\tLoss: 0.275795\n",
"Train Epoch: 3 [51840/60000 (86%)]\tLoss: 0.332906\n",
"Train Epoch: 3 [52480/60000 (87%)]\tLoss: 0.551177\n",
"Train Epoch: 3 [53120/60000 (88%)]\tLoss: 0.606337\n",
"Train Epoch: 3 [53760/60000 (90%)]\tLoss: 0.359024\n",
"Train Epoch: 3 [54400/60000 (91%)]\tLoss: 0.488740\n",
"Train Epoch: 3 [55040/60000 (92%)]\tLoss: 0.446344\n",
"Train Epoch: 3 [55680/60000 (93%)]\tLoss: 0.345605\n",
"Train Epoch: 3 [56320/60000 (94%)]\tLoss: 0.580558\n",
"Train Epoch: 3 [56960/60000 (95%)]\tLoss: 0.497444\n",
"Train Epoch: 3 [57600/60000 (96%)]\tLoss: 0.466277\n",
"Train Epoch: 3 [58240/60000 (97%)]\tLoss: 0.375065\n",
"Train Epoch: 3 [58880/60000 (98%)]\tLoss: 0.490512\n",
"Train Epoch: 3 [59520/60000 (99%)]\tLoss: 0.654881\n",
"\n",
"Test set: Avg. loss: 0.4058, Accuracy: 8941/10000 (89%)\n",
"\n",
"Train Epoch: 4 [0/60000 (0%)]\tLoss: 0.544170\n",
"Train Epoch: 4 [640/60000 (1%)]\tLoss: 0.424970\n",
"Train Epoch: 4 [1280/60000 (2%)]\tLoss: 0.312823\n",
"Train Epoch: 4 [1920/60000 (3%)]\tLoss: 0.357140\n",
"Train Epoch: 4 [2560/60000 (4%)]\tLoss: 0.338121\n",
"Train Epoch: 4 [3200/60000 (5%)]\tLoss: 0.398677\n",
"Train Epoch: 4 [3840/60000 (6%)]\tLoss: 0.409509\n",
"Train Epoch: 4 [4480/60000 (7%)]\tLoss: 0.406468\n",
"Train Epoch: 4 [5120/60000 (9%)]\tLoss: 0.488089\n",
"Train Epoch: 4 [5760/60000 (10%)]\tLoss: 0.554749\n",
"Train Epoch: 4 [6400/60000 (11%)]\tLoss: 0.337705\n",
"Train Epoch: 4 [7040/60000 (12%)]\tLoss: 0.445246\n",
"Train Epoch: 4 [7680/60000 (13%)]\tLoss: 0.526637\n",
"Train Epoch: 4 [8320/60000 (14%)]\tLoss: 0.593512\n",
"Train Epoch: 4 [8960/60000 (15%)]\tLoss: 0.459215\n",
"Train Epoch: 4 [9600/60000 (16%)]\tLoss: 0.477244\n",
"Train Epoch: 4 [10240/60000 (17%)]\tLoss: 0.372630\n",
"Train Epoch: 4 [10880/60000 (18%)]\tLoss: 0.403247\n",
"Train Epoch: 4 [11520/60000 (19%)]\tLoss: 0.555593\n",
"Train Epoch: 4 [12160/60000 (20%)]\tLoss: 0.347193\n",
"Train Epoch: 4 [12800/60000 (21%)]\tLoss: 0.507420\n",
"Train Epoch: 4 [13440/60000 (22%)]\tLoss: 0.461771\n",
"Train Epoch: 4 [14080/60000 (23%)]\tLoss: 0.381492\n",
"Train Epoch: 4 [14720/60000 (25%)]\tLoss: 0.518532\n",
"Train Epoch: 4 [15360/60000 (26%)]\tLoss: 0.509451\n",
"Train Epoch: 4 [16000/60000 (27%)]\tLoss: 0.359733\n",
"Train Epoch: 4 [16640/60000 (28%)]\tLoss: 0.323727\n",
"Train Epoch: 4 [17280/60000 (29%)]\tLoss: 0.371965\n",
"Train Epoch: 4 [17920/60000 (30%)]\tLoss: 0.609687\n",
"Train Epoch: 4 [18560/60000 (31%)]\tLoss: 0.666263\n",
"Train Epoch: 4 [19200/60000 (32%)]\tLoss: 0.426059\n",
"Train Epoch: 4 [19840/60000 (33%)]\tLoss: 0.203614\n",
"Train Epoch: 4 [20480/60000 (34%)]\tLoss: 0.358409\n",
"Train Epoch: 4 [21120/60000 (35%)]\tLoss: 0.325613\n",
"Train Epoch: 4 [21760/60000 (36%)]\tLoss: 0.539919\n",
"Train Epoch: 4 [22400/60000 (37%)]\tLoss: 0.262908\n",
"Train Epoch: 4 [23040/60000 (38%)]\tLoss: 0.249213\n",
"Train Epoch: 4 [23680/60000 (39%)]\tLoss: 0.341809\n",
"Train Epoch: 4 [24320/60000 (41%)]\tLoss: 0.300866\n",
"Train Epoch: 4 [24960/60000 (42%)]\tLoss: 0.419902\n",
"Train Epoch: 4 [25600/60000 (43%)]\tLoss: 0.347997\n",
"Train Epoch: 4 [26240/60000 (44%)]\tLoss: 0.348140\n",
"Train Epoch: 4 [26880/60000 (45%)]\tLoss: 0.385401\n",
"Train Epoch: 4 [27520/60000 (46%)]\tLoss: 0.377303\n",
"Train Epoch: 4 [28160/60000 (47%)]\tLoss: 0.410656\n",
"Train Epoch: 4 [28800/60000 (48%)]\tLoss: 0.447676\n",
"Train Epoch: 4 [29440/60000 (49%)]\tLoss: 0.411600\n",
"Train Epoch: 4 [30080/60000 (50%)]\tLoss: 0.304504\n",
"Train Epoch: 4 [30720/60000 (51%)]\tLoss: 0.615461\n",
"Train Epoch: 4 [31360/60000 (52%)]\tLoss: 0.488724\n",
"Train Epoch: 4 [32000/60000 (53%)]\tLoss: 0.502389\n",
"Train Epoch: 4 [32640/60000 (54%)]\tLoss: 0.424566\n",
"Train Epoch: 4 [33280/60000 (55%)]\tLoss: 0.251016\n",
"Train Epoch: 4 [33920/60000 (57%)]\tLoss: 0.398078\n",
"Train Epoch: 4 [34560/60000 (58%)]\tLoss: 0.722360\n",
"Train Epoch: 4 [35200/60000 (59%)]\tLoss: 0.505770\n",
"Train Epoch: 4 [35840/60000 (60%)]\tLoss: 0.321162\n",
"Train Epoch: 4 [36480/60000 (61%)]\tLoss: 0.385665\n",
"Train Epoch: 4 [37120/60000 (62%)]\tLoss: 0.350606\n",
"Train Epoch: 4 [37760/60000 (63%)]\tLoss: 0.418992\n",
"Train Epoch: 4 [38400/60000 (64%)]\tLoss: 0.380536\n",
"Train Epoch: 4 [39040/60000 (65%)]\tLoss: 0.321524\n",
"Train Epoch: 4 [39680/60000 (66%)]\tLoss: 0.372908\n",
"Train Epoch: 4 [40320/60000 (67%)]\tLoss: 0.257566\n",
"Train Epoch: 4 [40960/60000 (68%)]\tLoss: 0.386917\n",
"Train Epoch: 4 [41600/60000 (69%)]\tLoss: 0.258698\n",
"Train Epoch: 4 [42240/60000 (70%)]\tLoss: 0.548908\n",
"Train Epoch: 4 [42880/60000 (71%)]\tLoss: 0.521250\n",
"Train Epoch: 4 [43520/60000 (72%)]\tLoss: 0.397563\n",
"Train Epoch: 4 [44160/60000 (74%)]\tLoss: 0.288998\n",
"Train Epoch: 4 [44800/60000 (75%)]\tLoss: 0.522141\n",
"Train Epoch: 4 [45440/60000 (76%)]\tLoss: 0.264227\n",
"Train Epoch: 4 [46080/60000 (77%)]\tLoss: 0.297545\n",
"Train Epoch: 4 [46720/60000 (78%)]\tLoss: 0.335762\n",
"Train Epoch: 4 [47360/60000 (79%)]\tLoss: 0.441359\n",
"Train Epoch: 4 [48000/60000 (80%)]\tLoss: 0.515000\n",
"Train Epoch: 4 [48640/60000 (81%)]\tLoss: 0.396321\n",
"Train Epoch: 4 [49280/60000 (82%)]\tLoss: 0.597667\n",
"Train Epoch: 4 [49920/60000 (83%)]\tLoss: 0.425958\n",
"Train Epoch: 4 [50560/60000 (84%)]\tLoss: 0.431178\n",
"Train Epoch: 4 [51200/60000 (85%)]\tLoss: 0.306227\n",
"Train Epoch: 4 [51840/60000 (86%)]\tLoss: 0.445283\n",
"Train Epoch: 4 [52480/60000 (87%)]\tLoss: 0.442354\n",
"Train Epoch: 4 [53120/60000 (88%)]\tLoss: 0.540651\n",
"Train Epoch: 4 [53760/60000 (90%)]\tLoss: 0.375122\n",
"Train Epoch: 4 [54400/60000 (91%)]\tLoss: 0.350683\n",
"Train Epoch: 4 [55040/60000 (92%)]\tLoss: 0.505314\n",
"Train Epoch: 4 [55680/60000 (93%)]\tLoss: 0.373389\n",
"Train Epoch: 4 [56320/60000 (94%)]\tLoss: 0.403350\n",
"Train Epoch: 4 [56960/60000 (95%)]\tLoss: 0.280683\n",
"Train Epoch: 4 [57600/60000 (96%)]\tLoss: 0.494819\n",
"Train Epoch: 4 [58240/60000 (97%)]\tLoss: 0.344167\n",
"Train Epoch: 4 [58880/60000 (98%)]\tLoss: 0.407800\n",
"Train Epoch: 4 [59520/60000 (99%)]\tLoss: 0.343069\n",
"\n",
"Test set: Avg. loss: 0.3814, Accuracy: 8977/10000 (90%)\n",
"\n",
"Train Epoch: 5 [0/60000 (0%)]\tLoss: 0.440376\n",
"Train Epoch: 5 [640/60000 (1%)]\tLoss: 0.246016\n",
"Train Epoch: 5 [1280/60000 (2%)]\tLoss: 0.372876\n",
"Train Epoch: 5 [1920/60000 (3%)]\tLoss: 0.152530\n",
"Train Epoch: 5 [2560/60000 (4%)]\tLoss: 0.461249\n",
"Train Epoch: 5 [3200/60000 (5%)]\tLoss: 0.338112\n",
"Train Epoch: 5 [3840/60000 (6%)]\tLoss: 0.344228\n",
"Train Epoch: 5 [4480/60000 (7%)]\tLoss: 0.277568\n",
"Train Epoch: 5 [5120/60000 (9%)]\tLoss: 0.278701\n",
"Train Epoch: 5 [5760/60000 (10%)]\tLoss: 0.352110\n",
"Train Epoch: 5 [6400/60000 (11%)]\tLoss: 0.551467\n",
"Train Epoch: 5 [7040/60000 (12%)]\tLoss: 0.448395\n",
"Train Epoch: 5 [7680/60000 (13%)]\tLoss: 0.386086\n",
"Train Epoch: 5 [8320/60000 (14%)]\tLoss: 0.292033\n",
"Train Epoch: 5 [8960/60000 (15%)]\tLoss: 0.318571\n",
"Train Epoch: 5 [9600/60000 (16%)]\tLoss: 0.452842\n",
"Train Epoch: 5 [10240/60000 (17%)]\tLoss: 0.528022\n",
"Train Epoch: 5 [10880/60000 (18%)]\tLoss: 0.392916\n",
"Train Epoch: 5 [11520/60000 (19%)]\tLoss: 0.261237\n",
"Train Epoch: 5 [12160/60000 (20%)]\tLoss: 0.421092\n",
"Train Epoch: 5 [12800/60000 (21%)]\tLoss: 0.371231\n",
"Train Epoch: 5 [13440/60000 (22%)]\tLoss: 0.801004\n",
"Train Epoch: 5 [14080/60000 (23%)]\tLoss: 0.634258\n",
"Train Epoch: 5 [14720/60000 (25%)]\tLoss: 0.280379\n",
"Train Epoch: 5 [15360/60000 (26%)]\tLoss: 0.285845\n",
"Train Epoch: 5 [16000/60000 (27%)]\tLoss: 0.420509\n",
"Train Epoch: 5 [16640/60000 (28%)]\tLoss: 0.490641\n",
"Train Epoch: 5 [17280/60000 (29%)]\tLoss: 0.275802\n",
"Train Epoch: 5 [17920/60000 (30%)]\tLoss: 0.332785\n",
"Train Epoch: 5 [18560/60000 (31%)]\tLoss: 0.359173\n",
"Train Epoch: 5 [19200/60000 (32%)]\tLoss: 0.363823\n",
"Train Epoch: 5 [19840/60000 (33%)]\tLoss: 0.366682\n",
"Train Epoch: 5 [20480/60000 (34%)]\tLoss: 0.486425\n",
"Train Epoch: 5 [21120/60000 (35%)]\tLoss: 0.386097\n",
"Train Epoch: 5 [21760/60000 (36%)]\tLoss: 0.739181\n",
"Train Epoch: 5 [22400/60000 (37%)]\tLoss: 0.487370\n",
"Train Epoch: 5 [23040/60000 (38%)]\tLoss: 0.311916\n",
"Train Epoch: 5 [23680/60000 (39%)]\tLoss: 0.363188\n",
"Train Epoch: 5 [24320/60000 (41%)]\tLoss: 0.496899\n",
"Train Epoch: 5 [24960/60000 (42%)]\tLoss: 0.379592\n",
"Train Epoch: 5 [25600/60000 (43%)]\tLoss: 0.523853\n",
"Train Epoch: 5 [26240/60000 (44%)]\tLoss: 0.369336\n",
"Train Epoch: 5 [26880/60000 (45%)]\tLoss: 0.321644\n",
"Train Epoch: 5 [27520/60000 (46%)]\tLoss: 0.432370\n",
"Train Epoch: 5 [28160/60000 (47%)]\tLoss: 0.603944\n",
"Train Epoch: 5 [28800/60000 (48%)]\tLoss: 0.321047\n",
"Train Epoch: 5 [29440/60000 (49%)]\tLoss: 0.540413\n",
"Train Epoch: 5 [30080/60000 (50%)]\tLoss: 0.335787\n",
"Train Epoch: 5 [30720/60000 (51%)]\tLoss: 0.338214\n",
"Train Epoch: 5 [31360/60000 (52%)]\tLoss: 0.307868\n",
"Train Epoch: 5 [32000/60000 (53%)]\tLoss: 0.311590\n",
"Train Epoch: 5 [32640/60000 (54%)]\tLoss: 0.285531\n",
"Train Epoch: 5 [33280/60000 (55%)]\tLoss: 0.375049\n",
"Train Epoch: 5 [33920/60000 (57%)]\tLoss: 0.417083\n",
"Train Epoch: 5 [34560/60000 (58%)]\tLoss: 0.372413\n",
"Train Epoch: 5 [35200/60000 (59%)]\tLoss: 0.619531\n",
"Train Epoch: 5 [35840/60000 (60%)]\tLoss: 0.277805\n",
"Train Epoch: 5 [36480/60000 (61%)]\tLoss: 0.466849\n",
"Train Epoch: 5 [37120/60000 (62%)]\tLoss: 0.486229\n",
"Train Epoch: 5 [37760/60000 (63%)]\tLoss: 0.309154\n",
"Train Epoch: 5 [38400/60000 (64%)]\tLoss: 0.404888\n",
"Train Epoch: 5 [39040/60000 (65%)]\tLoss: 0.453257\n",
"Train Epoch: 5 [39680/60000 (66%)]\tLoss: 0.361263\n",
"Train Epoch: 5 [40320/60000 (67%)]\tLoss: 0.295195\n",
"Train Epoch: 5 [40960/60000 (68%)]\tLoss: 0.412795\n",
"Train Epoch: 5 [41600/60000 (69%)]\tLoss: 0.414311\n",
"Train Epoch: 5 [42240/60000 (70%)]\tLoss: 0.428503\n",
"Train Epoch: 5 [42880/60000 (71%)]\tLoss: 0.357411\n",
"Train Epoch: 5 [43520/60000 (72%)]\tLoss: 0.428151\n",
"Train Epoch: 5 [44160/60000 (74%)]\tLoss: 0.383007\n",
"Train Epoch: 5 [44800/60000 (75%)]\tLoss: 0.359972\n",
"Train Epoch: 5 [45440/60000 (76%)]\tLoss: 0.411746\n",
"Train Epoch: 5 [46080/60000 (77%)]\tLoss: 0.233550\n",
"Train Epoch: 5 [46720/60000 (78%)]\tLoss: 0.307277\n",
"Train Epoch: 5 [47360/60000 (79%)]\tLoss: 0.329553\n",
"Train Epoch: 5 [48000/60000 (80%)]\tLoss: 0.354325\n",
"Train Epoch: 5 [48640/60000 (81%)]\tLoss: 0.396153\n",
"Train Epoch: 5 [49280/60000 (82%)]\tLoss: 0.347568\n",
"Train Epoch: 5 [49920/60000 (83%)]\tLoss: 0.442707\n",
"Train Epoch: 5 [50560/60000 (84%)]\tLoss: 0.357499\n",
"Train Epoch: 5 [51200/60000 (85%)]\tLoss: 0.453966\n",
"Train Epoch: 5 [51840/60000 (86%)]\tLoss: 0.306474\n",
"Train Epoch: 5 [52480/60000 (87%)]\tLoss: 0.247665\n",
"Train Epoch: 5 [53120/60000 (88%)]\tLoss: 0.231308\n",
"Train Epoch: 5 [53760/60000 (90%)]\tLoss: 0.319081\n",
"Train Epoch: 5 [54400/60000 (91%)]\tLoss: 0.266016\n",
"Train Epoch: 5 [55040/60000 (92%)]\tLoss: 0.487024\n",
"Train Epoch: 5 [55680/60000 (93%)]\tLoss: 0.317053\n",
"Train Epoch: 5 [56320/60000 (94%)]\tLoss: 0.353949\n",
"Train Epoch: 5 [56960/60000 (95%)]\tLoss: 0.293953\n",
"Train Epoch: 5 [57600/60000 (96%)]\tLoss: 0.329859\n",
"Train Epoch: 5 [58240/60000 (97%)]\tLoss: 0.271243\n",
"Train Epoch: 5 [58880/60000 (98%)]\tLoss: 0.245314\n",
"Train Epoch: 5 [59520/60000 (99%)]\tLoss: 0.295283\n",
"\n",
"Test set: Avg. loss: 0.3646, Accuracy: 9015/10000 (90%)\n",
"\n"
]
}
],
"source": [
"for epoch in range(2, n_epochs + 1): # starts from the second iteration\n",
" train(epoch)\n",
" test()"
]
},
{
"cell_type": "markdown",
"id": "54cbd7dc-7bcc-4079-91fb-6ed6e5548268",
"metadata": {},
"source": [
"# Model performance"
]
},
{
"cell_type": "markdown",
"id": "91eb0cb8-9ff9-4d62-9035-ffbbedb9eb22",
"metadata": {},
"source": [
"Finally, we can inspect the results."
]
},
{
"cell_type": "code",
"execution_count": 21,
"id": "eaf16c5a-a610-4f69-96f7-42b37fc13521",
"metadata": {},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAjwAAAGwCAYAAACtlb+kAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjguNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8fJSN1AAAACXBIWXMAAA9hAAAPYQGoP6dpAAB9E0lEQVR4nO3deXwTZf4H8E96t/QCXWgL5ZZLTkEUkEO5BEHw1kUB8VhEQAREERUUFXRBEBVZjwWPVXEFXBQWAbkFVKT8RIHKyimUm5azLW3n98fjZJ6ZzCSZNGnS9PN+vfpKMpnMPJmkM998n8uhKIoCIiIiojAWEewCEBEREQUaAx4iIiIKewx4iIiIKOwx4CEiIqKwx4CHiIiIwh4DHiIiIgp7DHiIiIgo7EUFuwBlraSkBIcPH0ZSUhIcDkewi0NEREReUBQFZ8+eRUZGBiIi7OdrKlzAc/jwYWRmZga7GEREROSDgwcPokaNGrZfV+ECnqSkJADigCUnJwe5NEREROSNM2fOIDMz03kdt6vCBTxqNVZycjIDHiIionLG1+YobLRMREREYY8BDxEREYU9BjxEREQU9hjwEBERUdhjwENERERhL6gBz5QpU3D11VcjKSkJVatWRf/+/ZGdne32NWvWrIHD4XD527VrVxmVmoiIiMqboAY8a9euxaOPPorNmzdjxYoVKCoqQo8ePXD+/HmPr83OzkZOTo7z74orriiDEhMREVF5FNRxeJYtW6Z7PHfuXFStWhU//fQTOnXq5Pa1VatWRWpqqsd9FBQUoKCgwPn4zJkzPpWViIiIyq+QasOTl5cHAKhSpYrHdVu1aoX09HR07doVq1evtlxvypQpSElJcf5xWgkiIqKKx6EoihLsQgBiUrB+/frh9OnTWL9+veV62dnZWLduHVq3bo2CggJ89NFHmDNnDtasWWOaFTLL8GRmZiIvL48jLRMREZUTZ86cQUpKis/X75AJeB599FEsWbIEGzZssD0pWN++feFwOLB48WKP65b2gBEREVHZK+31OySqtEaMGIHFixdj9erVPs2Aeu2112L37t0BKBkREVHo69KlC0aNGhXsYoS0oDZaVhQFI0aMwKJFi7BmzRrUqVPHp+1kZWUhPT3dz6Wzp7gYOHUKKCgAfIjZiIioAvA08eWgQYMwb94829tduHAhoqOjfSyVMHjwYOTm5uLLL78s1XZCVVADnkcffRSffPIJ/vOf/yApKQlHjhwBAKSkpCA+Ph4AMH78eBw6dAgffvghAGDmzJmoXbs2rrzyShQWFuLjjz/GggULsGDBgqC9DwA4fhzo0weIiQE2bgxqUYiIKETl5OQ478+fPx/PPfecbvw59dqnunTpkleBjDedfSq6oFZpvf3228jLy0OXLl2Qnp7u/Js/f75znZycHBw4cMD5uLCwEGPHjkXz5s3RsWNHbNiwAUuWLMGtt94ajLfgpH5HCwuBoqKgFoWIqEJSFODixeD8edsaNi0tzfmXkpICh8PhfJyfn4/U1FR8/vnn6NKlC+Li4vDxxx/j5MmTuOeee1CjRg0kJCSgWbNm+PTTT3XbNVZp1a5dGy+//DKGDBmCpKQk1KxZE++8806pju/atWvRtm1bxMbGIj09HU899RSKpAveF198gWbNmiE+Ph6XXXYZunXr5hxXb82aNWjbti0qVaqE1NRUdOjQAfv37y9VeewKepWWJ8bU3rhx4zBu3LgAlch3clB+8SKQlBS8shARVUT5+UDHjsHZ9/r1+utAaTz55JOYPn065s6di9jYWOTn56N169Z48sknkZycjCVLluC+++5D3bp1cc0111huZ/r06Zg8eTKefvppfPHFF3jkkUfQqVMnNGrUyHaZDh06hN69e2Pw4MH48MMPsWvXLjz00EOIi4vDpEmTkJOTg3vuuQevvvoqbrnlFpw9exbr16+HoigoKipC//798dBDD+HTTz9FYWEhfvjhB4/Ve/4W1IAnnERHA5GRoi0PAx4iIvLVqFGjXGotxo4d67w/YsQILFu2DP/+97/dBjy9e/fGsGHDAIggasaMGVizZo1PAc/s2bORmZmJN998Ew6HA40aNcLhw4fx5JNP4rnnnkNOTg6Kiopw6623olatWgCAZs2aAQBOnTqFvLw89OnTB/Xq1QMANG7c2HYZSosBj584HEBCAnD2LHDhQrBLQ0RU8cTFiUxLsPbtL23atNE9Li4uxtSpUzF//nwcOnTIOb5cpUqV3G6nefPmzvtq1dmxY8d8KtPOnTvRrl07XVamQ4cOOHfuHP744w+0aNECXbt2RbNmzdCzZ0/06NEDt99+OypXrowqVapg8ODB6NmzJ7p3745u3brhzjvvLPPORiHRLT1cJCSI24sXg1sOIqKKyOEQ1UrB+PNn7YwxkJk+fTpmzJiBcePGYdWqVdi2bRt69uyJwsJCt9sxNnZ2OBwoKSnxqUyKorhUQanNUhwOByIjI7FixQr897//RZMmTfDGG2+gYcOG2Lt3LwAxddSmTZvQvn17zJ8/Hw0aNMDmzZt9KouvGPD4kRrhM8NDRET+sn79evTr1w/33nsvWrRogbp165b52HNNmjTBxo0bdW1vN27ciKSkJFSvXh2ACHw6dOiA559/HllZWYiJicGiRYuc67dq1Qrjx4/Hxo0b0bRpU3zyySdl+h5YpeVHzPAQEZG/1a9fHwsWLMDGjRtRuXJlvPbaazhy5EhA2sHk5eVh27ZtumVVqlTBsGHDMHPmTIwYMQLDhw9HdnY2Jk6ciNGjRyMiIgLff/89vv32W/To0QNVq1bF999/j+PHj6Nx48bYu3cv3nnnHdx8883IyMhAdnY2fvvtNwwcONDv5XeHAY8fqS30GfAQEZG/PPvss9i7dy969uyJhIQEPPzww+jfv79zwm1/WrNmDVq1aqVbpg6GuHTpUjzxxBNo0aIFqlSpggceeADPPPMMACA5ORnr1q3DzJkzcebMGdSqVQvTp09Hr169cPToUezatQsffPABTp48ifT0dAwfPhx/+9vf/F5+d0JmLq2yEsi5tB57DPjuO+DZZ4F+/fy6aSIiogotLObSChes0iIiIgpNDHj8iFVaREREoYkBjx+pGR720iIiIgotDHj8SM3w5OcHtxxERESkx4DHj9SAhxkeIiKi0MKAx49YpUVERBSaGPD4kTrSMqu0iIiIQgsDHj9ihoeIiCg0MeDxI47DQ0REFJoY8PgRx+EhIiJ3HA6H27/Bgwf7vO3atWtj5syZflsv3HAuLT9iLy0ionKouBjIygJOnAAuvxxo1QqIjAzIrnJycpz358+fj+eeew7Z2dnOZfHqhYT8jhkeP2KVFhFRObNqFdC3LzB0KPDMM+K2b1+xPADS0tKcfykpKXA4HLpl69atQ+vWrREXF4e6devi+eefR1FRkfP1kyZNQs2aNREbG4uMjAyMHDkSANClSxfs378fjz/+uDNb5Ku3334b9erVQ0xMDBo2bIiPPvpI97xVGQBg9uzZuOKKKxAXF4dq1arh9ttv97kc/sYMjx8xw0NEVI6sWgWMG+e6/NgxsfzVV4Ebbiiz4nzzzTe49957MWvWLHTs2BG///47Hn74YQDAxIkT8cUXX2DGjBn47LPPcOWVV+LIkSP4v//7PwDAwoUL0aJFCzz88MN46KGHfC7DokWL8Nhjj2HmzJno1q0bvv76a9x///2oUaMGrr/+erdl2LJlC0aOHImPPvoI7du3x6lTp7B+/frSHxg/YcDjR2rAU1QEXLoEREcHtzxERGShuBiYNs39OtOnA507B6x6y+ill17CU089hUGDBgEA6tati8mTJ2PcuHGYOHEiDhw4gLS0NHTr1g3R0dGoWbMm2rZtCwCoUqUKIiMjkZSUhLS0NJ/LMG3aNAwePBjDhg0DAIwePRqbN2/GtGnTcP3117stw4EDB1CpUiX06dMHSUlJqFWrFlq1alXKo+I/rNLyI7nqldVaREQhLCtLZHLcOXpUrFdGfvrpJ7zwwgtITEx0/j300EPIycnBhQsXcMcdd+DixYuoW7cuHnroISxatEhX3eUPO3fuRIcOHXTLOnTogJ07dwKA2zJ0794dtWrVQt26dXHffffhX//6Fy6EUJUHAx4/io7Wsjoh9BkTEZHRiRP+Xc8PSkpK8Pzzz2Pbtm3Ov+3bt2P37t2Ii4tDZmYmsrOz8dZbbyE+Ph7Dhg1Dp06dcOnSJb+Ww9j+R1EU5zJ3ZUhKSsLWrVvx6aefIj09Hc899xxatGiB3Nxcv5bPVwx4/IwTiBIRlQOXX+7f9fzgqquuQnZ2NurXr+/yFxEhLtfx8fG4+eabMWvWLKxZswabNm3C9u3bAQAxMTEoLi4uVRkaN26MDRs26JZt3LgRjRs3dj52V4aoqCh069YNr776Kn7++Wfs27cPqwLUANwutuHxs/h44MwZZniIiEJaq1ZA1aruq7WqVRPrlZHnnnsOffr0QWZmJu644w5ERETg559/xvbt2/Hiiy9i3rx5KC4uxjXXXIOEhAR89NFHiI+PR61atQCI8XXWrVuHu+++G7GxsbjcTbB26NAhbNu2TbesZs2aeOKJJ3DnnXfiqquuQteuXfHVV19h4cKFWLlyJQC4LcPXX3+NPXv2oFOnTqhcuTKWLl2KkpISNGzYMGDHzBalgsnLy1MAKHl5eQHZ/u23K0rr1ory448B2TwREfnLt9+KE7bV37ffBnT3c+fOVVJSUnTLli1bprRv316Jj49XkpOTlbZt2yrvvPOOoiiKsmjRIuWaa65RkpOTlUqVKinXXnutsnLlSudrN23apDRv3lyJjY1V3F3ea9WqpQBw+Zs7d66iKIoye/ZspW7dukp0dLTSoEED5cMPP3S+1l0Z1q9fr3Tu3FmpXLmyEh8frzRv3lyZP3++n45W6a/fDkVRlOCFW2XvzJkzSElJQV5eHpKTk/2+/YEDgR07gJkzgeuu8/vmiYjIn1atEr215ExPtWrAmDFl2iWdPCvt9ZtVWn7GsXiIiMqRG24QXc/LaKRlCh4GPH7GGdOJiMqZyEigTZtgl4ICjL20/Iy9tIiIiEIPAx4/Y5UWERFR6GHA42ecQJSIiCj0MODxMzXDw4CHiIgodDDg8TM2WiYiIgo9DHj8LC5O3DLDQ0REFDoY8PgZMzxEREShhwGPn7HRMhERUehhwONnrNIiIiIKPQx4/IxVWkRERKGHAY+fqQEPR1omIiIKHQx4/Ewdh+f8+eCWg4iIiDQMePxMbrSsKMEtCxEREQkMePxMDXiKi4GCguCWhYiIiAQGPH6mVmkB7KlFREQUKhjw+FlEBNvxEBERhRoGPAHArulEREShhQFPADDgISIiCi0MeAKAAQ8REVFoYcATAAx4iIiIQgsDngBgwENERBRaGPAEAAMeIiKi0MKAJwDUgIfd0omIiEIDA54AkKeXICIiouBjwBMAzPAQERGFFgY8AcAMDxERUWhhwBMAzPAQERGFFgY8AcBeWkRERKGFAU8AsEqLiIgotDDgCQBWaREREYUWBjwBUKmSuM3LC245iIiISGDAEwB16wKRkcCJE8Dhw8EuDRERETHgCYCEBKBJE3F/69bgloWIiIiCHPBMmTIFV199NZKSklC1alX0798f2dnZHl+3du1atG7dGnFxcahbty7mzJlTBqW1p3VrcfvTT8EtBxEREQU54Fm7di0effRRbN68GStWrEBRURF69OiB825a++7duxe9e/dGx44dkZWVhaeffhojR47EggULyrDknrVsKW5//TWoxSAiIiIAUcHc+bJly3SP586di6pVq+Knn35Cp06dTF8zZ84c1KxZEzNnzgQANG7cGFu2bMG0adNw2223uaxfUFCAgoIC5+MzZ8747w24kZoqbtk1nYiIKPhCqg1P3p/dmqpUqWK5zqZNm9CjRw/dsp49e2LLli24dOmSy/pTpkxBSkqK8y8zM9O/hbYQGytuCwvLZHdERETkRsgEPIqiYPTo0bjuuuvQtGlTy/WOHDmCatWq6ZZVq1YNRUVFOHHihMv648ePR15envPv4MGDfi+7mZgYccuAh4iIKPiCWqUlGz58OH7++Wds2LDB47oOh0P3WFEU0+UAEBsbi1g13VKGGPAQERGFjpAIeEaMGIHFixdj3bp1qFGjhtt109LScOTIEd2yY8eOISoqCpdddlkgi2mLGvAUFACKApjEYkRERFRGglqlpSgKhg8fjoULF2LVqlWoU6eOx9e0a9cOK1as0C1bvnw52rRpg+jo6EAV1TY5qVRUFLxyEBERUZADnkcffRQff/wxPvnkEyQlJeHIkSM4cuQILkpdm8aPH4+BAwc6Hw8dOhT79+/H6NGjsXPnTvzzn//E+++/j7FjxwbjLVhSMzyAyPIQERFR8AQ14Hn77beRl5eHLl26ID093fk3f/585zo5OTk4cOCA83GdOnWwdOlSrFmzBi1btsTkyZMxa9Ys0y7pwSQnm9iOh4iIKLiC2oZHbWzszrx581yWde7cGVtDfM4Gh0NkeQoLGfAQEREFW8h0Sw9HcsNlIiIiCh4GPAGkNlw2GQ+RiIiIyhADngBS2/Eww0NERBRcDHgCiIMPEhERhQYGPAGkVmkxw0NERBRcDHgCSM3wsA0PERFRcDHgCSD20iIiIgoNDHgCiG14iIiIQgMDngBiwENERBQaGPAEEBstExERhQYGPAGkjsPDRstERETBxYAngJjhISIiCg0MeAKIbXiIiIhCAwOeAFIzPAx4iIiIgosBTwCpbXgY8BAREQUXA54AYpUWERFRaGDAE0BstExERBQaGPAEEDM8REREoYEBTwBxLi0iIqLQwIAngDhbOhERUWhgwBNA7JZOREQUGhjwBJCa4cnPD245iIiIKjoGPAGUkSFud+4Etm4NblmIiIgqMgY8AVS/PtCvH6AowFtvBbs0REREFRcDngB76CFxu307cPZscMtCRERUUTHgCbC0NKBWLaCkBPjxx2CXhoiIqGJiwFMGrr1W3G7eHNxyEBERVVQMeMpAs2bidt++oBaDiIiowip1wHPmzBl8+eWX2Llzpz/KE5YSEsQtu6cTEREFh+2A584778Sbb74JALh48SLatGmDO++8E82bN8eCBQv8XsBwEB8vbhnwEBERBYftgGfdunXo2LEjAGDRokVQFAW5ubmYNWsWXnzxRb8XMBzExYlbBjxERETBYTvgycvLQ5UqVQAAy5Ytw2233YaEhATcdNNN2L17t98LGA6Y4SEiIgou2wFPZmYmNm3ahPPnz2PZsmXo0aMHAOD06dOIU1MZpMMMDxERUXBF2X3BqFGjMGDAACQmJqJWrVro0qULAFHV1UztjkQ6asBz8aIYddnhCG55iIiIKhrbAc+wYcPQtm1bHDx4EN27d0dEhEgS1a1bl214LKgBj6KImdPVWdSJiIiobNgOeACgTZs2aNOmDQCguLgY27dvR/v27VG5cmW/Fi5cyDV9+fkMeIiIiMqa7TY8o0aNwvvvvw9ABDudO3fGVVddhczMTKxZs8bf5QsLUVFAdLS4f/FicMtCRERUEdkOeL744gu0aNECAPDVV19h79692LVrF0aNGoUJEyb4vYDhgg2XiYiIgsd2wHPixAmkpaUBAJYuXYo77rgDDRo0wAMPPIDt27f7vYDhggEPERFR8NgOeKpVq4YdO3aguLgYy5YtQ7du3QAAFy5cQGRkpN8LGC44Fg8REVHw2G60fP/99+POO+9Eeno6HA4HunfvDgD4/vvv0ahRI78XMFwww0NERBQ8tgOeSZMmoWnTpjh48CDuuOMOxP7Z5SgyMhJPPfWU3wsYLhjwEBERBY9P3dJvv/12l2WDBg0qdWHCmTz4IBEREZUt2214AGDt2rXo27cv6tevjyuuuAI333wz1q9f7++yhRVmeIiIiILHdsDz8ccfo1u3bkhISMDIkSMxfPhwxMfHo2vXrvjkk08CUcawwEbLREREwWO7Suull17Cq6++iscff9y57LHHHsNrr72GyZMn469//atfCxguWKVFREQUPLYzPHv27EHfvn1dlt98883Yu3evXwoVjlilRUREFDy2A57MzEx8++23Lsu//fZbZGZm+qVQ4Uit0vrXv4AjR4JbFiIioorGdpXWmDFjMHLkSGzbtg3t27eHw+HAhg0bMG/ePLz++uuBKGNYUDM8BQXAK68AM2YEtzxEREQVie2A55FHHkFaWhqmT5+Ozz//HADQuHFjzJ8/H/369fN7AcNFTIx2nx3aiIiIypZP4/DccsstuOWWW3TLTp8+jQ8//BADBw70S8HCTW6udr9586AVg4iIqELyaRweMwcOHMD999/vr82Fnf79tfvFxUErBhERUYXkt4CH3KtZE5g9W9xn13QiIqKyxYCnDKk9tQoKglsOIiKiioYBTxlSAx5meIiIiMqW142WZ82a5fb5Q4cOlbow4Y6jLRMREQWH1wHPDC8GjqlZs2apChPu5NGWFQVwOIJbHiIioorC64CH00aUnlqlpSiiHY8aABEREVFgsQ1PGZIDHM6pRUREVHYY8JShyEhtxGUGPERERGWHAU8ZY8NlIiKishfUgGfdunXo27cvMjIy4HA48OWXX7pdf82aNXA4HC5/u3btKpsC+4HajocZHiIiorLj01xa/nL+/Hm0aNEC999/P2677TavX5ednY3k5GTn47/85S+BKF5AMMNDRERU9rwKeM6cOeP1BuVAxJNevXqhV69eXq+vqlq1KlJTU22/LhQw4CEiIip7XgU8qampcHg5aExxGcyM2apVK+Tn56NJkyZ45plncP3111uuW1BQgAJpLgc7wVsgsEqLiIio7HkV8Kxevdp5f9++fXjqqacwePBgtGvXDgCwadMmfPDBB5gyZUpgSvmn9PR0vPPOO2jdujUKCgrw0UcfoWvXrlizZg06depk+popU6bg+eefD2i57OD0EkRERGXPoSiKYucFXbt2xYMPPoh77rlHt/yTTz7BO++8gzVr1vhWEIcDixYtQv/+/W29rm/fvnA4HFi8eLHp82YZnszMTOTl5dmqfvOXJ54AVq8GnnoKuP32Mt89ERFRuXTmzBmkpKT4fP223Utr06ZNaNOmjcvyNm3a4IcffrBdgNK69tprsXv3bsvnY2NjkZycrPsLJnl6CSIiIiobtgOezMxMzJkzx2X5P/7xD2RmZvqlUHZkZWUhPT29zPfrK1ZpERERlT3b3dJnzJiB2267Dd988w2uvfZaAMDmzZvx+++/Y8GCBba2de7cOfzvf/9zPt67dy+2bduGKlWqoGbNmhg/fjwOHTqEDz/8EAAwc+ZM1K5dG1deeSUKCwvx8ccfY8GCBbb3G0zspUVERFT2bAc8vXv3xu7duzF79mzs2rULiqKgX79+GDp0qO0Mz5YtW3Q9rEaPHg0AGDRoEObNm4ecnBwcOHDA+XxhYSHGjh2LQ4cOIT4+HldeeSWWLFmC3r17230bQcNeWkRERGXPdqPl8q60jZ5Ka9484M03gT59gEmTynz3RERE5VJpr98+jbScm5uL999/Hzt37oTD4UCTJk0wZMgQpKSk+LK5CqVyZXF76hSQmwuU0/ETiYiIyhXbjZa3bNmCevXqYcaMGTh16hROnDiB1157DfXq1cPWrVsDUcawcvnl4nbjRqBbN2DRouCWh4iIqCKwXaXVsWNH1K9fH++++y6iokSCqKioCA8++CD27NmDdevWBaSg/hLsKq3sbGDAAP2yLVvKvBhERETlSplXaW3ZskUX7ABAVFQUxo0bZzo+D+mpGR4iIiIqO7artJKTk3U9p1QHDx5EUlKSXwoVzipXBiIj9ctKSoJTFiIioorCdsBz11134YEHHsD8+fNx8OBB/PHHH/jss89Mp5sgVxERgLFt97FjwSkLERFRRWG7SmvatGlwOBwYOHAgioqKAADR0dF45JFHMHXqVL8XMBwZJ5Tftw9ISwtKUYiIiCoE2wFPTEwMXn/9dUyZMgW///47FEVB/fr1kZCQEIjyhaXCQv3jffuAPwetJiIiogDwaRweAEhISEDlypXhcDgY7NhkDHiOHw9OOYiIiCoK2214SkpK8MILLyAlJQW1atVCzZo1kZqaismTJ6OErW+9cvPN+secV4uIiCiwbGd4JkyYgPfffx9Tp05Fhw4doCgKvvvuO0yaNAn5+fl46aWXAlHOsDJqFHDllcCePcAnn3BeLSIiokCzHfB88MEHeO+993CzlKZo0aIFqlevjmHDhjHg8UKlSkD//sCnn4rHzPAQEREFlu0qrVOnTqFRo0Yuyxs1aoRTp075pVAVBWdOJyIiKhu2A54WLVrgzTffdFn+5ptvokWLFn4pVEURFydumeEhIiIKLNtVWq+++ipuuukmrFy5Eu3atYPD4cDGjRtx8OBBLF26NBBlDFtqwMMMDxERUWDZzvB07twZv/32G2655Rbk5ubi1KlTuPXWW5GdnY2OHTsGooxhS63SYoaHiIgosHwahycjI4ONk/2AbXiIiIjKhk8BT25uLn744QccO3bMZeydgQMH+qVgFQHb8BAREZUN2wHPV199hQEDBuD8+fNISkqCw+FwPqfOsUXeYYaHiIiobNhuwzNmzBgMGTIEZ8+eRW5uLk6fPu38Y7d0e+Q2PIoS3LIQERGFM9sBz6FDhzBy5EjOn+UHapWWogAFBcDBg8Dnn7vOtUVERESlY7tKq2fPntiyZQvq1q0biPJUKGrAA4hqrdtvB4qLgXPngCFDglcuIiKicONVwLN48WLn/ZtuuglPPPEEduzYgWbNmiE6Olq37s3GmTHJUlQUEB0NXLokAp7iYrF861YGPERERP7kVcDTv39/l2UvvPCCyzKHw4Fi9apNXomPFwGP3FMrwnZFIxEREbnjVcBj7HpO/hMfD5w5ow94IiODVx4iIqJwxFxCkMXGiltmeIiIiALHqwzPrFmz8PDDDyMuLg6zZs1yu+7IkSP9UrCKQu2afuGCtowZHiIiIv/yKuCZMWMGBgwYgLi4OMyYMcNyPYfDwYDHJjXgOXlSW8aAh4iIyL+8Cnj27t1rep9KTw14jh/Xll26FJyyEBERhSu2FgkydSweOeDh3FpERET+5VWGZ/To0V5v8LXXXvO5MBWRGvAcOaItY8BDRETkX14FPFlZWV5tTJ5IlLxTv7643bhRW8aAh4iIyL+8CnhWr14d6HJUWPfcAyxbBuzerS2Te2wRERFR6fnchud///sfvvnmG1z8Mx2hcLpvn8TEAN266Zfl5wenLEREROHKdsBz8uRJdO3aFQ0aNEDv3r2Rk5MDAHjwwQcxZswYvxewIqhXT/+YGR4iIiL/sh3wPP7444iOjsaBAweQkJDgXH7XXXdh2bJlfi1cRWGceD4/H+BsHkRERP7jVRse2fLly/HNN9+gRo0auuVXXHEF9u/f77eCVSTVq+sfKwpQUKCN0UNERESlYzvDc/78eV1mR3XixAnEqhNDkS1mIyuzpxYREZH/2A54OnXqhA8//ND52OFwoKSkBH//+99x/fXX+7VwFckdd+gfsx0PERGR/9iu0vr73/+OLl26YMuWLSgsLMS4cePw66+/4tSpU/juu+8CUcYK4YkngAcfFN3UT51iTy0iIiJ/sp3hadKkCX7++We0bdsW3bt3x/nz53HrrbciKysL9YzdjchrERHAZZeZz55OREREpWM7w/Ptt9+ia9eueP75512ee/PNNzF8+HC/FKyiYsBDRETkf7YzPLfddht+/PFHl+UzZ87E008/7ZdCVWRqe/DcXPPnd+0Sf0REROQ92wHPjBkz0Lt3b+zYscO5bNq0aZg4cSKWLFni18JVRE2aiNtPPnEdi+fiReDee8VfQUHZl42IiKi8sl2ldf/99+PkyZPo0aMHNmzYgPnz5+Pll1/Gf//7X7Rv3z4QZaxQ7r8f+OorYMcOICsLaN1ae+7kSe3+xYsARwEgIiLyju2ABwDGjh2LkydPok2bNiguLsby5ctxzTXX+LtsFdJll4ksz48/AseP6587c0a7zwwPERGR97wKeGbNmuWyLD09HQkJCejUqRO+//57fP/99wCAkSNH+reEFVBiorj95BNg82bgmWeAqCjRXV3FbutERETe8yrgmTFjhunyyMhIfPfdd87xdxwOBwMeP0hOFrc7doi/q68GbrpJ35CZAQ8REZH3vAp49u7dG+hykCQpSf9Ybbtz+rS2jAEPERGR92z30qLAU6u0VIoiblmlRURE5BuvMjyjR4/G5MmTUalSJYwePdrtuq+99ppfClaRqVVaKrWBMqu0iIiIfONVwJOVlYVLly4571NgGau01KosVmkRERH5xquAZ/Xq1ab3KTAY8BAREfmX39rw7Ny5E3Xr1vXX5io0BjxERET+5beAp7CwEPv37/fX5io0BjxERET+xV5aIcjYaPn0adFw+eJFbRkDHiIiIu8x4AlBxm7pubn67A7AgIeIiMgOBjwhyDgpqKIAf/yhX8aAh4iIyHteTx5auXJlOBwOy+eLior8UiACzA6zcbBrTh5KRETkPa8DnpkzZwawGORJdrb+MTM8RERE3vM64Bk0aFAgy0EGkyYBGzYAFy4AGzeKSURlDHiIiIi8xzY8IapPH2DqVKBePfH4t9/EbWSkuGXAQ0RE5L2gBjzr1q1D3759kZGRAYfDgS+//NLja9auXYvWrVsjLi4OdevWxZw5cwJf0CCqXl3/uGpVccuAh4iIyHtBDXjOnz+PFi1a4M033/Rq/b1796J3797o2LEjsrKy8PTTT2PkyJFYsGBBgEsaPMEKeBQFGDkSePLJwO6HiIioLHjdhicQevXqhV69enm9/pw5c1CzZk1nA+rGjRtjy5YtmDZtGm677bYAlTK4jAFPtWriNtABz6FDou2Quq+4uMDuj4iIKJDKVRueTZs2oUePHrplPXv2xJYtW5yzuRsVFBTgzJkzur/yJD0diJA+JTXgKSgAPvwQWLUqMPstLtbuc8QBIiIq72xneEaPHm263OFwIC4uDvXr10e/fv1QpUqVUhfO6MiRI6imXvH/VK1aNRQVFeHEiRNIT093ec2UKVPw/PPP+70sZSU6GrjiCq1bulqldfQoMGsWkJoK3HCD//dbUqLdLyz0//aJiIjKku2AJysrC1u3bkVxcTEaNmwIRVGwe/duREZGolGjRpg9ezbGjBmDDRs2oEmTJn4vsHHwQ0VRTJerxo8frwvSzpw5g8zMTL+XK5C6d3cNeFR5eSI4ifBzrk4e2JABDxERlXe2L5P9+vVDt27dcPjwYfz000/YunUrDh06hO7du+Oee+7BoUOH0KlTJzz++ON+L2xaWhqOHDmiW3bs2DFERUXhsssuM31NbGwskpOTdX/lTc+e2v20NP1ziiLG6vE3BjxERBRObGd4/v73v2PFihW6wCE5ORmTJk1Cjx498Nhjj+G5555zaWvjD+3atcNXX32lW7Z8+XK0adMG0dHRft9fqEhPB0aNEtkcdVwe2blzrhOOlhYDHiIiCie2Mzx5eXk4duyYy/Ljx487GwSnpqai0Iur5Llz57Bt2zZs27YNgOh2vm3bNhw4cACAqI4aOHCgc/2hQ4di//79GD16NHbu3Il//vOfeP/99zF27Fi7b6Pcufde4NFHxcSixuqrc+f8vz8GPEREFE58qtIaMmQIFi1ahD/++AOHDh3CokWL8MADD6B///4AgB9++AENGjTwuK0tW7agVatWaNWqFQDRILpVq1Z47rnnAAA5OTnO4AcA6tSpg6VLl2LNmjVo2bIlJk+ejFmzZoVtl3QzDgeQkKBfJgc8xcXA2rXAqVOl2w8DHiIiCie2q7T+8Y9/4PHHH8fdd9/tnCE9KioKgwYNwowZMwAAjRo1wnvvvedxW126dHE2OjYzb948l2WdO3fG1q1b7RY7rCQk6IOcs2e1+wsWAK++CmRkAIsX+74PeZwfBjxERFTe2Q54EhMT8e6772LGjBnYs2cPFEVBvXr1kCg1ImnZsqU/y0gG8fH6x3Lw8+234vbw4dLtgxkeIiIKJz6PtJyYmIgqVarA4XDogh0KvEqV9I/lgMeid75tcsBjMaYjERFRuWG7DU9JSQleeOEFpKSkoFatWqhZsyZSU1MxefJklMij1VHAuMvwlCbgURQtuJEDHvk+ERFReWQ7wzNhwgS8//77mDp1Kjp06ABFUfDdd99h0qRJyM/Px0svvRSIcpLEXaPl0gQ8I0YAu3cDixYxw0NEROHFdsDzwQcf4L333sPNN9/sXNaiRQtUr14dw4YNY8BTBowBj9xoWQ54FAXYtw+oWROIjHS/zaIiYPNmcX/7dmZ4iIgovNiu0jp16hQaNWrksrxRo0Y4Vdq+0OQVbzM8H30E3HEHMG2a520eP67dT0rS99JihoeIiMo72wFPixYt8Oabb7osf/PNN9GiRQu/FIrc8zbgmTVL3P773563KffqKilhhoeIiMKL7SqtV199FTfddBNWrlyJdu3aweFwYOPGjTh48CCWLl0aiDKSgbuAx9d24zk52v1Ll9iGh4iIwovtDE/nzp3x22+/4ZZbbkFubi5OnTqFW2+9FdnZ2ejYsWMgykgG7gKeixd926Yc8BQVcRweIiIKLz6Nw5ORkeHSOPngwYMYMmQI/vnPf/qlYGTNqlv6pUv6tjd2yFVaDHiIiCjc2M7wWDl16hQ++OADf22O3DAbePCPP4AbbgB++823bR45ot1nwENEROHGbwEPlR1jhufCBWDOHN+rswDXKi3OpUVEROGEAU85ZGzDAwCHDpVum+fPa/eNjZYZ8BARUXnHgKcckgcRVLuhyxkaX8gZHVZpERFRuPG60fKtt97q9vnc3NzSloW8FCGFqZUrA6dOASdO+L49RWHAQ0RE4c3rgCclJcXj8wMHDix1gciz9HTtflKSCHhKo6BABD0qBjxERBRuvA545s6dG8hykA1pacDs2UByMjBlivev++474B//AJ57DqhfX1tu7MpubMNz5oxYFh1dunITEREFC9vwlFNt2wKNGokMj7fGjQN27AAefVRbZjZ2jzHg2bkT+NvfSldeIiKiYGLAU84lJnq3XnGxFsScPClup00TY/f873/6dS9ccH39zz/7XkYiIqJgY8BTznkb8Bw6pK/GOnIE+OwzMXbP55/r15WnqpAVFflWRiIiomBjwFPOeRvw3HqrPpOzcqV23ziuz9mz5tuwGtjw5Elg+nRg717vykJEFA7OnweWLzfPilPoYcBTzskBz9VXe/+6TZu0+8ZAxirDYxXwPPcc8OmnwIMPer9/IqLybuJE4OmngcmTg10S8gYDnnJODngeegh49VVgyRLPr8vK0u6fPq1/zirDY/UrZssWcZuX53m/vjhwALjlFuDLLwOzfSIiX6xZI25XrAhqMchLDHjKOTngSUsTjZCrVfP8OnlsHeM4PmrAE2H4dsi9uRQF2L5dpHSLi+2V2a5XXgEOHgRefDGw+yEiovDl9Tg8FJouXdLuV63q2zaMAY9apZWerp+jS87wfPMN8MwzQIMGvu1TVVgIlJQAcXHW65RmUlQiIiKAGZ5yTx4MMEoKX+X5tjwxjqSsZnj+8hfRg0sdZFsOPL7+Wtz+9pv3+zEqKQH69AF69NAHbmbrERERlQYDnnKue3egY0dgzBj98tLM8qEGPPHxQN264g/QBzzyVBR27NyptTHKzxfZpQsXgKNHrV/DgIeIiEqLVVrlXGwsMGOG6/K//Q1o2hR4/XXR6NcONZhRq5ni48Wt3YBn505g6VLRmDo5WSy77z5xm54O1Kljb3uh7vx58V6vuw4YNizYpSGiQIuOdp+dptDCDE+YiooCOnfWghVvGNe1G/DExOgf33ef6K7+r3+5rnvokH578lQWRuUlw7N0qaji++c/g10SIioLnF+wfGHAE+bcBRJGxkEM1UDH24BHbngsj8qsTmUh9/JKTNQ/PncOGDRINIQ2Ki8BTzhkqYjIe74EPNu3A9u2+b0o5AUGPGHO2CDZHWPA4y7DYxaEyA2l5VGd09LErTzeT3S0PuD5v/8Dfv0VWLbMdTLTQAYSe/YAw4f7Z64wOcPF4Ico/NkNeAoLgfvvF4O0nj8fmDKRNQY8YS5QGR4zcnD1f/+n3VfruOWAxzhLuzy688GD+u0GMniYOBHYvBkYMqT024qN1e5zqHmi8Gc34JHPx1YDvFLgMOAJc8YMj/wPKl+gASApSf/YGPDIF3GzDI/8z7xzp3ZfDWzcBTxHjmj39+/Xb1fel78HOVSr2/xBHqiRv96ooiopET8gxo4NdkkCz27AI5/LmAUue+ylFeaMGR6HQ7ufmKh/3lOVlnGkZaOiIhGQREbqMzbeBDxyt3RjrzJ5X5cu2RtjyJPKlYFjx/yzLbm3xrlzvg8ESVSe7d2rVRGr54NwJY99VlLiOjq9kdy2MdAj1JMrZnjCnDHDU6uWdt8Y4Nhpw2P160TdnxxIqfdzc7Vl7gIeY5WW/KvIThWdNy67TLtfXCyCsv/+117bJ5V8MvNXhicvDzhxwj/bIioL8o+qcO+yLWd4vDlnyOcI+T6VDQY8Ye7yy7X7HTsCU6Zoj+Pi9L++PLXhkau0rAIeNSCRAxOrDI8cQMlZFmOVlnxi8HfAI1fjHT8uZj5+9lng7bftb0s+4VnNOG+HogBduwI33sgqMio/5CyHv/9fQ40c8Bg7W5iRszq+/Kii0mHAE+ZmzgRatgTef18MUFi7tvZcZKT+H1DO/gBahichQdx6qtICtH9ieV2zgKewUL+O/EtQnr/LuC1/nyTk/R45Avz4o7hvNnaQnW35I0CRP5vDh0u/PQqcc+eARx4BFi0KdkmCTz43hPtFXX6v3sz5J/94C/djE4rYhifMNWoEvPee+XPGuvWaNfWP1cyOGvh4arQMuM/wGKu0rOqw8/LEiURNjcsBj79/Mcrbk4MKX8b+MbbhKS35hKgoojdZYqIYQZtCywcfiGD5xx+BW24JdmmCK5AZ2VAjn8O8yfCwSiu4mOGpgNR5th57TFvWrp2+AR6gBTxqhkf+BWP168Qs4PGmDY+sqEh7jaIENuCR30dOjms57AhkwHPsmBgvaPDg0m93/34xKjR7ifgPqxw18v9BuGcxShPwhPuxCUXM8FRAI0YADzwAVKokBsH65htgwgR9lRPgWqUln9St/lnNGi1bZXjcnSDOnRP7V3t+edqvmV9/FRkRuapOzhwZy3nypAj61JPSwYP6+b488XeVlrw9ORgrbc+X224Tt5GRQM+evm+HNJ5651QkFemizoCnfOG/aQXkcIhgBwAefRRYvFiMhmyV4VF7MuXmiovwuXPW/9zuAp68PG2ZNwGP/FpVQQHw5ZdinI/cXOCTT8R4H8ZszG+/iakq7r9fW7Z7t2gE/Prr+u2pjh7Vb2f3buvymQlkhkfetr96vsiDQ1LpyEF0qCsqAsaNA+bNC9z2VeF+UZffq91Gy+Hegy0UMeAhJ2PAo2Z4UlO157ZuBbp3tx67Rv2nN1ZplZToRxb1NeApLARefFGM8/GPfwCvvQasWQN8+61+vX//W9yeOaOddJ98Ujz+6COtOkc+IRu7w9sdnyeQGZ5AXERYpeU/pcnwFBXps5+B9u23wKpVwJtvBmb7FSngkQMYu42WGfCUPQY85CR3sXQ4RKCj3v/LX8T9V15x/49aWCgupMYMz/nz+obA69aJ8W6MqlUTt2pwZJbhURl7famKi4H167XHeXlizix5QMNnnwV+/12/PWPAc+aMa/ncMeuWfu4c8M47wL599rYF6I9zIHqqceAz/ylNhmfoUKBbN9fvX6AYe0H6m/y9ZaNlvYoUDIYiBjzkJGd4qlTRP1YDHuMoyEaFha4nufx8fXUWYN7NOiJCm2jUXZWWSs6iyMHa8eP6wfpyc/WTmQJiktLp0/XbMwZydgMeswzP66+LgOeOO+xtC9CfEOUecv66iDDD4z+lCXjUmbPNfgAEgrGtnr9VpIt6aQIeZnjKHgMecpIDHHnAQsD7aRLy810vyIWF3qXsMzKA5GRx312VlurUKe2+3IjXGKjk5pqPVvzDD+5PyHYDHvlkppZfbSfjS3BhVUUmLz99Gnj4YeDrr+1v35eu94G2fbvoRagGAeWFP6ZPKKsAVJ4/LhDfAXZLt8aAJ7gY8JCTHPDIUy4AWobHSkqKuDXL8AAi6+JJ7draaM/eZHjkX6rycmOgcvq0dpJv21ZbXreu+xNyaaq01ACvNBcxeXtywCOX+Z//FO2qJk2yv/1QDHgefBDYsUMEceVJeWq0LP9QCEQGpqJmeLxpw8ORloOLAQ85yQGP2n5H5UvAozZ6BvRzZVmpU0cLePLygLfeAlav1q9j7EZuttzYQyo3Vx/wqL1Tzp41P+moAzB6G/D8+qtooyP/YjtxovS/2K0CHnk/clWXv1y8GLyTsXpBCMVgzB054LHzucvrllWGRw54ApFlqEgBj92BBAOV4Slv/y/BwoCHnNwFPJ6qtNSAp6BACz7i44HYWHHfmx5PtWppAc/atcDcucDnn+vXMTZOVsmZIHdVWpdfrr03OWCSqWPvGNsdmTl2THR/v/1218aaZ8+W7iJmFdjIwZ06dIAv1OP32mtiUMOLF8W2r78e6NvX9+2GuqIiMSKyN1UQ3pJ7adlpDC5/xmV10ZK/96UNSPbtE/+rsoo68KDdgMdfx+aNN8R8e3Z7lVZEDHjIqTQZHrntjXpBjo11DXjU8X/M1K6tTea5d6+2PDJStO8BrKug5OVy93dAn+G57DItqLK6wDRsaL4dM/JEp8ayHT8emCotebl6fI3LrcjvWS3bJ5+IaSvefVe8n6Iicbz+8Q9g5Urfyh7K3n5bzHv1zDP+26bVDOGeqjm8mZ/On4wdCEp70b39dmDMGCArS1tWkTI8pQl4/DW1xAcfiKzdRx/5Z3vhjAEPOckNLytX1j/XvDnQowcwYICWzZGpAc/cuWJgQEBcjNVqLTXgMbYNktWs6TpjOyCqtnr0EPetxreRLxxmAY+c4THuIyJCH+DJAc/Fi+7bH8knLeN+SxvwWDVali8icpDqTUZKLm9Jif7x/Pn6gOjdd4G//9378pYXn30mbtes8d825QyPekzXrAE6dtTGhDIjB0Rl0YjV2HjfXwHJ9u3a/YraaNmfGZ4TJ8TgkD/84HvZyBUDHnKSf6UaMzzR0cDLLwOPP651HTc+r1JnjDYLeIy9vy67TExzMXy46ApvDEbuuANo0waIiRGPrUYwNsvwpKeL2xMntEbEl10mggR1ugxjOQEt4FEUYORIoE8fMY6PGfmkZaxK86ahtjveBDzy+/amJ5wx4DHOU2Zsa+VNEBVMOTnWn40V4wCb/qZ+bmPHittXXrFeVz7+3jR6VRUXiyyo3YDaWI3nryBL/h4GMsPzj38ATzwRGmNIlZTofyD4M+B59VUxOOSwYd6Xh9ObeMZDRDpqYNGihfU6ZtVSnTu7LouLc63SkjM8DRqI4OiRR7RJMY0Bjxp4eQp4zNrwZGaK2z17xIUhMlLbnlp1pm5bbiNTrZoWEGVliZOrsZ2CSi6PGhyogyf6kuE5fRrYtUvc96aXlvy+vQl45AucorhmzIxjtBQVBaZa4sQJ4O67RbYlP1+01TpyxP52+vYF7rzT3kjF/uhCvno1sGWL9li+8NkJIuTPz04D9BkzxI8BNVvlLWPZ/JWBkb8jgQx43n1XHPvvv/fvdn1hrBL3Z5XWH3/YLw8DHs94iPyluFicAZctE7eh8BPEB//9rxjTpUoV63XMGsp27gxs2KAPhuTMiXrikzM8iYn6TAugD0QArfpMDZzkHiYyswyP2ttKvRhWqaKdFOT9xMXpq6McDq2KTmUVaMmvUy9eansjXwKeJ54A7r3XtdeX1X05K2A34Ckqcs0qmI1XFIieYO+8IwaDnDZNdEF/9VXx690O+YJj5wJR2oDnyBHxOQ0dqi1zV7Uh7+/cOdFeSl1HPv52GlGrgc5bb3n/GsA1APFXhqesAh6VPxuc+8r4Ofszw+NL+x4GPJ7xEPnDqlXip+bQoaIl5NCh4vGqVcEumW0pKeZVVrJGjcyXx8UBLVtqj41VRZGRwJVXao/NAidj42g1I9Oggbj99VfzfZu14WncWL9OjRrafWOGxxiYGE+oVsPxmwVCdgMeOdOiTli6a5f1CdEqw+PNCLrGE64xmDELKP0xL5iRvN8dO8Ttxo3m65aUmPeok4+PnSDGuO7WrcBjj3kfNMm9YdTP192kkHJW84knRPXthx+Kx75meFTuOgGYMZbNXwFJWQQ88jEOhVHCjb9p/Rnw+PJ7uTyNBRUsDHhKa9Uq0brM2Cfw2DGxvBwGPZ7cf7+oRjD7B2vSRLsfGwv07Kk9vu020RNLZczuACLgkdsDqQFPy5bue4qZBTzp6VqWBwCaNtXuy1VnanWZzNho2yrgMevJpQZW3s5Z9PrrYhb3jRu14OKPP6x/fc+bJ6rYNm4U84Gp7GZ4vv9eNDKXmQUWFy6IC8zRo/670Jj1kLviCvN1X3tNfI/khrGA/jO3c7I3BjwPPwx89x0wYYJ3r5fLrl6YjBkeuWxytvTHH8Xt/Pni1tc2PCqz/yF3jBfZ0gQk8nfBqg2PPxstG6tjg620AY+77BoDnsBgwFMaxcUiJ+/O9OnltnrLSlyciOVatXJ9rlYt7X5sLNC/PzB1qmivMWyYPriQsz+qyEitsTGgBTyRkWKCRStmVVrJyfqMkhzwyBme2Fiti/KLL4rbkSOBLl1EWwnA+te/WcDTrJm43bNHfxEbONC87cHHH4sT4ciR2rI//rC+GJ04IboCjxypn5TUbqPl4mLXgR2tMjwLFgA33QSMH++fX+1mFyyri+POneJWDu6M69sZw8aq0bIv7SbU42kMeOT2SGbfczWb40vAIx+7YAY8VqMGB2ocHl8/70Axnta9Oc27ywTa3Rag/y6wSsszHqLSyMryPNrT0aP6QSrCiNlJR87gqO1uunUTPVYSE/XZG6tB86pX1+7LXeDV3lNmzBotJyUB9eppy90FPP37i27EN94olnXsKGLZq68Wj8+eNR952SzgqV5dZKOKi/UBxI4dwKOPao/37xdTQ5hxF/BYsZvhMWOV4VGzKytXAosX2yuXt6wCHvW4GwMCdxO/umN1YTD2uDFrz2S2nnyrlkUOeMzKpr4XbwMedeTxffv03zk54Fm/Xkwx4m47/gx4rDI5garS8vXz9sXFi54zNoHM8Hgb0MnbYMDjGQ9RaVidEX1dr5wx+xUiVyGZXcC8CXjkKgC5e7yxS7tM3Zc8tUVSkr5Nkdp7Sn1OpWadzMYAiovT2mD89pvr82ZteKKj9cGVldtvB2bPNn/ujz/sN1wMZMAjty+Rq+rWrgWefdZ+Ox9PGR755K0GPO4mkrVzAbRq71NYCBw+LO4/+qgIftU2VTKzgMfYS8tTwKPyNuB56SVR/fjQQ/rTifwdefxx0eFAbR9kxp+Nlq0yOXartPbsEW2bsrPdr2c1HIO/nTsnfqQNGuR+vUC24fH2f1/eBqu0PGPAUxrursC+rFfOmAU88i9Os27G3gQ8ckNMuTrA3WHMzxcnzTvuEI8dDhHAtGwpahWN3XeNGR531AlHX33V9URkluGJjtZXpZkpKXHfDuHECfdzeZlt35uRoT1d4MwCl/Pn9WWRg7wxY0TPPk81u0Zm792sTY6iaN39jQGBvL4/Ap6CAuDmm0Uny59+EsvUMaVkZr/SjVVa8mfh7sImvyd3jZY3bBC3p0/rAx6z17hLOlt1Sy8qEh1M7QwNYHXxtjtf1NSpomp1wAD368mftzH4/c9/RO83f8jKEsclO9v9/2hZNVr+9FPrz1Q+vgx4PGPAUxqtWnmeZKpaNfPGLmFgyBBxq1YDGZlNGCq34bFqSyFXY8n/xGaNlu++W9yePy9Ommr2oVIlLUvQuTNQv77+dddco9331HZj9GixvT17tDFyVFYBj9yWyYw3g+XJ7XOMqld3bWztzWSnVidld+nwCxf02zYLir76yvO+ZWYXEnkfalCSn2/ehRvQ/8q3U3UiBzxm5ZCnNZEDikuXRMPmZ5/Vlpm14bl0yfvqF2OGx6rthvz+PAU87j5LqwzP55+Ldmx33WX9WiP5u2Q1ua03mRhvAnVAX3b5uGVnA5Mni95v/iD/X7krvy8Bj7cjM8vrTZ+uHwJBZjW3IJljwFMakZHacKpWxozxz0hnIahLF5FCf+EF/XI1y/Lww66vkTM8VgHPHXeIaq3+/fXLjWPjTJ0K3HqruG+sijGua1S/vraOp5i1cmWtOky9wBw+DBw4YF2l5WlSz23b3D8P6C+8RsnJru/Rm1GRrS6+7o7B+fP6bVtVX1mNVWTGLNAwjoUE6Pdr/FXva5sO+XtndkGT3598f9Ys4Jtv9N3/rdrwyBciq7nPFMX1PXnTcFkewdtuwGPVLV0dEsBO1aTV5LZ22/DIw1zk5FivZzUcw8GDnvdhh3y6lr+TeXnAxInagJO+NFp2d2wuXjQf5gAQ5xozxnG1yD0GPKV1ww2irsN4xahWTSy/4YbglKuMpKW5nmDHjhVVATfd5Lq+NwFPlSqimsQ4uaOc7UlOFvXsZj1gAPP2OEbz54vgatQoz+uq+ykoECejm28WwZZZkBEdbV0u1S+/WD+nHiN3F4ukJPPBEYuKRHXZb7+ZnwCtToruAp4LF6wDHrldlLHbuDueuhWr5ZSzPu4CHjsne/mCZjaAnXzxVt9rSQnwxReu66oXHGO7HquAR67yPX/eNcDxJmiUg3tPAU9Ojhjg0aws8mNfej1ZZXjsBjzyeSArS3QeGDLENfNqFfD4+6Ivb08OeF5/HViyRMu2lHbgQXk/R46IjhJPPikee5utsapKJHNBD3hmz56NOnXqIC4uDq1bt8b69est112zZg0cDofL3y5jPUNZu+EGkdOfM0f0a54zR3RlCfNgx0pkpDatg9lzKndzGnlKiqkXQqvAwlOGBxBVZE8+6VrdZUb9ZV5Q4L4NusMhyu6pu7CxizUghs1//HHgr3/1XJ7ERPMs0rlzYvbkv/5VpMKNrDIhcuBidPq0/mIjX9zkC49Zg2crni6ExcXiTw541J4z6jJfq7Tki7tZhkcOQtQApLDQ/NhZdUu3alAtByO5ua4Bl1nAIweHSUn6DNOlS+JPfk/y/9Ujj4hqXzWDY5Xh8WVcG28yPN5Uacnr//67+MH088+imkr12Wf6Kiv5uFlVdfnKbIgLwLXTgj97af3nP+JWHbbN2+CFGR57ghrwzJ8/H6NGjcKECROQlZWFjh07olevXjhglb/7U3Z2NnJycpx/V1iNWFaWIiPFLJc33ihuw7Qay5/8MYmjVYNj4xQV/tpPfr77RqExMSLocVelVVLiWl0VGSkaWA8YYB0sypKS9O9d3d+ZM8Dbb4v7ZrN0+1KlZWzIajWvl9W0H2a8uRBeuuSa4Rk9WvyLHT7se5WWvK7ZBVJ+f2bj5cjsBjzycm8DHrk8ycmuvfHOn9dv59tvRa+uS5e0LMmYMebzopllqLwlX2AvXNC2YTfDYzVJrvw+jY3irXq35eeL7vneDvppxirgMX5W/my0bAw4femWzoDHs6AGPK+99hoeeOABPPjgg2jcuDFmzpyJzMxMvK2esS1UrVoVaWlpzr9IN8FFQUEBzpw5o/uj0FAeAx6zGcVl6ntyF/AcPux68oyL06rs5CkwrBgDHnVk6DNn3O/blwyP2lVbpV6UFEV/cfBmaguVN7/ECwv1Ac/RoyJTUVgopoPwNeDxlIGQsxWeAh51v8Zf7sayqRc0XwIeOaMYEeF6nC9e1F/0jx0TVcqffKIvw4kTrgGIWs7SZngURWSSCgvtZ3isAh65p56R2cjqgKgOe/xxoF8/z/u1Ih8jedvG9+KvgQdzc1237e3nwSote4IW8BQWFuKnn35Cjx49dMt79OiBjVaT6vypVatWSE9PR9euXbHaOFSswZQpU5CSkuL8y/Tm5zOVCXcXWSuPPSZuBw4Ut1FR5oGTN1VadsgBj7uuu2r7G3dBh9l8YPL63gY8cm8SNcBbu9Z991Rf2vCoAZ46XMC5c+KEXFSkPzEbMzybN2vdqY18yfDIWbHYWP8EPJ4yPOp9Txke4zg8ZpkURXHtbWXcrlmjYTngKSx0DXjM2gIBruOd5udr+1f/Z9Tj5kvAY/wu7dkjPiNjwONp254CHrNeXGYDjQLApk369Q4eFJPxrljhvgwy+Xvlrg2ZVYbn4kVR/WX2vo0ZnhMnRFvEjz7Sltv5LBjw2BO0gOfEiRMoLi5GNcNVr1q1ajhicUVJT0/HO++8gwULFmDhwoVo2LAhunbtinXr1lnuZ/z48cjLy3P+HfR3k36ybdo00bagTRv7r733XtF4VK7PN2vH402jZTvUgKew0LVKS54cUg1c3DVaVhssy+MKyesb5xMzk5ioD3jUrvxz57pv+Gp1UnTXs0x9jTrlR3GxuCgYgxb5Qnz2rPiMRo0yvxh7k+ExBjyy8+cDl+Exa6BdmiotQDRIXb1afzHbvVs7NmqS2lOGp6DAtUrrwgXz8u3f77qeWi41ePVXo2XV6dOuyz1Va1k1flYDHrOqUqtqJ+O+pk4VQ0mMH+++DN5s23iMrRot/+1vog3dd9/pn//6a/GDRHXpEvDDD67797WnXFGRqM77+GPvX18a8+eL4xsK85p5yw+VCqXjMPwcVRTFZZmqYcOGaCjNL9CuXTscPHgQ06ZNQ6dOnUxfExsbi1hPI8tRmerSRfz5wuHQT18BiOyG8ULh7wyP3EvLWKVVu7bWYLdzZ3HrrgmXOqJs48biBAXog43ISKBOHfORnVXGKi1v369VYHDpktimu27RaWmiUamimF+Y5QuTnMW6eNE1mPK2Ssuqq/2FC4Frw2Pc56VLnqu0jFUVxgtvcbGYf072yy/axa1aNVF16CngycvTLjDVq4u2Kps2Ae3aub7O+Nvu4kWtvImJYlulabTsbcBTUOB+cE+rDI/KrDG8VYbHGMB6O8aPzJuA59Il6wzPjh3idtky4LrrtOcnTXLdl9kx9GbEdJX8PTtzRlTnAcC113rXGaM0/v53cdutm28/XoMhaBmeyy+/HJGRkS7ZnGPHjrlkfdy59tprsdts/HcKT8XFYiCMZcvEbXGxbrJRVSAbLZsFPKo779TuWwU9aoNKuXbVGBDIs86bSUoCMjK0x1YBj7dTCTRq5Dkrlpqq9T678UZtNGKVnOGRu92b/cL3pkrL2IZHVpqAx06GR31sJ8NjFvCY+e037eKWliZuzS748gVQDUxiY7XM4rvvivGBPLl40TrDU5o2PE2aAN27i/unTlmP5uxpO4D5+zfL8Fi14TF+J3zpO2LWhsdYfXvhgvaZq9WDxuBFPgdZfR/MlttpCycfO7kliLcT0fqDnfG3gi1oGZ6YmBi0bt0aK1aswC233OJcvmLFCvSz0eIsKysL6WZXPAo/q1aJ+jC5TqlqVWRkvo+t0H8HAtlo2Vildc01okqgenX9XGJRUeYNGdWASQ5YjFVgjRsDX35pXZ7EROC++0S7iRtusJ6fNi9PP0K18WLUq5doD1Wjhudjlpws9qtelIzd3k+dEhcFh0N0K1aZXfC8rdKy+oUeyAyP8QR+441itAkzZgMPqlV+ViIjxbHOzdXKov7GM7t4mP3iT03VV0O5+66ojBkewD+9tKKitODLLMPj6bM29vYy8pThkQNU4/AJ3kyoqX5nzbahfv+MQYgc8MTGivdQXKx/L/L/k53BOn0NeGSBnGsM0Ad/5WmE56BWaY0ePRr33Xcf2rRpg3bt2uGdd97BgQMHMPTPkZ3Gjx+PQ4cO4cM/Z8ObOXMmateujSuvvBKFhYX4+OOPsWDBAixYsCCYb4PKwqpVrnUCAHDsGNKPfQVAP6xzoBotX7zoOg5PtWpiygGjqCj3Jx454DGemOUMT1yc60VDbb8zdaq4/fpr833k5uoDHuPFKDMTUEd18BTwpKToM1HqRTIlRVx0Ll0SJ/DERH2Vlln1jvFEnZTkGtzYqdKyMw6Pp0bLZswamsvb8tRoWRYTAzRooG+/4S7gMbsApqbqs2ipqe57DwLmGR5/9NKKitIm+bWq0vJmO4D+/btrw2OV4ZEzghcueM7wTJgg2lJ99JH+R41x28YyyAFPXJwIaIqK9J+VcZBJM2afra9VWjJ/jEfk7X7LUxueoHZLv+uuuzBz5ky88MILaNmyJdatW4elS5ei1p8TEeXk5OjG5CksLMTYsWPRvHlzdOzYERs2bMCSJUtwqzq/AIWn4mK3s1Omw3U8+kBleI4dc/01bDWpqadu93LAYzxpyPXv8vZr1tR6qsn69DHfh3zy/PFHUf0h71+ugvMm4JFP0Or7S07WLqCnToljJAcq48cDU6ZoE6aaXQDNAlR3jZaNAY+7HiqFhcC8edrM554yPGasAi9fqrRiYsRo4jI14DG7MJpdFCtXBgYP1h67GxtK5W2Gx9tf7Op7j47W3s+pU/YzPFZzb3kb8MjfEflYeRPwfPONyJLKvbu8DXjU96meG4qK9Nko+ThYVfuYvTerDI9ZYGGV4cnPFwP9P/986QOSBQvEqPJymzB5v8zw2DBs2DAMGzbM9Ll58+bpHo8bNw7jzH7lU3jLynJ7Rs/AYZdlnqZ2sEs9qalz/SQmatNqmE1qCngOeOTeXcaTUnS0Vnu3ZYs2gNzEiUCLFq7b6tZNnDyNcaEc8DzyiHZ/4EBgxAh9Kt9TGx41k6NSM12xseKCd/68WGYMaPbtE3/x8WKcFLNxQpOTXQeL8xTwGLMqsqIi4K23xLHavx94803x9+OP+hO0u7YOlSuL7vrZ2fYCHrNeWrLoaP0EufHxWsDnbZVW5crAAw+IY7ZihXcXtYsXtc9G/azN2vB4avvy7rviIqjOdRcVpQU8p0/bb8NjFayq3013VVpz5lhXbXpbpWUso1kbHuPnf/689pmr2daiItcedfL6ZswCHqsMT0GB63nN6nt29KiYEBYQ//ee5gs0k58vvvtTpojHr7+unV/k98aAh8if3M3lAH2GJzISaNjQtzF+3FFPNGq1weWXA//6l7hvdVJ1F/A4HPqshtkFS+3JpvbqAqy7q0dE6GeAV6knauNID9HRruP1mGV4EhK0dhVW1YSxsSLoO3hQ/Fm1u1G7y5rNTm+274sXrS8UFy7oy2888X/6qaim+Ogj/Qwvxour2WjUqpgYLdC1CrysBh70lOGRj2WlSvoxjoysqrTi40UvIG/HmJEzPO4aLXsaz+Uf/xC3770nbqOjtYEv5SqtmBixfTtVWjJ3GZ7iYhEIqWUwY8zwFBeLTN/ixcD77+szp1bTU5w8KY6N8XtobMMDiPXkiV3lbdqp0rLK8BQWeh/wyFPXnDvnW8Dz3HPaVBfGfcn3A91eyJ+CPpcWkUdWdUZ/qgat8cJ774m5pLz9ZectY/1+lSriZOrul7C7gCc5Wf9ad7/Q5Yuju/F5zLJa330n5iYyNmo1245pwBOtndlSEot11Siq2Fitp9q+febzhHliNgaQu6kqzMbhUavMADG9gkr+Lhgv5u72ERWlfe6eMjzGbJNaNrPu2MYMT2KilnExXhiLiswDSHV9O23V5DY8xgyP2QjAZswCPznDI1dpGdsJWfHU4Nwq2HzrLfevu3BB/9nn54tpVw4dEoGw/J6tMjwFBWL/xkygWcAD6BPR3gQ8dqq0zIIbq+BUnjBW/u4eOSICoBMnPB93OdgB9OcMf89fVlYY8FDoa9XK7U+UKBTjrwmL0KG9giZN3I807CvjhUv9ReuOu2DI+Hp3AY9c1eQuiDILGtavF9VIxl/CZtuRA55RN+3Gx5WHIyFPy56ljH4AjzRahXfe0b8uLg74s9kd9u0TbSLsko+vepFSE3tmVW1y9Yz6+K9/FYO+qal4lRww2OnNFR3tfcBjVaVlVvbYWOuAx5jhUas3HA59Faj6Wdlpq2aW4VEfy8fFOGWEzGxsqKgo7ft88aL2HtR92GnDY8YqY7h4sbi97DLgwQddnzdmAeWgpaTEuu2QMUA7ftw8w6Nmc+QfGnIm1V2VlloNZifgMTuO3mR41O/url1iyo3bbxc9D0eNMn+tFQY8RGUhMlKkKdwYPSkFr89yBGzOVmPAY2x0asZdcGIn4JEvanYzPFbMyiZfnK9b8hQand6MStDO1Mkn9yDyqXFocWqV7pezMcOjTgEhVyvWq6cNVGZGPr7q1BpqwCNnMeRMiHxB+b//E79qt24VwZ18MZPbgMjZgof1HftcmAU8xu+X1cCD6nKzz8tdhkedtkOlBjwpKfrPV/1OeDOiuBooueulZdYO5rvvgJ499dODmAU80dFie8ZjI+/j3DlxvD/91PX1xoyQTFE8Dx4YH2/+3T9/3vrCHBlpPZGnMYg4dsy1u/zixcDs2eK+POK53FNO3o4xkFU/N7PhAORqMZmxXL/9BvzZgdmF/Hmqg1U+95z4nqr/V99/b/5aK/J3Wd4+Ax4if7vhBtHtwJjpqVZNLJcbagSAvzM8xufcBStywOMuiLIzoLinKq0YiLNrAsSZPgqXnPcjZ0xH5VTtqiwHPH/8of2SlsckiosDrr/eum2VfAJVAx41UJEDHrV209hLS/71bujr4GxoDugb2no6XnLAozJWIZlleOSLo1m20awNj3oBlLNDRUXaBS01VV8WOxkes4DH2EvLbHb3xx4TGQg5E2CV4XE4XDOM6j7y84H//EcEo8axmwDtGMpBoLo8P9+1Uezkyfp9JSSY///IU2kAIsOhMgY8ZhdwNag3C3jkbckBj5ydcZfhMQvuVN4GPH/9q/U2ZKdOAQMGmGde9+wRnRfUEd/dkc9ZvvR0DAUMeKj8uOEG4KuvRNeMF18Ut4sXBzzYAVxPqKXN8KhVLs88IwIDdwksb6u0jBdXuVeWN2VzF/CkIA/OzR89iirxWoShjvorn8Tj4rSxWdR1AOs2J3LPFPX9ygHP7beL++rF12r+KFXVqmL+KsC8/UR0tOeMmFnAY3ZRBvQXZXl/Zm3JoqP1xyaxUgkSdmxxPj6XJzY2bx6wdKlYlpqqD1I9teGRA3I54DF2S1cvot5OQilXFarU75LxeMoZHvkCaczYqM/Jx0Qth5qRky+2kZHayNSAdcAze7Z+AMynntKXWX6fcsCsHgs18JYDHrMgOSpKK5+cQXSX4XEX8FjxNbDYscN6mppHHhFd8tUpKVRmGWeraj8GPESBEhkpJm658UZxG6g6LANfMjxmQYU6qLja5qB/f2DhQn02xEg+OXqaVFR2//0wbWQMmI9oaxbwqFVaydC3HK0Sp20gNlYEW9Wra89fdpm+rOoFyRgwqOSAR/3FrP7STU4GnnxStEVq2VIsKy52P6R95crWwwUA3md4jBdSY/nNqrTUgMfhML9wGNvwxKxYgohhQ1EJ4g2du3coji78DnPnautcd52+vGrAEhdn/i8gv3c1ODer0ioqEtUq3gziWFioZQnk6VSsAh61jMaJZuX2JcXFWrWOMeApLNSCIzmwu3RJP6ifVcCjbt9MZKT1lBZqWdWpX44f1/5fzPpPREZqx8Aq4DEG3b5MbmycN8tb6nAPVatqQZzKrMv/iRNaTzyZfM5gGx6iMOZLGx75V6jqqadEG5O77vJ+33L63k7AExEB1K1r/pxZI1xPGR5Z9Qyt8YF6bOT3e9ll+mPmKcMjl0d9j3LA43CIi4R8LNz1/klNdd8V158Znm3bXAe8A0TgZpYtUdu8qIovimOd+GfAc/5UPra9vAQFBWKIhR9/FIGrXHWiflYOh/n7kC/M6ndVHQ0Y0F9wCwpcu9UDru99zx4RQCQn6wfGVD8v4/pylZY8soQc8MhjL5kFk2pQK383a9XSH7/4ePMG++4U/3EYRQVaNCQHJOrFXA145AyPWcCTl6cFPHKmyNcqLStykLF/v/evO/znMGXJyfqG71bGjzfv7m+WBQMY8BCFHV8yPGPGAO3b65dFRooshadBCWXyydzO6wD9L2GZWTVPaipwx20luCf+S8RBnK1NMzzVqqF2a+3MbxXwyBdoswxPw4basqefFveHD9dnDwDXmeS9aZydkuI5w+NpO2ZZIONFeft21x5CavAWE2PeA6moCIhQtIttyZ+nYfVYn0MijkFEa3XrlDirKuXqMU/zNMkBufo5yFkB+YJrzJSpZZaPz9dfA/feK+43bGjerkxeXw7qPvwQWLRIe04NeJYuFSP4qowZHnkutcRE0ZX8pZeApk315XeX4bFSsPRbXHpYG/DWLMOjZkPkDI/Zd+rIEfMsm7tJUX0ZNkMOoNSOAd6Qe815E/BYzcsXDgEPBx4k8oLxwmdMDZu5/HJg1ixRbWU22J63atYEmjcXJ3ZPGR5jNYox4LnnHnGhGTDA/PVPjo8ArkkG/hzQPAniilMZUgpjzBjUjtPO2L4GPI88Ik6WLVqIC8m6daK8RUXiovbAA2I9eTuAOHF7Osl6k+Exfqb9+onqla++0vZr3Lcx4JHns1KpF7eYGPNy5ufjz6tKGwBA0Z+nYTXDIwc8VZWjwJ8T48oXGU8ZArnc6nGQA56EBO27Ygx41GxPXJwWvL3xhvZ8nTquwY26vio+3rrK8PffRRXmjBn65cYfEXLAk5QkJtRt3Fg8NmZ47AY8F5CA06e0fxSzgEedk/r0ae3/yCzDc/So+Xs1y/DExor3UL8+sHq1vTLL29u3z95rAe8DHivhEPAww0PkBfmEmp5uL9NS2rlsIiLEyLBvvOF5jCHj88aA5/HHxci8bgM2qUdcbyxFH3yFuzBf1yNOzsKoFzw54KlSRX8RUI+fXKWVni6mxFB/NatljYoSQdCcOUCnTlqDZZU31QEpKdrYQFbk8tWsCTz7rL4dkjdVWp62b1allZ8PXR1PA4gWpc4qLVTCUYjubFWjtUBTvuCZZRTkCWcjIsTEskOHaiNwyxep6GgtKLLK8Mhll9t6tG/vOcPjLgj56SfxucvVgJGRrtWdchseY280Yxseu1VEC3Eb/gZtQKnz58U/qTw+jxwouqvSOn/e/HwgH2/1vb71lqguMv5f1q/vmtl0tz15XitvJSa6ZtHk5wD37eLCIeBhhofIC/IJTa3b95bZWBt2eTuYonE9Y+rf61T6DTcAnTsjLSsLk06cAC4fIwaA/PNKKwc36sXQmOGRGyKrFz858+Bp+o82bcSfUZUqgDqnsFXD4NRU8zZUKuO8ROrna2wgbrxo2xnZWJ1awWzfuPxyfIR7sRHtcQfE/BZmVVrVasToX2fimWfE6OIvvKAFhw6HCCoA14apKSni+ehosU1jey71gm9WVfbhhyJDIQ+wZzfgMRMd7dqQVx5l2hjwGL/XxmChWTNR3eit86cKAcTqPi81EC8o0AYINAt4nnkG+Oc/XZcXFooAYuZM7Xip71EOpCMjxfhE338vqnRVERH6c4f8+asBVJMmonpLDUYiI7WG2hkZWvsdwHWspLg412DFXeaIjZaJKiBPmQMjfwQ83jIGNMbxSmxx0yPOOGQ/oA8wjOPGqPflk6MvPVUAfdWHVePxlBRRRquMzMWL+vKpF235GJm14bEb8JiNk5OfD6BVKzSuegoP4J+IhkilqBme6RiLX9AMAFD12jrO11n1nurfX7SRkS/68udj/NzV7J4afBpH3L10SRsDR1a7NpwjmcufnVmj5YQE++NCGY9VcbEWjBm/K8aAJypK39PR3RxbZi5cFL8U5KAiNVX7yqtBhLENz/Tp4vibZdwKCoAJE7RpXTp00D4jORhMTBTHNCND/3pj1so4mCAAjBypVcEC+mNo/GFmPIbXX++6bXcBj1WG5/ff9d3/QxkDHiKb6tWzt76nVLU/GQMe+aRpbI9SWmPHiouM2phV/vUbGWnehkftVl4acpAjt0mQT+hq6l5uxyPPJ5afbx7wGDM8Vj2PvBETI37d160rZmqX9202erga8MiqpmlXUjvTYsjfA+OYSMaAx+jSJfPsjjzLvdnYUPJFPC7OPOCxakgeFWUeHKqZFXcZHjWob9FCWxYZaa9h8LmCKCiKduFXu5obA2Zjhkd9P2ZVWnl5YowbQIwJ9Prr5sdKDXTUNkMqY6AqBxlqwJOSot+3fFwyMvSfQaVKQJ8+Irjq0AF44gng2mvFc5cuiaDObJwlVVGR+UCVADBkiL2u8sHCgIfIS0OGAG3bAn372nvdxIlA796i2qGsyRkef88xdvfdYgwhtd1LRISoRsnIEMdJvqCqJ96WLcXJ/+uvfd+vHPDI9xs00O6bBTxyu6XiYvMqLfl4ybOlq7zpnSe/vnlz4PPPtQsLIF1EDaOHy9N4qOT3Z2dWajkL6XDoj42ngOfoUTGup1GjRtp9T214zDI8XbpYNyQ3q9ICtKobT214ADF1RXw80Lmztk1vFZdEYOBAbX9qJk/O6DkcrhlF9fsiBx3y+y4pEZmWtm31r5PXUQMeY4BtrBJUq/cURQt4kpOtA57kZH2AlpgosrCrVgGvvSaenzZNe76gwHW6iZ49gZUrtcdqlsfsuyiPaB6q2IaHyEvDhnlex0zVqqJ9RVkw/qq12429tKZOFSf5iAjrk7fx5G+XVZVWvXpi+gJAC3juvVfMCXX11a7bkcuktgNyl+GJj/fcO69SJX2PHNkbb4gB3Z55Rlr4Z1spZGUh8atKwBL9a+x2X54zR1y01Ck0VFdcAfzwg7ivvgergODtt/WPhw4VwdFtt2nLvGm0LL//WbNE4+knnzTfp1mVFqC1P/Imw5OeDnzzjbZftY0SAMSgACnIw3FYd93buVOMSgxo3ys54DFrHK2+Z/n/LDVVP6dW69au+zLL8Ji9Vvavf4n2U9ddp1WxGTM8ciCoBjzqWEdqQKkbb0sKevfvF2MtRUSItlpr14rxwlJTtZGpL1wQ2zXLNh4/rg01EaqY4SEKI4GYKd4u9SItX1DttOfwxKpKS24ErWZRrr4a+Owz8YvWSC6TegFx14anShVx0fjsMzEruxm56siYQWnXTkwXIQ/aB8DZVirx6sa6xUOH6ld75RVxsTHLvqjatAEefdQ10JWro9S2Hd58JjVrinGGBg7UX+ztBjx16rivZlIU84BHbXRr7F1kbMMj31fb08jfv7Uvb8TSvwzGOLxiXoA/qVMwqN8ruUorIUF8pvKxNcvwGMt61VWu+5GPldwzUH6tWWP8b77ROgPExrqOtG38jOQqRLMMmvzDRO0m37SpyOb97W9aedRjrLbrMsvwjBoFTJrkujyUMOAhCiPqVBI9ewa1GADcp+dLw2yeKEDrfm3cX/365iPxyhcps4DHmOFRT/7161u345IzAt6Mxi2TL1Zz57oOaNi6NfDtt6INuV1ykOUpwyOzGsFYDk7U7uvGbJh8UVQ/J6t5ugoLzQMwtQrFOGK4VcAjk99fdI/r4fj6KzQY289lvbZtRdUjoLVhMcvwqMdC3p/6PZODDmPA4xLgwrxKy/haubPDiBHi9tdfteosdV35R458XGJj9QGPVdd9Y8BjHCwV0N77hQtiRIVt28y39fXX7ru2BxsDHqIwMmiQyCKEwi8tszY8/iAHEikpYibuBQtEun/yZNFA2CrTJV+Y5HXUC7G7gEfer1VVoRwIuOsWb0beptXcar5m8OrXF1mepk214MOYgTJrn2Sc6FMlHxc1KDG24ZEv5Oq+rALxwkLr95ac7NrY2W7AAwCIjERsi0am66ltXdwFPOo+1f05HNpxkD87YybFbPgFqwyPOtdevXrasAJ16oj2clFRohH38uViuVkPRGPAI/8gsAp41LKooze3a+e6jhrwXLwoqrncjfQsD1kQatiGhyiMREaKi1ooKIsMT3y8/oLRq5f718bFmfdAspPhAawDHvmiYjfgkV9rZ4BDb0RFiTYggBZYyAHP/feLX+/z5+tfd+yY+fbk4ESt5jD20qpZUwSfcrDSo4d4b48/rm8HYtXlHhAXf3fjS1llocy6ilvNdq4GPGpWSg145M9B7W2pfkfi4rRyWbWjAcyHMpDXl3tnde0q2mHVry+2U7++yFzGxoqG5zt2aJ+j2XdE3m5MjOcqLcD1O964ses6csBjNg+f7OhR86xWKGDAQ0QBEagMj3wBsaoisdK7N/Dvf+t7LQFawBMbqw3eZhx40JuAR76oeBpY0ahVK9FuR25v40/G9jPGYQOs5v3yxGxqCfWiL/dOA0SAcO21IqCQG+e663JvdvGUgwqrgMesys7sexgd7ZpBUjMj8ndNHQRT3bfVHHfGgMcsc1W9usjaGKdgcTj0g23K1Zft22uNqgHrgCczU4zE3LKlfn1PGR5A7NusnZX6nowZv0mTRGAvtzc7elS0PwqF9oRGDHiIKgi74weVltk4PP4gn5CtZoO38thj4hdshw765WrA43CIC9m5c64ZHmMPLjOlyfA4HK7tdgJJfg/x8eYj5vZzbfLiNG6cmP9MXcfYhscd40VVPf5mzL63CQmi+rKkxDpz4W3AI2d4VGqGR/4Oq72t5AyPSs4myQGP1VQODofLMEwePfCAKP9bb4nHZo28IyNFlq6wUBwX+dh4k+ExDn6oUgPCPXv0y2NiXAP77dvFmEO1aomhOEIp8GHAQxTm5s0DvvhCP2x9WQhUwAOInlI5Oa6ZGk/i4oCbb3ZdLl9wK1XSAh75PXgza72cqXA3eWkoMAYoxl/v48eLjJiVO+8UfypjLy137HS3t8p4eaq+9EeGRw5g1apTdZnZOE7G19htuO5OdDRw331awGNW3agO+Kl+b9UfBMnJ1uMuye/Dqrzqd9k4MGFMjOv3fOtWUW185kxoBTsAAx6isNe0aXDa9cgXAX8HPPXr+7edgFx1o/5Cj4627vJrFfDIPVT8PbK1vxkzPMby3nSTvc/NTsBjZeRIMWaPzNexXfyR4bn+ejGW09VXaxdvswyP/P2R25hZZXh8JX/vzLKbxnZLCQmiZ19UlHXwIR8Tq9nU1SyOMeCJjRV/H38MLF0KfPKJNu6P3LYuVLCXFhEFhDyOiD/b8ASCnOFRL1LGKgC5zYRVlZa7qplQY8zAye/JbJRpT8za8FixyvAMHAhs2GC9XTvUHk9ysG/2uUVF6TM8kZFawBMVJcaXkatAzdrwqL3qatYUDbNVZpONlta//y0amd9/v+tzZlnFlBT3s8l7k+FRAx7jRLTq8WzUCOjUSf+cVfVYMDHDQ0QBkZYmGlDGxPg/w+NvcqAycqSYA0ltszFgALBrlxjhVmWV4fnb34AffxRVD6HOWF0nvyd1RnU7jL203HG3bfm1dqaHMLrxRvH9kzMhZvtV58x6+GEx2nL37u5HKDcLeB56SPSmat1alPnBB8VQCer4Of5Up44YXFI2dSqwebPrCNve8CbDY9UeTZ5B3bhOKGZ4GPAQUUBERopfow5H6NXlG8kBT7Nm4k/1+OOu62dmit5GVauKhrPq3GB16oi5h0L9/QKuAY8cXNiZFV7ljyotoy5dfH+tw+FdVa76vh9+2LvtmlVpJSToe6QNHSqC37L6HnTrpo3bY5f8uXtqw2Mkd6nPyBCBo9pt3dM0LMHAKi0iCpioKPPxUEKN3aqoyEgxxsxzz7le3MtDsAO4tuG58krtsS+j5Vo14jXTpIn7x++8Iybpffpp++Wwy+58c02aiM/YWGaj8vI9UAeOBKzHf6pcWR8gt20LzJihb1AeEaGfRoMZHiKiEPLgg8B774ku1r4K9fZJVtxNjCpPa+AtOeBx12YEEF2yk5LE3Kl79ogB92RXXWU+B1Ug2K02a9dOTMNg1c27vJEH4rT6ceJwiCzPH3+Ix61aAR07uq535ZXaFBUMeIiIQsjf/ia6Vpem+7Dd8XZChVkvuvnzxfg2Q4b4tr1Jk8RoxZ4a66amajOnm81kH0ixsfp5vuxmeIDwCXYAfTscd664Qgt4rHqftW8vMp+Ab9WigcaAh4gqLIej9GOl3HabmGXbbA6i8kKtlqtXT4zb5Ks+ffxSnID6z3/EuFTvvScel6ZhdDgwm2rFTL9+WvbGqo1WgwbA22/7vzu+v7ANDxFRKURHi7Y83bsHuyT2yGPHhHovOn+6/HJ9VsmXDE84UXt83X23+/XkgN5dg+Srrw7c9CilVcE/aiKiiklup1MeGpb7k9x+qaIHPB06ACtWeM7KREaKzFh2tpinqzyq4B81EVHFVJ4GSfQ3ucdRRQ94AP3o0O7Urq3NGl8esUqLiKgCYsAjVPQ2PBUJAx4iogrIn3ORlTfM8FRM/KiJiCqgzp2BCROAxo2DXZKyx4CnYuJHTURUATkc2gSbFY0c8JSXEZGp9FilRUREFYoc8PgyqjSVTwx4iIioQpEDHkUJXjmobDHgISKiCiVCuvIxw1NxMOAhIqIKiwFPxcGAh4iIKqwIXgUrDH7URERU4dxzj+iS36VLsEtCZYXd0omIqMIZMybYJaCyxgwPERERhT0GPERERBT2GPAQERFR2GPAQ0RERGGPAQ8RERGFPQY8REREFPYY8BAREVHYY8BDREREYY8BDxEREYU9BjxEREQU9hjwEBERUdhjwENERERhjwEPERERhT0GPERERBT2ooJdgLKmKAoA4MyZM0EuCREREXlLvW6r13G7KlzAc/bsWQBAZmZmkEtCREREdp09exYpKSm2X+dQfA2VyqmSkhIcPnwYSUlJcDgcft32mTNnkJmZiYMHDyI5Odmv2w43PFbe47Gyh8fLezxW3uOxsicQx0tRFJw9exYZGRmIiLDfIqfCZXgiIiJQo0aNgO4jOTmZ/xBe4rHyHo+VPTxe3uOx8h6PlT3+Pl6+ZHZUbLRMREREYY8BDxEREYU9Bjx+FBsbi4kTJyI2NjbYRQl5PFbe47Gyh8fLezxW3uOxsicUj1eFa7RMREREFQ8zPERERBT2GPAQERFR2GPAQ0RERGGPAQ8RERGFPQY8fjJ79mzUqVMHcXFxaN26NdavXx/sIvnVpEmT4HA4dH9paWnO5xVFwaRJk5CRkYH4+Hh06dIFv/76q24bBQUFGDFiBC6//HJUqlQJN998M/744w/dOqdPn8Z9992HlJQUpKSk4L777kNubq5unQMHDqBv376oVKkSLr/8cowcORKFhYUBe++erFu3Dn379kVGRgYcDge+/PJL3fOhdmy2b9+Ozp07Iz4+HtWrV8cLL7zg89w0vvB0vAYPHuzyXbv22mt161SE4zVlyhRcffXVSEpKQtWqVdG/f39kZ2fr1uF3S+PN8eJ3S3j77bfRvHlz56CA7dq1w3//+1/n82H7vVKo1D777DMlOjpaeffdd5UdO3Yojz32mFKpUiVl//79wS6a30ycOFG58sorlZycHOffsWPHnM9PnTpVSUpKUhYsWKBs375dueuuu5T09HTlzJkzznWGDh2qVK9eXVmxYoWydetW5frrr1datGihFBUVOde58cYblaZNmyobN25UNm7cqDRt2lTp06eP8/mioiKladOmyvXXX69s3bpVWbFihZKRkaEMHz68bA6EiaVLlyoTJkxQFixYoABQFi1apHs+lI5NXl6eUq1aNeXuu+9Wtm/frixYsEBJSkpSpk2bFrgDZODpeA0aNEi58cYbdd+1kydP6tapCMerZ8+eyty5c5VffvlF2bZtm3LTTTcpNWvWVM6dO+dch98tjTfHi98tYfHixcqSJUuU7OxsJTs7W3n66aeV6Oho5ZdfflEUJXy/Vwx4/KBt27bK0KFDdcsaNWqkPPXUU0Eqkf9NnDhRadGihelzJSUlSlpamjJ16lTnsvz8fCUlJUWZM2eOoiiKkpubq0RHRyufffaZc51Dhw4pERERyrJlyxRFUZQdO3YoAJTNmzc719m0aZMCQNm1a5eiKOJiGRERoRw6dMi5zqeffqrExsYqeXl5fnu/vjJewEPt2MyePVtJSUlR8vPznetMmTJFycjIUEpKSvx4JLxjFfD069fP8jUV9XgdO3ZMAaCsXbtWURR+tzwxHi9F4XfLncqVKyvvvfdeWH+vWKVVSoWFhfjpp5/Qo0cP3fIePXpg48aNQSpVYOzevRsZGRmoU6cO7r77buzZswcAsHfvXhw5ckR3DGJjY9G5c2fnMfjpp59w6dIl3ToZGRlo2rSpc51NmzYhJSUF11xzjXOda6+9FikpKbp1mjZtioyMDOc6PXv2REFBAX766afAvXkfhdqx2bRpEzp37qwbDKxnz544fPgw9u3b5/8D4KM1a9agatWqaNCgAR566CEcO3bM+VxFPV55eXkAgCpVqgDgd8sT4/FS8bulV1xcjM8++wznz59Hu3btwvp7xYCnlE6cOIHi4mJUq1ZNt7xatWo4cuRIkErlf9dccw0+/PBDfPPNN3j33Xdx5MgRtG/fHidPnnS+T3fH4MiRI4iJiUHlypXdrlO1alWXfVetWlW3jnE/lStXRkxMTEge71A7NmbrqI9D5fj16tUL//rXv7Bq1SpMnz4dP/74I2644QYUFBQAqJjHS1EUjB49Gtdddx2aNm2q2z+/W67MjhfA75Zs+/btSExMRGxsLIYOHYpFixahSZMmYf29qnCzpQeKw+HQPVYUxWVZedarVy/n/WbNmqFdu3aoV68ePvjgA2ejP1+OgXEds/V9WSfUhNKxMSuL1WuD4a677nLeb9q0Kdq0aYNatWphyZIluPXWWy1fF87Ha/jw4fj555+xYcMGl+f43XJldbz43dI0bNgQ27ZtQ25uLhYsWIBBgwZh7dq1bstW3r9XzPCU0uWXX47IyEiXSPPYsWMuUWk4qVSpEpo1a4bdu3c7e2u5OwZpaWkoLCzE6dOn3a5z9OhRl30dP35ct45xP6dPn8alS5dC8niH2rExW0dN6Yfi8QOA9PR01KpVC7t37wZQ8Y7XiBEjsHjxYqxevRo1atRwLud3y5zV8TJTkb9bMTExqF+/Ptq0aYMpU6agRYsWeP3118P6e8WAp5RiYmLQunVrrFixQrd8xYoVaN++fZBKFXgFBQXYuXMn0tPTUadOHaSlpemOQWFhIdauXes8Bq1bt0Z0dLRunZycHPzyyy/Oddq1a4e8vDz88MMPznW+//575OXl6db55ZdfkJOT41xn+fLliI2NRevWrQP6nn0RasemXbt2WLduna7b5/Lly5GRkYHatWv7/wD4wcmTJ3Hw4EGkp6cDqDjHS1EUDB8+HAsXLsSqVatQp04d3fP8bul5Ol5mKup3y4yiKCgoKAjv75WtJs5kSu2W/v777ys7duxQRo0apVSqVEnZt29fsIvmN2PGjFHWrFmj7NmzR9m8ebPSp08fJSkpyfkep06dqqSkpCgLFy5Utm/frtxzzz2m3Rhr1KihrFy5Utm6datyww03mHZjbN68ubJp0yZl06ZNSrNmzUy7MXbt2lXZunWrsnLlSqVGjRpB7ZZ+9uxZJSsrS8nKylIAKK+99pqSlZXlHJYglI5Nbm6uUq1aNeWee+5Rtm/frixcuFBJTk4u027p7o7X2bNnlTFjxigbN25U9u7dq6xevVpp166dUr169Qp3vB555BElJSVFWbNmja4b9YULF5zr8Lul8XS8+N3SjB8/Xlm3bp2yd+9e5eeff1aefvppJSIiQlm+fLmiKOH7vWLA4ydvvfWWUqtWLSUmJka56qqrdF0hw4E6DkN0dLSSkZGh3Hrrrcqvv/7qfL6kpESZOHGikpaWpsTGxiqdOnVStm/frtvGxYsXleHDhytVqlRR4uPjlT59+igHDhzQrXPy5EllwIABSlJSkpKUlKQMGDBAOX36tG6d/fv3KzfddJMSHx+vVKlSRRk+fLiuy2JZW716tQLA5W/QoEGKooTesfn555+Vjh07KrGxsUpaWpoyadKkMu0G6+54XbhwQenRo4fyl7/8RYmOjlZq1qypDBo0yOVYVITjZXaMAChz5851rsPvlsbT8eJ3SzNkyBDn9eovf/mL0rVrV2ewoyjh+71yKEoZDrFKREREFARsw0NERERhjwEPERERhT0GPERERBT2GPAQERFR2GPAQ0RERGGPAQ8RERGFPQY8REREFPYY8BAREVHYY8BDRD7r0qULRo0a5fX6+/btg8PhwLZt2wJWpvJg0qRJaNmyZbCLQVShMOAhqgAcDofbv8GDB/u03YULF2Ly5Mler5+ZmYmcnBw0bdrUp/3ZsWDBAlxzzTVISUlBUlISrrzySowZMybg+yWi0BQV7AIQUeDJsxHPnz8fzz33HLKzs53L4uPjdetfunQJ0dHRHrdbpUoVW+WIjIxEWlqardf4YuXKlbj77rvx8ssv4+abb4bD4cCOHTvw7bffBnzfRBSamOEhqgDS0tKcfykpKXA4HM7H+fn5SE1Nxeeff44uXbogLi4OH3/8MU6ePIl77rkHNWrUQEJCApo1a4ZPP/1Ut11jlVbt2rXx8ssvY8iQIUhKSkLNmjXxzjvvOJ83VmmtWbMGDocD3377Ldq0aYOEhAS0b99eF4wBwIsvvoiqVasiKSkJDz74IJ566im3VUJff/01rrvuOjzxxBNo2LAhGjRogP79++ONN95wrvP777+jX79+qFatGhITE3H11Vdj5cqVuu3Url0bL774IgYOHIjExETUqlUL//nPf3D8+HH069cPiYmJaNasGbZs2eJ8zbx585Camoovv/wSDRo0QFxcHLp3746DBw+6/Yzmzp2Lxo0bIy4uDo0aNcLs2bOdzxUWFmL48OFIT09HXFwcateujSlTprjdHhHpMeAhIgDAk08+iZEjR2Lnzp3o2bMn8vPz0bp1a3z99df45Zdf8PDDD+O+++7D999/73Y706dPR5s2bZCVlYVhw4bhkUcewa5du9y+ZsKECZg+fTq2bNmCqKgoDBkyxPncv/71L7z00kt45ZVX8NNPP6FmzZp4++233W4vLS0Nv/76K3755RfLdc6dO4fevXtj5cqVyMrKQs+ePdG3b18cOHBAt96MGTPQoUMHZGVl4aabbsJ9992HgQMH4t5778XWrVtRv359DBw4EPI8zBcuXMBLL72EDz74AN999x3OnDmDu+++27Is7777LiZMmICXXnoJO3fuxMsvv4xnn30WH3zwAQBg1qxZWLx4MT7//HNkZ2fj448/Ru3atd0eAyIysD2/OhGVa3PnzlVSUlKcj/fu3asAUGbOnOnxtb1791bGjBnjfNy5c2flsccecz6uVauWcu+99zofl5SUKFWrVlXefvtt3b6ysrIURVGU1atXKwCUlStXOl+zZMkSBYBy8eJFRVEU5ZprrlEeffRRXTk6dOigtGjRwrKc586dU3r37q0AUGrVqqXcddddyvvvv6/k5+e7fX9NmjRR3njjDcv3k5OTowBQnn32WeeyTZs2KQCUnJwcRVHE8QWgbN682bnOzp07FQDK999/ryiKokycOFFX/szMTOWTTz7RlWXy5MlKu3btFEVRlBEjRig33HCDUlJS4rb8RGSNGR4iAgC0adNG97i4uBgvvfQSmjdvjssuuwyJiYlYvny5SwbEqHnz5s77atXZsWPHvH5Neno6ADhfk52djbZt2+rWNz42qlSpEpYsWYL//e9/eOaZZ5CYmIgxY8agbdu2uHDhAgDg/PnzGDduHJo0aYLU1FQkJiZi165dLu9PLlu1atUAAM2aNXNZJr/HqKgo3fFs1KgRUlNTsXPnTpeyHj9+HAcPHsQDDzyAxMRE59+LL76I33//HQAwePBgbNu2DQ0bNsTIkSOxfPlyt++fiFyx0TIRARBBgmz69OmYMWMGZs6ciWbNmqFSpUoYNWoUCgsL3W7H2NjZ4XCgpKTE69c4HA4A0L1GXaZSpOojd+rVq4d69erhwQcfxIQJE9CgQQPMnz8f999/P5544gl88803mDZtGurXr4/4+HjcfvvtLu/PrGyeymtWZqtl6uveffddXHPNNbrnIiMjAQBXXXUV9u7di//+979YuXIl7rzzTnTr1g1ffPGFV8eBiBjwEJGF9evXo1+/frj33nsBiAvz7t270bhx4zItR8OGDfHDDz/gvvvucy6TGwl7q3bt2khISMD58+cBiPc3ePBg3HLLLQBEm559+/b5pcxFRUXYsmWLMxOVnZ2N3NxcNGrUyGXdatWqoXr16tizZw8GDBhguc3k5GTcdddduOuuu3D77bfjxhtvxKlTp2z3lCOqqBjwEJGp+vXrY8GCBdi4cSMqV66M1157DUeOHCnzgGfEiBF46KGH0KZNG7Rv3x7z58/Hzz//jLp161q+ZtKkSbhw4QJ69+6NWrVqITc3F7NmzcKlS5fQvXt3AOL9LVy4EH379oXD4cCzzz7rMRPlrejoaIwYMQKzZs1CdHQ0hg8fjmuvvdayKm7SpEkYOXIkkpOT0atXLxQUFGDLli04ffo0Ro8ejRkzZiA9PR0tW7ZEREQE/v3vfyMtLQ2pqal+KS9RRcA2PERk6tlnn8VVV12Fnj17okuXLkhLS0P//v3LvBwDBgzA+PHjMXbsWGfVzuDBgxEXF2f5ms6dO2PPnj0YOHAgGjVqhF69euHIkSNYvnw5GjZsCED0vqpcuTLat2+Pvn37omfPnrjqqqv8UuaEhAQ8+eST+Otf/4p27dohPj4en332meX6Dz74IN577z3MmzcPzZo1Q+fOnTFv3jzUqVMHAJCYmIhXXnkFbdq0wdVXX419+/Zh6dKliIjgKZzIWw7F28pwIqIQ0b17d6SlpeGjjz4KdlFczJs3D6NGjUJubm6wi0JEElZpEVFIu3DhAubMmYOePXsiMjISn376KVauXIkVK1YEu2hEVI4w4CGikOZwOLB06VK8+OKLKCgoQMOGDbFgwQJ069Yt2EUjonKEVVpEREQU9tjijYiIiMIeAx4iIiIKewx4iIiIKOwx4CEiIqKwx4CHiIiIwh4DHiIiIgp7DHiIiIgo7DHgISIiorD3/58REVSn2hRdAAAAAElFTkSuQmCC",
"text/plain": [
"<Figure size 640x480 with 1 Axes>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"fig = pl.figure()\n",
"pl.plot(train_counter, train_losses, color=(0.2, 0.2, 1.0))\n",
"pl.scatter(test_counter[:-1], test_losses, color=(1.0, 0.2, 0.2))\n",
"\n",
"pl.legend(['Train Loss', 'Test Loss'], loc='upper right', frameon=False)\n",
"pl.xlabel('Training Samples')\n",
"pl.ylabel('Log Likelihood Loss')\n",
"\n",
"pl.show()"
]
},
{
"cell_type": "markdown",
"id": "42011104-4d54-4198-834e-20c4c76389cd",
"metadata": {},
"source": [
"It's always important to inspect anecdotes to convince yourself the model is behind as expected."
]
},
{
"cell_type": "code",
"execution_count": 22,
"id": "94401f54-1f96-4ab1-89a8-c181c7192e5e",
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"/tmp/ipykernel_2928/3418281087.py:10: UserWarning: Implicit dimension choice for log_softmax has been deprecated. Change the call to include dim=X as an argument.\n",
" return F.log_softmax(out)\n"
]
}
],
"source": [
"with torch.no_grad():\n",
" output = network(example_data.reshape(1000, 28*28))"
]
},
{
"cell_type": "markdown",
"id": "b3673745-19dd-464c-90b6-8a07c8adfd6e",
"metadata": {},
"source": [
"**Challenge:** Can you explain why we are using `torch.no_grad()`?"
]
},
{
"cell_type": "code",
"execution_count": 25,
"id": "0fb030fa-d62b-419f-ba72-5c1b6ac71b99",
"metadata": {},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAmYAAAGkCAYAAACb5OmoAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjguNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8fJSN1AAAACXBIWXMAAA9hAAAPYQGoP6dpAAAvlElEQVR4nO3de1yVVb7H8R/ITWCrIyKICiXemLwdUwc9BjjaeDQZ85Lly1HU8uiUFd5Kx5q8lWlqdRovpzMvNS95z6bkeJkmNFNUnI6NNJXWeEcTcVS8g6zzhwMzO9Yje8O+rA2f9+vlH3559vMs8Fny5YHF8lNKKQEAAIDX+Xt7AAAAALiLYgYAAGAIihkAAIAhKGYAAACGoJgBAAAYgmIGAABgCIoZAACAIShmAAAAhqCYAQAAGMJnitny5cvFz8+v9E9AQIA0atRIRowYIWfOnPHIGO677z4ZPnx46d937twpfn5+snPnTqfOs3fvXpk2bZpcunSpzNtSUlIkJSWlUuN0pUOHDskjjzwisbGxUrNmTalbt6507txZVq1a5e2hwQWYV95z4MAB6dmzp9hsNgkPD5du3brJnj17vD0suADzygy///3vxc/PT8LDw709FKcEeHsAzlq2bJm0bNlSbty4IZ999pnMnj1bdu3aJYcPH5awsDCPjqV9+/aSlZUlP/3pT5163d69e2X69OkyfPhwqVOnjt3bFi1a5MIRVt6lS5ekcePGMnjwYGnYsKFcu3ZNVq9eLUOHDpXjx4/LSy+95O0hwgWYV56VnZ0tSUlJ0qlTJ1m5cqUopWTu3LnSvXt3yczMlM6dO3t7iHAB5pX3nDlzRiZOnCgxMTFy+fJlbw/HKT5XzFq1aiUdOnQQEZFu3brJnTt3ZObMmfLhhx/KkCFDtK+5fv26hIaGunwstWrVksTERJee09lJ4266r4j69Okjx44dk3fffZdiVkUwrzzr5Zdfljp16si2bdtKP4Y9evSQJk2ayMSJE3lyVkUwr7xnzJgxkpSUJHXr1pWNGzd6ezhO8ZlvZVopudFOnDghIiLDhw+X8PBwOXz4sPziF78Qm80m3bt3FxGR27dvy6xZs6Rly5YSHBwskZGRMmLECMnLy7M7Z2FhobzwwgsSHR0toaGh0rVrVzlw4ECZa1s9Gt6/f7+kpqZKRESEhISESHx8vKSnp4uIyLRp02TSpEkiInL//feXPuouOYeuCF28eFGefvppadiwoQQFBUmTJk1k6tSpcuvWLbvj/Pz8ZOzYsbJy5UpJSEiQ0NBQadu2rWzZssXpj2t56tWrJwEBPtfr4SDm1T+5Y17t2bNHUlJS7D4B22w2SUpKkr1798rZs2crfG6Yi3n1T+78fLVq1SrZtWuX0U/07sXnP7N+9913IiISGRlZmt2+fVt++ctfyujRo2Xy5MlSVFQkxcXF0rdvX9m9e7e88MIL0qVLFzlx4oS88sorkpKSIgcPHpSaNWuKiMioUaNkxYoVMnHiRHn44YclJydH+vfvLwUFBeWOZ/v27ZKamioJCQmyYMECiY2NlePHj8uOHTtEROSpp56SixcvyjvvvCMffPCBNGjQQESsv/K4efOmdOvWTb7//nuZPn26tGnTRnbv3i2zZ8+WQ4cOSUZGht3xGRkZkp2dLTNmzJDw8HCZO3eu9OvXT7799ltp0qRJ6XF+fn6SnJzs8M8bFBcXS3Fxsfz973+XDRs2yPbt2+V3v/udQ6+F72FeuXde3b59W4KDg8vkJdnhw4dL3wdUHcwr93++On/+vKSnp8vrr78ujRo1Kvd4IykfsWzZMiUiat++faqwsFAVFBSoLVu2qMjISGWz2dS5c+eUUkqlpaUpEVFLly61e/2aNWuUiKhNmzbZ5dnZ2UpE1KJFi5RSSn399ddKRNS4cePsjlu9erUSEZWWllaaZWZmKhFRmZmZpVl8fLyKj49XN27csHxf3njjDSUi6tixY2XelpycrJKTk0v/vmTJEiUiav369XbHzZkzR4mI2rFjR2kmIioqKkpduXKlNDt37pzy9/dXs2fPtnt9jRo11M9//nPLMf7Y6NGjlYgoEVFBQUGlHy/4NuaVd+ZVu3btVPPmzdWdO3dKs8LCQtWkSRMlIur9998v9xwwF/PKe5+vBgwYoLp06aKKi4uVUnc/xmFhYQ691hQ+963MxMRECQwMFJvNJn369JHo6GjZunWrREVF2R03YMAAu79v2bJF6tSpI6mpqVJUVFT6p127dhIdHV3axDMzM0VEynz/f9CgQeV+6+7IkSPy/fffy5NPPikhISGVfE/v+vTTTyUsLEwGDhxol5estvnTn/5kl3fr1k1sNlvp36OioqR+/fqlj85LFBUVlXntvfzmN7+R7OxsycjIkJEjR8rYsWNl3rx5Tr43MBXz6i5Pzatnn31Wjhw5ImPHjpUzZ87IqVOnZMyYMaXn8/f3uf+aocG8ustT82rTpk3y8ccfy//8z/+In59fBd8L7/O5b2WuWLFCEhISJCAgQKKiorSP+0NDQ6VWrVp22Q8//CCXLl2SoKAg7XkvXLggIiL5+fkiIhIdHW339oCAAImIiLjn2Eq+9+/Kx6f5+fkSHR1d5iarX7++BAQElI63hG6MwcHBcuPGjUqNIzY2VmJjY0VEpHfv3iIiMmXKFElLS7N7LA/fxLy6y1PzauTIkZKXlyezZs2SxYsXi4hI586dZeLEiTJnzhxp2LBhhc4LszCv7vLEvLp69ao888wz8uyzz0pMTEzpr/e4ffu2iNz9DQOBgYEeXw1bET5XzBISEkpXuVjRNeV69epJRESEbNu2TfuaktZecqOcO3fO7j/HoqKiMjfVj5UUlNOnT9/zOGdERETI/v37RSll936dP39eioqKpF69ei67ljM6deokS5Yskb/97W8UsyqAeXWXJ+fViy++KOnp6XL06FGx2WwSFxcno0ePlrCwMHnwwQfdfn24H/PqLk/MqwsXLsgPP/wg8+fPl/nz55d5+09+8hPp27evfPjhh24bg6tUm+flffr0kfz8fLlz54506NChzJ8WLVqIiJSuMFm9erXd69evXy9FRUX3vEbz5s0lPj5eli5dWmYFyr8q+QFfR74q6N69u1y9erXMzbRixYrSt3tDZmam+Pv72/2AJqof5lXlBAcHS6tWrSQuLk5Onjwp69atk1GjRpX+YDeqJ+aV86KjoyUzM7PMn549e0pISIhkZmbKrFmz3HZ9V/K5J2YV9cQTT8jq1auld+/e8vzzz0unTp0kMDBQTp8+LZmZmdK3b1/p16+fJCQkyK9+9St56623JDAwUHr06CE5OTkyb968Mo+bdRYuXCipqamSmJgo48aNk9jYWDl58qRs3769dPK0bt1aRETefvttSUtLk8DAQGnRooXd99pLDBs2TBYuXChpaWly/Phxad26tXz++efy2muvSe/evaVHjx4V+ngEBARIcnJyud+3/8///E+pVauWdOrUSaKiouTChQuyYcMGWbdunUyaNImnZdUc88qeo/MqJydHNm3aJB06dJDg4GD58ssv5fXXX5dmzZrJzJkzK3RtVB3MK3uOzKuQkBDtLgTLly+XGjVqGL1DQRleXnzgsJJVLtnZ2fc87l4rMAoLC9W8efNU27ZtVUhIiAoPD1ctW7ZUo0ePVkePHi097tatW2rChAmqfv36KiQkRCUmJqqsrCwVFxdX7ioXpZTKyspSvXr1UrVr11bBwcEqPj6+zKqZKVOmqJiYGOXv7293jh+vclFKqfz8fDVmzBjVoEEDFRAQoOLi4tSUKVPUzZs37Y4TEfXMM8+Ueb9/PO6SY398HZ2lS5eqhx56SNWrV08FBASoOnXqqOTkZLVy5cpyXwvzMa+8M6++/fZblZSUpOrWrauCgoJU06ZN1UsvvaSuXr1a7mthPuaVd+aVji+uyvRTSikv9EEAAAD8SLX5GTMAAADTUcwAAAAMQTEDAAAwBMUMAADAEBQzAAAAQ1DMAAAADOHQL5gtLi6W3NxcsdlsPr0xKKoepZQUFBRITEyMz238zLyCqZhXgOs5Oq8cKma5ubnSuHFjlw0OcLVTp065dDNeT2BewXTMK8D1yptXDn0ppNt6ATCJL96jvjhmVC++eI/64phRvZR3jzpUzHgcDNP54j3qi2NG9eKL96gvjhnVS3n3qG/98AAAAEAVRjEDAAAwBMUMAADAEBQzAAAAQ1DMAAAADEExAwAAMATFDAAAwBAUMwAAAENQzAAAAAxBMQMAADAExQwAAMAQFDMAAABDUMwAAAAMQTEDAAAwBMUMAADAEBQzAAAAQ1DMAAAADEExAwAAMATFDAAAwBAUMwAAAENQzAAAAAxBMQMAADAExQwAAMAQFDMAAABDUMwAAAAMQTEDAAAwBMUMAADAEAHeHoC3NW/eXJu3b99em1+9elWbN2vWzOFrtmnTRpsPGzbM4XPci7+/vm8XFxc7dZ7HH39cm2/cuNHpMQGutHfv3jJZ586dtceOHz9em7/55psuHRMAuAJPzAAAAAxBMQMAADAExQwAAMAQFDMAAABDUMwAAAAMUW1WZVqtvszIyNDmDRs21OZ37tzR5jVr1tTmfn5+ZTKllPZYq9xZVqsvnT1/amqqNmdVJjxFt/pSxHoFps6CBQu0udV9fOrUKYfPDcAxDz74oDb/3//9X22+Y8cObT506FCXjclUPDEDAAAwBMUMAADAEBQzAAAAQ1DMAAAADEExAwAAMES1WZVptfdlYWGhNg8KCnLbWPLz8526ps1mc9tYRERycnK0+fLly916XaDE/PnztbnV6kvdysl///d/1x578uRJp645aNAgbQ6g4kaNGqXNIyIitHnLli3dORyj8cQMAADAEBQzAAAAQ1DMAAAADEExAwAAMATFDAAAwBDVZlXm2rVrtfnnn3+uzRMTE902FqtVmXPnztXm//Zv/+aS62ZlZWnzRx99VJtbjRNwtccee8yp42NjYx0+1uq+t7pm48aNtTl7aMLb4uLitPmKFSu0+VtvvaXNN2/e7KohOSwyMlKb6/aTvldeHfDEDAAAwBAUMwAAAENQzAAAAAxBMQMAADAExQwAAMAQ1WZVppXTp09r840bN7rk/MnJyWWyCRMmaI911erLXbt2afM33nhDm7P6Et5mtRJywYIFlT631Ry3MnDgQG3+5ptvVnosQGVYzQerfWJDQ0O1uTdWZVqt/ldKOZVXBzwxAwAAMATFDAAAwBAUMwAAAENQzAAAAAxBMQMAADBEtV+V6Sq61ZciIjt37iyTFRcXO3XugoICbf773/9em0+cONGp8wOeMm7cOKeOt9rrzxnO7sPZuXNnbc6qTHjK1KlTtXn//v21udXnlAsXLrhsTJXl769/DmQ1dvbKBAAAgNdRzAAAAAxBMQMAADAExQwAAMAQFDMAAABDsCrTSb169dLma9as0ea6FSfO7gFWs2ZNbR4UFOTUeQBvs1rxaOXUqVMOH+vsik8rjRo1csl5gPJYrb6cPHmyNrdawWj1OeXVV1+t2MDcwNmxb9q0yZ3DMRpPzAAAAAxBMQMAADAExQwAAMAQFDMAAABDUMwAAAAMwapMJw0fPlybh4eHu+2aNWrU0OZPP/20Nn/uuefcNhagMty54tHZPTGtOLtyFChPZGSkNh8yZIg2Dw0N1ebXr1/X5sOGDdPmn3/+uQOj8wxn977Mz89300jMxxMzAAAAQ1DMAAAADEExAwAAMATFDAAAwBAUMwAAAEOwKtNJy5Yt0+adOnXS5rt37y6TffTRR9pjJ0yYoM07duzo4OjuqlOnjja/dOmSU+cBXC0rK0ubW62ETExM1Ob79u1z+BxWNmzYoM1dtboTKDFlyhRt3qJFC21utX/kN998o803b95csYF5kNX75Oze0dUBT8wAAAAMQTEDAAAwBMUMAADAEBQzAAAAQ/DD/07atm2bNr///vsrfe7Tp09rc2e31XjppZe0+cSJE50eE+BKb731ljYfP368Nl+/fr02d+ZeXrBggVPXtDoeqKiHHnpIm/v765+NFBcXa/OhQ4e6bEzukpSUpM2d3ZKpOuOJGQAAgCEoZgAAAIagmAEAABiCYgYAAGAIihkAAIAhWJVpEN02MyIimzZt0uYDBw7U5h06dNDm4eHh2vzq1asOjA6ovFOnTmlzZ7dHWrduncPXbNy4scPH3sugQYO0udU8tDoe1c/XX3+tzdu3b6/NrbYpevPNN7W5SVsyde3aVZuzJZPjeGIGAABgCIoZAACAIShmAAAAhqCYAQAAGIJiBgAAYAg/5cCSiCtXrkjt2rU9MR5oWK1AGzBggDa32pMsNjZWm585c6ZiAzPI5cuXpVatWt4ehlOYV+UbN26cNvfGfpZZWVna/PHHH9fmVitQfQnzyr0OHjyozVu0aKHNw8LCtLnVp3GrzwXOHO/Oc9/reKvVziatQK2o8uYVT8wAAAAMQTEDAAAwBMUMAADAEBQzAAAAQ1DMAAAADMFemVVQbm6uNr99+7aHRwJUjtXegK5YlWm1P+eECRO0eVVYZQmzWO1rPGXKFG0+a9Ysbe7sfpNWx+/evdvhcyQkJDh17sjISKeOr854YgYAAGAIihkAAIAhKGYAAACGoJgBAAAYgmIGAABgCONXZUZFRWnzgoICbX79+nV3DscnWO2tmZeX5+GRAJXTuHFjt5170KBBbjs3UBmzZ892KjdJv379tPmmTZucOo/VKs7qgCdmAAAAhqCYAQAAGIJiBgAAYAiKGQAAgCEoZgAAAIYwZlXmb3/7W20+atQobf7pp59q87S0NJeNyVd9/PHH3h4C4BLp6emVPkdWVlblBwLAIZs3b9bmVntisldmWTwxAwAAMATFDAAAwBAUMwAAAENQzAAAAAxBMQMAADCEx1dljhs3Tpu/8sorTp2nT58+2rx9+/ba/IsvvnDq/N4wefJkbT5w4ECnzrNr1y5XDAfwus6dO1f6HOPHj3fBSABUhp+fn7eH4DN4YgYAAGAIihkAAIAhKGYAAACGoJgBAAAYgmIGAABgCI+vyjxy5Ig2v379ujavWbOmNq9du7Y2/+Mf/6jNx4wZo81PnTqlzfft26fNndW8efMy2dChQ7XHTpgwQZuzlxiqusaNG2tzZ1ZlbtiwQZu7ai4DqDj2ynQcT8wAAAAMQTEDAAAwBMUMAADAEBQzAAAAQ1DMAAAADOHxVZkZGRna/LnnntPmzz//vDZv1aqVNrdarbl27VptfuHCBW3+3XffaXNnNWjQoEwWGxvr1DmsVrIuX768IkMCjJOenl7pcwwaNKjyAwHgFvn5+do8IiJCm1fnvTV5YgYAAGAIihkAAIAhKGYAAACGoJgBAAAYgmIGAABgCI+vyrSybNkybf7RRx9p827dujl1/v/+7//W5lYrQqxyZ+lWlljtDXb+/HltbrWH5tatWys+MMCHWe2LCcBMH3zwgTZ/6qmntHl13kOTJ2YAAACGoJgBAAAYgmIGAABgCIoZAACAIShmAAAAhjBmVaYVq/21Nm7c6NR5/va3v2nzpKQkbW61n6XVnp7OOHTokDZPTU3V5mfPnq30NQGT7d+/36njFyxY4KaRAHCHvLw8bW61J+a7777rzuEYjSdmAAAAhqCYAQAAGIJiBgAAYAiKGQAAgCEoZgAAAIbwUw5sSHXlyhWpXbu2J8YDVMjly5elVq1a3h6GU5hXMB3zCq4SFxenzd977z1tnpKS4sbReFd584onZgAAAIagmAEAABiCYgYAAGAIihkAAIAhKGYAAACGMH6vTAAA4NtOnDihzavy6suK4okZAACAIShmAAAAhqCYAQAAGIJiBgAAYAiKGQAAgCEoZgAAAIagmAEAABiCYgYAAGAIihkAAIAhKGYAAACGcKiYKaXcPQ6gUnzxHvXFMaN68cV71BfHjOqlvHvUoWJWUFDgksEA7uKL96gvjhnViy/eo744ZlQv5d2jfsqBLy+Ki4slNzdXbDab+Pn5uWxwQGUppaSgoEBiYmLE39+3vjPPvIKpmFeA6zk6rxwqZgAAAHA/3/pSCAAAoAqjmAEAABiCYgYAAGAIihkAAIAhKGYAAACGoJgBAAAYgmIGAABgCIoZAACAIShmAAAAhqCYAQAAGIJiBgAAYAiKGQAAgCEoZgAAAIagmAEAABiCYgYAAGAIihkAAIAhKGYAAACGoJgBAAAYgmIGAABgCIoZAACAIShmAAAAhqCYAQAAGIJiBgAAYAifKWbLly8XPz+/0j8BAQHSqFEjGTFihJw5c8YjY7jvvvtk+PDhpX/fuXOn+Pn5yc6dO506z969e2XatGly6dKlMm9LSUmRlJSUSo3T1Q4cOCA9e/YUm80m4eHh0q1bN9mzZ4+3hwUXYF55D/Oq6mJeec/Vq1clPT1dYmJiJCQkRNq1aydr16719rCc4jPFrMSyZcskKytL/vjHP8qoUaNkzZo18tBDD8m1a9c8Ppb27dtLVlaWtG/f3qnX7d27V6ZPn6690RctWiSLFi1y0QgrLzs7W5KSkuTGjRuycuVKWblypdy8eVO6d+8uWVlZ3h4eXIR55VnMq+qBeeV5/fv3l/fee09eeeUV2bp1q3Ts2FEGDx4s77//vreH5jjlI5YtW6ZERGVnZ9vlL7/8shIRtWrVKsvXXrt2zSVjiIuLU2lpaZU+zxtvvKFERB07dqzS53K3nj17qqioKLuP4ZUrV1S9evVUly5dvDgyuALzyjuYV1Ub88o7MjIylIio999/3y5/+OGHVUxMjCoqKvLSyJzjc0/MfiwxMVFERE6cOCEiIsOHD5fw8HA5fPiw/OIXvxCbzSbdu3cXEZHbt2/LrFmzpGXLlhIcHCyRkZEyYsQIycvLsztnYWGhvPDCCxIdHS2hoaHStWtXOXDgQJlrWz0a3r9/v6SmpkpERISEhIRIfHy8pKeni4jItGnTZNKkSSIicv/995c+6i45h+7R8MWLF+Xpp5+Whg0bSlBQkDRp0kSmTp0qt27dsjvOz89Pxo4dKytXrpSEhAQJDQ2Vtm3bypYtW5z+uJbYs2ePpKSkSGhoaGlms9kkKSlJ9u7dK2fPnq3wuWEu5tU/Ma/gKsyrf3LHvNq8ebOEh4fLY489ZpePGDFCcnNzZf/+/RU+tycFeHsAlfXdd9+JiEhkZGRpdvv2bfnlL38po0ePlsmTJ0tRUZEUFxdL3759Zffu3fLCCy9Ily5d5MSJE/LKK69ISkqKHDx4UGrWrCkiIqNGjZIVK1bIxIkT5eGHH5acnBzp37+/FBQUlDue7du3S2pqqiQkJMiCBQskNjZWjh8/Ljt27BARkaeeekouXrwo77zzjnzwwQfSoEEDERH56U9/qj3fzZs3pVu3bvL999/L9OnTpU2bNrJ7926ZPXu2HDp0SDIyMuyOz8jIkOzsbJkxY4aEh4fL3LlzpV+/fvLtt99KkyZNSo/z8/OT5OTkcn/e4Pbt2xIcHFwmL8kOHz5c+j6g6mBeMa/geswr986rnJwcSUhIkIAA+2rTpk2b0rd36dKl3I+L13n7kZ2jSh4N79u3TxUWFqqCggK1ZcsWFRkZqWw2mzp37pxSSqm0tDQlImrp0qV2r1+zZo0SEbVp0ya7PDs7W4mIWrRokVJKqa+//lqJiBo3bpzdcatXr1YiYvdoODMzU4mIyszMLM3i4+NVfHy8unHjhuX7cq9Hw8nJySo5Obn070uWLFEiotavX2933Jw5c5SIqB07dpRmIqKioqLUlStXSrNz584pf39/NXv2bLvX16hRQ/385z+3HGOJdu3aqebNm6s7d+6UZoWFhapJkybaR8bwLcwr5hVcj3nlnXnVrFkz1bNnzzJ5bm6uEhH12muvlXsOE/jctzITExMlMDBQbDab9OnTR6Kjo2Xr1q0SFRVld9yAAQPs/r5lyxapU6eOpKamSlFRUemfdu3aSXR0dGkTz8zMFBGRIUOG2L1+0KBBZVr4jx05ckS+//57efLJJyUkJKSS7+ldn376qYSFhcnAgQPt8pLVNn/605/s8m7duonNZiv9e1RUlNSvX7/00XmJoqKiMq/VefbZZ+XIkSMyduxYOXPmjJw6dUrGjBlTej5/f5+7haDBvLqLeQVXYl7d5al5JXL36VpF3mYSn/tW5ooVK0ofVUZFRWkf94eGhkqtWrXssh9++EEuXbokQUFB2vNeuHBBRETy8/NFRCQ6Otru7QEBARIREXHPsZV8779Ro0aOvTMOyM/Pl+jo6DI3VP369SUgIKB0vCV0YwwODpYbN25U6PojR46UvLw8mTVrlixevFhERDp37iwTJ06UOXPmSMOGDSt0XpiFeXUX8wquxLy6y1PzKiIiosw1RO7+3JuISN26dSt0Xk/zuWKWkJAgHTp0uOcxulZcr149iYiIkG3btmlfU9LaS26Uc+fO2f3nWFRUpP0H/1clPzdw+vTpex7njIiICNm/f78opezer/Pnz0tRUZHUq1fPZdey8uKLL0p6erocPXpUbDabxMXFyejRoyUsLEwefPBBt18f7se8uot5BVdiXt3lqXnVunVrWbNmjRQVFdk9MTx8+LCIiLRq1cqt13eVavO8vE+fPpKfny937tyRDh06lPnTokULEZHSFSarV6+2e/369eulqKjontdo3ry5xMfHy9KlS8usQPlXJT/g68hXBd27d5erV6/Khx9+aJevWLGi9O2eEBwcLK1atZK4uDg5efKkrFu3TkaNGlX6A6ionphXlcO8gg7zqmL69esnV69elU2bNtnl7733nsTExMjPfvYzt17fVXzuiVlFPfHEE7J69Wrp3bu3PP/889KpUycJDAyU06dPS2ZmpvTt21f69esnCQkJ8qtf/UreeustCQwMlB49ekhOTo7MmzevzONmnYULF0pqaqokJibKuHHjJDY2Vk6ePCnbt28vnTytW7cWEZG3335b0tLSJDAwUFq0aGH3vfYSw4YNk4ULF0paWpocP35cWrduLZ9//rm89tpr0rt3b+nRo0eFPh4BAQGSnJxc7vftc3JyZNOmTdKhQwcJDg6WL7/8Ul5//XVp1qyZzJw5s0LXRtXBvLLHvIIrMK/sOTqvevXqJQ8//LD8+te/litXrkjTpk1lzZo1sm3bNlm1apXUqFGjQtf3OC8vPnCY1S/s+7G0tDQVFhamfVthYaGaN2+eatu2rQoJCVHh4eGqZcuWavTo0ero0aOlx926dUtNmDBB1a9fX4WEhKjExESVlZVV5hf26Va5KKVUVlaW6tWrl6pdu7YKDg5W8fHxZVbNTJkyRcXExCh/f3+7c/x4lYtSSuXn56sxY8aoBg0aqICAABUXF6emTJmibt68aXeciKhnnnmmzPut+0WDIlLmOjrffvutSkpKUnXr1lVBQUGqadOm6qWXXlJXr14t97UwH/OKeQXXY155Z14ppVRBQYF67rnnVHR0tAoKClJt2rRRa9ascei1pvBTSikv9EEAAAD8SLX5GTMAAADTUcwAAAAMQTEDAAAwBMUMAADAEBQzAAAAQ1DMAAAADOHQL5gtLi6W3NxcsdlsPrMJKKoHpZQUFBRITEyMz238zLyCqZhXgOs5Oq8cKma5ubnSuHFjlw0OcLVTp065dDNeT2BewXTMK8D1yptXDn0ppNt6ATCJL96jvjhmVC++eI/64phRvZR3jzpUzHgcDNP54j3qi2NG9eKL96gvjhnVS3n3qG/98AAAAEAVRjEDAAAwBMUMAADAEBQzAAAAQ1DMAAAADEExAwAAMATFDAAAwBAUMwAAAENQzAAAAAxBMQMAADAExQwAAMAQFDMAAABDUMwAAAAMQTEDAAAwBMUMAADAEAHeHgAAAPAt/v765zqdOnVy6jxHjx7V5vn5+U6PqargiRkAAIAhKGYAAACGoJgBAAAYgmIGAABgCIoZAACAIViVCQBANWK1otLPz69MppTSHjtz5kxtPmXKFKfGcurUKW0+evRobb5t2zanzu+LeGIGAABgCIoZAACAIShmAAAAhqCYAQAAGIJiBgAAYAhWZQIA4MPq1aunzd9++21tXqtWLW2+f//+MtmiRYu0xyYlJWnztWvXavMTJ05o85EjR2rzDz74QJtPnTpVm7/55pva3BfxxAwAAMAQFDMAAABDUMwAAAAMQTEDAAAwBMUMAADAEKzK9IL77rtPm6ekpGjzBx98UJsPHjxYm+v2OxMR6d27tzbXrcQBAJjFajXlf/3Xf2nzJ554QptfvHhRmx8/ftzhYx955BFtfuXKFW1uZePGjdrcapXljBkztHlxcbE2t1qZajKemAEAABiCYgYAAGAIihkAAIAhKGYAAACGoJgBAAAYwk8ppco76MqVK1K7dm1PjMdn9ezZU5vrVq4MGTJEe6yrPsZWqzLPnz+vzRMSErT5pUuXXDIeT7h8+bLliiVTVcd5ZfVvNHToUG3+4osvavPGjRuXyRz4r8yO1f09Z84cbb5s2TJtbjWvqgLmlXe0atVKm+/YsUOb16lTR5tPmjRJm7/77rvavLCwsPzBeUiLFi20+ezZs7V5YmKiNm/durU2z8/Pr9jAXKC8ecUTMwAAAENQzAAAAAxBMQMAADAExQwAAMAQFDMAAABDVPtVmVarWdLT07X5mDFjtPlPfvITbR4QUHY7UqsP+fXr17W51UoZq38Tq1WZVtdt0KCBNs/Ly9PmJmL1mFk6deqkzdevX6/NY2NjnTr/6dOny2TOrsqMiYnR5jVq1NDmGzZs0OaPP/64U9f1Jcwr92rSpIk237VrlzZv2LChNl+1apU2HzZsWMUGZjCr34CQkZGhzf/85z9r865du2pzT6xMZVUmAACAj6CYAQAAGIJiBgAAYAiKGQAAgCEoZgAAAIYou2SwimrevLk2X7t2rTZv06aNS66rW13z4Ycfao/95JNPtPnt27e1eVZWljavW7euY4P7B6vVab60KhNmmTZtmja3Wn351VdfafP58+drc90qtKKiIscG9w8TJkzQ5pMnT9bmVv+H1KxZU5vfuHHDqfGg+rFa5W+1+vLs2bPa/LnnnnPZmEy3fft2bf7FF19o844dO2rzPn36aPPNmzdXbGAuxBMzAAAAQ1DMAAAADEExAwAAMATFDAAAwBAUMwAAAENUub0yBw8erM3feecdbW61V6aV3Nxcp667Z88ep86vExYWps3/7//+T5vHx8drc6t/6gsXLmjzN954Q5tbrZTzJvb0844BAwZo83Xr1mnzEydOaHOrvTXz8/MrNrBKOHDggDbv0KGDNrdaxTl37lyXjclbmFeuYXWPzJgxQ5vfunVLm1utMPzmm28qNrAqpGXLltr8r3/9qza3Wglu9X+RK1dZs1cmAACAj6CYAQAAGIJiBgAAYAiKGQAAgCEoZgAAAIYwfq/M+vXra/NJkyZp8/Hjx2tzPz8/bX758mVt/pvf/EabL168WJu7k9VKHKvVl1bvq5XIyEht/sgjj2hzE1dlwjus9o/099d/zWe1Atgbqy9dxWoFHaqfBx54QJtPnTpVmwcE6D8F//a3v9XmrL60ZrXie/369dp80KBB2rx169ba3Gq1tjvwxAwAAMAQFDMAAABDUMwAAAAMQTEDAAAwBMUMAADAEMasykxJSdHmCxcu1OYtWrTQ5lb7QVrtK2m1h9knn3yizV3B6n21Wh1ptXrEgW1OPXo8UJ66detqc6u9Da1WTbvCq6++qs2bNWumzQ8ePKjNlyxZ4rIxwbc9//zz2txqv+Ndu3Zp8zlz5rhsTNWF1V6W7733njZ/7LHH3DmcSuGJGQAAgCEoZgAAAIagmAEAABiCYgYAAGAIihkAAIAhPL4q02pvsJkzZ2pzq9WXzurfv782v3nzpja32qMzJiZGm/ft21ebP/3002WyWrVqaY8NDAzU5t5y/fp1bw8BhrNaNd2zZ09tnpSUpM3Pnj2rzY8fP67NMzMzyx/cP3Tr1k2bW/3fYrXXbF5enjZnr0yUsPo8U1RUpM2tVl+yIt51tm7dqs2tPvf369dPm7NXJgAAQDVEMQMAADAExQwAAMAQFDMAAABDePyH/61+4LZz585uve6xY8e0ubt/yFL3g8Sm/WDn119/rc1//etfe3gk8DVXrlzR5unp6dp88eLF2vxnP/uZNm/ZsqVTuTt99dVXHr8mfEtERIQ2t/oB9G3btrlzOLgHq0U+Vv+GnsQTMwAAAENQzAAAAAxBMQMAADAExQwAAMAQFDMAAABDeHxV5sWLF7V5bm6uNrfaAslXLFq0qEz2hz/8QXus1cdg+PDh2nz8+PEVHte/2rBhgzY/ffq0S86P6ufQoUPavFevXtr8oYcecur8TZs2LZM1adJEe6zV/zkvv/yyU9c8ePCgU8ej+jl8+LA2Dw8P1+ZWq4u/+eYbl40Jen/5y1+0eZs2bTw8krJ4YgYAAGAIihkAAIAhKGYAAACGoJgBAAAYgmIGAABgCI+vyjx79qw27927tzbv2LGjNg8JCXHqurt379bmOTk5Tp3HnWw2mzZ/9NFHtbnVXl/+/vq+fenSJW2el5dX7tgAV7C6Bz/++GO3XXPt2rVOHV9cXKzNrcYOlNi/f782f/LJJ7V5amqqNmdVpvsdOHBAm48dO9bDIymLJ2YAAACGoJgBAAAYgmIGAABgCIoZAACAIShmAAAAhvD4qkwrVqsjTVo16W5PPPGENrfaA1Appc2tVpX97ne/0+aLFy92YHSAbwoMDHTq+C+++EKb79ixwxXDQRVmtQLYalXmjBkztPn69eu1+YkTJyo2sGps2LBh2nzo0KHa3NlV3O7AEzMAAABDUMwAAAAMQTEDAAAwBMUMAADAEBQzAAAAQxizKrM6eeCBB7T5q6++6tbrfvLJJ249P1AVWO13CJTHakXvZ599ps2TkpK0+ZIlS7T5yJEjtbnVHtTVidVvNViwYIE2r127tjb/6quvXDamiuKJGQAAgCEoZgAAAIagmAEAABiCYgYAAGAIihkAAIAhWJXpBenp6dq8bt26Ljn/gQMHtPlf//pXl5wfMFFISIg2b9WqlYdHgurq73//uzbv16+fNv/LX/6izf/jP/5Dm1vt1zp9+nRtfuPGDW1++PBhbX7y5Elt7k7R0dHavF69etp86tSp2nzQoEHa3N9f//xp7Nix2vzdd9/V5p7EEzMAAABDUMwAAAAMQTEDAAAwBMUMAADAEBQzAAAAQ7Aq080effTRMtnjjz/u1muuWLFCm+fl5bn1uoA3BQcHa/NmzZo5dZ6MjAxXDAcoZbVas02bNtr8D3/4gzbv2rWrNl+/fr1T47l27Zo2z87Oduo8rmC1d3T9+vW1uVJKm3/55ZfafO7cudrc6mN2584dbe5JPDEDAAAwBMUMAADAEBQzAAAAQ1DMAAAADEExAwAAMASrMt1sxowZZbKwsDC3XnP37t1uPT9gopSUFJec58KFCy45D1Aeq9Wa3bt31+a9evXS5gMGDNDmNWvW1OYtWrTQ5hEREWWy1q1ba491VnFxsTb/6quvtPmxY8e0+WuvvabNt27dqs0LCwsdGJ1ZeGIGAABgCIoZAACAIShmAAAAhqCYAQAAGIJiBgAAYAhWZbpI06ZNtbluHzCrvb6cNX/+fG2ek5PjkvMDvuS+++7z9hAAl7BaSfjRRx85lbtCp06dXHIeqz0o//znP7vk/FUJT8wAAAAMQTEDAAAwBMUMAADAEBQzAAAAQ1DMAAAADMGqTBdZs2aN28594sQJbT5v3jy3XRPwNUeOHHHJeaz2ETx48KBLzg/4kgMHDnh7CNUOT8wAAAAMQTEDAAAwBMUMAADAEBQzAAAAQ1DMAAAADMGqTBe5ePFipc9htT/arFmztHleXl6lrwlUFZ999pk2P3/+vDavX7++Nk9MTNTmq1evrtjAAMAJPDEDAAAwBMUMAADAEBQzAAAAQ1DMAAAADEExAwAAMASrMl1k8ODB2jwzM7NMFhISoj12xowZ2pzVYED5rl27ps23bdumzYcNG6bNhwwZos2t5uG+ffscGB0AOIYnZgAAAIagmAEAABiCYgYAAGAIihkAAIAhKGYAAACGYFWmi1jtldm2bVsPjwTAv5owYYI2f+CBB7R5aGioNo+MjHTZmADACk/MAAAADEExAwAAMATFDAAAwBAUMwAAAENQzAAAAAzBqkwAVVp+fr4279ixo4dHAgDl44kZAACAIShmAAAAhqCYAQAAGIJiBgAAYAiHiplSyt3jACrFF+9RXxwzqhdfvEd9ccyoXsq7Rx0qZgUFBS4ZDOAuvniP+uKYUb344j3qi2NG9VLePeqnHPjyori4WHJzc8Vms4mfn5/LBgdUllJKCgoKJCYmRvz9fes788wrmIp5Bbieo/PKoWIGAAAA9/OtL4UAAACqMIoZAACAIShmAAAAhqCYAQAAGIJiBgAAYAiKGQAAgCEoZgAAAIb4f3UkPrBGisG3AAAAAElFTkSuQmCC",
"text/plain": [
"<Figure size 640x480 with 6 Axes>"
]
},
"execution_count": 25,
"metadata": {},
"output_type": "execute_result"
},
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAmYAAAGkCAYAAACb5OmoAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjguNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8fJSN1AAAACXBIWXMAAA9hAAAPYQGoP6dpAAAvlElEQVR4nO3de1yVVb7H8R/ITWCrIyKICiXemLwdUwc9BjjaeDQZ85Lly1HU8uiUFd5Kx5q8lWlqdRovpzMvNS95z6bkeJkmNFNUnI6NNJXWeEcTcVS8g6zzhwMzO9Yje8O+rA2f9+vlH3559vMs8Fny5YHF8lNKKQEAAIDX+Xt7AAAAALiLYgYAAGAIihkAAIAhKGYAAACGoJgBAAAYgmIGAABgCIoZAACAIShmAAAAhqCYAQAAGMJnitny5cvFz8+v9E9AQIA0atRIRowYIWfOnPHIGO677z4ZPnx46d937twpfn5+snPnTqfOs3fvXpk2bZpcunSpzNtSUlIkJSWlUuN0pUOHDskjjzwisbGxUrNmTalbt6507txZVq1a5e2hwQWYV95z4MAB6dmzp9hsNgkPD5du3brJnj17vD0suADzygy///3vxc/PT8LDw709FKcEeHsAzlq2bJm0bNlSbty4IZ999pnMnj1bdu3aJYcPH5awsDCPjqV9+/aSlZUlP/3pT5163d69e2X69OkyfPhwqVOnjt3bFi1a5MIRVt6lS5ekcePGMnjwYGnYsKFcu3ZNVq9eLUOHDpXjx4/LSy+95O0hwgWYV56VnZ0tSUlJ0qlTJ1m5cqUopWTu3LnSvXt3yczMlM6dO3t7iHAB5pX3nDlzRiZOnCgxMTFy+fJlbw/HKT5XzFq1aiUdOnQQEZFu3brJnTt3ZObMmfLhhx/KkCFDtK+5fv26hIaGunwstWrVksTERJee09lJ4266r4j69Okjx44dk3fffZdiVkUwrzzr5Zdfljp16si2bdtKP4Y9evSQJk2ayMSJE3lyVkUwr7xnzJgxkpSUJHXr1pWNGzd6ezhO8ZlvZVopudFOnDghIiLDhw+X8PBwOXz4sPziF78Qm80m3bt3FxGR27dvy6xZs6Rly5YSHBwskZGRMmLECMnLy7M7Z2FhobzwwgsSHR0toaGh0rVrVzlw4ECZa1s9Gt6/f7+kpqZKRESEhISESHx8vKSnp4uIyLRp02TSpEkiInL//feXPuouOYeuCF28eFGefvppadiwoQQFBUmTJk1k6tSpcuvWLbvj/Pz8ZOzYsbJy5UpJSEiQ0NBQadu2rWzZssXpj2t56tWrJwEBPtfr4SDm1T+5Y17t2bNHUlJS7D4B22w2SUpKkr1798rZs2crfG6Yi3n1T+78fLVq1SrZtWuX0U/07sXnP7N+9913IiISGRlZmt2+fVt++ctfyujRo2Xy5MlSVFQkxcXF0rdvX9m9e7e88MIL0qVLFzlx4oS88sorkpKSIgcPHpSaNWuKiMioUaNkxYoVMnHiRHn44YclJydH+vfvLwUFBeWOZ/v27ZKamioJCQmyYMECiY2NlePHj8uOHTtEROSpp56SixcvyjvvvCMffPCBNGjQQESsv/K4efOmdOvWTb7//nuZPn26tGnTRnbv3i2zZ8+WQ4cOSUZGht3xGRkZkp2dLTNmzJDw8HCZO3eu9OvXT7799ltp0qRJ6XF+fn6SnJzs8M8bFBcXS3Fxsfz973+XDRs2yPbt2+V3v/udQ6+F72FeuXde3b59W4KDg8vkJdnhw4dL3wdUHcwr93++On/+vKSnp8vrr78ujRo1Kvd4IykfsWzZMiUiat++faqwsFAVFBSoLVu2qMjISGWz2dS5c+eUUkqlpaUpEVFLly61e/2aNWuUiKhNmzbZ5dnZ2UpE1KJFi5RSSn399ddKRNS4cePsjlu9erUSEZWWllaaZWZmKhFRmZmZpVl8fLyKj49XN27csHxf3njjDSUi6tixY2XelpycrJKTk0v/vmTJEiUiav369XbHzZkzR4mI2rFjR2kmIioqKkpduXKlNDt37pzy9/dXs2fPtnt9jRo11M9//nPLMf7Y6NGjlYgoEVFBQUGlHy/4NuaVd+ZVu3btVPPmzdWdO3dKs8LCQtWkSRMlIur9998v9xwwF/PKe5+vBgwYoLp06aKKi4uVUnc/xmFhYQ691hQ+963MxMRECQwMFJvNJn369JHo6GjZunWrREVF2R03YMAAu79v2bJF6tSpI6mpqVJUVFT6p127dhIdHV3axDMzM0VEynz/f9CgQeV+6+7IkSPy/fffy5NPPikhISGVfE/v+vTTTyUsLEwGDhxol5estvnTn/5kl3fr1k1sNlvp36OioqR+/fqlj85LFBUVlXntvfzmN7+R7OxsycjIkJEjR8rYsWNl3rx5Tr43MBXz6i5Pzatnn31Wjhw5ImPHjpUzZ87IqVOnZMyYMaXn8/f3uf+aocG8ustT82rTpk3y8ccfy//8z/+In59fBd8L7/O5b2WuWLFCEhISJCAgQKKiorSP+0NDQ6VWrVp22Q8//CCXLl2SoKAg7XkvXLggIiL5+fkiIhIdHW339oCAAImIiLjn2Eq+9+/Kx6f5+fkSHR1d5iarX7++BAQElI63hG6MwcHBcuPGjUqNIzY2VmJjY0VEpHfv3iIiMmXKFElLS7N7LA/fxLy6y1PzauTIkZKXlyezZs2SxYsXi4hI586dZeLEiTJnzhxp2LBhhc4LszCv7vLEvLp69ao888wz8uyzz0pMTEzpr/e4ffu2iNz9DQOBgYEeXw1bET5XzBISEkpXuVjRNeV69epJRESEbNu2TfuaktZecqOcO3fO7j/HoqKiMjfVj5UUlNOnT9/zOGdERETI/v37RSll936dP39eioqKpF69ei67ljM6deokS5Yskb/97W8UsyqAeXWXJ+fViy++KOnp6XL06FGx2WwSFxcno0ePlrCwMHnwwQfdfn24H/PqLk/MqwsXLsgPP/wg8+fPl/nz55d5+09+8hPp27evfPjhh24bg6tUm+flffr0kfz8fLlz54506NChzJ8WLVqIiJSuMFm9erXd69evXy9FRUX3vEbz5s0lPj5eli5dWmYFyr8q+QFfR74q6N69u1y9erXMzbRixYrSt3tDZmam+Pv72/2AJqof5lXlBAcHS6tWrSQuLk5Onjwp69atk1GjRpX+YDeqJ+aV86KjoyUzM7PMn549e0pISIhkZmbKrFmz3HZ9V/K5J2YV9cQTT8jq1auld+/e8vzzz0unTp0kMDBQTp8+LZmZmdK3b1/p16+fJCQkyK9+9St56623JDAwUHr06CE5OTkyb968Mo+bdRYuXCipqamSmJgo48aNk9jYWDl58qRs3769dPK0bt1aRETefvttSUtLk8DAQGnRooXd99pLDBs2TBYuXChpaWly/Phxad26tXz++efy2muvSe/evaVHjx4V+ngEBARIcnJyud+3/8///E+pVauWdOrUSaKiouTChQuyYcMGWbdunUyaNImnZdUc88qeo/MqJydHNm3aJB06dJDg4GD58ssv5fXXX5dmzZrJzJkzK3RtVB3MK3uOzKuQkBDtLgTLly+XGjVqGL1DQRleXnzgsJJVLtnZ2fc87l4rMAoLC9W8efNU27ZtVUhIiAoPD1ctW7ZUo0ePVkePHi097tatW2rChAmqfv36KiQkRCUmJqqsrCwVFxdX7ioXpZTKyspSvXr1UrVr11bBwcEqPj6+zKqZKVOmqJiYGOXv7293jh+vclFKqfz8fDVmzBjVoEEDFRAQoOLi4tSUKVPUzZs37Y4TEfXMM8+Ueb9/PO6SY398HZ2lS5eqhx56SNWrV08FBASoOnXqqOTkZLVy5cpyXwvzMa+8M6++/fZblZSUpOrWrauCgoJU06ZN1UsvvaSuXr1a7mthPuaVd+aVji+uyvRTSikv9EEAAAD8SLX5GTMAAADTUcwAAAAMQTEDAAAwBMUMAADAEBQzAAAAQ1DMAAAADOHQL5gtLi6W3NxcsdlsPr0xKKoepZQUFBRITEyMz238zLyCqZhXgOs5Oq8cKma5ubnSuHFjlw0OcLVTp065dDNeT2BewXTMK8D1yptXDn0ppNt6ATCJL96jvjhmVC++eI/64phRvZR3jzpUzHgcDNP54j3qi2NG9eKL96gvjhnVS3n3qG/98AAAAEAVRjEDAAAwBMUMAADAEBQzAAAAQ1DMAAAADEExAwAAMATFDAAAwBAUMwAAAENQzAAAAAxBMQMAADAExQwAAMAQFDMAAABDUMwAAAAMQTEDAAAwBMUMAADAEBQzAAAAQ1DMAAAADEExAwAAMATFDAAAwBAUMwAAAENQzAAAAAxBMQMAADAExQwAAMAQFDMAAABDUMwAAAAMQTEDAAAwBMUMAADAEAHeHoC3NW/eXJu3b99em1+9elWbN2vWzOFrtmnTRpsPGzbM4XPci7+/vm8XFxc7dZ7HH39cm2/cuNHpMQGutHfv3jJZ586dtceOHz9em7/55psuHRMAuAJPzAAAAAxBMQMAADAExQwAAMAQFDMAAABDUMwAAAAMUW1WZVqtvszIyNDmDRs21OZ37tzR5jVr1tTmfn5+ZTKllPZYq9xZVqsvnT1/amqqNmdVJjxFt/pSxHoFps6CBQu0udV9fOrUKYfPDcAxDz74oDb/3//9X22+Y8cObT506FCXjclUPDEDAAAwBMUMAADAEBQzAAAAQ1DMAAAADEExAwAAMES1WZVptfdlYWGhNg8KCnLbWPLz8526ps1mc9tYRERycnK0+fLly916XaDE/PnztbnV6kvdysl///d/1x578uRJp645aNAgbQ6g4kaNGqXNIyIitHnLli3dORyj8cQMAADAEBQzAAAAQ1DMAAAADEExAwAAMATFDAAAwBDVZlXm2rVrtfnnn3+uzRMTE902FqtVmXPnztXm//Zv/+aS62ZlZWnzRx99VJtbjRNwtccee8yp42NjYx0+1uq+t7pm48aNtTl7aMLb4uLitPmKFSu0+VtvvaXNN2/e7KohOSwyMlKb6/aTvldeHfDEDAAAwBAUMwAAAENQzAAAAAxBMQMAADAExQwAAMAQ1WZVppXTp09r840bN7rk/MnJyWWyCRMmaI911erLXbt2afM33nhDm7P6Et5mtRJywYIFlT631Ry3MnDgQG3+5ptvVnosQGVYzQerfWJDQ0O1uTdWZVqt/ldKOZVXBzwxAwAAMATFDAAAwBAUMwAAAENQzAAAAAxBMQMAADBEtV+V6Sq61ZciIjt37iyTFRcXO3XugoICbf773/9em0+cONGp8wOeMm7cOKeOt9rrzxnO7sPZuXNnbc6qTHjK1KlTtXn//v21udXnlAsXLrhsTJXl769/DmQ1dvbKBAAAgNdRzAAAAAxBMQMAADAExQwAAMAQFDMAAABDsCrTSb169dLma9as0ea6FSfO7gFWs2ZNbR4UFOTUeQBvs1rxaOXUqVMOH+vsik8rjRo1csl5gPJYrb6cPHmyNrdawWj1OeXVV1+t2MDcwNmxb9q0yZ3DMRpPzAAAAAxBMQMAADAExQwAAMAQFDMAAABDUMwAAAAMwapMJw0fPlybh4eHu+2aNWrU0OZPP/20Nn/uuefcNhagMty54tHZPTGtOLtyFChPZGSkNh8yZIg2Dw0N1ebXr1/X5sOGDdPmn3/+uQOj8wxn977Mz89300jMxxMzAAAAQ1DMAAAADEExAwAAMATFDAAAwBAUMwAAAEOwKtNJy5Yt0+adOnXS5rt37y6TffTRR9pjJ0yYoM07duzo4OjuqlOnjja/dOmSU+cBXC0rK0ubW62ETExM1Ob79u1z+BxWNmzYoM1dtboTKDFlyhRt3qJFC21utX/kN998o803b95csYF5kNX75Oze0dUBT8wAAAAMQTEDAAAwBMUMAADAEBQzAAAAQ/DD/07atm2bNr///vsrfe7Tp09rc2e31XjppZe0+cSJE50eE+BKb731ljYfP368Nl+/fr02d+ZeXrBggVPXtDoeqKiHHnpIm/v765+NFBcXa/OhQ4e6bEzukpSUpM2d3ZKpOuOJGQAAgCEoZgAAAIagmAEAABiCYgYAAGAIihkAAIAhWJVpEN02MyIimzZt0uYDBw7U5h06dNDm4eHh2vzq1asOjA6ovFOnTmlzZ7dHWrduncPXbNy4scPH3sugQYO0udU8tDoe1c/XX3+tzdu3b6/NrbYpevPNN7W5SVsyde3aVZuzJZPjeGIGAABgCIoZAACAIShmAAAAhqCYAQAAGIJiBgAAYAg/5cCSiCtXrkjt2rU9MR5oWK1AGzBggDa32pMsNjZWm585c6ZiAzPI5cuXpVatWt4ehlOYV+UbN26cNvfGfpZZWVna/PHHH9fmVitQfQnzyr0OHjyozVu0aKHNw8LCtLnVp3GrzwXOHO/Oc9/reKvVziatQK2o8uYVT8wAAAAMQTEDAAAwBMUMAADAEBQzAAAAQ1DMAAAADMFemVVQbm6uNr99+7aHRwJUjtXegK5YlWm1P+eECRO0eVVYZQmzWO1rPGXKFG0+a9Ysbe7sfpNWx+/evdvhcyQkJDh17sjISKeOr854YgYAAGAIihkAAIAhKGYAAACGoJgBAAAYgmIGAABgCONXZUZFRWnzgoICbX79+nV3DscnWO2tmZeX5+GRAJXTuHFjt5170KBBbjs3UBmzZ892KjdJv379tPmmTZucOo/VKs7qgCdmAAAAhqCYAQAAGIJiBgAAYAiKGQAAgCEoZgAAAIYwZlXmb3/7W20+atQobf7pp59q87S0NJeNyVd9/PHH3h4C4BLp6emVPkdWVlblBwLAIZs3b9bmVntisldmWTwxAwAAMATFDAAAwBAUMwAAAENQzAAAAAxBMQMAADCEx1dljhs3Tpu/8sorTp2nT58+2rx9+/ba/IsvvnDq/N4wefJkbT5w4ECnzrNr1y5XDAfwus6dO1f6HOPHj3fBSABUhp+fn7eH4DN4YgYAAGAIihkAAIAhKGYAAACGoJgBAAAYgmIGAABgCI+vyjxy5Ig2v379ujavWbOmNq9du7Y2/+Mf/6jNx4wZo81PnTqlzfft26fNndW8efMy2dChQ7XHTpgwQZuzlxiqusaNG2tzZ1ZlbtiwQZu7ai4DqDj2ynQcT8wAAAAMQTEDAAAwBMUMAADAEBQzAAAAQ1DMAAAADOHxVZkZGRna/LnnntPmzz//vDZv1aqVNrdarbl27VptfuHCBW3+3XffaXNnNWjQoEwWGxvr1DmsVrIuX768IkMCjJOenl7pcwwaNKjyAwHgFvn5+do8IiJCm1fnvTV5YgYAAGAIihkAAIAhKGYAAACGoJgBAAAYgmIGAABgCI+vyrSybNkybf7RRx9p827dujl1/v/+7//W5lYrQqxyZ+lWlljtDXb+/HltbrWH5tatWys+MMCHWe2LCcBMH3zwgTZ/6qmntHl13kOTJ2YAAACGoJgBAAAYgmIGAABgCIoZAACAIShmAAAAhjBmVaYVq/21Nm7c6NR5/va3v2nzpKQkbW61n6XVnp7OOHTokDZPTU3V5mfPnq30NQGT7d+/36njFyxY4KaRAHCHvLw8bW61J+a7777rzuEYjSdmAAAAhqCYAQAAGIJiBgAAYAiKGQAAgCEoZgAAAIbwUw5sSHXlyhWpXbu2J8YDVMjly5elVq1a3h6GU5hXMB3zCq4SFxenzd977z1tnpKS4sbReFd584onZgAAAIagmAEAABiCYgYAAGAIihkAAIAhKGYAAACGMH6vTAAA4NtOnDihzavy6suK4okZAACAIShmAAAAhqCYAQAAGIJiBgAAYAiKGQAAgCEoZgAAAIagmAEAABiCYgYAAGAIihkAAIAhKGYAAACGcKiYKaXcPQ6gUnzxHvXFMaN68cV71BfHjOqlvHvUoWJWUFDgksEA7uKL96gvjhnViy/eo744ZlQv5d2jfsqBLy+Ki4slNzdXbDab+Pn5uWxwQGUppaSgoEBiYmLE39+3vjPPvIKpmFeA6zk6rxwqZgAAAHA/3/pSCAAAoAqjmAEAABiCYgYAAGAIihkAAIAhKGYAAACGoJgBAAAYgmIGAABgCIoZAACAIShmAAAAhqCYAQAAGIJiBgAAYAiKGQAAgCEoZgAAAIagmAEAABiCYgYAAGAIihkAAIAhKGYAAACGoJgBAAAYgmIGAABgCIoZAACAIShmAAAAhqCYAQAAGIJiBgAAYAifKWbLly8XPz+/0j8BAQHSqFEjGTFihJw5c8YjY7jvvvtk+PDhpX/fuXOn+Pn5yc6dO506z969e2XatGly6dKlMm9LSUmRlJSUSo3T1Q4cOCA9e/YUm80m4eHh0q1bN9mzZ4+3hwUXYF55D/Oq6mJeec/Vq1clPT1dYmJiJCQkRNq1aydr16719rCc4jPFrMSyZcskKytL/vjHP8qoUaNkzZo18tBDD8m1a9c8Ppb27dtLVlaWtG/f3qnX7d27V6ZPn6690RctWiSLFi1y0QgrLzs7W5KSkuTGjRuycuVKWblypdy8eVO6d+8uWVlZ3h4eXIR55VnMq+qBeeV5/fv3l/fee09eeeUV2bp1q3Ts2FEGDx4s77//vreH5jjlI5YtW6ZERGVnZ9vlL7/8shIRtWrVKsvXXrt2zSVjiIuLU2lpaZU+zxtvvKFERB07dqzS53K3nj17qqioKLuP4ZUrV1S9evVUly5dvDgyuALzyjuYV1Ub88o7MjIylIio999/3y5/+OGHVUxMjCoqKvLSyJzjc0/MfiwxMVFERE6cOCEiIsOHD5fw8HA5fPiw/OIXvxCbzSbdu3cXEZHbt2/LrFmzpGXLlhIcHCyRkZEyYsQIycvLsztnYWGhvPDCCxIdHS2hoaHStWtXOXDgQJlrWz0a3r9/v6SmpkpERISEhIRIfHy8pKeni4jItGnTZNKkSSIicv/995c+6i45h+7R8MWLF+Xpp5+Whg0bSlBQkDRp0kSmTp0qt27dsjvOz89Pxo4dKytXrpSEhAQJDQ2Vtm3bypYtW5z+uJbYs2ePpKSkSGhoaGlms9kkKSlJ9u7dK2fPnq3wuWEu5tU/Ma/gKsyrf3LHvNq8ebOEh4fLY489ZpePGDFCcnNzZf/+/RU+tycFeHsAlfXdd9+JiEhkZGRpdvv2bfnlL38po0ePlsmTJ0tRUZEUFxdL3759Zffu3fLCCy9Ily5d5MSJE/LKK69ISkqKHDx4UGrWrCkiIqNGjZIVK1bIxIkT5eGHH5acnBzp37+/FBQUlDue7du3S2pqqiQkJMiCBQskNjZWjh8/Ljt27BARkaeeekouXrwo77zzjnzwwQfSoEEDERH56U9/qj3fzZs3pVu3bvL999/L9OnTpU2bNrJ7926ZPXu2HDp0SDIyMuyOz8jIkOzsbJkxY4aEh4fL3LlzpV+/fvLtt99KkyZNSo/z8/OT5OTkcn/e4Pbt2xIcHFwmL8kOHz5c+j6g6mBeMa/geswr986rnJwcSUhIkIAA+2rTpk2b0rd36dKl3I+L13n7kZ2jSh4N79u3TxUWFqqCggK1ZcsWFRkZqWw2mzp37pxSSqm0tDQlImrp0qV2r1+zZo0SEbVp0ya7PDs7W4mIWrRokVJKqa+//lqJiBo3bpzdcatXr1YiYvdoODMzU4mIyszMLM3i4+NVfHy8unHjhuX7cq9Hw8nJySo5Obn070uWLFEiotavX2933Jw5c5SIqB07dpRmIqKioqLUlStXSrNz584pf39/NXv2bLvX16hRQ/385z+3HGOJdu3aqebNm6s7d+6UZoWFhapJkybaR8bwLcwr5hVcj3nlnXnVrFkz1bNnzzJ5bm6uEhH12muvlXsOE/jctzITExMlMDBQbDab9OnTR6Kjo2Xr1q0SFRVld9yAAQPs/r5lyxapU6eOpKamSlFRUemfdu3aSXR0dGkTz8zMFBGRIUOG2L1+0KBBZVr4jx05ckS+//57efLJJyUkJKSS7+ldn376qYSFhcnAgQPt8pLVNn/605/s8m7duonNZiv9e1RUlNSvX7/00XmJoqKiMq/VefbZZ+XIkSMyduxYOXPmjJw6dUrGjBlTej5/f5+7haDBvLqLeQVXYl7d5al5JXL36VpF3mYSn/tW5ooVK0ofVUZFRWkf94eGhkqtWrXssh9++EEuXbokQUFB2vNeuHBBRETy8/NFRCQ6Otru7QEBARIREXHPsZV8779Ro0aOvTMOyM/Pl+jo6DI3VP369SUgIKB0vCV0YwwODpYbN25U6PojR46UvLw8mTVrlixevFhERDp37iwTJ06UOXPmSMOGDSt0XpiFeXUX8wquxLy6y1PzKiIiosw1RO7+3JuISN26dSt0Xk/zuWKWkJAgHTp0uOcxulZcr149iYiIkG3btmlfU9LaS26Uc+fO2f3nWFRUpP0H/1clPzdw+vTpex7njIiICNm/f78opezer/Pnz0tRUZHUq1fPZdey8uKLL0p6erocPXpUbDabxMXFyejRoyUsLEwefPBBt18f7se8uot5BVdiXt3lqXnVunVrWbNmjRQVFdk9MTx8+LCIiLRq1cqt13eVavO8vE+fPpKfny937tyRDh06lPnTokULEZHSFSarV6+2e/369eulqKjontdo3ry5xMfHy9KlS8usQPlXJT/g68hXBd27d5erV6/Khx9+aJevWLGi9O2eEBwcLK1atZK4uDg5efKkrFu3TkaNGlX6A6ionphXlcO8gg7zqmL69esnV69elU2bNtnl7733nsTExMjPfvYzt17fVXzuiVlFPfHEE7J69Wrp3bu3PP/889KpUycJDAyU06dPS2ZmpvTt21f69esnCQkJ8qtf/UreeustCQwMlB49ekhOTo7MmzevzONmnYULF0pqaqokJibKuHHjJDY2Vk6ePCnbt28vnTytW7cWEZG3335b0tLSJDAwUFq0aGH3vfYSw4YNk4ULF0paWpocP35cWrduLZ9//rm89tpr0rt3b+nRo0eFPh4BAQGSnJxc7vftc3JyZNOmTdKhQwcJDg6WL7/8Ul5//XVp1qyZzJw5s0LXRtXBvLLHvIIrMK/sOTqvevXqJQ8//LD8+te/litXrkjTpk1lzZo1sm3bNlm1apXUqFGjQtf3OC8vPnCY1S/s+7G0tDQVFhamfVthYaGaN2+eatu2rQoJCVHh4eGqZcuWavTo0ero0aOlx926dUtNmDBB1a9fX4WEhKjExESVlZVV5hf26Va5KKVUVlaW6tWrl6pdu7YKDg5W8fHxZVbNTJkyRcXExCh/f3+7c/x4lYtSSuXn56sxY8aoBg0aqICAABUXF6emTJmibt68aXeciKhnnnmmzPut+0WDIlLmOjrffvutSkpKUnXr1lVBQUGqadOm6qWXXlJXr14t97UwH/OKeQXXY155Z14ppVRBQYF67rnnVHR0tAoKClJt2rRRa9ascei1pvBTSikv9EEAAAD8SLX5GTMAAADTUcwAAAAMQTEDAAAwBMUMAADAEBQzAAAAQ1DMAAAADOHQL5gtLi6W3NxcsdlsPrMJKKoHpZQUFBRITEyMz238zLyCqZhXgOs5Oq8cKma5ubnSuHFjlw0OcLVTp065dDNeT2BewXTMK8D1yptXDn0ppNt6ATCJL96jvjhmVC++eI/64phRvZR3jzpUzHgcDNP54j3qi2NG9eKL96gvjhnVS3n3qG/98AAAAEAVRjEDAAAwBMUMAADAEBQzAAAAQ1DMAAAADEExAwAAMATFDAAAwBAUMwAAAENQzAAAAAxBMQMAADAExQwAAMAQFDMAAABDUMwAAAAMQTEDAAAwBMUMAADAEAHeHgAAAPAt/v765zqdOnVy6jxHjx7V5vn5+U6PqargiRkAAIAhKGYAAACGoJgBAAAYgmIGAABgCIoZAACAIViVCQBANWK1otLPz69MppTSHjtz5kxtPmXKFKfGcurUKW0+evRobb5t2zanzu+LeGIGAABgCIoZAACAIShmAAAAhqCYAQAAGIJiBgAAYAhWZQIA4MPq1aunzd9++21tXqtWLW2+f//+MtmiRYu0xyYlJWnztWvXavMTJ05o85EjR2rzDz74QJtPnTpVm7/55pva3BfxxAwAAMAQFDMAAABDUMwAAAAMQTEDAAAwBMUMAADAEKzK9IL77rtPm6ekpGjzBx98UJsPHjxYm+v2OxMR6d27tzbXrcQBAJjFajXlf/3Xf2nzJ554QptfvHhRmx8/ftzhYx955BFtfuXKFW1uZePGjdrcapXljBkztHlxcbE2t1qZajKemAEAABiCYgYAAGAIihkAAIAhKGYAAACGoJgBAAAYwk8ppco76MqVK1K7dm1PjMdn9ezZU5vrVq4MGTJEe6yrPsZWqzLPnz+vzRMSErT5pUuXXDIeT7h8+bLliiVTVcd5ZfVvNHToUG3+4osvavPGjRuXyRz4r8yO1f09Z84cbb5s2TJtbjWvqgLmlXe0atVKm+/YsUOb16lTR5tPmjRJm7/77rvavLCwsPzBeUiLFi20+ezZs7V5YmKiNm/durU2z8/Pr9jAXKC8ecUTMwAAAENQzAAAAAxBMQMAADAExQwAAMAQFDMAAABDVPtVmVarWdLT07X5mDFjtPlPfvITbR4QUHY7UqsP+fXr17W51UoZq38Tq1WZVtdt0KCBNs/Ly9PmJmL1mFk6deqkzdevX6/NY2NjnTr/6dOny2TOrsqMiYnR5jVq1NDmGzZs0OaPP/64U9f1Jcwr92rSpIk237VrlzZv2LChNl+1apU2HzZsWMUGZjCr34CQkZGhzf/85z9r865du2pzT6xMZVUmAACAj6CYAQAAGIJiBgAAYAiKGQAAgCEoZgAAAIYou2SwimrevLk2X7t2rTZv06aNS66rW13z4Ycfao/95JNPtPnt27e1eVZWljavW7euY4P7B6vVab60KhNmmTZtmja3Wn351VdfafP58+drc90qtKKiIscG9w8TJkzQ5pMnT9bmVv+H1KxZU5vfuHHDqfGg+rFa5W+1+vLs2bPa/LnnnnPZmEy3fft2bf7FF19o844dO2rzPn36aPPNmzdXbGAuxBMzAAAAQ1DMAAAADEExAwAAMATFDAAAwBAUMwAAAENUub0yBw8erM3feecdbW61V6aV3Nxcp667Z88ep86vExYWps3/7//+T5vHx8drc6t/6gsXLmjzN954Q5tbrZTzJvb0844BAwZo83Xr1mnzEydOaHOrvTXz8/MrNrBKOHDggDbv0KGDNrdaxTl37lyXjclbmFeuYXWPzJgxQ5vfunVLm1utMPzmm28qNrAqpGXLltr8r3/9qza3Wglu9X+RK1dZs1cmAACAj6CYAQAAGIJiBgAAYAiKGQAAgCEoZgAAAIYwfq/M+vXra/NJkyZp8/Hjx2tzPz8/bX758mVt/pvf/EabL168WJu7k9VKHKvVl1bvq5XIyEht/sgjj2hzE1dlwjus9o/099d/zWe1Atgbqy9dxWoFHaqfBx54QJtPnTpVmwcE6D8F//a3v9XmrL60ZrXie/369dp80KBB2rx169ba3Gq1tjvwxAwAAMAQFDMAAABDUMwAAAAMQTEDAAAwBMUMAADAEMasykxJSdHmCxcu1OYtWrTQ5lb7QVrtK2m1h9knn3yizV3B6n21Wh1ptXrEgW1OPXo8UJ66detqc6u9Da1WTbvCq6++qs2bNWumzQ8ePKjNlyxZ4rIxwbc9//zz2txqv+Ndu3Zp8zlz5rhsTNWF1V6W7733njZ/7LHH3DmcSuGJGQAAgCEoZgAAAIagmAEAABiCYgYAAGAIihkAAIAhPL4q02pvsJkzZ2pzq9WXzurfv782v3nzpja32qMzJiZGm/ft21ebP/3002WyWrVqaY8NDAzU5t5y/fp1bw8BhrNaNd2zZ09tnpSUpM3Pnj2rzY8fP67NMzMzyx/cP3Tr1k2bW/3fYrXXbF5enjZnr0yUsPo8U1RUpM2tVl+yIt51tm7dqs2tPvf369dPm7NXJgAAQDVEMQMAADAExQwAAMAQFDMAAABDePyH/61+4LZz585uve6xY8e0ubt/yFL3g8Sm/WDn119/rc1//etfe3gk8DVXrlzR5unp6dp88eLF2vxnP/uZNm/ZsqVTuTt99dVXHr8mfEtERIQ2t/oB9G3btrlzOLgHq0U+Vv+GnsQTMwAAAENQzAAAAAxBMQMAADAExQwAAMAQFDMAAABDeHxV5sWLF7V5bm6uNrfaAslXLFq0qEz2hz/8QXus1cdg+PDh2nz8+PEVHte/2rBhgzY/ffq0S86P6ufQoUPavFevXtr8oYcecur8TZs2LZM1adJEe6zV/zkvv/yyU9c8ePCgU8ej+jl8+LA2Dw8P1+ZWq4u/+eYbl40Jen/5y1+0eZs2bTw8krJ4YgYAAGAIihkAAIAhKGYAAACGoJgBAAAYgmIGAABgCI+vyjx79qw27927tzbv2LGjNg8JCXHqurt379bmOTk5Tp3HnWw2mzZ/9NFHtbnVXl/+/vq+fenSJW2el5dX7tgAV7C6Bz/++GO3XXPt2rVOHV9cXKzNrcYOlNi/f782f/LJJ7V5amqqNmdVpvsdOHBAm48dO9bDIymLJ2YAAACGoJgBAAAYgmIGAABgCIoZAACAIShmAAAAhvD4qkwrVqsjTVo16W5PPPGENrfaA1Appc2tVpX97ne/0+aLFy92YHSAbwoMDHTq+C+++EKb79ixwxXDQRVmtQLYalXmjBkztPn69eu1+YkTJyo2sGps2LBh2nzo0KHa3NlV3O7AEzMAAABDUMwAAAAMQTEDAAAwBMUMAADAEBQzAAAAQxizKrM6eeCBB7T5q6++6tbrfvLJJ249P1AVWO13CJTHakXvZ599ps2TkpK0+ZIlS7T5yJEjtbnVHtTVidVvNViwYIE2r127tjb/6quvXDamiuKJGQAAgCEoZgAAAIagmAEAABiCYgYAAGAIihkAAIAhWJXpBenp6dq8bt26Ljn/gQMHtPlf//pXl5wfMFFISIg2b9WqlYdHgurq73//uzbv16+fNv/LX/6izf/jP/5Dm1vt1zp9+nRtfuPGDW1++PBhbX7y5Elt7k7R0dHavF69etp86tSp2nzQoEHa3N9f//xp7Nix2vzdd9/V5p7EEzMAAABDUMwAAAAMQTEDAAAwBMUMAADAEBQzAAAAQ7Aq080effTRMtnjjz/u1muuWLFCm+fl5bn1uoA3BQcHa/NmzZo5dZ6MjAxXDAcoZbVas02bNtr8D3/4gzbv2rWrNl+/fr1T47l27Zo2z87Oduo8rmC1d3T9+vW1uVJKm3/55ZfafO7cudrc6mN2584dbe5JPDEDAAAwBMUMAADAEBQzAAAAQ1DMAAAADEExAwAAMASrMt1sxowZZbKwsDC3XnP37t1uPT9gopSUFJec58KFCy45D1Aeq9Wa3bt31+a9evXS5gMGDNDmNWvW1OYtWrTQ5hEREWWy1q1ba491VnFxsTb/6quvtPmxY8e0+WuvvabNt27dqs0LCwsdGJ1ZeGIGAABgCIoZAACAIShmAAAAhqCYAQAAGIJiBgAAYAhWZbpI06ZNtbluHzCrvb6cNX/+fG2ek5PjkvMDvuS+++7z9hAAl7BaSfjRRx85lbtCp06dXHIeqz0o//znP7vk/FUJT8wAAAAMQTEDAAAwBMUMAADAEBQzAAAAQ1DMAAAADMGqTBdZs2aN28594sQJbT5v3jy3XRPwNUeOHHHJeaz2ETx48KBLzg/4kgMHDnh7CNUOT8wAAAAMQTEDAAAwBMUMAADAEBQzAAAAQ1DMAAAADMGqTBe5ePFipc9htT/arFmztHleXl6lrwlUFZ999pk2P3/+vDavX7++Nk9MTNTmq1evrtjAAMAJPDEDAAAwBMUMAADAEBQzAAAAQ1DMAAAADEExAwAAMASrMl1k8ODB2jwzM7NMFhISoj12xowZ2pzVYED5rl27ps23bdumzYcNG6bNhwwZos2t5uG+ffscGB0AOIYnZgAAAIagmAEAABiCYgYAAGAIihkAAIAhKGYAAACGYFWmi1jtldm2bVsPjwTAv5owYYI2f+CBB7R5aGioNo+MjHTZmADACk/MAAAADEExAwAAMATFDAAAwBAUMwAAAENQzAAAAAzBqkwAVVp+fr4279ixo4dHAgDl44kZAACAIShmAAAAhqCYAQAAGIJiBgAAYAiHiplSyt3jACrFF+9RXxwzqhdfvEd9ccyoXsq7Rx0qZgUFBS4ZDOAuvniP+uKYUb344j3qi2NG9VLePeqnHPjyori4WHJzc8Vms4mfn5/LBgdUllJKCgoKJCYmRvz9fes788wrmIp5Bbieo/PKoWIGAAAA9/OtL4UAAACqMIoZAACAIShmAAAAhqCYAQAAGIJiBgAAYAiKGQAAgCEoZgAAAIb4f3UkPrBGisG3AAAAAElFTkSuQmCC",
"text/plain": [
"<Figure size 640x480 with 6 Axes>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"fig = pl.figure()\n",
"for i in range(6):\n",
" pl.subplot(2,3,i+1)\n",
" pl.tight_layout()\n",
" pl.imshow(example_data[i][0], cmap='gray', interpolation='none')\n",
" pl.title(\"Prediction: {}\".format(output.data.max(1, keepdim=True)[1][i].item()))\n",
" pl.xticks([])\n",
" pl.yticks([])\n",
"fig"
]
},
{
"cell_type": "markdown",
"id": "7133bd4a-b40d-48ef-9bf7-c1dbaa01df50",
"metadata": {},
"source": [
"**Challenge:** Re-do the plot above, but showcasing only miss-classifications (i.e. cases in which the model did wrong)."
]
},
{
"cell_type": "markdown",
"id": "026ad18c-2610-4c79-b4da-7112e03304de",
"metadata": {},
"source": [
"## Loading trained models"
]
},
{
"cell_type": "markdown",
"id": "9cacec6b-1e09-4c41-92cf-404a4b0a601c",
"metadata": {},
"source": [
"Eventually, you will want to load the model you trained in the past for either running inference or continue the training procedure. The functions we developed above save artifacts contain all of the metadata and data about the model, assuming you have the right model class. Let's inspect those files: "
]
},
{
"cell_type": "code",
"execution_count": 26,
"id": "8ced5e87-eb77-4882-9780-57009de7cf0b",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"model_iteration-1.pth model_iteration-5.pth\t optimizer_iteration-4.pth\n",
"model_iteration-2.pth optimizer_iteration-1.pth optimizer_iteration-5.pth\n",
"model_iteration-3.pth optimizer_iteration-2.pth\n",
"model_iteration-4.pth optimizer_iteration-3.pth\n"
]
}
],
"source": [
"! ls results"
]
},
{
"cell_type": "markdown",
"id": "d1b7317a-647b-4737-9ea4-03a84ee0bc83",
"metadata": {},
"source": [
"To load a model:"
]
},
{
"cell_type": "code",
"execution_count": 27,
"id": "756fdf17-564c-488a-8dfa-1800f051e3c1",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"<All keys matched successfully>"
]
},
"execution_count": 27,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"trained_model = FeedforwardNeuralNetModel()\n",
"model_state_dict = torch.load(\"results/model_iteration-1.pth\")\n",
"trained_model.load_state_dict(model_state_dict)"
]
},
{
"cell_type": "markdown",
"id": "288a08cf-70e3-45a7-a127-a6ddd50b5d07",
"metadata": {},
"source": [
"Before proceeding, let's inspect `model_state_dict`"
]
},
{
"cell_type": "code",
"execution_count": 28,
"id": "05af704e-3d15-4e6c-8e60-ad64cf68c930",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"odict_keys(['fc1.weight', 'fc1.bias'])"
]
},
"execution_count": 28,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"model_state_dict.keys()"
]
},
{
"cell_type": "code",
"execution_count": 29,
"id": "1a281f3e-3a71-42d3-b8c7-3848a0e651f7",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"torch.Size([10, 784])"
]
},
"execution_count": 29,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"model_state_dict['fc1.weight'].shape"
]
},
{
"cell_type": "code",
"execution_count": 36,
"id": "e6a59542-0642-4a1a-87f9-b2147d05c5a0",
"metadata": {},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAn4AAAJ8CAYAAABgGKxrAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjguNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8fJSN1AAAACXBIWXMAAA9hAAAPYQGoP6dpAACZP0lEQVR4nO39d3iWdfo1/C5I74E0QgoJEHrvUgQFC6KIqGNv41jHNjpFnT6O3RnHNuqMvffesNB77yUQSCghpJEE0gPsf/az9/POu9Yt8xzH3uT3Xuvz5xnP5L6v+ypf74N1fjscO3bsGMzMzMzs//E6nugXYGZmZmb//+GFn5mZmVlAeOFnZmZmFhBe+JmZmZkFhBd+ZmZmZgHhhZ+ZmZlZQHjhZ2ZmZhYQXviZmZmZBYQXfmZmZmYBEX68/+GXGw/Q+u8/2iR7Jg7pSutLt/DfBQBZafG0npoYLXuKyw/TekpilOzZX91A66f0TZc9JQebaL1jB9mC6kPNtB4XHSF75q/YQ+sFPVNkz7gC/rMuifrvbNjfSOvZSZGy53DLEVrfVcmPJwBs2VlN6+Hh+v87/nxuP1q/94ONsicjLY7WN20ukz17nj5X/uxEGX7fHFrvnp0ke355cnda/6G4UvaUHWql9cLSOtmzb/8hWh8W4ropKq2l9SvG58ie4mp+3YzMSpQ9n2+uoPWVm8tlT2pqLK2XlfH3CQD3nM/PzZKD/DUDQHE1v9bW76iSPU9eNITWd9Xx+x0AzN/Jj/WUnp1kz7tr+PXRO5PfiwEgNZY/Or7ZyD8DAPjm5tHyZyfKDe/z59eMfmmy5+UVe2l9yUp+3waAsyb2pPVunfUz6vnPttB6YmKM7ImI4PfUob31+wkTD7AL+mfInrfX7af17xftkj1TJ/ag9SsHZ8me33y6gdb379fXgHquTBqZLXsONfJ7YccQD/cJPZJpfU9Ni+z5YO5OWh8r1koAMHtpCa0/ce0I2TNjUKb8GeBv/MzMzMwCwws/MzMzs4Dwws/MzMwsILzwMzMzMwuI4w533PMe/0eWf71wgOwJ68D/YWRWiPDAM5/wf9A6TfzjdQBoaOL/MHNyv1TZM3cx/0eoU/rrf6S+aM0+Wh85oIvsmVDQmdbfWbxb9twxk//j8a/W61BMU9tRWl+zt172TO/D/8Hv5ir9D2e/XV1K6ycP1Mdg7n7+D85j43Vg5wMRJrrtLP6PpAHgqa930HrfvvofKbdH0dFhtL5puw5qPNDGQzd9u+owxOKN/B/1dxDXLQDcc14fWn93Ff/H3gAwKJ8Hj6LD9P93psXxUFJxDQ9JAMCF4hwcnpsgexbuqKH15uY22bNyL78+Lhuo/0H1H7dvo/WrJnWTPTe+vorW/3XlcNlTlsb/Yflj32yXPV1SeSiqXzoPvgBAbjzvKa3j9+L2KjWen2efb9XX2vmD+f1kY6HuWbaJX2sTzukte8LD+X2grk5fA49cPYzWF5XowNZH3/Fz88xeOkyY1Ynfu5+5QQd4fvc+D+bVh7jWYmL45xMfr9cQ08fm0vq5vfRz4N+reDAnIUovkT5dy59RTSIACQD3XsCf7fWt/PkNABuKeMjq1eV8PQI43GFmZmZm/29e+JmZmZkFhBd+ZmZmZgHhhZ+ZmZlZQHjhZ2ZmZhYQXviZmZmZBcRxj3O5bVoBrf/+A71Xb1gYHwtRLfbJBYBfXtif1reW831yAaBHJh9ZsWxnjex57Gcjab3t6DHZk5DA91VMD7En8PvL+L6OPbOSZc/XG/jeoqH291296yCtNzTpqPy5ffk4l09W6Jj4bWfwcSriowYAlI7Np/VUcTwBYNEGPv5geI4ezfGHGX1p/d53+Cii9iozhY/KuH86vzYAoAP4B/D1Dr136kXj+diDL9fqvY17JPHRAqO7671gR3VNpvWl+2pkT88Uvh/p55v0+xmRyf9O/xQ90qarGCmUGcfPWQD4/Td85NT8ZD1i4vpx/Fg/MZvv3QkAV53KX8NfviuUPdMG8XFUD583UPa8KsZU/XuBHjn10NliLEWIURbtUWEZH81TdUg/b7bsraH1Ry8dLHs+2sTv6bO38/s2ALx181haf3yh3g93/i4+OmvOSv4cAoAPfjGJ1i96aoHs+cvl/L0++BUfDRNKXooeGzS1Lx/JtqNSj7QZIvaYfm6FPp/3VfE1SecQz6irx/C9f5/6QV/Tf3xzPa3fch6/ngA90iYvXe+l/WP8jZ+ZmZlZQHjhZ2ZmZhYQXviZmZmZBYQXfmZmZmYB4YWfmZmZWUAcd6p3zb56Wo+I0GvHzp14Mu+XZ/eSPWqz9yMh0rbnD+Obs1c16ETri4v5psyJMTqZd+HYHFqvb9YbLO/ff4jWJ/Tj6TsAKCnnPR1CJGd/Kl5b52j9fn7+Mt8E/pSR/HcBwONf8c3e87N0cvKwSBav3cKTbgBwSGxEHh2uD8Kb4tw5dkyfO+1RUhz/zN7byDcFB4DEaL6he3aIpGmfzjwhvT1LJ+bKG5ppfVx2Z9mzpZpvED8iM0n2vLuef5b7Q0wE+MNXPG372zP0/ebt1fzvHG5qlT1nDODX7pYy/drUhIFQ1/ScbVW0HhPJP2tAXx8VjTqlemoBT2T/Y0el7Fm0t5rWI8P/Z32XUNvQQuu/OKWH7EmI5I/Nj0Lcz64Y0pXWH569Q/b85Ameqr3irN6yp7SGX58F+Tp1P3c3P89GDuavGQCe/pYnV7dv0enhR2+ZQOufrNX3td5pfA2x+6A+n/fX8c90TDf9jHq5hKerK0Pcb2oKkml90oAM2dN5ZBatF1boe25SPL+Hfxzi3Hl8eh/5M8Df+JmZmZkFhhd+ZmZmZgHhhZ+ZmZlZQHjhZ2ZmZhYQXviZmZmZBYQXfmZmZmYB0eHYcc666Hfvt7QeFaUnwtw5reC/fkENbXyT7++38PEBALB4RQmt33f1MNnz4jzec/WEbrInPymO1ucU69f2wzoxlmI/H3EBAPdczDdU/zRE7H3bdh7J7yRG6gBAVBQfC9HaqsfTqPE9XVL4sQGAmUN4vD02XI+lKDrIY/RFlTrGPzSbv4Z3l5fKntm38Y3QT6SlO2po/fPtelxElwS+kXe3JP35P/Al31D9wRkDZM+SfXzsQesRfRuZ2pOPP7n/h0LZc8kIPkriu0J9rR1q5CNY9lUclj3DC9Jofds+vtk9AByq5+MiBuanyJ6MRP75hDpuOw7wEVqNLXpM1b2n8nvuXR/xzeEBYFD3VFq/ZigfPQEATy4ppvVQ43bm3N7+rrXF2/n5/Kdv+bUBANeNy6X1UKOzbnt9Na1fOUWPjclM4L/vzWX7ZE9+Bh/RNDZPjzKZV1RD63sr+fkHAIU7+POme3c91imrcyytv//vz2TPFT8/j9a37dXX56Z1fFTbL68cKXv6pPLX9s6aMtmzbiu/H58/MV/2NLbwZ+vqXfq+lpsWT+stYq0EAG9fOVT+DPA3fmZmZmaB4YWfmZmZWUB44WdmZmYWEF74mZmZmQWEF35mZmZmAaEjuf8hO4unggbmJMuelxbxdE1EiI28K6t4KuxIiPTbjefxFOxf3toge84/tSetr9uvU2kq1dszNVr2pI3NofWSGp4MBIA/vbyS1k8e2132XCTSYZ8u5OllAIiO5qnapiadGrzmFH7c5mzTqaR9dXzj8LomnUoa1pWn00LZfZAf07Y2nVJuj/62gG+AvitE0nTioExaH5im03wnD+hC67//YrPsuWUyPwc/3aATx+pzvnAYf80A0NDKz8GUOJ2cvHk0T+Q/OHu77CkRid/UJH1NH6zlyfKkGH07zU2OovVQG9T37MKvgW2leiLAzW/y9OjvZvSVPUv38N937cvLZc/TVwyn9WWlNbKnPfr1pxtpPTJSTxxQ0wi2Vev0eEQE71krjj0ALBAp9eVLi2TPhjie4v+8QwfZU71sNq3/5JfXyZ5l5TtofXGJTsGm5/B0f7/J42XP+5+to/WrLtATOwbm8WTxC1/pKQJ3nNeH1sf1SJY9h5v455MRzxP8ALB2Hz9HHp3eX/bc9j4/Bo/M4Oue4+Fv/MzMzMwCwgs/MzMzs4Dwws/MzMwsILzwMzMzMwsIL/zMzMzMAsILPzMzM7OAOO5xLrFR/D/dGmK0QOFWPqqgQ0cdLe+cwjckHtqHR8EBoGcKH72QmKhHMnSK5e+nqkGPMmlo4z9btVdvZp3XmY9xeOX9VbLn3mvH0PqBQzw+DgCzVvKNu/sX8A3YAWD5Wt4zZWye7Cmt4yNTwsP0Z/rG7F20/tuZesTE03N5T60YpQEA5Qf4ufjANXz0RHu1aXslrXfuzEc1AEBZHT8uJXX63IyN4P/f91iIMQHqc757kh4bc8fHfKxSaU2j7BmczX/f9F76PvD3hXwMzlUjs2XP66v4NTCpZyfZc/nQrrS+rUqP81gvxkRdPVq/tleW7aX1cwZnyJ752/m4mxeX8NFaAHBavzRaL8jnYzEA4M31+2m9cG+N7Ln7VD5y6kTqnsnPs1tOypM9ja18PNGTn62VPX/4CR/XsXCXfn4qx47q0WbJnfnzs/JAjew58+araP1AiOuzuYH/LD2Hj4gCgMhI/sw9bRi/ngAgfGQWrX+8oFj2/HxaL1r/QdzvAOCZL/iol3vO7yd7bpuQT+u3v85HKgHAXy8eROuHmvW643dn8VEz/1ymR7WN76XvX4C/8TMzMzMLDC/8zMzMzALCCz8zMzOzgPDCz8zMzCwgvPAzMzMzC4jjTvW2HeEb3V8yUidythTydOJPz+KpGwD4ZClPn4WFSAIv23OI1icN44kgACg8wNOOeyp1CnJvFU/mVdTq9NOonFxaj0/i6SsAePClZbTe2qJTvW0tPBXUJ18nWnNzefLnrZe+lT133zWd1ucs0BuHP3rDaFpvC5FOU351tj53VokNsB/6aIvsuXJEzn/9Gv5/7aFLePJrZYgEfddEnub8x1d8M3UA+OdlfKPzMpHYA4ClYlP5pSKJDAA5qfxcv39qb9nTeoSfGw/M0e/nnP48ndorLUH2XDQ0k9a/2Fwhe4Zk83thj05xsqdzLN+4vb5Vp/nGF6TQ+kaREAaAuGh+S79yuL4X7qzl97w7RGoRAO79dBOtHzyoz532qFU81256Uycz6+v5ZINskUQHgK83V9H6vBDJzOum80Rp5DR+fwCAT79aT+s/OXeI7PluMX8N2dlJsmf4OJ40HRti+sbEvGRav/2VEMf6ED/XH/jZSNlTUt1M68P66zR8qXi2767hvwsAosL492adOunpC8p1LyyXPzv9pG60vqdcTxH4Mf7Gz8zMzCwgvPAzMzMzCwgv/MzMzMwCwgs/MzMzs4Dwws/MzMwsILzwMzMzMwuI4x7nUlhUTev3bOCbdQPABafzcQ0FKTruXCEiypkhxhF0SeCjEhbsOCh7zhcbnT/8OY/dA0B1NR9VcOtZPWVPndh8edhgPQZnmzjWmZl6LEVOKh8lsU6M1AGACUP4a8i85BTZs7OSH4OmhibZc+c/F9P6VecNlj156XwEyPPz9fiD9GR+Xv3sDP35tEd3v81HMvTpwcd7AEBCj2RaP1Nscg4ATy7eReu3j9VjPKLD+fX5+HkDZU+SGGWycq++PvOS+Pn8x9P0OJ8Ve/h1s6BEj2ZZs4+PMhmTr0dz5CbE0vp768tkz7jufDRGp2g+hgcANu/nx2dMvh6zsbWcX5/1bXpszMgufKzTq2tLZc89YhTP14X6/tketYmxQb3z9Cb3G8Q9daAYjwUAs1fto/W+IcafpMTxx/Nr3/LfBQBP/2ISrT/xjR6DdOnpBbS+eZ8eHzW6O3+ve0KMP9lfz58RBT06y56aOn6txUfopcsTbyyg9efunKT/TjMflfb1Rv38fGcuv3+ODDE2JrIj/65tXIjRc6eIe/uWvTWy58f4Gz8zMzOzgPDCz8zMzCwgvPAzMzMzCwgv/MzMzMwCwgs/MzMzs4A47lRvYmIUrcfF6VTaoSaeJJu/q1a/oIgwWv9g7k7Zc/cFfDPrZat1+qlBpG2vP6277Hnqi0Jar6zXibm//HMe/ztXnCR7du7mx6euTiemisVm4wdKdXKySqSpqg/rv5ObyhOa3Xryze4BYFBBKq2vLdGvrV92Mq1HR+pTdqVIVdaKTdUB4M6J+vM+UZ68Yhit//6zTbKn3+hsWv/dh7onIYFfu7/+bKPs6ZwYTetNbfz8A4B+Gf99Mi8ynP8/aec4fv4BwLxdNbTeI4W/ZgBYvqWc1lcVdpA9TeK+dvs0no4EgH6deUr4ueW7Zc+wHJ7iD5WcvG1sHq1/sFEndHsk8wR9drK+ty/bx+9RZ/fm13p71Smev0eV9gWARy/m0wju/2ab7Kk52EDrf5jJn10A8OyCYlr/5fl99d9p4unUI+L5AACXDOTTHdam8+sWADaW8TR8WEd93WTF86kLN4zLlT3PL+LXR0Orfub2GcDvhXtCPD/LD/HjtnWHTqm/et1oWi8T6WUA6BTFz7cd+2pkz+/W8Wv3wUv1VIwf42/8zMzMzALCCz8zMzOzgPDCz8zMzCwgvPAzMzMzCwgv/MzMzMwCwgs/MzMzs4A47nEuZ47gmwh3jtG/onsnHt/+1etrZc/5p/Sg9V0VPD4OAHMK+ViQX100QPY89zXftPrxnXyjdwDIzOTjFXZU8o3RAWD6OUNpfXWRjomfP74b//tJepRF+SEeb58xnEf1AeCbjXyURbQYqQMAS9fvp/V4MRYBABJi+Ov+4OPNsifs1P60XhgiXj/tZD6aJTbif9b/3yRG8eM1rp/e/Pve9/UIFuWuKT1pfVcNHz0BAGOzU2g9xBQHHBDjDd5YrUeM3HMKHzGyed8h2aNGSYQaf7JxBR/RhEo9ZgXJfHTR3F5psmXURD466b4ze8ue6sN8DNF932+XPX/8lo8USYoJMXZLjLYa2TVJ9tzzIT/fwjvoE+GsAenyZyfK6m0V/3XPVUP5szApxD2wR09+3awr0+dzbS0/b2saj8ieTDGiaVgffeyTxYiks/p0kT0RHQ/QenhHfa/de4g/J/OS9NiY9Rv4PWJwbrLs6duNX2spsXqtMiazE63vr9GjWXbWHqb1fy/U946SkhpaH9hXfz4HDvC1T3iom+6P+J/1RDQzMzOz/2Ne+JmZmZkFhBd+ZmZmZgHhhZ+ZmZlZQHjhZ2ZmZhYQHY4dO6Z3o/7ffLmRp3hW79eppA8WltD6oxcNkj3LS/nm3x8v2SN7Wlp4yikiRJqzR04yrdc18CRdKImxOs21sZCnxs4dnyd7RmTx9PBGke4BgAsH8KRhUTVPHgHAK8v30fquvfwzAICtcxbR+tjzp8iebVv5udO7j06pniw2e0+J08msrzfwYx0XrXveu3qY/NmJsmInP/7fFOkEYlwkP9dL6/T5XCyS8pEhUt23jOGJ8xVlNbJntEjMfVrIzwsA6J8RR+ux4fq1Ld9bR+uri3nqHwASReL8y9e/lj23//JCWo8M1ym7EV0Taf39dfoY3C0mHISH6fva8r089b66VN87Vu2opPXGRp72BYCwMP5eu2fpJPBbVwyRPztRJvxtIa2HeixePi6H1rdX6AToSbn88/9+hz43fy6uteZWnertkhxN68UVOqnfeIR/zmN78HswAJTX8cTxBxv5MwUAcpL4a1tYzK9bAMiI59fn1gP6uTahRzKt//rJ+bLn7LP4mqS2Xt8/244cpfUDISaQPHQh/zv/WqqTwLePz6f1B3/Q6f5vbh4tfwb4Gz8zMzOzwPDCz8zMzCwgvPAzMzMzCwgv/MzMzMwCwgs/MzMzs4Dwws/MzMwsIPSci/+wvZpHlLt1ipI9vz+vL63/9Bk+EgQALjyjD62fNEBvGK1i1aEG1eSn8Gj5jkq+kTQAfPI135i8e4F+bZeeykcy9Evn4yoAYHEJH+cRapTJW+v5ZtbDuvLRMACQncI3x06O0+NpCrpNo/WIcP3/EIu3bKb12kw9+uFDMQpoQIEeMXDjOD7+QJ277dXS0mpaj47Q40KyEvl1uPmAHuNw7Wg+luKtNftlz/1ihEBzmx4xcV4/Pmqo9Yi+QNUG8ZWH9HgFNc6lLMToh+VFZfwHNaIOYJzYIP6LrXwsCgDMquef6RXDusqe2bv4+J7UWD7iAgC6xsfQenSuHoOTHM1/NlFsdg/oe+sHm/V4mvbouon8nvHmMj2WJDeB3zffXLxX9sxew+/Paan8dwGAutqfW6FHm90rRgCN6M5HKgHAKyuKaT1+tz7P1NiYG8fw0SMA8NIKfk+flK+fAx068KOwqVSPkYsRz6K8An2tnT+IjxY73NIqe0ZmpdD6d0XlsmdHDb8XHQjxjJq/m987Qo2N+TH+xs/MzMwsILzwMzMzMwsIL/zMzMzMAsILPzMzM7OA8MLPzMzMLCCOO9X7zUaeMOuTxTefBoDTe/DUS5++PEEDAJVi8+fEEEm22ct5ykkEggAAu/N4ymnuJ3zTbgBI6ckTU9kZOjmblcRf95hcnZjbU8uTxTP761RSutgA+/21OgGWlcTTu0srdVro4GH++dTU6A3K4/P4cdu0eIPsGTllGK0PzdHH+sZnF9P6qWO7y572qFsST2Y2hkjOrtrH02JTCnSa7zfvrqf1u87pJXvSY/l51hDitbWI9G7vNP4+AeCddTwheeXwXNlzxZAsWn/i2e9lT3Zffm5UdjxJ9yTy1/3Eef1lz7qSGlpPjNH3tc+28ZRwVnf+GQDAilI+EaBvqp4ioBKSKlkPAD85OY/WT+upz7f26M9v83vQyEE8iQ4A2w/y++NfpvEpFgDQdIRfH5WN/H4KAI0tvCc+Sie0D9bzFGrpQX1/HpDCU7X9s/Wz/c7P+KSGG0fySQEAcHqPdFp/e4NOUE/I5efT4Sadtj0iIudniSkGgH7mfriCp7EB4KWO/NlaGeL5OV7co5qa9P3zpOxkWh94nr6mf4y/8TMzMzMLCC/8zMzMzALCCz8zMzOzgPDCz8zMzCwgvPAzMzMzCwgv/MzMzMwCosOxY2q77f+rXr/+htYLuuuxJJu28s2KU0OMFrh0At80+2/vrJM9mVnJtD64IFX2rC3koxImDOKbwwNA21Fen1qgj8FXhVW0fvNoPZYiJpLH9UNF8tXomp4Z8bLnyUW7aH3RNj66BwBG9eTH9KlH3pI9p19xNq0fOapPvdP6pdH639/fKHuGihEMG7boTbOL/naW/NmJsnwnH8lxzYvLZE+vfH4Oqk3OASAtkY8FKRejDQAgXYyaCeuo/05MBP//yzML+LgnQI/MyIiLkj1tR/kF+uy8YtmzbO4mWo+M1aNmfnf9WFq/KsSomfJaPrZjbom+1qrq22h93lbdk53C762REXoEiProrh2WLXvWV/BzdG2pHmXxzMx+8mcnyvjHFtB61xDPqFbxIDhQ3SB7jop7XZcQf+fqUXz0R0RH/X3NhnI+1mnZzoOy5+Lh/L4ZE67PmddX8jEn1WLcFwCM6sHvUckxeqrcqMxkWn9hpR5TVtfAR73cOj5P9kSG8WOaFq/vNze+s5bWQ402O3RI/0w5/9SetL6muFr2zL6N36P+F3/jZ2ZmZhYQXviZmZmZBYQXfmZmZmYB4YWfmZmZWUB44WdmZmYWEDpO8x9GDORp1x17amTP5Wfyzd4Hd9FJ0zdW8A2b+w/Qm2ZXH+QpxF0H+ObjAFBZydNP78/aKnuysvmG0VcP6Sp7+qTzdKDagBsAtlby1x0qgH1mX/753PD+etlz13i+Qf3HC4plz+zDLbQe31Mn9qJEonD2vELZ06cr3yD8iZ+NkD1Pzy+m9c6dY2VPe1TZwJNxz181Uvbc+wVPp3YVKU8AuEQk2L/byZPoAFBUzlObibERsqesjp8zOcn6c6lv5YnWk3vwtDcAPLmQp9RDScniG8dndk2WPQXifIoM1/8frcLVIzL4PQUA1NV+end9DB6YvZ3WVRIVAGpF0nBPiGkFrywoofVLxubInvaoTRyXYblJsievE0/D/+HtDbJnxsn5tL6ltE72HGrh10B4iAR9m0gPd0vT94Envy+i9Tpx3QLAyP4ZtP7NN/oYTO43jtZnb9YpdTUtoDnE83P7Tp52TTy1QPb8cxk/n28bmyd7fjKap65TQ9wLF+zin/epPZJlzxsiQX3/Wf/nKXl/42dmZmYWEF74mZmZmQWEF35mZmZmAeGFn5mZmVlAeOFnZmZmFhBe+JmZmZkFxHGPc5m3hI9KOPsUPrIFAF79Ygutp6YlyJ4Lx/ONzg816/j2k3P53xk1vrfsKd9dRus3XzNB9nyxqJjWP9zCfxcAnFPAx0XsqtWbmZ/em0fl91bzsTUAsGBHJa3/dITeaP319TwmPjbEeJo331zIfxCmI+yff7aG1qNi9QbYw7P4yB81sgXQG6Hffibf5Lq92lrFRw2p0QYAMKEvP896p/FxQgDw/qYDtH6e+F0AUFXPN0C/Y1ye7KkVm6bHRupN4Dcc4NfHRxvKZc+YfD6C47AYQQQAVUt+oPWz7rlB9hxs5u8nWowtAoB1B2povbBCX9MTuvFRL/sO6x51ivToou+5TSl8PM3KfXoc1vWT8mg9KUrfB9qjXHFcEqP1ZxkmZvOo+w8A7KluoPULh/GRSgBQJs7bA4f4+QcAnWP5I319yUHZkxDH78OrF2+TPcrUqQPlz+574H1aTx8wQPaEh/HvpjZt1feB2NhIWlejbgDgimH8mbf9oL4Genfmz6iFe/Sx3lDMR81M76NHNP1mEn9+tRzRI5p+jL/xMzMzMwsIL/zMzMzMAsILPzMzM7OA8MLPzMzMLCC88DMzMzMLiA7Hjh3TUZf/zcTHF9H6BSN1AnTONp5gqW/SqaQtIq0zZRzf5BoAjoi0znuP/kv2IH8oLccl6c2s+w/iG5Dffmp32bN4dy2t/2piD9mzrKSK1rdV6STwsp01tD6pt95ovaqebwL+xnc7ZM/lp/GEUVSIDeqjw3kKbk+NTlseauavbVg2T1IBQHUD7+kXYoPyGYMy5c9OlD53z6L1n5yir4HPluyh9YgQSdPfTOOJ/I0iUQsAAzL4sWwKkTBbupsn4zrH6tc2vRdPtm+q5NcTAKzax193SQVPSQPA7B820/rXf5kueyLC+PkcGeIaqKhvpvWOIiEKAN/v5PeBLgk6OTtnG+/JT9fXTXI0T4JWiAQ3AGwUKdE+2cmy5/kL+8ufnSivr+TXzZOzimRPtUjoNtY3yZ6TT+LX7ul99f355/e+Q+s33KrPzY++307rR9r0VIyYuGharwuRaJ1+Rj9aXyEmBQBAdiZPUM9++2vZc+H1M/hra9Tn5pr1+2k9KlpfN7+9kL+fsI76mr7zn4tpvVt3PRXhhin8PPjdiytkz/O3jKf1u17j0zIAYMdjU+XPAH/jZ2ZmZhYYXviZmZmZBYQXfmZmZmYB4YWfmZmZWUB44WdmZmYWEF74mZmZmQXEcY9zGfrn2bR+6lA9zqVvBt8g/vP1eoPl84fyTat/+ewS2dMoNi2feS4f2QIAVYd49D4hRke+C8Qoi+oGHZW/ZUwurbe06fEXr67ZR+tjchNlT9lhPi4iOsSIib99spXW09P1+JM+Ocm8J4FvjA0AczaU0XqY2IAbANrE8UmM5xuKA3oj+vF5+rhdMixb/uxEGf/YAlqfMlBv6H64hZ+Dc9fx0QYA8Nxlw2j9y+36+lyyg49o+suZfWRPVSM/N2ua9TifOUU1tL45xGbz5eV8nEt1Bf9dAHD5jMG0HuLyxM2j+TW9dG+l7BnZNYXWO8Xp+83sIv45LC6ukz39u8TSek2jvkc1i1E8a4prZI+61i7qr8/RcQWd5M9OlEF/+J7WO4QYs3PeWP75b6/QY5Cm9k2ldTUSBABS0pNovXjFWtlzxsWTaX3Wa5/LHsTzc7PXSD1+p3AVf3aMO5PfUwAgQtzvO4W4p8eLUUPri/jYIgCYMYrf0ytDjCd69eN1tJ6Vyz83ABjel49tqT7E73cAECvez9Zd+r7WIu7tbSFG9Gx58Az5M8Df+JmZmZkFhhd+ZmZmZgHhhZ+ZmZlZQHjhZ2ZmZhYQXviZmZmZBQSPmBAq/Ft5WCfzwjJ5wmzLdp3IeUok8+69aoTsWVbMN27ftlsnZaKi+FufKzZtB4DbrxlL61/M0xt6D8/mCdkPV/OkKwAMyOZprooGfaz/+WUhrV93ZoHsOXtcN1pfGyI5+dXcHbR+0Rk61Tl9RBatJ0SFyZ5DzTyx1Nii45aqpyrEht7t0W+n9qb1WTv0dZMYzY9lszgmALB8P/+cVXIXAB6fMYDWYyL1Z1lSx6/pvMR42TMwk39mB2p4gh8ArhifQ+sPvLVB9ozI5unULzdVyJ4NB2r478rsLHsyk6Np/autOnXdL4Wn0d9aXip7rhvBj8HqMn1Nd47hifxRWToN3yAShd/t1Mnm9pjqve8Cfj4v2s2fKQCwqfQQrY/K5/dtALj7pZW03m8gvzcCQHEx/8yuuGGa7Hn96Q9pfdrV02XPinX8fCrolix7Cj/bQuvbtuopCWr6xr3XjJI9P2zh51PHjjp1/ewn/BleW60/08HD82n9cIj1zVez+TN33Og82ZMcy6+1+2fy8xAAkqN48n97DT8Pj4e/8TMzMzMLCC/8zMzMzALCCz8zMzOzgPDCz8zMzCwgvPAzMzMzCwgv/MzMzMwCosMxNaflP5z+zFJa75IcI3smFSTT+tEQf/GrjTy+HR6m49vzF++k9YLeesPwfLHJeEuI3dmrD/PNl1MT+agGAGhsbqP13pn87wPAyl18nMaIfD0u4p3vttN6Tk6y7Bkkft/H3/INuAGgd19+TM8fkSl72o7wD/y1ucWyJz6ex94PHDgseyaO4KMEFq/TIzM2/vU0+bMT5bZP+KiEplY9mmVfVQOt/2S4vgbSY/l5W97QJHteX7aP1q8bxzeuB4B+aXzMxa0f8I3RASArlY9BCnW3io7gI2V2ltXJnjsn96D1jzeWy54wsdn8z0frY/DhFj6+qaVNv6H1u2to/coxegRIajTf8L4sxGeaFsN7vi/SY32+Wbab1kN9PpsfOF3/8AS56q31tB7qeaPUNeixUWrc0ck9k2XPrXf+i9Zn3HCB7FmwrITWq0r4dQsAUUn8+hw2urvsObl3Gq3/4/VlsmfamXxkybK1+v7cPZ+PAGoSz1UAaGriP1PPFABISeD3wuVr9HErX7mY1uN6D5Y9bS38tcUm8NF3AJDUid8L/3Bhf9lzyTA9VgfwN35mZmZmgeGFn5mZmVlAeOFnZmZmFhBe+JmZmZkFhBd+ZmZmZgFx3KnesY/Mp/X4OJ2UGZSbTOvvf88TqAAwcnBXWs9L48kWACiu4JvAz5q1UfaER4bzv9NTp1O3/jCP1jsPGCJ7zjutD60Py9Ipnoc+4KnOyjKdsrvlshG0/sWyvbKnW1e+CXtmJ/3aoiL4/yts2as3wI4WibbiED0pKfw1tLToZOuePTW03qsgVfZ8+/Mx8mcnykkP8/NsWAFP0gFAjPhcVhZVyZ5ocQ2c3l//nXSRjFu4S3+W2ck8NTo4QyfbD4gU6isLeJoUAJLi+d/5xUSdTqxq4kn97Hh9DRwRt8znRNIVAO6fyu8DTy4ulj3XimTezoM62b67rpHWi6v5+wSA1Di+CXxWor63q4dGfAQ/pwBg5mB9bz1R3lrN749Ldx+SPSOy42l9V7VOTte38GkR/3xtiewZMoonzle9/5nsueyuq2g9I0F/lh/M5VMx4kI8288dk0Pr/dL1c3qeuEfkdebXLaC/mXp7ob7WhovE8asvfid7rvjpFFpvbNbPm8Ui8dvSrNPdE0Z3o/XLh/J1DwB8s4Pfw7uHOG53nKzveYC/8TMzMzMLDC/8zMzMzALCCz8zMzOzgPDCz8zMzCwgvPAzMzMzCwgv/MzMzMwC4rjHucx8cRWtD87hI0EA4N25u2i9f4jxGjvFiI+Kch2vnzG5gNa/mM//PgCMEqMFPv14hezBMR7JHztFb8q8fz9/3QVi82kAiBKbzTe36mh5Sxt/bQWZ+vOZlJ9M62X1evTD+lI+OiczxOiH9+cX03priPcTE8NHTCQk6Aj7z0/Np/Vfv8TPXQDY/dR0+bMT5auN5bTeclQfr9++u4HW75nZT/bkio3B//AVHycEAJmdec/+6gbZ89ez+Gs43Ko3Wj/cykcixITzawMAyhv5ebtxv35tu6v4z341kY/SAIDnV/BREtcM1Rujf7urgtbrmvRnGh3O/798f62+PrunxtD63C387wNA9y58rM6qLfw8BIBhfdJp/fTenWXPxUOz5M9OlL/P46NMapr0ubmllI/T6d9Vjyfqk86vm+KDegRMV3FPjQrT39fc+8Y6Wu/YsYPsUSuAM8by0SMAcEYB/5yzxT0FAF5bx8efXDVEnxe/+Jjf164dnyt74sRIoZ01fNQRAPzluYW0Hhmln2u9+/MRLFlpfNwPACxbzY9BeojRVmFh/LO756zesufcQV3kzwB/42dmZmYWGF74mZmZmQWEF35mZmZmAeGFn5mZmVlAeOFnZmZmFhB6R+3/cOVInmBZWarTtnfP7Evrry7hG2MDwG+n883Md9fqRE58FE/69Tqf/30AKD/MU4ORsTwVBwCjx/H0cNHOatlzzdRetH7/U9/LnjPPGUHrM4ZkyJ4H399M6wNzkmXPL19dQ+t/vXyQ7JnRl6f5altaZE9qKk969cvRyeapvVJo/cVle2TP99sP0nrPECny9uiR77fT+pB8nZhUiqr0dXP/B5to/b6LBsqeFxbz43/nZJ2C/fmbq2k9L1snzqtqedrxb+fp19YjhafpFhTxSQEAcO4gfj7f/0Oh7LlyFE/vbq/V98JIkcTcUVYje5QxPfR50DmW39LHFPDrCQDWlvDX8MQlQ2XPhkp+TPMT42RPe7RqTx2tL17F05cA8ORP+f35pueWyJ4nrh/9370wACv38vRwubg2AGDcMJ6QvX6kTsG+s6mM1num6AkK0WH8mds9XX/+O8r49dHQopPtaiLAz15aLnt+d2F/Wn/tB57gBoD8Xvy4/Wq6Ts4mRfLJE5VNOnXf0MzT4hs2HZA9503uSeuF1XzCxvHwN35mZmZmAeGFn5mZmVlAeOFnZmZmFhBe+JmZmZkFhBd+ZmZmZgHhhZ+ZmZlZQHQ4dkxt0fx/9Y/5PAqdEssjzQDwyKfbaP2qKd1lzyvfFdF6S4jI9+9/wuPbFfV8ZAsArNzNY/xXijg8ADwhjsHE3v/9uJC8TtHyZ8/OLab1n03QkfykKP45rNmvR0zkJPMNqHMTdCT/QTFqZHK/NNnztng/oU69O87hY3Bajuie8kP88/58uR4ftOr3p8ifnSg3ijErmWLTdgBYtL2K1g/W6NEPf7uQj+0prtNjAvbW8lEFh5r19dklgb/u3p31Zub1bXzsQUWDHpUwa1MlrV8+Ul/Tr4lz4+5T+OgmAPhiezmtT+2hr4HoCP7/2G9u2C97Rmbz4/NMiLEUra1Hec/FejTLp4V8lER8JB/ZAQA9O/OxV7HhekLYWQP46JwT6dG5/HnTO0XfA29+bimtz5iiR3/M6MPPjcY2fd089A1/fnZK1M+O1jb++V84IlP2JItnx6oQo9oKUvlryIzT49Dml/BxW1N76usmoiO/btaU18ievp0TaP2zwgrZU1PPx5H9sLhY9gwbzEfcZXXm48sA4CwxpqykTo/dKq3jr+2k7CTZM22AHv0G+Bs/MzMzs8Dwws/MzMwsILzwMzMzMwsIL/zMzMzMAsILPzMzM7OA0BGs/7C+lCf9ckOkU8PD+bryifc3yJ6ERJ4K+vPFA2SP2sy6QwfZglUb+cbUpxR0kj2XjOQpnqUlOv3UNYknGt9eWSp7emTyzes7hHhDf/p4M61HhkjmdU3jqcFrR+vPVKWr351fInuyu/KU1dieeuP42YU8Ada3i06CvvTFFlo/epQn3dqrk7rx4/XxOp4mBXSS7MhRnYL+didPwa7axY89AOSk8rRjvwydZKtr4udMaqxOKW8o4an7vGSdGrx0BL8+Qzl7IE+atoY4Z9Qx/X4XP54AkBrPb7Xz1ur7wOqiKFrvnqnTfCf3TKb1l9fskz3TevPr8KFZPMEPADUiLX7RxDzZ0x5TvavFdIfXZ++SPTm5/BkxMFOfm8tLa2j92Y95gh8Arj2nH62fGSI9/sEW/lzbWNYge4Z25dd0XIhnR9/O/Bx8crE+bteOzOE9C3XPzr21tH7fTD7JAwCKavl6oL6ZTwoAgIQYnmzOzkmWPfnp/FlUWacnD2ys4K9tQSGfygAAYR35c39St86y58f4Gz8zMzOzgPDCz8zMzCwgvPAzMzMzCwgv/MzMzMwCwgs/MzMzs4Dwws/MzMwsII57nEukGM3SrTMfOQAAHUUMOVRE+piYPqGHUgDrdvPxE1P66fEBj1w2hNYPterI94sLd9N6hDg2ADC1Vz6tf7lKj1coKqmh9eViBA0AxMfz0Rhj+upjMGc1HyVx13Y9lqKTGN/zy3N6yZ6iKj764fXvdsgede5sLtKx9xtn8PEH5YdbZU979OwPfLzB4cN8s24AmDG+G62PzdejPzaX843B1cgWAPjp0Gxa/6pIj5rJTeb3iF99pkdZTOiTSuvPLdBjg2pr+Xl2w5TusmfFHj6KSW2MDgD9xeiamHA9/mLpHj42pG93PZJhoBiDtLNKb+j+1jJ+X0mO06NzXlzGf19eBv/7AHDlWb1p/VCLvn+2R3vK+XiN88X1BACvfsPH3AxN16PAPtjM790FvfRolk+X7qH1UPez8fl8FFhajH5Oq2fuwx9vlT2TruYjgCb10sfgvm+20frRECOnnrx4KK3/c5m+D0SJ5/Hc5fx4AsC5k/g9IjaWj3kBgDAxXW3Rav1sz0zuQetXjMqSPZ9urKD1sgZ+vzse/sbPzMzMLCC88DMzMzMLCC/8zMzMzALCCz8zMzOzgPDCz8zMzCwgjjvVO16kA2uadMKovp4n4+4+Z6DsWSbSb2+v3C97LhieSetfrNdJw721/LVtLKmWPTeczJNeH649IHuOiTxyk9i4HgBqa/iG2j+bzlOrALBhr9jUvpNOc916Vk9aP9ysX1ubCGCFOgZdkvnm5cMHZMie6QN42u1wi35tKTE8uZiT9D8rafgX8TnfP6tQ9kwVG7fXh0ipv7FkL62Hh0ipdxBJtsXbddo6cSD/nPNDpEaHZ/J0YnSI17ajgqdTX5pXLHtiY/k5M7mnTieq9O7yffwaBIDSGp7Aq6rTybzJY/lEgIZWfa0lRPGU8JCuOqn91LdFtH7ZBJ1sTYzkacf/aaneLp15QvvjxXyCAwA897ORtF4RImWp7qmD83SqO2swPzf/9aW+D2zdx6+b6UO6yJ5Vu3my/bwJebLnQCN/rx+u0pMnfjGZJ2c/FqlVALj17TW0/sylPO0LAK+u5dMqjqn4MoD1JXwyyAUjusqeJTtraf3Nm8fKnqeXFtP6R3N3yh6lc7x+tl8+PHSvv/EzMzMzCwgv/MzMzMwCwgs/MzMzs4Dwws/MzMwsILzwMzMzMwsIL/zMzMzMAqLDsVAZZzMzMzP7fwx/42dmZmYWEF74mZmZmQWEF35mZmZmAeGFn5mZmVlAeOFnZmZmFhBe+JmZmZkFhBd+ZmZmZgHhhZ+ZmZlZQHjhZ2ZmZhYQXviZmZmZBYQXfmZmZmYB4YWfmZmZWUB44WdmZmYWEF74mZmZmQWEF35mZmZmAeGFn5mZmVlAeOFnZmZmFhBe+JmZmZkFhBd+ZmZmZgHhhZ+ZmZlZQHjhZ2ZmZhYQXviZmZmZBYQXfmZmZmYB4YWfmZmZWUB44WdmZmYWEF74mZmZmQWEF35mZmZmAeGFn5mZmVlAeOFnZmZmFhBe+JmZmZkFhBd+ZmZmZgHhhZ+ZmZlZQHjhZ2ZmZhYQXviZmZmZBYQXfmZmZmYBEX68/+E5/1pB64Nzk2RPQlQYrZdUN8ueCfmJtP7asn2yp7n1CK2fPiBD9rQeOUbrAzPiZM+nmypovaymUfYU7qii9VdvOEn2NB3h7+eDTQdkzw/LdtP6KaNyZE/1If45xEVHyJ7MpCha/251qew5epQf61+d00v2dOzQgdYTI/RrW11WR+vlh1tlz5Mz+sqfnSjXvruB1tWxB4CNe/l7j47g1yAA9OrCz/WWNv55AUBdEz83p/TsJHvm7qqh9Ql5+t7x0OfbaL1Tp2jZMziXv4bFW/R10y+vM60vXb9f/50+6bReVdckezKSY2h9UkGy7OkaF0vrmysPyZ4355fQenKyPm6n9efvp2tipOy5//1NtD5uaJbseenigfJnJ8ra3fxYRkfo70T+vnAXrZ/RK0X2dI3jn/9dH62XPd26JNB6c+tR2XP9mFxa31lbL3v21bbQen5nfb9ZvIvfb07vza8nADh6jN9XGsTzGwAyYvl5mx6rX9uGylpaT4vRPW3itcWG6/vnBxv5feXk7vq+9s5Kfl/pl6V7xnXjP3ts1nbZs/jXJ8ufAf7Gz8zMzCwwvPAzMzMzCwgv/MzMzMwCwgs/MzMzs4Dwws/MzMwsII471VtUfJDW7zmlp+y59Z21tP6Lswpkzw87ami9d1ee9gWAVSI5OzxT99S3ttH6t9v5+wSA4vLDtH7FSTrJ9rlIVS7cWy17xmfzZFRLm05zXX4a/xy+W18me1padJpKKQzjadv7zusve5KieBL3xVV7ZU9NPU+a/Wy0Tim/M5en7c4SSbf2akY/nrLcXMnPPwBITeTpt6RofYknip8tLdLXwMwhPCn/4pI9sueyUV1p/cHPtsoeZc8entgDgLsn8/tKZpJOp9Y08vvA3y4dKnv+MW8nrU8dyD83ANheyRO/76zQ6eEbxnej9XX79Hlw5aQ8Wn97EU/9A8Cbc/h1c774+wDwuwv59f7CAv132qOig/xY9uwUL3sqavlnGSo1+sjcHbSelKB7LhzchdYjOurva74orKT1+iY92WD/QT6VovVIsuzpncET58+LVDkAjCngqeef9M+UPYXVPHW9plxP0th8gP/splGpskcMkUC5mHwBAPkpPKn9ybpy2fPQtH60/k8xlQMAdoqpIVFRx718+7/xN35mZmZmAeGFn5mZmVlAeOFnZmZmFhBe+JmZmZkFhBd+ZmZmZgHhhZ+ZmZlZQBx3Hvj3F/AY8m3vrpU99WIkx7db+fgVABiUzTem3rxfbzL969P5GIc/fr5F9hwTmzKfMYSPngCA6VP437nngw2y55Gf8I3Jy+r1hu73fM43QL9uoh6v8P02Ph6mtlbH0aeP5WNOYkJsUP6q2Bj6nXV6LEVtAz8PJobY1HxwBo/4v7pqn+z57cy+tN5yRI/BaY/q2/iIkVe+5SMhACAmho/MyUiLkz35KXz8SMl+PkIBAP5eyjdnP3OEHmnUKo7/mP58XAUArNjKRyLExurRLGokw5tzi2XP3dN70/p7G/QYpPOG8ZE2MSE2dE+O4Z9pzy565FS4eEMDuurPVI1tGdVHj5rp34WPpXjmK32+PXLRYFo/tb/+O+1RajQfp7JDjHkBgG7imkqN16NZwsP4PfWWMXo81ZK9fHTRWT3SZM/2Ut7TLydZ9vRI46NZxmTpnuRofr/pKK5BAJiUy8epLN+vR5u9J8YdDc/nI88A4OS8JFrfXqXva19t42uSG0bqz2daAT/X0+L1suohMdbnxtH62V5cx8/F34j1yPHwN35mZmZmAeGFn5mZmVlAeOFnZmZmFhBe+JmZmZkFhBd+ZmZmZgFx3KneFJF+UsldALj/kkG0/ubKUtmzejdPDRaW6I3jd4gk06/P1KmXfy7gm0l366yTWR9v5knDK8TG6AAgwsP4cO0B2XPHlB60/uvX1siefr150uv6M/jvAoC0OJ6QzI7jKS8AyPkJ74mP5CkvAPhqK984fHkx/9wA4N/f8PTwmEEhNvSu5JtZd03USdD26KFPt9L6SYN14rxvF/6ZJUfrS7y0jl+7N0zJlz0byxpo/eMFxbKnSxee1J86iKdjAWBbIr8O/3gGT+ECwKwifp7Fx+vPf95Ofg5eESLd//4mfu0WlfF7FwCM6M5TiNvEvQsApvXmqfdPVuhk+1/P7U/rf5tbJHtO68Ff28UT82TPrCKegowKCxHrbIeeWVJM69eP5hMPAGCwSFX/bcFO2XPNyGxaf+CbbbLnr+fwSRrljXoixJT+/DkwJEOnx3M78ffT0MyT6AAQFcET7LM28mckAESH8++ZBqfzFC4ADOnG32tuJ31Np8VE07r6rAHgrgndab31iHiAA7j/h0Ja7xwi3R0Tye/HH27R64Eze/L7QLdO+jn9Y/yNn5mZmVlAeOFnZmZmFhBe+JmZmZkFhBd+ZmZmZgHhhZ+ZmZlZQHjhZ2ZmZhYQxz3OZZ4Yp5KRES97MmN5rPr28XpcxEOz+RiPC8fpeL2KXNc2t8qeC4fzsSBtIeLbXZN4hDwhSm/O/kUhHzGxdHmx7BlfwMcrhImNvgHgprF8k+f3Q2w2nyLGXCR311H5DWKcx+w1ekTPL87iY3Ue+HCz7JkxIY/WU+JCjI1Zx99rZSV/zQBw3Ri9OfaJMnlYFq2flKtHMrSJuUFP/6BHTGSn8Wu3pFrfFjKT+KiCQWKcEABEiPM21LU2NK8TrauRLQCwdAcfMfLUhYNlT70YWVHTrMdUtRw5Suu/mKRHJ1WIERw5yXo80XdFfPP6e6fqkTYvLN9D62Xl9bLn4Vl8LMUvTuspe44e479vzpYK2dMeje2eTOuvrtIjcxpb+DkzfVC67DkGfq5ffBIf8wIAD8/eQeszh3aRPd9t4sd/YJq+d7y3gd+7pxXo97NsH7/WuoYYMXJmAR/f9KvP9XNA3Tt2HDgke7qN4a8h1HgaNU7l1Hz+LAaAkn18fNN9146SPTe/u5bWR/bS98+XxDWdnaKP9T/O7St/BvgbPzMzM7PA8MLPzMzMLCC88DMzMzMLCC/8zMzMzALCCz8zMzOzgDjuVO9H83fRekqIZEnLUZ5+m1vM02oAkJ4UQ+sRHfXm3+8t3E3r/UIkcgZk8UTj0iKeXgaACLHJ9GMvbZA9U88cSOt53XWK5+VveZqrOURKeftBnrJbun6/7Pn9hXxD96MiIQoAO0U68L6Z/HcBwCPf8ffTt2eq7OmfwTcOLz6oNyi/ZBRPw26v1D3tkUqsbdpbK3vCxPUxZWCIBKA4N6qrG2VPT3FNDcjWqcENe/jr/mI1T8UBwO+m9qH1z7fp1Ohdp/BU7VKRQASAEV14eripgd+7AGBwV35utor7HQDEhPPk/4vi3gUAp/TjqcpfvrlW9pw/kU9MWBki3T+kF78Oa0Mkm8sP83vR6QN0ErQ9Oq0Hf71bDuhr4LaxebReVq/vM4daeaJ0SFqy7EkZzScYfLpBXwM9u/DrMCOep/EBYLu4p38Xrv9OUjQ/n0/umSx7qg/z8+nhs/vJno8383vUrA08hQsAqTH8ve4sqdGvrZZ/dl8v44laAAgX64HL/rVM9kwY0pXWT83n9yEASOzNz9HE6ONevv3f+Bs/MzMzs4Dwws/MzMwsILzwMzMzMwsIL/zMzMzMAsILPzMzM7OA8MLPzMzMLCCOOw986sgcWldjJACgsrGZ1sdkJ8ue7inRtP72Cr6RNAAkJvL4dlQkj5wDwKcr+CbcVVUNsmf3Uh7TvuC6GbJn534+yqJWxMcBICKCv+4hA/RojgdfW0Xr50/VUfkP1vBI/NT+eszKqnX8c7hyJI+pA8AlY/iYlddDjLJ4feleWv/9aXqD+l99vJ7Wb5rcXfa0R4kxkbRefYRfTwBwy4Q8Wn97rR7nExXFL/8bz+olezaV8dEPi7dVyp6bJ/HX9sgXhbJHbWo/KoePYQKApftqaH18jh6V0C2Vj6NKE/cUANhcxsfQhBqDVCjuKzU1+j5Q13SE1nOyk2RPlwQ+AmTmJH0NfLmkhNbLavRIk+4ZCbQeauxWe/T1jnJaz07Wn/+nhfy+eU6BHmXz8Fx+P7vrZP25DM3k5+0Li/SIkSE5fJzLn77dJnsm9Eqh9U9X6mfu+eJ+372THu/2r5X8dfdK1z3DM/n7yRTnOQD89Xt9X1GunJBL65X1fAwPADz/ySZaPxbiPqDuERWN+j7wpbi37hRjvwDg25+PkT8D/I2fmZmZWWB44WdmZmYWEF74mZmZmQWEF35mZmZmAeGFn5mZmVlAHHeqNyGKJ03Xhtj4uKmVp9IWr9NJwwtPzqP1cwZnyJ4PV/Hft1RsQg8Apbv5BtQXnDNI9hSJNF1KAk9hAkDfTJ5+ihQbPANApdgAvUJscg0AiZ142vFgve7ZuaeG1h/ewpNuAPDwNcNo/cP1uqemgb+Gx88fLHvWlB+k9V+8v072/GF6X1r/5RtrZM+VI3ha/UQ6d0AarT+3kKcvAWDHQZ62TYzV5+YVvXhq8MAhfv4BwFERWBvavbPsmVNUQ+unD9dJ8NdW8dR9v0yeJgWAkiqejBsf4iNOiOHpwOp6fQxK6nhCd1iGTg8fPcYTeIN78c8aAHKS+Wf35uc8IQoAuWn8PjCmmz5uQ87nyf829WEDyIzj0xfu+XCj7LlzYvtL14/P4uetSu4CwO5qnnb+sE33HDzME/lpCTo9/Mj8nbR+7Th9QmfF8YRsdYNOpyZE8WdRfYhnR/80fj7N2lEle6b35ef614W6Z81unqBvaTsqezrH82N6++VDZc/f5hTR+oCcZNkTn8CvgRhxTwGAIVlxtJ4cpe/T0/vw45Yzppvs+TH+xs/MzMwsILzwMzMzMwsIL/zMzMzMAsILPzMzM7OA8MLPzMzMLCC88DMzMzMLiOMe51IpxhukhNjMfNFavsnzM1cOlz1X/XMxrf/9pyNkj9rofPTALrJnhdhMPE5sXA8Aj5zTn9Y/3lome1YW19L6Gf35xtgAMKJLKq3/a7nenDs3l4+aKa/VG61npPFo+ej++rh9uYlvGN0tNUb2tJXz6P1r6/Qm4Op1nz5MjwBZIEYLxYYYadIe/ekDvvn3/T8ZKHuqmvi4iMJSfv4BQFocP9ff+oGPNgCA28/tQ+s1jXpcREEKHzGx86A+Nw818vtNt076fnPRQH5ufLZNX58DuvLrpibEKIuKw/y9flvPR0QBwA+b+Lij68Tm8ABwqIX/nad/Plb2PPDZVlpPi9cjJrbtP0zrPxmm7wPvbuCjSxrF59ZeVTTy6+akHH5eAHqcS3WDfu+Pn8ev3VWl1bInrzMfFxIbzkerAcCX2/k5mJ4QYsRIWjKtHw0xzqf1CP/Z8Cw+TggA1uznI406xepnrnrdIV4aZvTmo9+aQ4yAKdnD75OXj86SPdWDMml91rwdsucPLyyn9axc/swHgHvFPfeZpXq813MX8LXK/+Jv/MzMzMwCwgs/MzMzs4Dwws/MzMwsILzwMzMzMwsIL/zMzMzMAuK4U71HjvBEzPUjdSrteZG8ufZfy2TPJaf3ovWHvtgme/5wbl9aLw6RaG1o5om5ywfp1KhKOZ3VQ2+0fk5BOq0fbNIJsF99sIHW87MSZY/amHrxit2yZ+AAnkrq1lknJ9+Zt5/WT+3NPzcA+GQhTx8lhkiEjxCb1xeV18uekws60fr95w+QPe3RuMH8HPzDhzztCwB3n9ub1nNSeXIbAI4c4+fz41cMkz3FdQ20vq9Wp2CHZPDz9rOV+2TPrVN60PrmEJ//7hqeEr9wgL6mD4r0bm2zvj4ndutM608tKpY9SXE8Wf6v+TqZ1yBSoj1DbByvzvV/LdH3gQkF/P18vaVK9tQ18OPWr6dOJ7ZHn2/lKdhrh2XLnttOyqP1BXt1QvfXn/Nr946J3WVPfQs/119Yoqc7hIlpFbsqdBI4nLcgMlL3RITxpjlFNbInVvy+Mbn6ufbGcn6P+OcFg2RPjbhuHp6rpxU8dCn/fd+HeD9HxHogIlInqE8+KZ/WR+XpFHnHDvxYnyau2+Phb/zMzMzMAsILPzMzM7OA8MLPzMzMLCC88DMzMzMLCC/8zMzMzALCCz8zMzOzgDjucS4z+vOxJP9YsFP21IkNu6eO55FmAEgVm4lfObGb7AkXEfZx2TruvKeGb87dM0OPv3hnHY/RJ0Xp+HZ+Iv99q8v4ptAA0DePjyVJjOUjIQBg9gr+2m6/kG8ODgArivlraGnTO2DfdwEfF5EZHyN7oqOP+zT7//h+OX8/PcSxAYBP1pTRepsYRQQAZw3g5/WJtEZstN7WdkT2hHfk/w83PCdB9ny5vpzWq+r5qCMAqKhrovWLhvLRQACwan8drffMSpY9v31jHa3/YiYf3QQAc7fpcRrKmWIU0+oy/poB4Jv1/DzbIzZ6B4BRg/nxCU+Mlj0p4vgUHdCv7WAzH7PSSYx7AoBvNvLzINR1M31IF1r/ZDUf99Re/Wx4Dq2/to6PBgKAAV34vW58ln7e5CfznkOtemxQYys//nefUiB7rn9lBa0/c+Vw2XP5E/NpfeiQLNnzy3fW03pSkj7P7pvWj9ZT4/Vz7aLh/BhUHdbjo5Ji+fP40XP43weAiHB+/9xaycdXAcC0kfzcmTlQP1OeW8jHKq0o0feOPHHu9E3TY3B+jL/xMzMzMwsIL/zMzMzMAsILPzMzM7OA8MLPzMzMLCC88DMzMzMLiA7Hjomd2v/D0D/PpvWBPVNkj0rKrNnGU4sAkC42lf/5hDzZ8+IyngAN9db+OpWnAwu6xMue7WWHaV1tPg0A059YwP9Ovk6AqQ3Ql3z0g+xJ6sXfT26e3jS9vJy/n7d+Pk72PDx3B61fO4YnnADg08388x7fXW9M/chHW2j9okl6U/M9NTxxesGADNkzVaTVT6TPNvDU6EbxeQHAB4t4Wuynk3WCfkMpT6zNW7VX9gzrr4+lsq+SbzY/trc+N2Mj+DX16qztsmfy6Nz/7oUBmFLAU+IfrT+g/05vfu3O31Eje3JTeDKva6JONL48p5jWLxqv32dWAv99B5t0Unv3QT7hIDlGp/GHdeGJwj99wa9bAFj865Plz06Un76zgdb3VvFzFgB+dWoPWn/wO31u/v703rT+5MJdsic5jn+W143Q99qnlhTT+pj8ZNmzes8hWl+/vVL2jBb3gfmr98memBietv312b1kz/5D/Ny8cax+Dmwr5e+ne7qe2LGznH/ekWINAwBHxfqiqFrfp/M78fVF8UF9vkWIiQ33faWvtUW/Cn2t+Rs/MzMzs4Dwws/MzMwsILzwMzMzMwsIL/zMzMzMAsILPzMzM7OA8MLPzMzMLCB0Xv8/xMby//Sm0d1kz92fb6L1jh07yJ4DFTzW/PR8HXu/WYx62V/Px3sAwOr9fEP3rsl60/RX1/AxF9N765EgR47wyPesZ16RPbmnn03rnfr0lz0H9/ON1g/E6E2zExP5iIn6Nj36oaBLAq3nJMbqvyNi/ANT9TiXCcP4BuHztupRQH88k49M+G5nlexpj+Ncvi86SOvjuulNuXvkJNN6XdMR2dM1iY+L+PNFA2RPWAd+7T7yVaHsOW1oV1r/+wt81BEAXHLhKFrvns/HrwB6NEt2fKhzk9/X4obpW+MT83bS+u69eqP1tn58/MUXS/koKgC4dgofWVHdoK/P97bzUUC79/MRF4Aee/XQBQNlT3UzHznVo6u+ptujSwdn0vrWan28aptbaf1v5+rr5s31pbT+28kFsufZ5XxE0846Pfrj4GH+uTz7pb4+S7aL8U0718ietrYZtH7DVP1+1L1oQLo+Z5ra+HO6qUXf19T64sXlxbLn5Fw+WqriMB8nAwAvrODX7q0n5cmeiDD+2t5fz69bADjUyM+3Kyf89+Or/hd/42dmZmYWEF74mZmZmQWEF35mZmZmAeGFn5mZmVlAeOFnZmZmFhDHneq9bBxPkFz3ygrZc9YY3rNoi94A/Xdn9aH1WTt0MvMFkYybOVRvKN8zmadTt4gNngFgw+4aWp+9hie2AODoUZ6YQwbf6BsAeubzTeD7ZuXJntIangBctla/ticvG0rr5Y06Dd0m3s8zy3gCDQBqRDLqiUXFsue6UXwj8pXpOqF576c8Rf4/LWl4RBzjR0Mk8xISeHq7R2d9Dfzx3Y20nt8tWfbs2VdH66mp+nP5+98/ofWRU8fKnvc+WkXrKoEKALFR/HZ20bAusqfqAE/MfbBCXzf9cnh6ODxM/3/01l08nXj6iGzZU3aIv7b0eJ6SB4CWtqO0/tAFOnHaKiYP3PMhPz8AYLJIal88RB/r9igllifb9xfzYw8ARVX8fpabGCd7pnRPofWFe/RzrbaeJ3SLD+r78559PFleXaET5+lZfLJBn1Mvlz3qKly6q0b2xETy67O2QR/rSd35a1sfIkHfJZFP5ggXiVoAiI4Mo3X1vAOAPl34571or/5MoyP4PeJIiPvavafypPT7m3US+Mf4Gz8zMzOzgPDCz8zMzCwgvPAzMzMzCwgv/MzMzMwCwgs/MzMzs4Dwws/MzMwsII57nMs7S/lGzuHheu24ZFsFrV8xXm8uXNHIo/J5nfi4CgDokcLj21EhxitUNfG/s61Kb4DdeoSPSrjm1HzZ89Bba2n9mb/MlD3vrNhP66NzEmXPjR/wDbUf/bkemTGvhI+Y+HBhiezJyuRjcA7V682s++Ty8RfXj+AjWwCgUpwHby/UY2Oe+MlgWr/638tkD8B7TqTt+/nIlNvP7Cl7Vuw9TOtPz94le26b3pvWQ1w2+HdVA62HmEaAlF69aH3F6+/InpiB/LxNStGjeSb34SMz6lraZE+aGOexdTO/BgG9CfySdz6XPYjnr+2kAXr8yfIt5bR+3hg9AmaXGDn1x8+3yJ7cDH5NJyfz+2ooc3YdlD+bNkCPFjpRvt3Jn1GXD86SPS+s5OPDQo0aqm3ho1kGput7+pkF/Hj95fvtsqdfz1Raj+6rj31+WgytP/r8Atlz6UWjaH1qb36eA0BeEh9/sra8RvYkxPDRRdXi+Q0AGWKcS2a8Pp8jxTomLoqPeQGAC/rzkUb1zfp+8/tvttL6T8foZ+FRcV4t3srvDwCAs/m9/X/xN35mZmZmAeGFn5mZmVlAeOFnZmZmFhBe+JmZmZkFhBd+ZmZmZgHR4VioKNL/5nff8A3id1XylB8ApImN40NtMt6xA0/MbSrlqUUAmFiQTOtVDTpds0JsJn1aP51KGtGlM60fbOSJLQD466xttN4jU6e5UuJ40nBcrk40RnTka/grHv1B9vz+2tG0vqdGv5+VRXwD6vJy/fncLtKjwzN42hcAnlxcTOspCfzYAECc2Gj7NLFBOgBMCpFCO1F+/QU/Z3I76feem8iTefNCbprOz5k3xLUOAAf2HKD1c84ZKnv2inPj4MFG2dO/gKcT127WSbbWVn69F4jfBQCV4v4ValpBdheegi0NcQ0MEmnLltYjsic3hX+mXyzlqVIAuGxSHq3XN/OJBAAwJDOe1reJBDcAbN7Ppx+0tOn38/aV+hw5UR6aXUTrZ/ZMlz1Hj/JH5jdF+tw8vXsarb+5QafHbxrFp19ER+hz8+8LeYq/Tzo/lwBgWTGfIjA6Tz+jhqQl0/rr60tlz5WDeQr2yx36uE0Tn0N5g071qsRvZpw+Bgv38DR6mFiPAMC4HP78SozS65u/fMfv7WcP0ufbgUOttD45T9/XxvfSz1bA3/iZmZmZBYYXfmZmZmYB4YWfmZmZWUB44WdmZmYWEF74mZmZmQWEF35mZmZmARF+vP/h1yv20fqAHnocRkkFj/wv28rrAFBdzUc83HAW3+gdAHZV8/j29F46In1uH745enOrHnuwXmwmPaIrH/MCAJP68tcQajTHkHQexf5wS5nsWbSNbzZ+8bmDZc9fX15O6x1CRNjbxMiMp+6YKHse+phvEH/aKL0xdWIsj8SLSQoAgKFiLMW6cj6uAGif41y+Wrqb1rtl63E+vbry0QsLN+hz5meT82h9ohgjAQCb0/gxXraa3x8AID2Djz9pqNdjgxYs3knrWbn680pP5dfh8Hw92qAwjo+cCuuor4FFK/jnM2QQH1cBAPur+WiUVav57wKAp28eS+vzxJgsAKiq59dn/4xY2fP79zfReqiRNrdO7UHry0r0SJv2aElRNa3P3sLvpwAwsjs/zxYXVsqeTfv5calv0iPHahr4GI8QpyYu7p9J60nifgoA1WLs2aQ8PoIGAFaW8vEnI3L4/QEAYqP4cqOuSY8AihTn4HficwOArftqaf32ifmyJyGKjwKbu5WPLwOA6b0zaL2mkX9uAPDY9P60fs+X/BkZSqhBfB7nYmZmZmYAvPAzMzMzCwwv/MzMzMwCwgs/MzMzs4Dwws/MzMwsIDocOxYqG/L/9e1mnnKaU6zTNTsO8PTuT4byRC0APDu/mNb/OrWv7HlkHt9o+8GzdM/aMp5KWrlXp9L+eDpPFh8JETWtFcmsthA94SK29efvCmXPdSN4Qvb6V1fKnikjs2m9WKSxAWByH56qzIwPkTRs4unN9aV6E/htpTyZFROpg+htR3giu2QP/10AsPmB0+XPTpR/zOeJ1j4pPB0LAI/O3kHroS7vqiqeoL/1rJ6yZ9MB3vPDyr2y56zR/Nx84yudZBs/qhutN7boFGStSAmXl+vzOSyM/79vz27Jsueb9+bS+swrp8ieqjo+eeDOSd1lz756fqzfWV4qewZ342m+pjY9raBXWjSt5ybqTe3v+2wrrd85VZ87lwzj95sT6Q+z+D31koFZsudvC3fR+kUDecoTAF5fzT+zm0bz8xzQ6d3nV+yRPZN6JtP6kAxeB4D0RH7vTojRSeBXV5bQ+hk99TEorDhE68tFChcAhmbye15YqMkT4tk6Zxd/5gNAdjI/BqGe7WWH+LP9mqH63HlM3Nun9NaTQdRnt+ugvq+dO0ivsQB/42dmZmYWGF74mZmZmQWEF35mZmZmAeGFn5mZmVlAeOFnZmZmFhBe+JmZmZkFhJ6N8R++2s43oP50Lo8nA8C/rx9N6+vL62TPdeP5BvFFtXrMynVj+LiI5hAjDF5dyjeVv+YkPXLgvXU8Rj8+N1X2FFXx192vS5LsSU2IpPVTC/TGy/d+uZnWC/J0z8diBMhRMRYFAJYuK6b1J24+SfbUNPJNuEMNEtq7l58jbSE+00euGML/jv4z7dJbC3fT+o1T9Cbj1dV89Mfo/nq8wqQJfJTEAx/rMStJSXz0x2/P06OTosP4Buj9rxkue+pb+Tkzb7seyXDDSfz9PLOwWPZsFfe1feX6fvPTG8+i9W+X8BEXAPDr8/vxniK9Cfy0gjRa75YeL3t+0p+PcXh8ER9BAgBd85NpPSacf24AcJG4Tz/+Db+nAO1znEvnGP4IvPn9dbKnRYwUaurLPy8A+PkYfm7+6tONsqe2lo8AyumqxzqNzeXjdFbu02PXimv453w0xA36vP5daX3dXj2apUs8v3fsqSmXPVPy+bP1rg/Xy57eufyZF6bm4wDIzk3kr622SfaoUS+NLfzeBQA90mJp/ftt+vNZXMyfhTvK9DrK41zMzMzMDIAXfmZmZmaB4YWfmZmZWUB44WdmZmYWEF74mZmZmQXEcad6P5/PU2EzT+khe/aLTcZ1LhP4w9sbeE+IzZLbRALwhnN10vC6cTyVFhEi+TOnqIbWz+uvN2X+fFsZrfdO5ykiAJhTWEHrryzZK3vG9+GJsh0H9EbOaWk8HVgRItF45Ag/1lFis3sAeOVbnvS7fXpv2dM2nB/TST2SZc87a/bT+h3jdBq2PeokkrMp0XwjcQDo3DmG1rOSdE9DK08n/uvKEbKnuqmF9yzlSWQA+NPp/HN+dyPfuB4ALuiXSetn9eZ1ACir4Qm8Hl10CjIumt8CQyUAy0XS75fn6/tNdSM/1kdCRM43VvLUXnGI6/MOkXZsauJ/HwD+vJ0ni/928WDZ0yLS9RER/7O+Sxgo7sNn/kSn4bdX8uOfFBUhex5fyJ+fOSES2ndN4Qnd8gae9gWAPdUNtL5gl07b5nXm94iMeD5dAgA2lfJzM9R9QF1Tv59SIHvKD/P3OqB7iuy5QEwy+Gwrf64CwOwdfFrA5YN5ehkAvt9USOvN/fS5o1LK+Z31NZ0ew58H61J4/Xj8z7pKzczMzOz/mBd+ZmZmZgHhhZ+ZmZlZQHjhZ2ZmZhYQXviZmZmZBYQXfmZmZmYB0eHYsRA7Mf9v3l/LRy/UNLXKnn/9wCPsCQk6Jn7GQB6F/niZHmWyYdVOWn/r91Nlz73v8rEH2V31mJWZw/nGx4NTk2XPa+v4catv1uMVTuvdmdab2vTmz0VVfMTE+t06xl8YYoN4ZXDfdFpfuZ6PUgGAEYP4CI6IcP3/HWf147H3ssN8nAgAvL94D63Hxenzbc7tY+XPTpRxj86n9d45fPNxAKip58clOcR77yjGK4zN0+NPjohZTMUH9YiJqnp+j7h+RI7sWbCHn5uFFXxEFAAMz46j9TmFNbJHnYNXDNFjHMob+bX22SY9LuLMPnz8RLM6oABaxKyXXp30CJAfdvHjdqBOXzeZYuRP33S+oTwAhIlpNw9/tk32rP3TZPmzE2Xxdj7GQ41fAYA/n8HHExWK8TsAsHQP/1mpGEEEAOu3V9J63+78+QAARXv5/f7GKXqk1b5afn0eOKyf7VeIe7oa9wQAs3bwc/PyQfpa++WnG2n9pol5sqdejKl6c5keH3XhCP5+1uzV49DW7+SfT6jnTccO/MJ5bHp/2XPL++tofUiIkTZPztCjpQB/42dmZmYWGF74mZmZmQWEF35mZmZmAeGFn5mZmVlAeOFnZmZmFhB8h3Ji1b5DtD4yW6dgH71gIK3/c0mJ7GkVSbbkRL0h8ajxPGW1/SDfsBoAqip4ymr8kCzZ0z2JpwZDJZkmdk+i9fVlOi30yboDtL5uS7ns6dQphtanjdDvZ0wPng5rPaqThu/8wBPUM07pIXsu6s8TU1uqdeK4czRPRs0WKTwAOHkgT11H/w/bOP7QIX4+jcjVadtO0fxS3lKhr4GCFH7O9EwOlerl1+d3W3UK8ucn5dH6h1vKZE/PVH69z+ibJnsSIiJofVmU3gB9grg+DzaHSI+v4a/7zP48iQ4A763mPfHicwOAK4bxa3dHjX4/pTU8XR0Xpf/O7iqelK5v0VMEEqPDaP3Mkdmypz16bzP/XC4YwqdLAEDlYX6MsxN0Cjo1nh/j8/rov9MkUu9XPL1Q9kwc3Y3Wyw7phG6XBH7dzN6snzdF3fg9oqZJT6uY1otfH8W1+lkYF81f2x/e2SB78nKTaX1QNz0VYVsFT1dfFSLdv1PcO74Mke6/YxxPVz88r0j2PHA2T/x2iuPH5nj8z3oimpmZmdn/MS/8zMzMzALCCz8zMzOzgPDCz8zMzCwgvPAzMzMzCwgv/MzMzMwC4rjHuXy1bA+tD+mqNwN+edU+Wh8UYgTMnC08Cn3mwHTZs3gHH/Fx/wtLZM85Zw6g9X3VOlp+z4d8w+gRffRraxQjEQ7U6M3mG5t5JL5XD70pc3Ut/31dEnXke+kuPtLmQjEWBQAWZPHP7ltxfgDA2b34CI6MWD2i57aXV9F6fDzfUB4A0tL4OIUbxvMRB+1Vl4x4Wm9q1WN2Spr4iImPFujRSYoaDRTKX8/uJ39WI0ajXDSAj/kBgPpmft18VshHHQHAtv18zMmfz+DjngDgV59tovUp/fXYmMJifr8Rk24AAGcP4r/v+W/5eCQAKO7Jx0/UNekxK6Pz+PX52Vp93KLEuKOTeybLnvAwvtn87G163FJ71CYuqadn6/FEHfhbx6Pn8mcKAJzenX/+f5y1Tfbce2oBreeIcSUA8N387bReN1LfAxct5qNE/nHLeNnz4MdbaT0uxIiRhgY+UmZEPz3SpkE8Cx++bLDs+aawmtbL6/jIFgDITOLPolfW8jUMAHQAPxGqDvF7MQBEhvNrLT3EuDo1juyNdfq1PZil73mAv/EzMzMzCwwv/MzMzMwCwgs/MzMzs4Dwws/MzMwsILzwMzMzMwuI40719hBJosUlPBkKAGu38YTu5l18g28AGCYSoO8s2C17rpjIE0tbduhN01uP8DjXD2/Pkj3nXD2N1ltUNAx6c3SVDAOAjh35D1MSdKK1Vm0cHq8TmqcU8M9h6b4a2TOmgCeLOw/WyazH5vDUWEubTidednpPWj/UHOJYR/L/j/lmW5XsuTDEJtwnijo1vtmoN02/eQK/Bgry9Mbk15+US+vL99XKnlYRXY0Q5ywAvLGqlNbvGM83LAeA73ZW0vqp+Z1lT1U9Tw0+t1zfO1Q6dXu5Tt2fL1Li9S363Hz8E57e7JbDN3oHgIrD/P3ERen7554afh+Y3E/fCzuIm9Hjn+nE6e3Te9H68g37ZQ+gk5gnyt4qPsXh4en9ZU9NM/9cQrnj4w20funoLNmz73ADrV98Uo7sKRvAJzIMyOATDwBgdHd+j4gO098LqWdUuEitAnoNcZJIogPA2X35efvnj7fInscuGkTrnaIjdc8Cnq4/1Pjff9ajeuh71PL9PHF8w0j9ma4s5Un5A3V8WsLx8Dd+ZmZmZgHhhZ+ZmZlZQHjhZ2ZmZhYQXviZmZmZBYQXfmZmZmYB4YWfmZmZWUAc9ziXM/rzWPXeWh0pvu8CHon/zVvrZM+yTWW0ftfZfHwAAHwoNiC/bHIP2bN+Lx9D8+BfLpM9n63lry05TsfER+fG0/r7X/J4PwCMH8tfd+GeGtnTL49HyL/ezuPjADA6h7+2jXsPyZ7KWj7mYmxvvan9ZaP4yJRHPtXjIhry+WiMcbk6+v/EDzySv3evHk+Ciwbqn50gByr4iInbp/FN2wHguYV8ZMntE/XIlPc38OtmSi89AkYNbXl6aYnsOXaMj4BZfaBG9sRH85El76zn1yAA3DCSj6d5d6MeMXKLGCmzeC8foQAAb8/m59nFp3SXPVeL8UR9U+Nkz52vrqb1PgV6NMvVY7JpfcOBw7Ln86V7af2JK4bKHvU5/PUSPkqjvbp+DD9nKhr5WBwA+FBcNzMGpMueswfzMSvjc/VnuWIfv3cXVTXJnvI6/rqPiDFMAFB+iD/D527lI5UAYIAY6xUZYgRM4d4aWg/1fr7cz0fC9e2uR6akxvKxZ02tenxYQ3MbrR+s06/tN2fwNcnXhXp82PIi/pmOytTvJyGCL9N2HdDP6R/jb/zMzMzMAsILPzMzM7OA8MLPzMzMLCC88DMzMzMLCC/8zMzMzAKiwzEVufsPPX/5Na1HhdgwvLWVJzNfvX6M7Hlg9nZaLyzSSZm/XMyTme+s0mm+4XnJtL4/xMbHm3bzpF/3LjppOjiLJ2cPNeuE0drdPIV64VCeDAOAp2fvovVHzx0ge8obeWLprdX6uKUmRtP6hmKdHo6J4qmkbmn82ABAYwtPWW3Yoc+Dpy7hKcSVZTqhecfJOol5onywtpTW15bpZObakhpaD/X+iut4ejg1RqfUV+/nSbKsJN3zyRqeghzSLVn2pMRF0PqEbJ1++3YnTyGe3l0nJxvb+HX4zyU6pXzneH5MP9rK3ycAVNXz+0pavD5ui7byROPkARmyZ5Y4d647JU/2JEfx19Agjg0AvChS5N27JOiedpigf23lHlpfuVdfax1FtD0uUj8LF2wpp/XwcP3dy+Mz+PF6ewP/jAFAhXcTRUoeAAZl8Pvw90X6vnlaDzFFIkSiNTORn2eFB/h9CABO68P/TrxIugLAy8t4Sj1OPIcAID2JP9duGsVT3wDwj0XFtJ6fwn8XAMzoq5/hyrvi8+7eWf+dK0bkhPyd/sbPzMzMLCC88DMzMzMLCC/8zMzMzALCCz8zMzOzgPDCz8zMzCwgvPAzMzMzCwidb/4PqamxtP7Hc/rKnsdmF9H67e+tlT1d03m0/P4Qm383HeFjY6IidIQ9I56Pi1gqNlEGgPTkGFoPNRHn+a/5eJqXrx0le07L5+Mn7vxovey5+CQe3w41q2fnwQZa37pLH4Npo/nfCTWW4HADH2XRM03H0b9cyzeBz8vSo3P2NzTSem2THkvRHi3dU0froQYvndqHnzMvreSjDQAgPZFvZh7WgR9HAMjtxHsSI/WtJCsljtYPNfPrFgCmdOef8+3vr5M9ORl8lMii7XrEhBrjMCY/WfY8uaSY1uubWmVPWEd+ffTtwo8NAFw1nl9r6/fz6xYAmsWYKDXmAwCKa/jnHRupr+k7TuEjbV5YysejtFdvr+CjMiaJ6wkATu+eRuvry2tkz+50fm5mhxjJcbu43yfE6hFAvcQ4nR828XEyADAlj7/X7p35tQ7okTaTe3aSPep509Sq788R4g/9EGLUzFFxsg/I0qOGuiby9UB9Mx8rBgCXDsqk9VVlNbLnjXX7aL1HiBEw5/fjf6emQd9vfoy/8TMzMzMLCC/8zMzMzALCCz8zMzOzgPDCz8zMzCwgvPAzMzMzC4jjTvWGhfF0zdMLimVPtkjzZXbXG62flJNE68W1Omn49iK+YfhJ/fRm5gliw+a0RJ2uuXQIT9fMDpGCPV2kYNtCxOwONDTR+l2nF8gelbK67pUVsqdbNj/Wf57ZX/bsr+ev7atq/fk0NvL00dYDOp3YL4enw6b1TpE9D3y9jdanj8iSPe1Rxw78w6ys5+loAPhycQmtjxqoNwVfsJ4np/uHuD6VD1fwtBqg0/B5aTrR+t6GA7R+9cRusmdbBT83l2/Ticay8sO0vnqL7vnjBfz6mLerVvYUlx+i9Y/EhvIA8FPxXieFSBzvruQb3qv7AwA892Uhrf/uJ/o+8Iy474dKw7ZHyXE8IZsQpSdCxETyn43oqq+bzDh+DXRN5HUAWL+bn08qtQoAfTP479tVoRO63xRV0PrwrnqCwp5D/H6fGaefnz068ckgq0r4FAMAONjEU7XNbXoiQEoCf68b9/FrEACiwvl7nVNSKXtW7+av+4bRubJn7QH+mW4qC5HUF+81NsTUkvHQ6WrA3/iZmZmZBYYXfmZmZmYB4YWfmZmZWUB44WdmZmYWEF74mZmZmQWEF35mZmZmAdHh2LFQW7+bmZmZ2f9T+Bs/MzMzs4Dwws/MzMwsILzwMzMzMwsIL/zMzMzMAsILPzMzM7OA8MLPzMzMLCC88DMzMzMLCC/8zMzMzALCCz8zMzOzgPDCz8zMzCwgvPAzMzMzCwgv/MzMzMwCwgs/MzMzs4Dwws/MzMwsILzwMzMzMwsIL/zMzMzMAsILPzMzM7OA8MLPzMzMLCC88DMzMzMLCC/8zMzMzALCCz8zMzOzgPDCz8zMzCwgvPAzMzMzCwgv/MzMzMwCwgs/MzMzs4Dwws/MzMwsILzwMzMzMwsIL/zMzMzMAsILPzMzM7OA8MLPzMzMLCC88DMzMzMLCC/8zMzMzALCCz8zMzOzgPDCz8zMzCwgvPAzMzMzC4jw4/0PR94/l9YjI/Xa8fpJebS+prRe9vRKi6H1F3/YJXuyu8TTelhH/dr2HjhE6wO6p8iepev30/qk4dmyZ8nGMlqPidGHfu+eGlqfenIP2XOgtpHWrx+TK3tu+vcyWh81JEv2VIi/c8ep3WXPN4XVtP71gp2y5+YZ/Wj9u03l+rVVNNB6WFgH2bP6D6fKn50o3W7/nNafv36M7Hl09g5a37mTH3sAuP28vrT+6pxi2aOOZUZqnOy5ZUIerf/58y2yZ89u/rrPObWX7NlYzHv27+fXOgDcd9kgWk+KjJA9D83aTus9uybKnn6Z/Pj0T02QPfsO82vt/VX8PgQAvbom0fqUHsmy54utVbR+5MhR2XPhwC60fvd762XP+r9MkT87USY/tYTWR3bvLHvG5SbT+tPz9TNqQu9UWt9f1yJ7Nu0+SOtNTW2y57zR/Fl0qPmI7Gk7cozWt4a4bmIiw2j9QA0/ZwHg6rE5tB7qWitvaKb1UNdA9UH+Gu45p4/seXnpHlovKuafAQCkpMTS+tg+6bInr3MUrT/52TbZM3IAv9am9efnFABcPFQ/wwF/42dmZmYWGF74mZmZmQWEF35mZmZmAeGFn5mZmVlAHHe4o3Qv/0eOd1zE/3E0AKh/Grxhd43saWrlXbecqYMNKdGRtH7biytlz23n8fDA4+9vkD1/vGIIrT/7rQ4pTBP/2PbdH4pkT0sz/we/X8/XPT+f2Z/Wfwjxj/vvuXggra/dp8M3VYea+O96W/+j7qtP70nrCQnRsmdPDT8G6Uk8/AMAF43oSusfruYBm/aqqZ4f49tfWyV7Hr18CK3vH5kpe5Ki+D+qrq7Sn/8tM/l18/wXhbJnVU/+D+UjIvg/EAeA+AT+OfcU4S8AWL+T/yP1zp35P8IGgNeX7aP1MT30P+7/+an5/9XvAoCZAzJoffHeGtmzYU8tre8vOyx7isT1fkr3obKnZyo/pjsq9T/Uf2cd/8f1p43QQbf2qGcXHsiZlKc//2Pg51l5JQ+XAcCSMP653Du5QPasyeCfS6N4RgJAnxQedIwIEXT8+1z+XGlt03/nJ8N4aHBQerLsuf6t1bR+/3T+7AKAreJedPtEfg0CQEMbD7I8NUc/pw8f5s+bhy8dLHse+prf807voQOiydH8njtuKH92AUBBGr9/vbFc328c7jAzMzMzAF74mZmZmQWGF35mZmZmAeGFn5mZmVlAeOFnZmZmFhBe+JmZmZkFxHGPcxk1nO+z98p3esRIm4iDt7TofQMrRSS+PsT+hOsKK2g9PFyPi3j2C743Xly8HjGyfj9/bV3S9D6lBxv46z56lI8EAICxo3lU/fR+Oia+rLiO1run6lEWL4j9j/ft0SNgThqVR+ujxuvXVn6oldbvncn3igWAp8R51SCOJwCc0YePYNhVovdbbI/uv24UrW8s0+MiZu/k77GvGAkBAM/PL6H1v145RPYkRPJbRt9eet/I2Aj+/5fFu/gesQDwl6v4+JGwDnrf5RLx+0YM0yNGVqzie3Ru2nxA9qhxS/GJ+j7w+zK+7+nPz+SjjgCg7QgfNZKRrD/TRrEn669eXSN78vM70fqdp+j9t5+cx+8dLSFGgLRHm0rE+JueybJn/k4+ZqdfiBFAeWJf15VlNbJn3T4+tufnY7rJnu938WfhqMxk2fO3cwfQ+tUvr5A92fH8/Wyr0vv7qud+iEsacZH83vGnT/U+3/lZ/LpJiuNj3wAgP4PvmV1Yre+56hn+6/f1aLNLTuaf3Sk9+TUIADFiHVNxmD9Xj4e/8TMzMzMLCC/8zMzMzALCCz8zMzOzgPDCz8zMzCwgvPAzMzMzC4jjTvUePNxM63/5yUDZ8+cPN9H62CF6Q+L+Xfkm0+8u3C17rpzM02c/bOYJJwBYs5qn+VIzkmTPlwt4ku3iKTqZ13qEJ3/CwvSae+eeGlr/xfvzZQ/qynm9o042x/bgm2OfeZpO224r5unR8oM6/XT2MP55f7dNp4fTO/PU2IxT+Wb3APDpev55P3yZ3mi7PXpxbjGtnzNCb7z9vEi5DQ+R0C0pqaH1u7fpROsT14+m9QuG6s/lkU+20vo/RHoZAGYV8oTuqFyevgOA3Dyeqlyzfr/siU/k51mo1H1WNr9H/H6avm6eWVhM6395Y63sSUvn7/XmM3rInpdFUrt3gU7dzxzahdbbQhyDc4bwzzs1Vicn26O/ncefX5Hh+v48MF18/t/wSREAUNPIpxFs269TsMO68b9zsJGnygE9QWFdGE8iA0BqPf/M0kQSGQAONvHXsE6k1wHg0QsG0frGSj6RAgBa2vg5+PTFQ2TPc8v5WuGcAWmyp09nngS++8vNsufeab1p/bfvbZQ9CVH8efzIp/weCQDPXjGc1od01RMufoy/8TMzMzMLCC/8zMzMzALCCz8zMzOzgPDCz8zMzCwgvPAzMzMzCwgv/MzMzMwC4rjHuRw6xOPbJTWNsic5OZrWI8WmwwAQHc53bO4sfhcAZMRF0Pq+EFH5CWP5CJhDjXrj4xIxZuWDuTtlT/3hJlrv2FGvuQu687EU4eOHyJ5Rffl4hQ+/5iN1ACAhmY/O+eijVbInNoFH/E+dxKPtANDUyjduX7SmVPZ0z+ObVlc16Aj71P6ptB4ZYqRNezR5cCatry7RIxluP5+P5lm3v172XHo6H0P0wzo9/kQN+Lj3JX3OKL95fa38WUwMv6ZTxLUOALedxsec3P+R3tD99ev4eJpQbn5nDa1XNfGRVwBQV8/vn49eO0L2hIvd648c02NW+uTy66ZNjJUCgNI6/to+WavH+uwV99ZLJ+XJnvbomheX0/oNUwtkT3YifxZFR+r7zKUD+TX9x1l6jEd+J/537v5wg+xJTIyi9cUhnms3iXFol43SY9cqGvm5vqNcj/W6fHA2rT+xkI9JA4DqOv78HN4lWfYcFNfae6vLZE/neD6m7JS+egTMo9/uoPUBIUYnqWdhdLReikVH8PMqMsRIuB/jb/zMzMzMAsILPzMzM7OA8MLPzMzMLCC88DMzMzMLCC/8zMzMzALiuFO9kZF8jfjJKp3MjBQpp1XbymXPgMw8Wt+4UScN/3mYJ4y6ZPDUKgDkig2oJ+XzVBwAvLhsD62H2tBbKa3UacuTe/NU0N9WFMue1M78/dx/3SjZU1jBE1NrQxyD/WWHaX1UPt9QHAA6iHTihOFZsmdm/3RaX7ZPJ1vjxTl64/NLZc/up6bLn50oXyzl59l5Y3Nlz5freQJzTE+dMEsTCdninZWyp7GNJwBHDOOJPQC482Setv1ziERjWxtPv100QCcNm1qO0Ppnt4+XPZ9t4feVCwfqc/Pckfy9fhgiBXu7SE5+tbVK9twwMofWHw+RglT3tbnr9f3z23n8NfztBp14XrirjtaTY/5nJejfuekkWr/z042yp1UkM8vL+b0RAJJP5Qn6vPQE2ZMSzRO6D54/QPYkRvJrOjWB/y4A2FBWQ+svL9sre7ql8Wfr0Bz9fn73Db/e89P1czojKYbWZz4wS/a8dNcptP7swhLZc8uYbrT+0VadBM4V64v+mfr9vLlgN60PLuATKQBgWWk1rZ+Uxad/HA9/42dmZmYWEF74mZmZmQWEF35mZmZmAeGFn5mZmVlAeOFnZmZmFhBe+JmZmZkFxHGPc7nqZB53Xl7CN+sGgPkreRz8zHF5sufvH26m9V9fOkT2bCtvpPXGljbZMyY3kdYf/Hqb7BnRi2/YfMkAvgE3ADwwezutXz1Bj+ZIiuKR/EkT+EgAAEiOi6T1nsk6Wr7lAD9uZw7go1QAoPsEfh48t5DH1AGgbzYf9bInxEibecU1tL5ypx5/ERnOR0lERB73ad4uxMfzz/Jfn+gRE+dO7k3rr3y+SfYkJPLRH9On9JE9//iKn8/V1Xpz9uej+fn8SzHiAgBeEKOTmlv5yBYAmF3Cx9Cc07uL7Fmxm48lObuPvqbP7cN/36Fm/doGiU3lX1zC3ycA/FDM30+oYzAmh9/XmsR4HEBvEP/kt0Wyp38+HyXx1zfWyZ4bxuTJn50o1Q0ttB4T4p7R1sZ7UlPjZM+yffy+dd1wPQbplnfW0vpLV42QPf9YxEf91Nbz1wwA4WH8+5+NWytkz9FjvN4vgz8jAaChmT+Pl4cY75abwcfD5HbPkD1//oSvIS6dmCd71NiWhCg9nmhCj2RaH5zG6wCQncTH6pTXt8qeUZl8vFrrEfEhHAd/42dmZmYWEF74mZmZmQWEF35mZmZmAeGFn5mZmVlAeOFnZmZmFhDHHXdMi+VplPhongwFgKSkaFrPTuKpRQC47/LBtP7Y5zptmyA2oC4t5Yk9ACir4a97+gi9OfuQDJ6YK29okj2dRNq2UWz0DQDPfsvfa7eu/O8DwEl5/Gevry2VPZcP5hve17XohNHba/hm71eN0cdtTlENrQ/P52klAHj3e54ozMoKcQzERtfXj9MJ6vboqIjMXTGtn+yZmMeP5ZfzdDKzqYl/zjmd9Ibuu1J4Eriurln2fP0tTxaHdegge1Zv4OfZx11CpNTF9d4Q4lqLEknwG99bK3sG5/JjPXuNvtZykvl94Gdj9bn54Od8U/tJg3XieKe4r+0/qO/TqUkxtJ6byj9rAMjvzO/t/S8cKHvaozvf5SnkxER9DTx27gBar2/RaesXVvD09qpYPdng6YuH0Pq8Ep22vW54Dq1/sUMnZ0eK54qaFAEAh8S9IzZCp2B37+PX581Tdbp/9rZqWp82WqehB2TwdPVT4pkCAP3zeEo9OUYvkdaX8kkGuQn6utlby++TRZX6+vxsFb+v3Dmlh+wB+CSN/8Xf+JmZmZkFhBd+ZmZmZgHhhZ+ZmZlZQHjhZ2ZmZhYQXviZmZmZBYQXfmZmZmYB0eHYsWPHtdPveS+spPVYscE3AOwS4xU6dtRjHO45sxett6ldoQGsP3CY1kd21ZHmjzYfoPWte2pkT0UFj97PPDlf9pwiYuJ//IJvJA0Av53ah9b/vXS37PndFH7c0kKMJVhUzMcCpMbonlQx1ud5Ma4AAG4YyUcMzCvhm9ADQLzYHDvUuIDoMP6z8gY9auTGk/Lkz06UWz7eQuul1Xx8AABcNZKP07nlX8tkz+CBfCzI4UY9zqe09BCtt7TwDdgBoL6OXzcxcXyMCABERPL7yoA+ehP42d/za6pt33bZg8R0Wr7imlNly+vPfEzrMbl6vMItl42g9X+8slT2pGby8UShPPezkbS+vLRW9mwv5+fV2O56dNLSXfzeHhcdIXuePq+v/NmJsn4PP5/31urxGm3H+HiglSHGhw3K4GOIHvpCjym766wCWq9t1tfa0PRkWr/yX/o8O3KEP1ufvHq47PliG793V9Tp0Wa5YhTUjaP1SKMdlfzZXtPcInsGimPQEGLczl0fb6D16yd2kz3PzdlF6+P6Zsiew838NYSH6TVRlhh/lxan114/HaVfN+Bv/MzMzMwCwws/MzMzs4Dwws/MzMwsILzwMzMzMwsIL/zMzMzMAkLHQv7DwcM8GTmxF0+tAsC8JcW0PvO03rLn2UUltF6yVyemCsQG9WojcQBIFGnkSf15yi/U71tQpBNzVU38uMXH6g2w61p5qrKlTW82Xy9Slf0TdTJvxkCeBA3lu608DT08m6fWAGBPHU8NJoXYADs+gv+sqlGnubaKv5Mer5OG7dHmPQdp/fIx+vP6ZBPfhL2lSR+vlat4Sjypk/4sB4lUbWOIxNzs51/jPxg6QfZ0DOMbrYdKjfYeyNOBg84bJnve/vcXtP76OzoNndiTp+7r9ujU/XPvraX1zFx9vzlcx5Old108WPa8t5Ffn3OW6dd25rg8Wl8U4r62dhufCNA7Xz8P2qM5xTydGhuhvxPZVsE/l1nL9+q/k8yfHddN0RMhnp/Pn4UxUfq++Wod75k6Vqc8VWr0rjfXyJ6e4nP++7n9Zc8nW/bTescOOtGaHseP28EQqd6/L+Rp2/IQSe0zBneh9Tk7amRPbkYCrb/7XaHsGSYmKWzeridcpKfz+3GoxPGP8Td+ZmZmZgHhhZ+ZmZlZQHjhZ2ZmZhYQXviZmZmZBYQXfmZmZmYB4YWfmZmZWUAc9zgX5cMVpfJnN88cQOub9vONlwFgSLdkWh8hRrYAQGUDH2Wy6QDfHB4APltQTOsnj8iWPfM28xEGe/fqsQe7xSbTLSHGX7wpjumRo3wzbQDokcbHX8wv5K8ZANrE7yut17H3iI48ev/Wsn2yp7KSj1l5NMRYigMN/DW8NKdY9sTE8FEfdXV8pA4A3DJOj1M4UfLFmICnvi6SPbGx/FJ++pZxsud3b6+n9ZycJNmzZhMfFxIeHuL/IfOH0vJNF/E6ALz6Fd+8fv7SYtlz9Tn9eM9mPuoGAGKz82g9oRP/DADgwFox5qJO/51DNWW8HquPdVgavxet3XtI9iSKMVH3X66vtdX7+D1q+349QisyMozWQ0zmaJcSo/n7CAvxRn5Yze/P547j44QA4NNFfJzOrE1VskddU8f0YwA/nZhH6y+L0TAAMHFABv9dZ/SUPU9/vJnWD5+hR7XVNPJn3keb9RpiRJdkWlfPLgCoqeejXjrHR8me3GT+s2c/3CB7evXhxy03N8RapbaJ1lNSYvXfyeb3iPxEPXbrx/gbPzMzM7OA8MLPzMzMLCC88DMzMzMLCC/8zMzMzALCCz8zMzOzgDjuVO++fTzhdbiOJzYB4IsjR2m9aypPoALA61/zNN+0k3vIniixoXbfdJ2UaW5qpfUx3XSaLzqCJ8DWreSbQgPAddN4yumlb7bLnjEnp9F6RJhOmt33ww5aDw/Ta/skkWjrnRYje+Ij+ClTU8PTSgAQHc17Xly+R/Y0tvCkdlwcTy0CwNVjeQoyIsQxaI+Wb+TJ2eoqneaMi+ef2dNz9LmpNloflKuTphvX8fT2tCk6zXegO/87r32tNzM/Ku4dl5/VV/Z8uWwvrTc28msdAJ66fQKtzy2qkT1zxH1g73Z9Pl980Rhar23Qr23hYp7inr1Yf6Y3nscnKTz6hT7WvcQkhcn9+H0IAD5Zzo/1yvU8vdxePfzhFlp/9brRsmfikExa752q75txcWLiQCNPoALATRPyaP1PH2ySPf+ezachPHbBINlz/UsraH1ov3TZc8N0fh1Ovf9b2ZOVk8L//ml6skJ6Ak/bfhZiWsX24oO03qlTtOxJGsgTui3N+vPJTuHrmKE5eg0xLpvfC//wzVbZc/Awfw1/+lb3fN/rJPkzwN/4mZmZmQWGF35mZmZmAeGFn5mZmVlAeOFnZmZmFhBe+JmZmZkFhBd+ZmZmZgFx3ONcHryMb/L95kq9wfKpfXh8e1GIUQmPXD2M1v/6EY/dh3JsRJb82dBBPJK/tESPzDgWands4c+P83h7Vu882XN+Xx4tf2C2HgETE8k/yukD9UiGWVuraf1gAx+lAgDDsnmE/ZwxfJQKAOwo5yN/lq7R587wgV1ofbQYDQIA9760itYTk/VYn0uH6dd9ovxmRh9abznCNzkHgPve3kjrncQ4BADYuoNvEH/OYD3G4c/XjqD1h97VIyZOGc03r7/4tALZc3I3vtH5uyHGhZw/jv+dK4bqz/jxhXw0Slq8Hht0/qTutP50CR/DAwCz5vFxSxFRfMwHANx7FT/WL32/U/YMyuAbt0ef3E32fLexnPeE6+8FGsQ9IilJj8xoj7p04cfr+RV6NM9No/h51ipGEAHAYTGSo0e2Hp3UJ5WPBXnpmpGy565PNtB6bbMeG1Qg7qnqmQIAz3zA/86Ek/i1AQB1DfwYJIT4OzOfXEjrN03rJXuevWI4rT8qrkEA+PeS3bT+wh0TZU9VUzOtP/0NH8MEAGGn8JFsoVYWhbv4c7pnHr9HHg9/42dmZmYWEF74mZmZmQWEF35mZmZmAeGFn5mZmVlAeOFnZmZmFhDHnep9ei5Pv91xqk7xPPYtT9EUbecpMgDIT+MJzGum6L+zZg9P4rbpkBU2buWbPDcc5ptcA0BzI0/xTJmsN44fnMNTWyUHm2TPoRaemOueoTd/7pfONwh/ZQnfTB0Adu7im1lHR+vT4tV3+O+LitHp0TuvHEXrm3bonvmLeDJqu9jkGgAeu46n3ZIidUKzPXp31X5a3yZSuAAwYQRPrqYn6mO8fjO/Dn8QaW8AOKU3P/53nt9P9qwW12e/DJ22jgjj6bex3RNlz8WDc2i9vI5ftwDw0DR+7b6+skT2vLOCp9F79OV/HwA68LeD1lad1P5ITEzo2kXfBzZX1NP6v7/cJnuuPIOnq99coI9BeVkNrY8ZqdPD7VFYGP/u47oR+rOsa+EJ2W3Vh2XPr87tzX9Xs/78//xdIa0/cd4A2TMwj1+fLy7TKeUIcQxmL9Wff/Xm9bS+s2uy7OkuEsxVDTpx3NTIf7ZhPz/PASBc3Dt+N1kngWeXVNJ613j+XAWAfhn8XpR0rn7efFvI760Pn63vn1e9uJzWU0JMbPgx/sbPzMzMLCC88DMzMzMLCC/8zMzMzALCCz8zMzOzgPDCz8zMzCwgvPAzMzMzC4jjHufSNSWO1m96ZrHsmTKhJ62fNqiL7Oktxrm8sXyf7Nm8jUex1ZgCAGhs4ONU4hJ0fPtgGf87C5fwUTcAkBzfh9aXbdCbzU/szmPvH/2wXfasyecx/t27a2XPbTP4awvrKGZPAPhUbGpeU6PH0xRVNND6Vafmy56Kej6e5OWP+ebgAHDn88to/YVbx8ue9uiM/qm03ivEGI+J+cm0XilGEAHA9r5ptL5stR4BlJ/G7wPREfqc6dOF9wxM0xvUx0SG0frQHL0xebE4z5pCjEz5roiPtDmtR7rs+funW2l95sn6fH7xIz7+4uoZA2VPp1h+e+7RSd+jfv3aWlp/6qd843oA+M07/LXV1erRVvdfO4LWH/pgi+xpj26byD+zL0KMHOubzp9Rb4cYnfX4TP453/XJRtnzD9Hzmy/0MX70HD4WJCpCf8dTuJ+PoXkxMVr2FBfwe1R2Kr/WASAqnF/Tv77vI9nz/dM/pfWNVfq5dtWIPFq/45NNsufXE/m4uJveWyd7wsUYnL+d21/2xITxY1B0UI8Cysrk9/2h2fxZfDz8jZ+ZmZlZQHjhZ2ZmZhYQXviZmZmZBYQXfmZmZmYB4YWfmZmZWUAcd6p3UBZPlhT34MlAAFi5iSdXl6w9Knv+ftVQWj9QyRN7AHDzuTydeuToMdlz9Ah/Dcd0C9DMX0PP3nyjdwDYV8U3k75mCk8RAcC3W/lGzrUhkkzDJ/MEc3K83sh5T00LrfdI1WmuYWIT8Kf++qLsSbliJq13jtObWeen8NcdE2LT7JGDM2n95eU6bTdtQIb82Yny4Btraf2FW8bJnn2HeQLzwfd0ku1XF/AE4K49+jwb0pWn9irq9Ubr43NSaH19eY3sGdeN31cON7XJntom/hpSQpxnfTvz+9rC3TzBDwDVFXW0/vdH3pM9UVl5tJ4co2/BmQn8ddc0hzjWw7No/cttVbKnbw/++VQd0kn9f3zBJwwkJel7R3s0t7iG1iflJcueqiaelD8W4uGhJiXMHNFV9ny8lT8/bxrdTfbMFSn14Vn8vh3Kt4uL5c8iRep+/pfLZc+lV51C6288cqnsKRPTN1bs1ilYoJhWu4jrCQDSE/nzpkeXRNlzw8gcWv/NF5tlz/lD+USTxz7bJnv+fskQWo/o+H/+vZ2/8TMzMzMLCC/8zMzMzALCCz8zMzOzgPDCz8zMzCwgvPAzMzMzCwgv/MzMzMwC4rjHueQm87jzJWN5pBkA0sQYhVdCbGa9TYw/mRoi9r6ljPc0tujN2XsU8E3Yk0KMPxnQZzytHxZjJACgrJy/to799Rgc5eFb+N8HgPWlfNTMryb2kD3XvbyC1vfvqZA9V1/IN3tPG6lf2+iefJTAxwt3y56D1Tyuf/oE/X5Wb+WjDM4eo8/R9mjAAD6WZuHuGtlz4BAfzXPxlJ6yJz6SX/6nipEgAJCfyDcG/9d8PTZmwXY+nuicgfoaaBPjlmrq9TiXcjH64b1NB2TPup18zMmhQ3xkBwAcPcpf27jzJ8uekd35NdDcpkdb7a3lr2FjqR5lER/NP9MlG/loEAA4eJDfO/56xRDZ8+zsnbQ+eRAfV9Feje+WROsrSvnIHgBIjeXHuEMHPrIFAH75Gb8+7pikx3q1HOGf87zdejRPdDh/DYtDjCeKDOPf/xysOCh7mjctpfUZv7hW9nz0xXpaT4geJns6iXFHN4zQ9/SjYqxO9yT9nC49yO8d3UOMNntt7T5ab27V1/T7q/l1+IeZfLQWAHRN4iPMnluun58Te4ce3+Nv/MzMzMwCwgs/MzMzs4Dwws/MzMwsILzwMzMzMwsIL/zMzMzMAuK4U717RMLsu008SQkAl47iSdxxBXxTcAB4ffYuWs/M4GlCAKis5hvUD++jU4OrFm3lP6jXG9TjiEgFhevNn2+99Wxa31GlU4NtIjW45QB/nwCweU8Nrf9CJF0B4OyxfLPv59/SKcg3PllH6+PH6rRt5xAb0StDBvFzp7SaJxABoLmZJz7bjujN09ujg7U8YTYuJ1n2LNjNE3hfLdcJ+qah/BgP6BIre+78kH/+D84YIHtiI/jnv6VKX2v/XMYTa2pjdAD4ehtPO94xLl/2nPktvw+cf0Yf2TNPnE5jQ9zX+qXH0fo9r62VPUP6Z9D6HRN0EvQY+IuLDA+TPQvXldL6378slD298zrR+oY9Og3bHpUe4tfal8v0dXNEJM4TEvRzoLaWPzvWHTgkezLiI2h9Yp5+rs0p5vf7FxfoBOiF4jndXKWTwOg2iJZP7aXTpB069Kf13ZX6nr5WfD4pcfqZMiEnldZfXcvPcwC4cwJfX0zK1df0N0V8+sXdk/UkhUfnFNH6e2t16v78wfya/tnwbNnzY/yNn5mZmVlAeOFnZmZmFhBe+JmZmZkFhBd+ZmZmZgHhhZ+ZmZlZQHjhZ2ZmZhYQHY4dEzsa/4c/zdpO66tKamTP8G7JtN4lkcfUAWBACt80+0iIl/lDMY+dv/ntDtkzbQIf8fDmx2tlT8P6RfwHXXvLnvQeebQeagPsN+49ndb//MkW2dPaeoTW9+/VG3ofPshHL/QcoMdfTBjCo/8h9ifHoK58PEhLm/5Mu4mNqe95h2/0DQBJSXxD7bOG8dcMAL+boqP3J8pbq/koiQc/FiOIAJw0qAutn16gxyuoa6qwUo8NGtmVX5+1LS2y56kfdtL6FeP1aJbhGXxcyMNz+TgEANhbzkdjdEnho1QA4IAYD1QrRuoAQP8CPi6ivomPEwKAuGg+fqIixN/JSObXQFKsHhvSWYy5+GLJHtmTksL/TliY/l5g3z5+7+ghxrwAwJc3jpI/O1FWFfP38f6m/f/172o7qu9n1wzjozeeX6E/l1vG8HFbL6/WPWLSDGoaxSgyANtL+TFQ47EAIDudjz+ZOZiPIAKAwRnJtN4knl0A8PFWPubkQJ1+P3eMy/uv/05VA79/qfFIALC16jCthxq7Vrifj7CaMZTfvwFgYCq/54Yysbe+7wP+xs/MzMwsMLzwMzMzMwsIL/zMzMzMAsILPzMzM7OA8MLPzMzMLCD0Tsf/YXtFPa1Hheu1Y2E573l/Pt9IGgAyMxNo/bdTesmez5fyFORd5/eTPZtF8qZhw2LZk3Pa2bS+p2if7Ckv4T+79NJxsufKh76n9Y4hUna/vIon5t5boOO2lTFRtL67SCfavhXpsJEiVQoAD7zNk5iXnK4/0xUlPGn2i+k6Qd3YyiNte2p04rQ9WlvKr5uuXXiSDgD6pPPk9H0f6yT4BeN5anDrAZ5WA4CSgzyFetOoXNnzwmXDaP2ReTztCwBn986k9etG6yTwA98W0vreEO9n4/xVtP78/T+RPa8s5qnKS0ZlyZ6FO3maLzZK34JrRdLw7lN0En2xSPGHhen7QAcRyb9pAj8/AODPH26m9aJiPa2gPXp2+W5aj44Ikz2n9eDJ5S2V/LoFgJvfW0froaYhrMrlz8JFWytkT3Yav0f0FPcHAFi8jr/uPt11MjRdTF0Y0iVZ9mw4wK+BgRk6tbp0RzWtP3HeQNmjjukzy/hnDQD9MvjxyRGTIgBgyS7+fvZW6PtNYyNPSh9uFnFsAAv28GMwZ4s+D37ofZL8GeBv/MzMzMwCwws/MzMzs4Dwws/MzMwsILzwMzMzMwsIL/zMzMzMAsILPzMzM7OA6HDsmNip/T98up5vlvzVNj4+AAD2VvGY+EUj+KgGAEgTI0bu/WCj7PnjeX3F79JR7Dkl/HWX1urRH4eb+CiTjz9bK3sSOvFIfoeOOsdfvYOPP+nco4fuWbuM1m/97TWy56kX5tF6bm89mmPmxHxa/36NHgGjNrw/aUhX2VNZx3sG5ujof/kh/tkdCbF5+quXDpI/O1EeE2NOqur1xuT9M+JoPSLEeVYhxoXUt+jRAo3iZ2p0EwDUi+vmwmH6PrB8zyFaLxfnEgC0tPHX1rWzHmXxgxjn0SvEKIvbJvBrYO+hBtnTKTqS1mdt56MaAOD0nvw1vLKcj68CgLx0Ps4jxDQXrCzi98Lqar3ZfGoqP6Zx0RGy5/PrR+oXcYL0/s03tH5riLFR4eKaahLjpACgpvEIrZ/dK1323PTWGlqfPipb9nwrntNbN+mRY7368uvwtin6efPOKn6/P3+oHus1KJ3fu3//zVbZ8/iMAbT+x1l8dBMA3HISH0P0iw/Xy57nL+Ujpwqr+H0IAJ6cze/T/XP5uB8AuG0sf23q3gUAjS383PlDiOM26+dj5M8Af+NnZmZmFhhe+JmZmZkFhBd+ZmZmZgHhhZ+ZmZlZQHjhZ2ZmZhYQx53qfXQuT5rOL9SptNHdebrlsxCptB7ZPPlz7Ui9OfsfPucbho/tqxNTBxt40vCWMXpj8nMemU3rEZE6ydZRJMDy8nTyZ+M6fnzCwvXG4Z1TeXr40CGdgnz+ep78CREExZ1vraX1yEj92qYM45vXL9x8QPbU1PDXfTREQveaM/jm9fUiFQUAfzmjl/zZifLTdzbQemxUuOxZvoUfy3EDdMrumyUltJ6crNPw/7x4KK1/vJWnCQFgWCY/N7dX6xTsimK+AXpBBk+tAkCnWH581uypkz0FaTwNPXuTPjdjxOcwsodOAp/Xh38Os4rKZc+bIjU4dYxO3W8t5e/16tE6Cfrikj20vqukRvYMEffW5DieXgaA5y7oL392oqzcxc+zb4oqZM/oLP6M+vOXOmWZlhxD69eF+CxbjvL71pEQT+zaZp7UD3HbRNkh/iycv61S9jw5cyCt/3sFP5cA4CbxbF25T68hIjvy76aeW8jvXQCQFMvPwan9U2TPoLRkWj/c0iZ7MhL4ffLdDaWyZ9lO/l7PHKDXKt9s5PeIK0bz5yoAXDpMX++Av/EzMzMzCwwv/MzMzMwCwgs/MzMzs4Dwws/MzMwsILzwMzMzMwsIL/zMzMzMAkLPh/gPYR34jI/eYlQDADz7IR9LMXl8d9kzvX8ard/1zjrZ01dsqB4dode11w3n42GeXqpj4nP+cDqt/2NRsewZmMk3M4+P1Ie+7zl87EGokRnjc/h4mOYjepTJC8v4BvU3nKRHDHQQ58GREDMGZq3g42n6hRh/cesUfo4s26M3zVYO1PFxBe3V7srDtH7bxHzZU1vPxzjEhrgG6mr5OJVzxuXJnmtfXUnrd5xVIHtiw/m5vquqWfZ0ioui9TN6pMqehXv4qISoEGOQPlvGx0/ExuoRTVeM4aMSeibpUTPXvcaP261T+QgiAMgRY0PO6KnHUjS28s3eX16qR2gV7qii9bsv1ONXYiL4Md0txjC1V8+J8SMX9M+QPWX1/D2Gh+trrfwgv9a+2s6PPQD0y+AjYKrq9YiRk3P5PfXL7Xo8TWcxBmnvPj0GaW4x/32fzOMjiABgzlo+5uQ35/SWPSvEa9izl4/hAYCxk/iz49VF+hqoqiqk9WeuGC57Hpi9g9b31+gxVbdP5K8tPkKvB+ZG8HOkoVU/23+Mv/EzMzMzCwgv/MzMzMwCwgs/MzMzs4Dwws/MzMwsILzwMzMzMwuI40719uzM06lNbTxFBgDnTuFpndpGnkAEgKViQ/X8HJ5wA4Az+vGUW3oMTwYCwNFjPIVaUsETlQDwjEj8/vZUncxT1u/TqSSVRu4coz+uqDDes/aATmad0Y8nJDuAJ3cBYOponob+YHaR7IkQ7+fMvjqd+OJinrYr3KbTaWOG87RlQoxOaLZHe/fyz2xHtU6LXTa0K62vKNXnWVgYT2ZOESl5ANhSyl/bAx9slj1paXG0XlFRL3suPoWn3656donsufsinkJ9ZdY+2RMXxzd0rwmRTn3wI/1elUiR4s+K48lNAIiP5j2tIRL028TnfWpfPi0BACb14dfh/B01sudwE0/KnzVA/532qE8Gf67tb2iUPcO78AkK/XL0xIHIcH5P3VGme0bl8JT4P7/aLnsmXMFf2+pinngHgDUrimk9NoEfGwD4y6urab1zqp7yUVfHU/xX3/a87LnwxvNpvbFRT2oYmZVI6+vFfRUA7prSg9ajQyS112+vpPXbp+n1QLGYpHCoWSd0K2r4udgtUX8+P8bf+JmZmZkFhBd+ZmZmZgHhhZ+ZmZlZQHjhZ2ZmZhYQXviZmZmZBYQXfmZmZmYBcdzjXG799wpaHzeSj/cAgE5iVMLcxbtkT14+Hy1w9YRusueRj7bQ+tGjeuyBGq/wmxl9ZI9yw3vr5M9axbibB6f1lT23fbSB1jOS9eiHTWV8NMbQbL1xfPFBHq9/aV6x7MnJ4HH9UBuU//I8/l63V+mRGd3F32lq0huUl4vYe9EePdKkPXr0siG0fvPzS2VPv758U/l16/TG5AMGZdH6HW+skT33XTSQ1t8O8fnPnruN1kefxEcoAMB2MeolTIwtAoDlJXw0xoTh/H0CQIcOfMzG8o1lsuf6M/i4hme+4Bu9A8ClE/n9q7FNj3E4d1A6rb++Wo+nqa3l1/S+Wj1C65zefATL+r16tNWylbtp/VQxGqa9ykrgz6hN5Xp00pYDpbR+zVB9nj0yj4+7emhaP9lz6b/49Z6aqsd4vL/pAK1XVOj38/nvzqT1617hz3wAuFvc0298+HvZc+MVJ9H6Z3p6GFqP8Ofno1cPkz3ljfwaKNylR9ocG8k/u6eX8BFuABAVxcdhNbTqEXdqbMuOCj0+KCKC/5231+l71GkhxjcB/sbPzMzMLDC88DMzMzMLCC/8zMzMzALCCz8zMzOzgPDCz8zMzCwgjjvVe+eFA2h9W7lOo6wXKZqbZvJkIAC8/t0OWv/bJ1tlT2xsBK2HSgAeEWmhpEj+uwBg3QGeGty9T2/+fN3pfLP5Z5bxVBwATBvEE5oqhQsAcZH8vVbV6xRsYRlP7ZWGeD89s5JpfebEfNnzwLsbaf2Gc3SCenk5f225Iu0LAAdEqrdNJKvbqxeX7aH1viK5CwDnDetC61edlC17Hv+aX2vXnK7Ttq8s4ynhJYt5ahEAUrt0pvWVYnN4ADhcxtOJY84YIXs+/Iyn65tL9d+55IZzab3moE5BLiqqofXbp/eWPTEi9fzCUv5ZA8CF4jPdVnxQ9ozoy5PAkSFS1zUtPPFbVKrT8L+7kqcq/xHiPn3reH2POFHWi2kIM3rra03ZXKWP163j8mh9yd4q2ZOVye91v5jInykA8MR8PjHj7RvGyJ4/zuKp++evGil7fvXxelpf+Nj5skcl6Bdv4dc6ABwTgzl6p+jnQOsR3nRM/TIALy/n97UHztLTN1aW8vVNeX2r7Bksnl+hntMzTuWfd0mtXnv9GH/jZ2ZmZhYQXviZmZmZBYQXfmZmZmYB4YWfmZmZWUB44WdmZmYWEF74mZmZmQXEcY9z+XI9j1xnddYbRquRKSuLa2TP5FG5tL6rnI9SAYDJ/fiGxMt26r9zSm8+YuKhb3i0HQCKd/Lo/bUz+sue3MQYWn/8oy2yZ7DYAHtuiM3Z1bG+8GQ9QiEygq/7r5uuI+wvfMHHNfzl8sGyZ+gAPpYiRvx9ANhRxKPyNSE2KC/bz8fQ3DhDb4TeHiXF8o3jzx+iR0z86e0N//Xf+dtVQ2l976Em2TMqvxOtTxuox0V8sGo/rV93VoHs+cd7fFzEhD6psmdyfz7K5NNleqRN4d4aWu/UOU72jMlPpvV7n10se/5x2wRa37BZj7JYvYaPmOg/IFP27K7g40mmDeLHBgD+tYBvRL9vnx5P0iUuitabGvlomPbqUDO/b1736krZc/+FfBzZqn382APA0C78ujkKPWKkX1YSrde16HEhw8W5eeuH/HoCgLCOfMxKWYMeF/LgDD7e7a5P+eguAJjYhz+n/33ZcNmz/xB/DY/N3yl7ahv4OXjJKXoMzoJtlbR+0bNLZM/U0Tm0vjDENV1by++tkZF6KfbFomJaf/U6fc/9Mf7Gz8zMzCwgvPAzMzMzCwgv/MzMzMwCwgs/MzMzs4Dwws/MzMwsII471dvSeoTWp/XVKbvVWytove8AvcHy0u08OVtfr9Ni8ZF8/bpjd43sGZDFX0PHjnotfPV0nt4tFBt9A8DmfTyN3CaOJwB8uraM1rtl85QXAOSm8hTi6BA9keE8zVXfwpNuAJCbm0zrc3fUyJ6a+mZabz2qE23PXDuC1h+cVSh7UtPiaf3DxXtkz72Te8qfnSj9Mvln+ebyUtnTWaTre+boz79MfC7hIuUHAJtK+fk8rkey7PnpWJ5+S4qMkD2tzTy5OExsXA8AlSJRKvaGBwCUl/Nrt7qiRva8vYjfNqNieNIVAA428k3Yu3XjaU8A6JTAf9+1Ik0IADc9x1OIzSHuN4cP8+PWOUSy+el5u2j9zgt52rO9mlrApztMytfXzcebymk9M0l//p9u5UnPnZUNsucng/g0hDteWy17Prh1PK0v28GnJABA3WF+H/hsE39+A8D67TwFGx6un5/qufKHWXqSxnmDeRq9KcT53NzK/873G3Xa9q9n8ckPd3+mU8rpCfz+FRamj4FK784cy6eZAPqet6LsoOwZ0zNZ/gzwN35mZmZmgeGFn5mZmVlAeOFnZmZmFhBe+JmZmZkFhBd+ZmZmZgHhhZ+ZmZlZQHQ4duyYnqlhZmZmZv+P4W/8zMzMzALCCz8zMzOzgPDCz8zMzCwgvPAzMzMzCwgv/MzMzMwCwgs/MzMzs4Dwws/MzMwsILzwMzMzMwsIL/zMzMzMAuL/BfGRaIOQQx9gAAAAAElFTkSuQmCC",
"text/plain": [
"<Figure size 800x800 with 9 Axes>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"f,axs = pl.subplots(3,3, figsize=(8,8))\n",
"\n",
"c = 0\n",
"for ax in axs:\n",
" for sax in ax:\n",
" sax.imshow(model_state_dict['fc1.weight'][c].reshape((28,28)), \n",
" cmap = pl.get_cmap('Blues'))\n",
" c += 1\n",
" sax.axis('off')\n",
"\n",
"pl.show()"
]
},
{
"cell_type": "markdown",
"id": "17fe103d-7860-45f1-8302-ddd8155a9f61",
"metadata": {},
"source": [
"Let's do the same for the optimizer"
]
},
{
"cell_type": "code",
"execution_count": 37,
"id": "d642116a-6edb-4c0c-9e74-5f2088d65e21",
"metadata": {},
"outputs": [],
"source": [
"optimizer = optim.SGD(trained_model.parameters(), lr=learning_rate, momentum=momentum)\n",
"optimizer_state_dict = torch.load(\"results/optimizer_iteration-3.pth\")\n",
"optimizer.load_state_dict(optimizer_state_dict)"
]
},
{
"cell_type": "code",
"execution_count": 38,
"id": "38237ef4-a4fd-4417-a787-9c67c1985917",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"dict_keys(['state', 'param_groups'])"
]
},
"execution_count": 38,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"optimizer_state_dict.keys()"
]
},
{
"cell_type": "markdown",
"id": "19fd44fe-de50-4aa7-bde7-f0c9e0a288f1",
"metadata": {},
"source": [
"**Challenge:** Can you make sense of the data in this dictionary?"
]
},
{
"cell_type": "markdown",
"id": "a5090364-a89d-4092-a3b6-95db76b1bc14",
"metadata": {},
"source": [
"## Final challenges"
]
},
{
"cell_type": "markdown",
"id": "322747a3-a689-4596-a254-6ab112ad2117",
"metadata": {},
"source": [
"* What happens if you use only 10% of the available training data? Plot the difference in performance of the network.\n",
"* What happens if you remove 80% of all samples with label 5. Do you see a difference in performance? Is this difference homogeneous?\n",
"* What happens if you change parameters like the learnign rate and momentum? Plot the difference.\n",
"* Can you add more layers to this neural network? Start with one additional layer (often called \"hidden layer\"). What changes can you observe in doing so?\n",
"* Can you add regularization to this model? Look for L1, L2, and drop-out regularizations. What changes do you observe?\n",
"* [stretch] Can you change this model and turn it into a convolutional neural network?"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.10.14"
}
},
"nbformat": 4,
"nbformat_minor": 5
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment