TensorFlow.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# TensorFlow Tutorial"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "-"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.12.0-rc0\n"
     ]
    }
   ],
   "source": [
    "import tensorflow as tf\n",
    "\n",
    "print tf.__version__"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## How to Create a Tensor?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[ 4.  5.]\n"
     ]
    }
   ],
   "source": [
    "# More on session later\n",
    "sess = tf.InteractiveSession()\n",
    "\n",
    "a = tf.constant([1.0, 2.0])\n",
    "b = tf.constant([3.0, 3.0])\n",
    "\n",
    "# Add an op to subtract 'a' from 'x'.  Run it and print the result\n",
    "add = tf.add(a, b)\n",
    "print add.eval()\n",
    "# ==> [4.0. 5.0]\n",
    "\n",
    "# Close the Session when we're done.\n",
    "sess.close()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Session Object\n",
    "\n",
    "A Session object encapsulates the environment in which Tensor objects are evaluated\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2.0\n",
      "2.0\n"
     ]
    }
   ],
   "source": [
    "a = tf.constant(1.0)\n",
    "b = tf.constant(2.0)\n",
    "\n",
    "c = a * b\n",
    "\n",
    "with tf.Session() as sess:\n",
    "    print sess.run(c)\n",
    "    print c.eval() # syntactic sugar for sess.run(c)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## TensorFlow Variables\n",
    "\n",
    "“When you train a model you use variables to hold and\n",
    "update parameters. Variables are in-memory buffers\n",
    "containing tensors” - TensorFlow Docs.\n",
    "\n",
    "All tensors we’ve used previously have been constant\n",
    "tensors, not variables.\n",
    "\n",
    "TensorFlow variables must be initialized before they have\n",
    "values! Contrast with constant tensors."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[ 1.  1.]\n",
      " [ 1.  1.]]\n",
      "[[ 0.  0.]\n",
      " [ 0.  0.]]\n"
     ]
    }
   ],
   "source": [
    "W1 = tf.ones((2, 2))\n",
    "W2 = tf.Variable(tf.zeros((2, 2)), name=\"weights\")\n",
    "\n",
    "with tf.Session() as sess:\n",
    "    print sess.run(W1)\n",
    "    # Note the initialization step\n",
    "    sess.run(tf.global_variables_initializer())\n",
    "    print sess.run(W2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Updating Variable State\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0\n",
      "1\n",
      "2\n",
      "3\n"
     ]
    }
   ],
   "source": [
    "# Create a Variable, that will be initialized to the scalar value 0.\n",
    "state = tf.Variable(0, name=\"counter\")\n",
    "\n",
    "# Create an Op to add one to `state`.\n",
    "new_value = tf.add(state, tf.constant(1))\n",
    "new_state = tf.assign(state, new_value)\n",
    "\n",
    "# Variables must be initialized by running an `init` Op after having\n",
    "# launched the graph.  We first have to add the `init` Op to the graph.\n",
    "init_op = tf.global_variables_initializer()\n",
    "\n",
    "# Launch the graph and run the ops.\n",
    "with tf.Session() as sess:\n",
    "  # Run the 'init' op\n",
    "  sess.run(init_op)\n",
    "  # Print the initial value of 'state'\n",
    "  print(sess.run(state))\n",
    "  # Run the op that updates 'state' and print 'state'.\n",
    "  for _ in range(3):\n",
    "    sess.run(new_state)\n",
    "    print(sess.run(state))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Fetching Variable State"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[21.0, 7.0]\n"
     ]
    }
   ],
   "source": [
    "# Build a graph.\n",
    "input1 = tf.constant(3.0)\n",
    "input2 = tf.constant(2.0)\n",
    "input3 = tf.constant(5.0)\n",
    "\n",
    "intermed = tf.add(input2, input3)\n",
    "mul = tf.mul(input1, intermed)\n",
    "\n",
    "with tf.Session() as sess:\n",
    "    result = sess.run([mul, intermed])\n",
    "    print result"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src=\"Demo.png\">"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Inputting Data\n",
    "\n",
    "All previous examples have manually defined tensors.\n",
    "How can we input external data into TensorFlow?\n",
    "\n",
    "Use **tf.placeholder** variables (dummy nodes that\n",
    "provide entry points for data to computational graph).\n",
    "\n",
    "A **feed_dict** is a python dictionary mapping from tf.\n",
    "placeholder vars (or their names) to data (numpy arrays,\n",
    "lists, etc.)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[array([ 21.], dtype=float32)]\n"
     ]
    }
   ],
   "source": [
    "input1 = tf.placeholder(tf.float32)\n",
    "input2 = tf.placeholder(tf.float32)\n",
    "output = tf.mul(input1, input2)\n",
    "\n",
    "with tf.Session() as sess:\n",
    "    print sess.run([output], feed_dict={input1: [7.], input2: [3.]})\n",
    "\n",
    "# [output]: feth value of output from computation graph\n",
    "# feed_dict: feed data into compuation graph"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src=\"Feeddict.png\">"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Linear Regression in TensorFlow\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/usr/local/lib/python2.7/site-packages/matplotlib/font_manager.py:273: UserWarning: Matplotlib is building the font cache using fc-list. This may take a moment.\n",
      "  warnings.warn('Matplotlib is building the font cache using fc-list. This may take a moment.')\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<matplotlib.collections.PathCollection at 0x1181ac6d0>"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX8AAAEACAYAAABbMHZzAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3X90XOV95/H3Y1vzwxrJshOFXwYJzA87IYDMcUoSssjE\n/AjdDSRtIO5JS4JO1tSBUNoEDDTF5zhOIYTjdZIlsomDSYtlNz9a2LNsFHyw2s1pHbkgcDbC/CjI\nMaFB02zxYipbAn/3j3tndGfmjjTSzGhmNJ/XOTqMrubeeeYC3/vc7/M83+vMDBERqS9zKt0AERGZ\neQr+IiJ1SMFfRKQOKfiLiNQhBX8RkTqk4C8iUocKDv7OuW3Oudedc/sD277unHvOOfeMc+5Hzrnm\nwN/ucM696P/98lI3XEREpm8qPf+HgCuytv0UeJ+ZXQC8CNwB4Jx7L3AtsAz4GPCAc84V31wRESmF\ngoO/mf0M+PesbbvN7Lj/615gsf/648BOM3vbzIbwLgwfKL65IiJSCqXM+d8APO6/PgU4FPjbr/1t\nIiJSBUoS/J1zdwFjZtaT2hTyNtWREBGpEvOKPYBz7nrgKuDSwOZXgVMDvy8GXsuzvy4KIiLTYGbT\nHkudas/fEejVO+euBG4DPm5mxwLvewz4tHMu4pw7HTgT6M93UDOr2Z+777674m1Q+yvfjnpsfy23\nfTa0v1gF9/ydczuATuBdzrlfAXcDdwIR4Al/Ms9eM1trZoPOub8BBoExYK2VorUiIlISBQd/M/uD\nkM0PTfD+vwT+cjqNEhGR8tIK3yJ1dnZWuglFUfsrq5bbX8tth9pvf7FcpbMxzjllhEREpsg5h83g\ngK+IiMwCCv4iInVIwV9EpA4p+IuI1CEFfxGROqTgLyJShxT8RUTqkIK/iEgdUvAXkYIlk0n27dtH\nMpmcFZ9TzxT8RaQgPT27aGtbymWX3Uhb21J6enbV9OfUO5V3EJFJJZNJ2tqWMjKyBzgP2E88vpKD\nBw/Q2tpac58zG6i8g4iU3dDQEJFIO15ABjiPhoY2hoaGJt13KimcYj5HpkbBX0Qm1d7ezujoELDf\n37KfsbGDtLe3p98TFuSnmsIp5HOkRKrgaTQmItVvx46dFo8vsubmDovHF9mOHTtz/rZgwfL034aH\nhy0eX2TwrIEZPGvx+CIbHh42M7Ph4WHr7+9P/17I58g4P3ZOO/Yq5y8iBUsmkwwNDdHe3p7OwefL\n0//d3/Vw7bV3cPjwU+n9m5uXs3v3Fl566WW6utYSiXg9/W3bHmD16usm/BzJVGzOv+gHuItI/UgF\n4lQOvrW1lS1bHmRkZBHZeXogkMLxLgpjYwdJJBJ0da1lZGQPIyPe9q6ulaxadWn6+K2trQr6Zaac\nv4gULDuHv2XLg2zceB/wW7Lz9B0dHWzb9gDx+Eqam5cTj69k27YHOHLkiAZ1q4DSPiJSkLD0TjT6\nESKRJbz55u3AWqANeJ4NG+7iz//8zvR+AwMDAHR0dABoOmcJaKqniMyI8GmYpzE6+gqwDDgAfIlY\nLMKaNZ9P77d795Ncc81qrr32DtralrJ795OhdwQK/DNLPX8RKUhYzz8Wu4Q/+ZM/ZvPmLTQ0tDE2\ndpBt2x5g1apLGRoaIpFIcOGFF4f28gEN6hZBPX8RmRGtra0ZPfaGhos5ftz4znd6MTvOl7/8++mg\nnhoX6Oi4CDiFsPx+a2srK1asUOCvkIJ7/s65bcB/Bl43s/P8bQuBXXiJviHgWjM77P/tm8DHgLeA\nz5rZM3mOq56/SA1J5fCvuWa136M/CXiCWOwLPP30PwZ6+icBPwL+FNib874jR44U1OvPN+2z3qeD\nzmTP/yHgiqxt64DdZnYO8CRwh9+ojwFLzOwsYA3QPd0GikjlBVfvtra2snDhQj///xywFLifo0dH\n2bz5W1nbHwSOM2fOB4F24G5GR49x/vkX8dGPrpl01W++FcIq/lYCU1kRhtfD3x/4/QBwgv/6ROA5\n/3U3cF3gfc+l3hdyzBKtdxORcsi3ejcWazFYmLGCNxZrCdm+xyDu/7418Dp31W9QvhXCg4ODE64c\nrhcUucK32Jz/e8zsdT+C/wZ4j7/9FOBQ4H2/9reJSA1JJpPpBVmHDz/FyMgeurrWAnDXXV8G3kUw\nnx+JnM7116/O2t4InIqX9vkScHbGPvPmnRY6xz9fkbf+/n6tEyiBcq3wDctD5U3sr1+/Pv26s7OT\nzs7O0rdIRKYsFYC9lbgQDLRr1nyer33tfkZGMlfw3nLLI3z/+7sC29/C6ws+gZc8OERw1e/o6FBo\n4bbMIm/eeMGxY/9CPB4PXTk824u/9fX10dfXV7oDTuU2gdy0Tzqdw8Rpn3R6KOSY5bsvEpGC5Cuy\nNllxtnxF2LK333TTF/100HyDew0WGZxnELfu7q1527Vjx05raGjy9zvJIG7x+PutoSFhkciCKRd/\ny/c9axFFpn2mGvzbgV8Efr8XuN1/vQ64x399FfA//dcXAXsnOGY5z4+ITCIspx/292CgDQbRQl6b\neYF3w4aNFo8vskTiXItGm+2+++7P+/7UPt7FZ49/wcgcX+jt7S04kE/2PWvNjAV/YAfwGnAM+BXw\nOWAhsBt4Hu+eriXw/m8DLwHPAssnOG65z5GI5JHbs99j0WizDQ4O5rwvFZjzBdFCg2vqWN3dW9Pv\nb2hoskhkQc6g8vbt262pqcOg32C530bvp7m5w/r7+yf8bsELy2wbJJ7Rnn85fhT8RSqnt7fXGhvP\n9wPiTr93fbZFoy2hwbtUM3AyjzOcM2uooaHJ4vFF1tT0fn92UG7Pf6LjZ1+INmzYaAsWTO3iUe0U\n/EVkWnbs2BnIwxcWXPv7+0OD6Pbt26cUXDOPk92rH/bblGrLvQZxi0bbDeIWiy21aLQ571hB2AUq\nFmtRzz/rR+UdROpQagrn0aN/D3wP+Djj0zOTwDHmzj05Z/pkvscsfuADHwjdnkgkQp/fm3mcduCV\nwL5PACczPpXzNhKJJWzZsp777vsqZr8hElnCrbeuC13cFTZFNBI5nTvv/DMVkwsq5spRih/U8xcp\nq1Tue3BwMJ0Dz+3BD2bNxDk/ZyZOdq5+8hk+txQ8kBycvROLtVgksmDaqaWJ8vua7aO0j0hdSAXY\nePyM9DTJeHyR3Xff/RaNtmQEyHnzGi3f6tvsHHp399a8U0NTF5pCA3XYbJ+wGUZhKadE4lzbvn17\nQfvPNgr+IhIq/zTJzBx6PH5uelDUm1ljft693xobl9o3v/nNKefL840NTGWANWyqaGY7vO/R1NRh\n8+Y1WkNDszU1hU9HnY0U/EUk1HgADg6oDmddCMando4H11Tqx7s4RCJnGJw5pUBerqmVqR59InGu\nTade0GxSbPDXgK/ILDU+qPoWXsX1/f4/T2V8MLSTaHQJR44cobW1lU2b7gHW45Vi/n/AXkZH9xL2\njN6Jyilk1/6faIA1WDF0MqtXX8fBgwf49re/RFPTUqZaL0gCirlylOIH9fxFyibVU47FxqdJTtRL\n7u/vz7OoaqfBfGtsPK+k5RSmu+p2/M7iEYP3W/Y01Wi0ZcI1BrMhHYTSPiIykezZPtmzdYKDt6Us\np1BIu8Lm4xf6GZnrFIL1gmL2xS/eEnqc2VTiQcFfRKYsrMRCKhhm3y2kBoRLHShzB4W9u4v58983\n4SKu7O8RrBc0d+58mzNnvn9BONMikQXpi9tsew5AscFfD3AXqVNhD2TPfrh6IpEo+HGLxX3+ScA5\nePUh78Ubl3iB7u7NrFnz+UmPMzAwwBtvvMH116/h6FEH9Pnf6S+Ab5BInMPY2MvMmdPml5r2NDcv\nZ/fuLaxYsaKk320mFPsYR/X8RepUKaZjFit1l9HYeLbB6VPK3Qf3X7BguUWjzRaNLgmMVWTPAtoz\nq2YFodk+IjId+Uo1pGbxTGUWznSlZu/8+MffIhr9LZkzkc4jEmlPz9rJbk/2U8aOHXuUY8d+jVcq\noo/cWUCdRKMnEI1eohIPoJ6/yGxW6GybfKUaZnJgtLs7/3z9sPaE3bnEYu02d26jQTTkTmKPNTQk\n7Gc/+5lm+5gGfEVmrWDAjMVabMOGjXkflD7xStqZS490d2+1aLTFmpouyFipO5Uy0oODg7Zr1y6/\nfEVqFlCbP9W1PIPXlaDgLyI5MgPmTvPq5Z9ZUOCr9FhA9sVoovZMVMNnfDxh4rUNtUrBX0QyZD4B\nK7ucw+SBr9qeejVZeyZKbWWei8pczMpFwV9E0lK93fEnYD1i+R5/OFHQrLaqmMW0p9ouZqWi4C8i\nZpav6mXMMp+K9Wx6Ve9kA7rVVgahmPZU28WsFIoN/lrkJTJL7Nu3j8suu5HDh59Kb0sk3s9nPvMR\nHn54Fw0NbYyNHWTTpnu49dZ1oYu7ZvO0x2QyydDQUFkWrFVCsYu8FPxFZolCVuy2t3vz5rMvErW8\n0rVeFRv855WyMSJSGalerderX5nu5QcXMQV7u+OLu7yLxGQlmmX2UfAXqXE9Pbvo6lpLJOKt2N20\n6R6WL78gb3ojVWu/qyv8IiH1oSRpH+fcrUAXcBz4BfA54GRgJ7AQeBr4QzN7O2RfpX1Epik31dNH\nNHo1AwN7WbZs2aT7zqYceL0pNu1TdG0f59zJwM3AcjM7D+9uYjVeab77zewc4A28i4OIlNDQ0BCR\nSDte4H8Q+ATHjp1AR8eH6OnZNeG+ra2trFixQoG/TpWqsNtcoNE5Nw+IA68BK/GeBQfwMPCJEn2W\nSN1LFTlLJBJ+/v4vgFuAvwde4Nixv6era21Zi7JJbSs6+JvZa8D9wK+AXwOH8dI8b5jZcf9tr+Kl\ngUSkSD09u2hrW8pll93IhRdezB/8wSeBr6Pn2MpUFD3g65xrAa4G2vAC/w+Aj4W8NW9if/369enX\nnZ2ddHZ2FtsskVkpWMZ4ZMSbqfPXf/0RGhvP4q23DhGcwTM6OqQZPLNIX18ffX19JTte0QO+zrnf\nB64ws8/7v/8h8EHg94ETzey4c+4i4G4zy7koaMBXpHBf/erX+MpXHgJeTG9LJN7P2NirHDt2B95Q\n22LgxYKegiW1q+IDvnjpnoucczHnnAM+CvwS2AN8yn/P9cCjJfgskbqVTCbZuPE+4LcEH8Dyzjuv\nsXnz14nH7yWROJlodEiBXyZVdNrHzPqdcz8EBoAx/59bgceBnc65Df62bcV+lkg9GxoaIho9g6NH\nb8ObT9EGHOCP/uizfPKT1/DJT16jqZtSMJV3EKkRuQ883wB8l6ampbz9trdQa/Xq69Lv1YVgdquG\ntI+IzIDUytx4fCWJxKXAd4G9vPnm04yM7ElP7QzOBmprW5ox338mnssrtUE9f5Eak0wmefzxx7n5\n5s28+ebT6e3Nzcv5wQ/u4ZprVocWd9u9+8mMMhDBOwWpPer5i9SZ1tZWrrrqKt5++yDBgd+xsYMA\ngRW/AOfR0NDGwMBAeoro4cNPZdwpSH1S8BepQa2trWzadA/R6CU0NXUQj69k27YH6OjoCFTshMku\nCloEVr8U/EVqUE/PLm69dR2RyKmMjr7Mpk33sHr1dRnjAs3Nyye9KGgRWP1Szl+khiSTSQYGBvLm\n9VMze8Jm+6RKPwfLOCvnX7v0JC+ROpEK3nPmtPLWW+8QXOVb6JO4NAV09lDwF6kDuXP8zwH6qKdn\n8EomPcZRpA6k6vZ7xdwAvgN8kMbGMzl+/FU9iUumTMFfpAa0t7dnPXd3GbFYhB//+D46OjoU+GXK\nNNtHpIqlVuQCObN4vve9bi6//HIFfpkW9fxFqtSWLQ9yyy23EYmcnq7dc/DgAQ3YSklowFekCm3Z\n8iA33ngLsJepPphd6oNm+4jMMslkklNPPZNjx04HngF2AWuBdxONDvPQQ92any+q7SMy23gze9qA\nQ3jTOdfiPRvpeT2YXUpGwV+kyrS3t/P2278Gbsd7PPa7UU0eKTUFf5EqM16f514aG0/GuwNQTR4p\nLeX8RapUqhTD008/w623rlNNHsmgAV+ROqCaPJJNwV9EpA5pto+IiEyZgr9IFdED1mWmKPiLVIme\nnl20tS3lsstupK1tKT09uyrdJJnFSpLzd84tAL4LnAscB24AXsBbmtgGDAHXmtnhkH2V85e6l1mv\nP7NGP6DBXslRLTn/zcDjZrYMOB84AKwDdpvZOcCTwB0l+iyRWSdVrz97MdeWLQ/qbkDKouiev3Ou\nCXjGzJZkbT8AXGJmrzvnTgT6zGxpyP7q+UtdCU7bBC/wJxIJLrzw4oyefyx2Cc7NmfBZvVK/qqHn\nfwbwb865h5xzTzvntjrn5gMnmNnrAGb2G0D/tUrdC+b1TzllCYsXn8Vll93IhRdeTFfXZzLq9d91\n15dD7wZU2kFKoRT1/OcBy4EvmNk/O+c24aV8Cu7Or1+/Pv26s7OTzs7OEjRLpLokk0m6utYyMrKH\nkZHx5/COjnq9+m3bVvLUUz/jyJEjJBIJDh06xOjo/Yw/vUulHepZX18ffX19JTteKdI+JwD/ZGZn\n+L9fjBf8lwCdgbTPHn9MIHt/pX1k1ksmkzz++OPcfPNm3nzzaWAfcCPwVPo9zc3L2b17Cy+99DJd\nXWuJRNr5j/94AefmEoudodIOkqHiD3D3g/sh59zZZvYC8FHgl/7PZ4F7geuBR4v9LJFa1NOzi66u\ntcybdwpvvvkSXk++HXiF7F59IpEI3B2M5/5/8IN79KxeKalSPcbxi8AjzrkG4GXgc8Bc4G+cczcA\nvwI+VaLPEqkZwVSPF+S/DlxEU9M5HD06hnP/KaNXf+TIESKRdj/wA5xHJHI6CxcuVOCXkipJ8Dez\nZ4EVIX9aVYrji9Sq1BTO8WB+G4nEX/Gtb/0JV111Vfo9qTn8yWSS0dEhlOeXctMD3EXKqL29PSeY\nv/POa1x11VXpnnywR5+q5d/VtTKjhLN6/VJqquopUmapnP9U6vGrhLNMRiWdRSpkKgE67L0K8FKM\naljkJVJ3plqErbW1lRUrVqSDfPb+W7Y8qGqeMqPU8xeZoomKsBXSg8/d/y+Ab5BInMM77/xKc/ml\nIOr5i8ywfEXYCi27kLn/g8A3gL0cOTLAyMgeurrW6g5Ayk7BX2SKMmfwwGTTMbMf0DK+fx/wJeBs\ngheSefNOU/0eKTsFf5EpSk3HDBZhyzcdM2xsILV/NHo18G7gEN6FJAns4NixlzWvX8pOOX+RaZps\nts5kYwPPPfccHR0f4tixO4CvAu8AJ9HQkOThh7cq7y8TUs5fpEKyZ/Bkm2xsYNmyZTz0UDex2F/i\nBf5/Al5ibOx/K+8vZafgL1ImhYwNrF59HY8+uovGxrNQ3X6ZSQr+s0T2oKJUXqFjAx0dHRw/nsr7\ng+r5yExQzn8WSJUPiES8nqbmiVeXQlbyTqcEhNQ3lXeoc8UsOFJ5geqifx8yFRrwrXPTXXA01fIE\nUn6TDSCLlJJ6/jVuOj3/YssTyPSpdy+lop5/nStkUDF7MLjY8gQyPbrbkmqinv8ska9HGTYYvGrV\nper5zzDdbUmpqecvQHi+OPj82MOHn0oXDQMKLk8gpbFly4OMjCxCd1tSLfQYx1ks9/mx4wFn9err\nWLXqUuWfZ0AymWTjxvsAh57NK9VCwX+WCaZ/wp4fGww4ra2tBQf9ZDLJwMAA4C1K0sWicENDQ0Sj\nZ3D06G3ASqANeJ4777xL51EqRmmfWSR7QHH37icz0jux2CXceeefTXqc7AHinp5dnHLKEq644hNc\nccUXWLz4LA1WTsH4RXgZcAD4ErFYhDVrPl/Zhkl9M7OK/nhNkGINDw9bPL7I4FkDM3jW4vFFNjw8\nbMPDw7Zhw0aLxxfZggXLLR5fZDt27Aw9zo4dOzPe19291WKxFoOFoceW/IaHh62/v9+Gh4fT57W5\nuWPC8y9SKD92Tj/2FrNzxoG8u4ingcf839uBvcDzQA8wL89+5Ts7daS/v98WLFjuB2fvp7m5Ix18\nsi8MsViL9fb2ZgTwsPdFo802f/7ZBpnHbmw8z/r7+yv4jatXvott8GIgUqxig38p0z63AIOB3+8F\n7jezc4A3gK4SfpZkmaiCZO68/uc4enSUT37ytoz55kNDQ8yb10bmjJTTePvt3wCvZBz7+PFXNVgZ\nYsuWB1m8+Ey+8pWNobOstIJXqkYxV47UD7AYeALoZLznnwTm+K8vAn6SZ9+yXRnrTb7UQmaPftBg\nQWgKp7t7q0E852/d3VutoSFhMN9giUUiC3LSFvXaqw1+7/Hz94hBR+hdmEipUA1pH+AHwAXAJcBj\nwLuAFwJ/Xwzsz7Nv+c5OHcoXhHfs2GkNDU0GUYOzcgJTb2+vf4G412CRwXkGcevu3po+bm9vb06q\nKHXsQsYTas1kF7TU925qer9FIgmbO7fR4HyD8ItovV0YpbyKDf5FT/V0zv0u8LqZPeOc60xt9n8y\nbjLyHWP9+vXp152dnXR2duZ7q0wi3/TNVasuZd68BsbGHgN+j+zpn2+88QZz5pwK3AZ8DhgikbiB\n5csvSB/38ssvT08lTW0LLiTz1hPsp6trJatWXVrT6Y3JymSPf+/b8TKcJwFjwEHgdmA93rTOxcCL\nbNq0uabPh1ReX18ffX19pTtgMVcO7+LD14BfAS8D/wocAf4aGCYz7fO/8uxfxmujpGQOCO/0e/dn\nWSTSbJ/61Kf9GT3zJ+ythvXwcweah62x8Wzr7e2t4LctzkQzp1J/3759uzU2nuufx2cNhv0ZUTcZ\nnJk+F9BvicS5SvlIyVENaZ/0wfy0j/96F3Cd//o7wI159inXualr2SmL3IC2x+bOnW/R6IJA0N/p\nB7AlOembzP2HDR6xWKzFBgcHA9tT+585Yfqn2scHJpo5FUz1eCm08wPv2+mneya+iIqUQjUH/9OB\nnwMv+BeChjz7lO3k1Kt8OfjggHAs1mKRyAJ/cDJ/zz0VqHt7e/2AmLprWG4w3zZs2Gg7duws6M5h\norZNZKYvFvl6/pkXumGDrpzcfizWYuvW3aE5/VJ2VRX8p9UABf+SKiRlkRnMhwOpi8z3BwN1LNbi\nDxiHL/bq7e21xsZgLzh3hstkbQtTicHk4Dz9YAAfvyMIXgAjNndukzU1XZAzw6qa726k9in415FC\nAsqGDRsDOefwIJw61ngg3mre9M/xdE9YoJ47N5732IUE9onSKfm+71QvFtOROq+Dg4MZi7NisRbb\nsGFj+vMGBwctEmnOuQCGLZgTKTcF/zqR3QsPBqWU4eHhKZViGJ/+Od/gdJs3rzE9tTMsUCcS51o0\n2hKaDtmwYWPGWoCGhqacNk41mE/1YlHMeY3HzzCI5U1dpd4XjZ5ssKSsbRIphIJ/HcgMmvkHVXPT\nEh3pvPzkx80Mdvn+1t29NSMdctNNt2Tl+4cNbjaIW1NTbs57KjVuyt3zHz/+Hv98ZY9/ZK+BSH2/\n3Atg6pwp1SMzRcG/DowH9fz5ebP8M3LyBabJZrXkW9UbTJN4nxcMmuFtHBwcTH/+REEy+2/lLIg2\n/v37/faHt318fCR1nnYazLfGxvPSbZqtC92kein414HxoB7eMw2mHMKCZb7AVPislvGLSFD4Ranf\nMqc/msVip1s02jLliqJhg6el7F3n9vzDp7tOVhhvpsYmRIIU/OvERNMpg71qs9xgOVFgCrtY9Pf3\nW1PT5LVpwtNR7ZY5/fFvrZBSB4UE0HL0rlPHjMW8dsfj54aOqUx0BzITYxMi2RT860jYFMSbbrpl\nwoBYSGDK7k3nK/CWPXjb39+fMQaQCpqpbbHY6eYthDp70sA4WTvL2bsOprHy1S8KO0/B7er5y0xT\n8K9DuTn3iYP0VALT+PvDC7yZhT/wJTsoDg4O+jOD9oTm0acaQAu9GylGMXcWeliLzDQF/zpWaK8+\nbMFSYcfMrU0TViYiGm22wcHBCY4zXksoGm2ZNOfvTSltzrjgTOVupJBt2T38UvTeNdtHZpKCfx2b\nLBAXsjZg8mPm9sBzg/rZOUG90ItEtu7urRaNtmRME53O3Ui+ge7MtQ1npmcxKW8vtUbBv05l59y9\n/Hrc4vH3Z8zHn05PdqIURvgMmfDjT5YKyR6YzpxPP37MzKmW43cjvb29edNfsVhL6LZ8i+AKSaGJ\nVBMF/zoTTOOkerT33Xd/zsrbaLS5qBz5RCmMHTt2WjTaXNBAbr7jBHvlDQ1NFokssMbGcyysfETY\nRaGhoSm9fzTabLHYuRn7NTaenVNrqLHxbIvFzrJ8zyNW3l5qiYJ/Hcmd7unNwY9EEjmBPl8phlL1\nZMcHdCc/fvYFIHcx2sKQ15nHDK9ImnrfV3LGA6ba8w+bJitSzRT860TuQq9gZcm4NTQ05wS07FIM\npe7JFpLWyb5L6e7eatu3bw9crFKra1MXrtzVs8HjZVYkNfOeSdxiYeMB+Ra8ha1cVtCXWlNs8Hfe\nMSrHOWeVbkO1SiaTDAwMpH+/9to7OHz4J8DZeE/J7CP1KMaGhg8zb16EhoY2xsYOph87mHrsYnt7\ne1keI5jv+D09u7jhhhs5enQU+Ce/nV8H1tPYuIS33voXYC/e4w/P8b/LScATRKN/zPe//yAtLS10\ndHTktDuZTNLWttR/hOJG4ETgeSBJ6vGTTz75PVasWJHTvtQ5feONNwBoaWnhlVcOcuut6/I+slGk\nGjnnMLPsx+UWrpgrRyl+UM8/VPaslFRe3Ovdh5dtTg2AVrr3Gl6OIrtuzr0GcWtsPNfmzo3bnDnz\n09917txGi0QWZMzSyb8QbY8Vso7ALHycwXsi19SnkIpUGkr7zD75SjM3NCTS8+ALKZlQKYXW/IlG\nT/MHeoPfJzfvHxzcDS9BMfk6gvzjDLntCg5cq2CbVCsF/xqWr0fZ39/vz3zJnZWS6t2XO59fjMJq\n/uwJ/B4MwNljAMOWr55RvnUEk1cwDX5G/kqpKtsg1UzBv0ZN1KPM1/OvpXRE9uycYM2f5uYOi0QS\nFo2+LyQAZ/f8HwlNceWbmllYBdPsz/BSUNmPYpxstbNIJSn416CJSimngvlE9fRrRb7SChs2bAyp\nUDoegBsaEhaJLMgzrTP/1MzJzmvw4hP8jHz1iXJXFp9v2SuLRSpFwb8G5fYoe62h4YR0zftUT3my\nKpO1KDx5t0WNAAANjklEQVQltCQnAAeDeqGLr8JKNGQ/SyDfZ+RTSE0hkUpQ8K9BmT3KJj+4zM8K\niLmPaZwNcgP0sDU2nm29vb0T7ldIoA6rJ1Rs4J6JaqIi01Fs8J8z7TmiMmXJZJJ9+/YBsGnTPcDd\nwDzgu8BSvHnua/HmvL/IyMgeurrWkkwmK9PgMmhv9+bSw35/y79y/Pi/0dHRMeF+ra2trFixYsK1\nCq2trWzb9gDx+Eqam5cTjV5NPH4m3hoDgPNoaGhjaGhoSu19++2DgfbuZ2zsIO3t7QUfQ6QaFR38\nnXOLnXNPOucGnXO/cM590d++0Dn3U+fc8865XufcguKbW7t6enbR1raUyy67kba2pSSTSebPPw04\nHbgMGAKeANopJlhVu+wAHY+vZNu2B0q2AG316us4ePAAu3dvYWBgL/Brignc5W6vSKUUvcLXOXci\ncKKZPeOcSwBPAVcDnwN+a2Zfd87dDiw0s3Uh+1uxbah24ytS95BakRuLXQLA0aOplbrPAWuAMcZX\nxO4nHl/JU0/9jCNHjpRtlW4llHvlcUpPzy66utbS0NDG6Ogr3HXXl1mz5vNT/syZaq9IoapuhS/w\nd8Aq4ABwgr/tROBAnveXPBdWbfLVit+wYWPGjJ6Ghib71KeuyxjcnOwxjTL5eEBYjSGdR6l1VFNt\nH+dcO1439lzgkJktDPztt2b2rpB9rJRtqEZhPf94fCUHDx4ASNfvSdWxSfUyE4kEF154ceh+9dL7\nDOtxB7ft3v0kXV1rJ6zLM9H5r5fzKLNPsT3/eSVsSAL4IXCLmR1xzhUc0devX59+3dnZSWdnZ6ma\nVRVSeeOurpUZhddSgefyyy/PeT/A448/zrx5bYSNAdRD0EqlbIKBHUhvO3bsZY4fN0ZH/4GRES+o\nd3WtZNWqSzPOz9DQEJFIu/8eqLfzKLNDX18ffX19pTtgMbcNqR+8i8hP8AJ/attzZKZ9nsuzb+nv\nh6pUoStyU/PaCyk6NluFLdjKrdGff/Vv6hiFPuhepNZQJVM9vwcMmtnmwLbHgM/6r68HHi3RZ9Ws\n1HRFgH379oVO4Uwmk3R1rWVkZA9vvrkfWA9cRFNTR13NNEn11oN3PXPnvoc5c04NbLsMeI2w2TzB\n2VUXXngxXV2f0YwdkaBirhzexYcPA+8AzwADwNPAlcAiYDdeofUngJY8+5fz4lh1JqsSGTY4nEic\na9u3b6+rnmphPf/xSqfB1b+FlM+YSjuqtX6S1De0wrf6FZJ+UIoi10SF27KDfTBA55tdNdVVuSrn\nLNVMwb+KZU8xnDt3fmiOOnsa4k03fbFqyzXPtHzF4Sab2lnsBVTlnKXaKfhXqfCHrS+w7DLNYamM\n6aYoZFyhxeDyKdXdg0i5FBv89QzfMhifV/4t4H68Rc/7gBuB2/Dq97QBz7NmzfXs3PlzDh9+Kr1/\nc/Nydu/ekh4clukpZlWu1gZItSt2nr8Ku5XB+EyVVM2e/Xg1e14BluEtfv4SsViEW265OavQmQqH\nlUohxeAm2lc1fWQ2U8+/DDJ7jc8BfwwsoqHhdZybSyx2Rnqh1+rV12XUnwlul8pTTR+pVsX2/BX8\nyyRfQTEgNJgoyIjIVCj4V5HsAK6ALiLlouBfJcLq0Ch1IyLlouBfBXJnhvQRjV7NwMBeli1bVunm\n1SXddclsp9k+VSCzDs0u4Pc4duxEOjo+RE/Prso2rg5lPzVN/w5EcqnnXwLjPf8fAb8HaG54pWh+\nvtQL9fyrQGpOeDR6NfBuZvMzeKtdWDVQ/TsQyaXgXyKrV1/HwMBeotFhtGCrctrb27VoTqQACv4l\ntGzZMh56qFurQitIK3NFCqOcfxkkk8mc5/LKzNJsH5ntNNWzwsKCjOb8i0i5KfhXUFiQX7XqUs02\nEZGy02yfCgk+a/fw4acYGdlDV9daBgYGQmebDAwM5H1ur4jITFPwn6Z8UwqBnNkmIyMvcc01q7Xo\nSESqhtI+0zTRYqLdu5/MqOh5/LgxOvoPOe9TGkhEpktpnwppbW1l06Z7iEYvoampI2NK4erV1/kX\ngS08+ugu4vElaNGRiFSTeZVuQK3q6dnFrbeuIxI5ldHRl9m8+RsZM3paW1vTZZ3H00Bez1+LjkSk\n0pT2mYap1o/Rk7pEpNSqPu3jnLvSOXfAOfeCc+72cn/eTMgd7D2JOXPezcDAAMlkMmdWTzANdPDg\nAQV+Eam4svb8nXNzgBeAjwKvAfuAT5vZgcB7arznn3pG77v8Z/TOIR5fosVdIlJW1d7z/wDwopkd\nNLMxYCdwdZk/s+xS9WNisUuAG4A+4B8ZG5vH6Og/ZMz717x+EalG5Q7+pwCHAr+/6m+reatXX8ej\nj+6isfEsvPTPEHA6mtUjIrWg3LN9wm5JcnI869evT7/u7Oyks7OzfC0qoY6ODo4fP4Q3k6cdeAXN\n6hGRcujr66Ovr69kxyt3zv8iYL2ZXen/vg4wM7s38J6ay/kHBWfyjIy8iHNzicXO0KweESmrqi7s\n5pybCzyPN+D7r0A/sNrMngu8p6aDP2RW9gRUSlhEyq6qgz94Uz2BzXjjC9vM7J6sv9d88BcRmWlV\nH/wnbYCCv4jIlFX7VE8REalCCv4iInVIwV9EpA4p+IuI1CEFfxGROqTgLyJShxT8RUTqkIK/iEgd\nUvAXEalDCv4iInVIwV9EpA4p+IuI1CEFfxGROqTgLyJShxT8RUTqkIK/iEgdUvAXEalDCv4iInVI\nwV9EpA4p+IuI1CEFfxGROqTgLyJSh4oK/s65rzvnnnPOPeOc+5Fzrjnwtzuccy/6f7+8+KaKiEip\nFNvz/ynwPjO7AHgRuAPAOfde4FpgGfAx4AHnnCvys6pSX19fpZtQFLW/smq5/bXcdqj99herqOBv\nZrvN7Lj/615gsf/648BOM3vbzIbwLgwfKOazqlWt/wek9ldWLbe/ltsOtd/+YpUy538D8Lj/+hTg\nUOBvv/a3iYhIFZg32Rucc08AJwQ3AQbcZWb/w3/PXcCYmfUE3pPNimyriIiUiDMrLiY7564H/itw\nqZkd87etA8zM7vV//wlwt5n9PGR/XRRERKbBzKY9llpU8HfOXQncD/wnM/ttYPt7gUeA38FL9zwB\nnGXFXmlERKQkJk37TOJbQAR4wp/Ms9fM1prZoHPub4BBYAxYq8AvIlI9ik77iIhI7anYCt/ZsEDM\nOXelc+6Ac+4F59ztlW7PRJxzi51zTzrnBp1zv3DOfdHfvtA591Pn3PPOuV7n3IJKt3Uizrk5zrmn\nnXOP+b+3O+f2+u3vcc4VezdbNs65Bc65H/j/Xf/SOfc7tXT+nXO3Ouf+j3Nuv3PuEedcpJrPv3Nu\nm3Pudefc/sC2vOfbOfdNP+4845y7oDKtHpen/SWLm5Us71DTC8Scc3OAbwNXAO8DVjvnlla2VRN6\nG/hTM3sv8EHgC3571wG7zewc4En8fw9V7Ba8dGLKvcD9fvvfALoq0qrCbAYeN7NlwPnAAWrk/Dvn\nTgZuBpab2Xl4KePVVPf5fwjv/8+g0PPtnPsYsMTMzgLWAN0z2dA8wtpfsrhZseA/CxaIfQB40cwO\nmtkYsBO4usJtysvMfmNmz/ivjwDP4Z3zq4GH/bc9DFxTmRZOzjm3GLgK+G5g86XAj/zXDwOfmOl2\nFcI51wR8xMweAvD/+z5MDZ1/YC7Q6Pfu48BrwEqq9Pyb2c+Af8/anH2+rw5s/76/38+BBc65E6ig\nsPaXMm5WS2G3Wlwglt3OV6nOduZwzrUDF+D9x3OCmb0O3gUCaK1cyya1Cfgy/poR59y7gH8P/M/w\nKnByhdo2mTOAf3POPeSnrbY65+ZTI+ffzF7Dm9n3K7z/Jw8DTwNv1Mj5T3lP1vl+j7+9VuJOUFFx\ns6zB3zn3hJ8fTP38wv/nfwm8p1YXiNVKOzM45xLAD4Fb/DuAqm8zgHPud4HX/buX1Ll35P57qNbv\nMw9YDvx3M1sOvIWXgqjW9mZwzrXg9Y7b8AJ8I156IVtNfJ8QNfX/cyniZlkHZ8zsson+7i8Quwrv\n1j3lVeDUwO+L8W4vq82rwGmB36u1nWn+7foPgb8ys0f9za87504ws9edcycCw5Vr4YQ+DHzcOXcV\nXsqhCfhveLfnc/zeZzX/O3gVOGRm/+z//iO84F8r538V8LKZ/V8A59zfAh8CWmrk/KfkO9+1EndK\nFjcrOdvnSuA24OOplcG+x4BP+zMJTgfOBPor0cZJ7APOdM61OeciwKfx2l7NvgcMmtnmwLbHgM/6\nr68HHs3eqRqY2Z1mdpqZnYF3rp80s88Ae4BP+W+r5va/Dhxyzp3tb/oo8Etq5PzjpXsucs7F/IHE\nVPur/fxn3x0Gz/dnGW/vY8AfATjnLsJLZ70+M02cUEb7Sxo3zawiP3gDEgfx8oZPAw8E/nYH8BLe\noOTllWpjAd/hSuB5/7usq3R7Jmnrh4F3gGeAAf+cXwksAnb73+MJoKXSbS3gu1wCPOa/Ph34OfAC\nsAtoqHT7Jmj3+XidhmeAHwMLaun8A3f7/0/uxxssbajm8w/swOv9HsO7eH0OWJjvfOPN3nsJeBZv\nVlM1tr9kcVOLvERE6lC1zPYREZEZpOAvIlKHFPxFROqQgr+ISB1S8BcRqUMK/iIidUjBX0SkDin4\ni4jUof8PIuI3d0Kq/6gAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x10f3a9490>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "%matplotlib inline\n",
    "\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "# Define input data\n",
    "X_data = np.arange(100, step=.5)\n",
    "y_data = X_data + 20 * np.sin(X_data/10) + np.random.normal(0, 5, 200)\n",
    "\n",
    "# Plot input data\n",
    "plt.scatter(X_data, y_data)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# Define data size and batch size\n",
    "n_samples = 200\n",
    "batch_size = 100\n",
    "\n",
    "# Tensorflow is finicky about shapes, so resize\n",
    "X_data = np.reshape(X_data, (n_samples,1))\n",
    "y_data = np.reshape(y_data, (n_samples,1))\n",
    "\n",
    "# Define placeholders for input\n",
    "X = tf.placeholder(tf.float32, shape=(batch_size, 1))\n",
    "y = tf.placeholder(tf.float32, shape=(batch_size, 1)) \n",
    "\n",
    "# Define variables to be learned\n",
    "with tf.variable_scope(\"linear-regression\"):\n",
    "    W = tf.get_variable(\"weights\", (1, 1), initializer = tf.random_normal_initializer())\n",
    "    b = tf.get_variable(\"bias\", (1,), initializer = tf.constant_initializer(0.0))\n",
    "    y_pred = tf.matmul(X, W) + b\n",
    "    loss = tf.reduce_sum((y - y_pred) ** 2 / n_samples)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src=\"Regression.png\">"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX8AAAEACAYAAABbMHZzAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJztnXt4VOW1/z9vkrmRSbiEKIo2sdrT2p5qoz88nCNKKhep\nnlpqEbmoqFFDRUWttuLlQMWKFm201Uq0EbRKULReWq1RlFitx0IVqke0rVYQVMgoFwkMmWSyfn+8\nM5OZZCYJmUkyk1mf55lnZu/Zl3c24bvXXu+6GBFBURRFyS5y+nsAiqIoSt+j4q8oipKFqPgriqJk\nISr+iqIoWYiKv6IoShai4q8oipKFdFv8jTE1xphtxpi3otb93BjzrjFmvTHmcWNMYdR384wx/wx9\nPzHVA1cURVF6zv5Y/kuBk9utex74hoh8C/gnMA/AGPN1YCpwJPAd4NfGGJP8cBVFUZRU0G3xF5FX\ngR3t1q0SkdbQ4uvAIaHPpwErRKRFRDZibwzHJT9cRVEUJRWk0ud/PvBs6PNIYHPUdx+H1imKoihp\nQErE3xhzHdAsIrXhVXE20zoSiqIoaUJesgcwxswCTgFOilq9BTg0avkQ4JME++tNQVEUpQeISI/n\nUvfX8jdEWfXGmEnAj4HTRKQparungWnGGKcx5jDgCGBNooOKSMa+5s+f3+9j0PH3/ziycfyZPPaB\nMP5k6bblb4xZDpQDRcaYj4D5wLWAE3ghFMzzuohcLCIbjDGPAhuAZuBiScVoFUVRlJTQbfEXkRlx\nVi/tZPtFwKKeDEpRFEXpXTTDN0nKy8v7ewhJoePvXzJ5/Jk8dsj88SeL6W9vjDFGPUKKoij7iTEG\n6cMJX0VRFGUAoOKvKIqShaj4K4qiZCEq/oqiKFmIir+iKEoWouKvKIqShaj4K4qiZCEq/oqiKFmI\nir+iKN3G5/Oxdu1afD7fgDhPNqPiryhKt6itraXkiBImTJ1AyREl1K6o7XqnND5PtqPlHRRF6RKf\nz0fJESX4Z/hhBLAVPMs9bHp/E8XFxRl3noGAlndQFKXX2bhxI85hTivIACPAMczBxo0bu9x3f1w4\nyZxH2T9U/BVF6ZLS0lIC2wOwNbRiKzRvb6a0tDSyTTyR318XTnfOo6SINOhGI4qipD/La5eLp9Aj\nhaWF4in0yPLa5W3fLbffDS4dHPmuoaFBPIUeYTbCAoTZiKfQIw0NDSIi0tDQIGvWrIksd+c8Shsh\n7eyx9qrPX1GUbuPz+di4cSOlpaURH7zP56Pk8BL8M2P99E+ufJKplVPZde6uyP6FywpZ9egq3n//\nfSpmV+Ac5iSwPUBNdQ3Tp03v9DxKLMn6/JNu4K4oSvYQFuKwD764uJjq6mr8Dn8HPz3Q5sIJ3RSa\ntzfj9XqpmF2Bf4Yf/wg/bIWKygrGjxsfOX5xcbGKfi+jPn9FUbpNex9+9b3V/OzWn8FeYv30nzdT\nVlZGTXUNnuUeCpcV4lnuoaa6hsbGRp3UTQPU7aMoSreIF4bp+q0L53Anu0fthmeAwcDnsHD+Qq6/\n7vrIfuvWrQOgrKwMQMM5U4C6fRRF6RPCYZj+EX67ImSxBz4PwHDgEuADcD/vpvKiysh+q1at6uDf\nr6muoaKyAscwB83bm6mprlHh72PU8lcUpVvEs/zdD7m5/NLLufOuO3EUtQn5+HHj2bhxI16vl2NH\nHxvXygd0UjcJNMlLUZQ+obi4OMaH73jAQWtrK/fU3oOIcPUFV1tRFyLzAmWjyqCAuP794uJiRo0a\npcLfT3Tb8jfG1AD/DWwTkaNC64YCjwAlwEZgqojsCn33S+A7wB7gXBFZn+C4avkrSgYR9uFPnjLZ\nhncWEHH3vLnmzTZLvwDYANQBF9Bhu8bGxm5Z/YnCPrM9HLQvLf+lwMnt1l0DrBKRrwIvAfNCg/oO\ncLiIfAWoBJb0dICKovQ/0dm7xcXFDB06FGeREz4D7gL+F/Y17ePOX95pI3nC698EBHLuz4E7gHoI\nBAIcfezRjDtjXJdZv4kyhLX4W/Lsl8/fGFMC/D7K8n8PGCsi24wxI4DVInKkMWZJ6PMjoe3eBcpF\nZFucY6rlryhpTG1tbYcJ2/HjxvOlL3+JfS374Fza5gB+6wZgXzBq/YfAw1jr/2Pgj6HPXUT6JCry\n9sbrbyScR8imJ4D+9vkfEBZ0EdkKHBBaPxLYHLXdx6F1iqJkED6fL5KQtevcXfhn+KmorADgumuu\ng0HE+POdxU5mnT0rdr0TGwJaADwPFMXukzckL26Mf6Iib2vWrOlZnkAw2PlyltFboZ7x7kYJzfsF\nCxZEPpeXl1NeXp76ESmKst8kCu/cuHEjlRdVcvOtN+Pf6o/J4J172VwefPjBtvUB4AvgA+xN4Ati\nsn4D2wNxC7fFFHkLzRc0+ZrweDw2vLRd5nCnxd+CQRgzBhYtgvJyqK+HefPg1VchNzc1F6uXqa+v\np76+PnUH3J9CQNiJ3beilt8FDgx9HgG8G/q8BDgzarv3wtvFOWZP6xopipIiEhVZ66o4W6IibO3X\nX3LpJeL2ugUHwngED8KBCA5kSfWShONaXrtcHB6H3c+LkId4RnrE4XGIc5Bz/4q/rV4twaIi2XLB\nBRIsKhJZvbrH1ysdIMnCbvsr/qXA21HLtwI/CX2+Brgl9PkU4JnQ59HA650cszevj6IoXRCvImfM\n93EEPvpm0Z3PIvZGsvCmheIp8Ij3UK+48l2y+LbFCbcP7+Mp8AizQjeMqJuQ2+uWurq6DjesCE1N\nMYsrHnxQbnLmiYDc5MzL+GqhfSb+wHLgE6AJ+Ag4DxgKrAL+DrwADIna/i7gfeBvwDGdHLe3r5Gi\nKAnoYNnPQlz5LtmwYUOH7cLCnOhm0dVNpP2xlixZEtk+bMm3Lwm9bNkyKSgpEC5EOCg0xtCrsLRQ\n1qxZE/+HNTVJMD9fNl1xhTQ0NMjuhQtlD0iDG5k/1r5PHORMfOPIAJIVf83wVZQs5vnnn+f0C05n\nT8Ue+D9sfZ5B4GpysfS+pTFlliF1ETgxxykAfkVM1JDjAQd5eXnkDclj99bdMAN4FJhFt45fW1vL\nm+edw61NLbyaYzihVdicazjnLOHlw2Dsh/DzFbmY+lcZNXp0Ki9pn6G1fRRF6RG1tbWcf9H57Gva\nZ8MxnyEirk1bmzqUWYbEE8DhCJx4E8PxxDnmOB9jfQjh6J0CaG5ppnlWs133KrAcXIUummqacBe5\nkd1C1S+q2o4dDEYmbn0+HxdWns+e81o49Y9Qvkn4U47hFI+LPZ59ALzsgZOcDj48/PDUXtQMQss7\nKEoWEg7h3HfWPvgeUAt4sGK7BwhC7uDcDuGTidosHnfccXHXe73euP17Y44zBNhB274fEFsSYgx4\nR3ipvr2axTcvRhoF53AnV1x9BSuWP9wWyVNfD8Egnz/+OC8Fmrn8X3DiJqgvgTGtwu9O+HZMeen7\n7r0/q/ICOpCMzygVL9Tnryi9StjHvmHDhojffs2aNTK4dHCbD30OXUbiRHz11Uu6HeHT3Ynk6Ogd\nt9ctzkHODhFGGzZsiJmfyLsAeT3XyI4nnrCRO4WFIiNHSnDYMPmO2yFfOJC5J9v9r3LlSTA/Xxq2\nbIkb1ZSJ0JfRPr3xUvFXlN4jPAnrKfYIDhsm6Sn0yOLbFovL64oR2Dx3npBH3LDO9pO5S6qXJAwN\nDd9oOgsRbb99+2ifeBFG0TesvOuR1w5Brh7sEH9Bgew94wxpNUYEpPGqq2R57XIp8Lpib1Dton8y\nnWTFXyd8FWWAEplU/b4/drL0VeBlcA120bSrCc8BHtgN1/74Wn5+38/Zfd5u6/rZCfnP5LPomkX8\n5LqfdOjR21k5hbVr1zJh6oS4/XtHjRrV7fFHF27zbd1KyVe/TNM0P68+C4+64dr34VOH4ahmwQ/c\nMdjJBV8EeOv66znq0ssGdOG3/i7voChKmhIpj+DE+tXD/vw/AxXQdEkTzIDWna288fobVF5UScuO\nFntzuAt4FPY07OGqG6+K26O3s3IKieYGOs3CbUdMyedgkOLvf5/fz/0RrhUentjt4Ofvw8dD4ahm\nYQ/Q5IC6yQHO+G/w3vwzCAa1ZHQnqPgrygAlIsABYCdWiHcChbQJ+WHgKnbR2NhIcXExVYur4GVg\nKna/CyBwXqBjj94uhLx97f9w/95EYZ/xJoUjBAI2kmfRIsbdswRfxWxubHXzsiuHb+2A9cBeAwtO\ngkUvwitlcPLB+WzcvDn+8RRLMj6jVLxQn7+i9Bph37m72JZWcI8IlVhI4Itfs2ZN/KSqKXYCOP/Q\n/O6XU5DEZSMi40uUGNbSYt+bmkS8XpHbbrPLJ58sAuL/wQ8kALL0UKTBIHPzkNcOtHMBzEZcXlfC\nc3Y1pkwBnfBVFKUz2kf7tI/WiZ68jWT8zupBOYUejKv9pLDb65a6Z5+VwLHHttXemTPHStVRR9n3\nCRNE8vLkr2fNFLfXLWNzkdcGI3muUJRSHnLZ3MvijrW7WciZQLLirxO+ipKFhCdT33zzTa748RUx\ntfoBKiorEJewb+e+yIRwTXVNh4zfZOgwKfx/kPMkuA8YxPHbm3kqz4Xniivgnnvg4IPhrbfg6KNh\n/Xp48UW4/np8TzxBdc1vuOWWn2GG5uFv8CMIra2tUADOfU5+WfVLjik7ptN+wpk4L5DshK+Kv6Jk\nKYlKNUQ3V/d6vd1ut5jU+Qsg7074kwfm+eHlobBsmw1QYsIEWLUKTjgBXnkFfvELuPzySFZvuK3k\nzp07mXX+rNhGMi8Br4H3YC/NnzeTMyQH/4X+yBj2NwIpndDyDoqi9IjOavX3RZRMeFK4orKCvEGG\nuuBeHtkDK/PgDS+cvA2WG5j+wguYOXPgrrvgjjvghhvg4ovB6YzpMrbPt882kcmzv4U3gNeAC6Bx\nRKMtYbGc/esDMIDRaB9FyVK6CsfsMgonWQIBpk+bzqb3N/HYA09wY04u1wXhnQNg0gdQdzjMPrSA\nv//qV/DGG/i2bmXt8cfje+89cDo7dBlrmtJE044mWyriQzp2DTvM1gdyPeTqMgIpG1DLX1EGMO0T\npaKJtrwdwxw0b2+OiGG8vr2p9PcTCEBRESxcSPHllzNxwwbKc/JYIUHO+QjqvwTHfgz/mdNE0Zln\nsmLoUM7/6pdjxnPE4UfEPrkcBu6hbpq/aCb4UNDWB4ruGvYhtO5tZfULq3E6nQM2+avbJDNbnIoX\nGu2jKL1CdGSL2+uWhTctjBupE6/pSndKM/SYcBhnVZWIMSInnmjfZ8+WYE6OPJibI74cI1e58sT3\nlSOk4dNP444nUQmJDRs2yCOPPCKufFdbraLBNgrIfbA746N8wqChnoqitCdGwKcguBGGIZ6CroWv\nQ9G3rhqndIew4Le0iIweLbJqlV0++mgrQyecEFnf0NAg795zjw33bGrqdDyJ2kiKtOU45B+cn7Bm\nUSaTrPirz19RBhg+n49nn32WvKF51vXxDDb65TLwz/RTUVnRqR8/FaUZYggE2kou5+bClCkwaRJM\nnAh/+5sN33z1Vbt+3DiKi4v52uzZOP7yFwi5ZxKNJzxnsOrRVWx6f1OMayr83d03303ByIL9Kk+R\nFSRz50jFC7X8FSVlhF09BYcW2Eze0xO3P+ws07Uzi3q/CFv6VVUiw4eLzJolkpcnMmOGtfhPPtlu\nV1VlM3kTVN5MZjy97sbqJ1C3j6IoInFEbrz1c8cr5xDdPzeRmKasDMLq1Vb4x45tE/zhw0Wuu86+\nhzN5uyi5nMx4UnYzSyOSFX9N8lKUAUK8MsreGi9nnXoWDzz8AI4iG9FTtbiKK66+om8zXc89Fx54\nAE48EV57DRYvtola9fUwb551+4TaMPYWnUU+ZSKa4asoCtC9jN3S0lI2btyYdK39uET10Y1ZfvFF\n6+OfOROeeQauuQYee6xN8Nvvp3QLreevKErEqq1aXBW3jHJ0bfyUT+hCbB9dsO9jxtjJ3uuvh+ee\ng2XLYOVKK/wvv9wm+Cr8/YJa/oqS4bRPyKpaXMUxZcd06t6oXVHbIbkr6SSu+no44wyYMwfuvtsK\nfXl54icCJSnSwu1jjLkCqABagbeB84CDgRXAUOBN4GwRaYmzr4q/ovSQDq6eD8H1mIt1a9dx5JFH\ndrlvyn3gCxbAT38K8+fbz0qv0e9uH2PMwcClwDEichS2ZMR04FbgdhH5KrZ/UEWy51IUJZZIq8Zw\nIbNHoMndRNlxZdSuqO1035g2id0lGLSveMv19dbinz/fvoddQEpakiqffy6Qb4zJAzzAJ8C3gcdD\n3z8AfD9F51KUrCdcdM3r9Vr//UvAH7HJXJdC01lNXSZz7TfBIBx/PHzzm1bY6+vhqKPsukDARu2s\nXGkt/pUr7XL0jUJJK5Iu7CYinxhjbgc+wnb6fB7r5tkpIq2hzbZg3UCKoiRJex//jKkzqFlaA8OJ\nyWLNG5LHxo0bU+PSCfvpb7kFJk+GU04BYyAvD556CpzO2HDN8vI+Cd9Uek7S4m+MGQJ8DygBdgEr\nge/E2TShY39BlG+wvLyc8vLyZIelKAMSn89HRWUF/pl+W81yKzz024fIH5HPnu17YmrVB7YHUlOr\nPhzJs2iRFfXJk23MPlgXT/j/a3uhV+FPKfX19dSn0JWW9ISvMWYKcLKIXBhaPhv4T2AKMEJEWo0x\no4H5ItLhpqATvorSfW666SZuqLoBLmtb563x0ryjmabRTfBnoBDYDkvuWkLlRZU9O1EgYK35MM8/\nb+P0Tz0VHnzQfpeb22b5q8HW5/T7hC/W3TPaGOM2xhhgHPAOsBo4I7TNLOCpFJxLUbIWn8/Hz279\nmXWuRsXoB3cGufP2O/Gs8eAd7sXV6Epe+IuKbNcssO8/+AFceKG1+IuKbNz+M8/AIYfYpC317Wce\nydSGCL+A+cC7wFvYyV0HcBjwF+AfwCOAI8G+SVa4UJTsIFLaeEqoRv0IW7uncnalNDQ0pK4Wj0hb\nrf2xY+37nDm2Ds8NN4gUFbXV42lpaSvXrPQpaG0fRckO2jc852XgTSgYWUDLjpaYRK39iuFPlIRV\nXm4zcU880T4NhH3+fViPR0lMOrh9FEXpA8JtFz3LPXhXeGEdcAHsPm83/hltdfpra2spOaKECVMn\nUHJESUy8f4e+vNG19sHW4RkzBm6/Hf70Jxg7Fl55xdbaD/v1NZJnYJDMY0MqXqjbR1H2i4aGBlm2\nbJkUlBR0qNNfV1eXsHZ9dFtHT6FHah9+KH6t/WeftbX1q6rsCbuota/0D6jbR1Gyj0QVPJ9c+SRT\nK6d2qNi5snolk8+YHNne+RHkPubhkweWM+TCC+FrX7PW/KxZtgBb+2if9stKv6NuH0XJQoqLi6la\nXIXrIRcF9xdEKniWlZXFrdgJRMpAuPfB3qVwR2sL/xw5EoYNs8I/ZoyN4Kmv7yj0KvwDDrX8FSUD\nCWf55g3OI7A9wJ2/uDMS2hmvYuf4/zqekm9+jaZpfl59FrYamLwZWgsLyf3iC/i3f4Pt2+G66+CR\nR9SnnwGkRVXPZFDxV5Tu4/P5WLduHZOnTMY/M3Enrphon2AQRo7k7+NOouwvf2blPj+nBKDZ5cTZ\nFIAhQ2DHjrYonpdfVks/A0hW/JMu76AoSt8QtvZz8nPwO/wxdXwcwxwxdXyKi4spHjYMGhutuB94\nIF99YRWNDgemGVqHDMG5cycUFsLOnXDJJXDXXWrxZxHq81eUDMDn81ExuwL/DD97ztrTIcs3phNX\nuMzymDGwbh3813/Btm0A5DQ3Y3Jzyd25E0aNshb+5Mlwzz3g96vwZxEq/oqSAcTU7c8H/huogfya\n/Jh2jfj9VvRfecUmZZ12Gnz0UezBgkH47/+GNWts6eWtW2HXLvB4+uGXKf2F+vwVJQOIF9rpfsjN\nU48/RVlZWZvwH3CADdd85BEoK4MXXmg7SE4OtLbCgQeCz2ddQh6PtlXMUDTUU1EGMOGMXCCS3Rtu\nzn7/ffczceLENuE/6SQr/L/+tRX6F16wVTfBivuLL9rGK9u2wccft1n6KvxZiU74KkqaUl1dzdyr\n5uIsckZq92x6f1PHmj2BQKzFn58PDQ1W+PPz4aGHwOttq8fT2AiDB/fvj1P6HXX7KEoaUl1dzexL\nZ9vO191pzH7HHXDllVbUd+60Yr9nD1x8MbzxhhV9UCt/AKFx/ooywPD5fBx62KE0eZvgh8D/Ac8A\ng8DV5GJZdQ3TZsxs2yHssz/qKHj7bRvamZcHZ55p6+83NOhk7gBEff6KMsCIRPZ8AXyIFf5ZwKXQ\nPKOJw845m51PPmk3rq+30T2LF1vh/+Y3beTOmWdai1+FX0mA+vwVJc0oLS2lZVcLHA+sALxEErpa\nD4Ybh3t48rzzYP16uPtuePhh22nrttvgRz+yLqAbblDhVzpF3T6KkoaE6/Pk5Rt2+/bCBcAIyPkE\nXCs8+C6aQ/5tt9kG6gsWaBXOLER9/ooyQPFt3YpnwgRemjiBab9Zwrc9hv/5zE/jNfMYV30vzJlj\nLf+VK7WBehai4q8oA43opKv6epgyhT3nnYfzN79hT00NQxYv1paKioq/ogwYgkH7PmaMFfcTTrBl\nGs4+G7ZsaXPxJOq5q2QVKv6KMhAIF2JbtMguT55sY/UbG0EErrhCXTxKDBrqqSgDgdxcWLSI1ilT\n+Pjhh5FAwJZg8Hrhqaesxb9ypXXxhJ8QFCUJVPwVJU2o/fRTFu3excjf/Ia9fj9v/+B0aGpq26C8\nXH37SspIifgbYwYbY1YaY941xrxjjPkPY8xQY8zzxpi/G2PqjDFaTERREuDz+Vh2wblcZFrYXADN\nDrjqj39g53330XzVVax9/XV8Pp8Kv5IyUmX53wk8KyJHAkcD7wHXAKtE5KvAS8C8FJ1LUQYcGz/4\ngIXBIGdMg9IrYPIMWBgMcvfbbzH0H+8wYfokSo4ooXZFbX8PVRkgJD3ha4wpANaLyOHt1r8HjBWR\nbcaYEUC9iHwtzv464atkFdH9dcGWc/B6vYz6j2PYM3NfpF7/oN+6kJycTnv1KtlLOkz4fhn4zBiz\n1BjzpjHmXmPMIOBAEdkGICJbAf1rVbKe2tpaSo4oYcLUCYwsGckhpYcwYeoEjh19LOede0FMvf55\n867HWeSM26tXUZIlFbV98oBjgDki8ldjTBXW5dNtc37BggWRz+Xl5ZRrKJsyUIgqs+Dz+ai86Dz8\nZzXhL/DDr4BZEBgRgK1Q80ANb7z+Bo2NjXi9XjZv3kzgVvtd2PKP6dWrZBX19fXU19en7HipcPsc\nCPyviHw5tDwGK/6HA+VRbp/VoTmB9vur20cZmAQCUFQECxfimzmTD+fO5esrahl6HbRsA/4AVLZt\nXriskFWPruL999+nYnYFzmFO9m7bizEG9wFumrc3U1Ndw/Rp0/vrFylpRLJun6Qt/5C4bzbG/JuI\n/AMYB7wTep0L3IotSPtUsudSlIzC6YSFC5Err+TdH13JmFbhyhxo+QwYAuygg1Xv9XqpmF2Bf4Yf\n/wh/pFfvyuqVbb16FSUFpKqk82XAw8YYB/Av4DwgF3jUGHM+8BFwRorOpSjph98fWz45tOybOZN3\nf3QlJ7YK9SVw51eAGig4qIB9Zh/mwVirvrGxEecwpxV+gBHgHO5k6NChKvxKSkmJ+IvI34BRcb4a\nn4rjK0raEgxa947XCz/8Idx5J8ydC/fcA42N+BctYkxI+E/cBHO/BjUjvPzqxl9xyimnAMT05PX5\nfAS2q59f6X20to+i9JToejyPPWZr7zid9mYwZw784he0DhvGT1qauO28FuZ+CAtfgpEuFx/8a3NC\nSz5cy98xzKF+fiUhWthNUfqaOCWXueQSuPFGW4StpATC4ZiBALW/ezwi5vJZE9X3Le1SzKNzAdTd\no8RDxV9R+pJAAMaOhUWL8H3jG3y+ciVfuekmcj/91H5fUgKbNlnL/667IrvFE3MVeCUZVPwVpa8I\nu3nOPJN9//M/rPTvYXpLK43AYMC4XPDcc9YFFPL5J+qhW1tbGwnnDGwPULW4imPKjtEbgdJtVPwV\npS+pr6d1yhRe3bGdE1uFBg9MHQ/rX3Ky8aFHGHLrrbbyZiCQUPh9Ph8lR5TgnxEq2/AS8Bp4D/YS\n3BlUH7/SLdKhvIOiZA/l5WwfPToSumkMMAyk2M0/R45sK7mcQPjBRvc4h4XKNrwBvAZcAI3nN+Kf\n4aeissJW8FSUXkTFX1H2hxdfpKiujt/m5fCNbfCzE2DRcxD8PGDDMeOUXPb5fKxduzYi6KWlpTac\n80PgeaCImPo9eUPytH6P0uuo+CtKNIFA4uVgEK6/HvPcc+T99iHOynEyoz6Xk/e4uO/e++P66qML\nuYVLMhcXF1NTXYPrMRcMAr7AxvXvAd6Cps+bNK5f6XXU568oYaJq8XD55XDHHXDDDfD555HibNFh\nnj6fj40ffEDp4YfHFf4Ovv12JZnfffddyo4ro2l0E/wJWwqxABz7HDxQ84D6/ZVOUZ+/oqSKUC0e\nrrzStky88kq7HBZ+iHHrFBcXM2r06ITROTG+fehQkvnII49k6X1Lcb/utsJfAVwGzec0q99f6XVU\n/BUlmssvhxNPhJdftu+XX97jQ0V8+1tDK+KUapg+bTpPPf4U+SPytW6/0qeo+A8Q2k8qKj3kjjvg\nT3+yiVx/+pNd7iFh3350g5aa6poOTwplZWW07mrt9CahKClHRPr1ZYegJMPy5cvFU+iRwaWDxVPo\nkeW1y/t7SOlLS4t9xVtuahLxekWqquxyVZVdbmpK6pQNDQ2yZs0aaWhoSLjN8lr7b1hYWqj/hkq3\nCGlnj7VXJ3wznK4mFbvaN6vKCwSDcPzxsHu3LcIGcPHFUFgIf/6z9edHdd4COi73Iln376EkhU74\nZjldTSomIl4I4oAnNxduuQW2bIFTT4VTToFPPrHrwhO57YW+j4QfQhPIo0ap8Ct9gop/htOdScX2\n+Hy+SLeoXefuGrhZpcFgx+XycrjiCti71zZcufxyu66P0LkZJV1Q8c9wujOp2F5wevq0kFGEi7CF\nG17X19vlF1+EqioYNMiWYLjjjrZtepmsfNpS0hb1+Q8QEvmL21ePrKmuYfy48T2eJ8go6uvhjDNs\neeW774Y2nfbqAAAgAElEQVQVK+C66zr3+fcSPp+PksNL8M8c4Ndc6TP6vYG7kh4UFxd3EJFo9064\nGXhFZQWb3t9ETXVNh25RA06Eysut8P/0pzB/Powb1+biCQv922/HLvcS1dXV+B3+uE9bA+66KxmB\niv8AJuzeiW4GHhac6dOmM37c+IEdXVJfby38+fPte3l5R/9+L4s+2Jvwz279GbQQ25v3c43lV/oP\nFf8BRrT7J2YyOE4z8HhPC50dd926dYBNSkr7m0UwCPPmwcqVbaI/b15byeU+ZOPGjbiGu9j3//bB\nA9jOL5/DtfOvTf/rqAxY1Oc/gIjn3wci7p3AZwGuu+Y6Ki+q7FR02s8f1NbWMqtiFs0tzVAAzn1O\nltUs6//CY9G9dHuy3EfE5GIUAB+A+3k3H/3rIxV/pcck6/PXDN8BQkNDg3gKPcJshAUIsxFPoUca\nGhqkoaFBFt60UDwFXWcBt88WXlK9RNxet+Am9tgFnk4zVnudlhaR0aNFVq+2y6tX2+Xo7N1+Jjqz\nVzN4lVRDkhm+KXP7GGNygL8CW0TkNGNMKbACGAq8CZwtIi2pOp8SS2f+/dLSUm7++c34Z7ZN/J5/\n4fkUDSuKceHEmyCee+VccgtywRAzWZkzJKfvJyujLffcXLjppthonpUr+8Wyb4/P56P63mpuvvVm\nnEVtT2Gb3t80sOdYlIwilXH+c4ENUcu3AreLyFeBndiCtUov0VmyV4e4/s9gX9M+Tr/g9Jh4840b\nN5I3NK9DRErL7hbYQcyxW3e29u1kZSAQG7f/4otw/fXwwx/aaJ45c/o0WSsR1dXVHHLYIdzw0xvw\nz4xNogM0g1dJG1Ii/saYQ4BTgN9ErT4JeDz0+QHg+6k4lxKfzpK9Ym4MPuD3QAXsqdgTk9375ptv\nsvuT3TEiH9wV5Je/+CUO44Aa4JfgfNBJzb2dJ5KllEDAVtk880xr6Z9zDkyaZD/fc09bNE8fJWtF\nE/27q6urmX3pbAInBzq0ZhxwSXRK5pOMzyj8AlYC3wLGAk9j//T/EfX9IcBbCfbtJY9YdpKoguTy\n2uXi8DiEXIRhId996FVYWih1dXV2zmA8ggfhQAQHsqR6SeS4dXV1UldX1/HYvVVVNFxxc/RoW2Fz\n+HCRo48WAZGzzup1n39X1TjDv7vg0AJxDnJKrivXXrfvIuQRd/5FUVIF/e3zN8acCmwTkfXGmPLw\n6tAr5j6T6BgLFiyIfC4vL6c8DR7fM5VE4Zvjx40nLy+P5qnN8Cgdwj937txJzuAcGAOUATvB+5yX\nY8qOiRx34sSJkUig8LpEiWTjx41Pzr0RLs+waJF9fe970NICf/sbHH00PPeczdgN/62Ul6c0jDNe\n5FR0dFPkdx/nhz8DHqAV2AW8AJRjn3cLgc+h6u4qdfcoSVFfX099Kp9uk7lz2JsPNwMfAf8CPgUa\ngYeABiAntM1o4I8J9u/Fe6MSZs2aNTK4dLC1RKeErPsixJnvlDOmniHufLfg6NxajWfhxxx3AcLV\nSP5B+VJXV5f8oFevttb+/PkiLpe1+I8+2q6rquq16J7OIqfC3y9btkzyD8m313G2/d24EUZFPVld\njXAh4j3UK2vWrEn5OJXshiQt/1SHbY4Fng59fgQ4M/T5HmB2gn1669pkNe1dFh0EbRaS68oVV76r\nTfSnhARsmA3ljHbfxOx/NcLpiNvrlg0bNrSt72T/zsbWKfPn2z9TY0TOPjtW+JNsspKIDje0kGts\nzZo1Ma4eckPusfB2U0Luni5uooqSCtJZ/A8D/gL8I3QjcCTYp9cuTraSyAcfHWvu9rrFOcgpnI5w\nEAkt97BQ19XVWUEMPzUcZEVu4U0LZXntcpsL0A3Rizu29iIeXg5Z/i0HHSTN+fmy44kn2nz7vST8\n4d8cz/KPudFdjVDW0bfv9rrlmnnXiKdAY/qV3iWtxL9HA1DxTykNDQ3iKejcZREj5lfT5rpot320\nULu9bjthnCDZq66uTvIPzY9rLceMrZ2oFnhdEszP79g6ce9ekdGjZdX110t+gVtOPTBfXs81Uvvw\nQ72eyBWdFBct4JEngugbYA6S686Vgi8VxAj9fj3dKEoPSFb8tbZPBtGdNn9dVY8Mv3w+nw3/3A2M\nB5YC+eBp9lBzb6gsRLuJ3NyluZBP7LGL7LHLyspo3dmasI4QtCWiNQ/30xLa3wx3seW7s/jSlVfC\nk0/apum/+AV4PPieeILvfvXL+Gfu45kR8MdPwPXDCxk3YWJKJ0/D19Xr9fL47x6PJGeJCFdfcHWk\nHMa7776L3+eHPwDntv1Ox28dPHbfYzEJc/tTN0lR+gNt5pIhRDcC+dKXv8RNP7upQ0x9pHrkXmKT\nveJUjwznBTgecMBzgAfy9uRRdVsV06dNj9vwxXOAB1eTq0MimdfrpfrealqaWyK5AI4HHFz742tj\nzllaWgo+P9tvhbn/a/ev/GQPh9x/v43sefllOPFE210L2Lh5c8wYWg9Ofbx8+LqOPXUsXz/q6zHJ\nWfvO2sfNP785st2xo4/FOAwMIua6OIudDB06VMVeySySeWxIxQt1+3RJjLukk0nVDm6JEW1++YTH\nTeAiSvTdkuolMTVqLrn0klh//9UIx1lfeEFJrCtEWlrE95Uj5NeOHAmCrDdIK4icdpqd0B071r6H\nXEBdRd2k7LrOCl2v9vMf7XMgoqN6ElwzdfUofQXq8x/4RES9E/+8SOKInETC1FVUi8PjsKI+DHEO\ncnbwZ0cmQKNFM84Y8wtsVNCaNWtkxxNPSHDYMNk3YoT98/vmN62PPyT4uxculBaPRxq2bBER6dWC\naJHff2Fo/Amub2R+JDqqx4HkH5ofGVOvJbopSgJU/LOAiKgnsEyjJ1XjiWUiYep2VEvUTSSauDel\nC2PDH/OuR17PMzLB7ZDBpYNlkschTR6P/dMrKbGW/uLFduyhcRZ1MnmaSuu6g+Wf4Mkq3nVye92R\nbOfefkJRlHio+GcJnYVThq3q9hE93RGmeDeLNWvWSEFJQac3mfB5OrijhrSFP+b8D/JaETLXIA1u\nZOnRSDPW1dP07W/bmP05c0S8XmnYsqVLAe0N6zr8+93F9tp6RnrE7XXLwpsWxp67kyeQzp6gFKW3\nUPHPIuKFIF5y6SWdCmJ3hKm9Nb1kyZIuY/bD+0TPAYRFM7zOXeyWsTlIg0FWlyACsm0Q8t0D3Pb8\n4Zj9vXu7HGdvWtfRbqxE9YviXafo9Wr5K32Nin8W0sHn3oVI748wRbZPUOBNJH7DlzVr1kjDp59G\nttmwYYN48p3CLGRpjhX+1SVIgwuZOMjZdv5QzH5X4+zu00gyJPNkoc1alL5GxT+L6a5VHy9hqVvH\njFObJl6ZCFe+Sza8/XZMlc1377lH/uLKlZPOQgLG3gAaDPIjR674vnJE3EStsIB6D/WKK98Vc8PZ\nn6eR7qxrb+GnwnrXaB+lL1Hxz2ISCvGGDSIiHTJ02/uxu3XMOBZ4dIG4HDdCEeLyumTVDdeLFBWJ\nzJ8vwaIimeRxyGsHIt8+x45tgtshe44+utPSDEuWLBGX1xUTJtqTp5FEE92JopjUb69kGir+WUp7\nn3v0hGV0PH5PLNnOXBgNDQ2SX+AWZlnhf+1A5Ntn2+NPHOSUloMOsn9W8+fL8trlkl/gjj1OlMXf\nfmK6rq4ubm5BTKhl1NNIXV1dQveX2+vucCy3122rl8YpUdEdF5qipBPJir+Wd8gw4vWHvfF/buT6\nBddDBTG9d53DnQnLPHTG9GnTGT9ufNxSEsXDhrFxxCHMeGQTL+Q388i3oO5huHkMXO4PIOaLSGet\n6StXMv6Dj+IeJ7pe/t5tezHG4BjsiFuaAmjrRDYC2A1NnzUxecpknEVO9vn2YQabmP1yC3Ihjw7r\ngsEg5NKhH3FjYyM11TVUVFbgGOageXtzpBOaogxEjL2B9OMAjJH+HkOmUFtby/kXnc++pn22I3IB\n8AE4/+jEdYCL3eftjmzrrfHSvKOZprOaIjVoPMs9bHp/U88FLRAApxPq62k5/XR+vnsnF+YJz3wV\nzv0bbDHg/d0TDJk82bZUnDcPXn0V3/btMTcAn89HyREl+Gf47W/4FbZWTvTndmNe9eKqiDAHPgvQ\n2tpK4JyA3e4l4DXggrb93A+5Mcbgn+mPWYfAvuC+2HM87GHTB5siY9Mm60omYIxBRNo3zeo+yTw2\npOKFun26RYdEr+jKknlYP3YXpRiSikBpaorJxJWTTxYBeSjXiC/HyE3OPPEXFLS1VRSRhk8/jUw2\nR0cGLVu2rC1yJ5xd20n2bPQ1iKlIugBhTii/IM58QNyEt9r4Pn+drFUyDZJ0+6jln8b4fD7WrVsX\nWZ5aOZVdZ+yy1jHEVpZc5iDPkYejqM1lMX3a9NRasnfcAVdeCUcdZdspTpiArF7N5ksvxTNvHsXv\nvBOx9msffTT2KWUE8CrwMuQfmM+ebXvanl6iLf8PwFXn4sH7H2TIkCExlTKjr0vJESW2heIrgBe4\nFNhDpP3kS4+/xKhRozr8/vA13blzJwBDhgzhww8/5IofX5GwZaOipCNq+Q9Q2kelODwO23xlNsJJ\nJGzCnjLrNVGDlRNPlEg7RRGRVati2ym2tMQvR9G+bs74kIU/Ml9yXbmS48qJ/NZcV644BzljonQ6\nJKJVh0I/Z9FpvaP21zQc/RO+ngWHFvQohFRR+huStPy1pHMa4vP5OP+i82mWZmsdXwbNs5qRVsHz\nsAfvP73wBR1KK5eVlTFq1KjkLfxAAIqKrKUP9r2oCBYvhldesWWX33rLrh83LrZxem5uWznow4Gd\noXHuxDYzD0+0jgFXoYvmHc24i920Blvtb62AoAkSOCfArnN34Z/hZ9b5syg53JazLjmihNoVtRxT\ndgwFBxfYfnGnYpul/wpcD7niTtRGN5rfdcYumqWZwDkBdk/aDcOIOzEOsaW0w+dWlIGARvv0I4lc\nMhs3brTRKtChbvzvfvM7hg4dypvr3uSKq69IbWRKeELX6YSFC62LJ9xg5dZb4cYbbaOVyy+3wn/D\nDXDxxXb7KEpLS9saxZwKLAPcQCNtETsfQtMXTVABgWAAng6t/xgYGvW7C6C5pZnmWc2RSKaKygre\neP0NWna02OP9O5APrsdcrFu7juHDh7N27dqY6xq+IflH+GPPsYe2G2m7JjTRN4zoc48fN14ng5WM\nRy3/fqIzi7K0tJTg7iDsIMa6b93ZGrHuKy+qtFEwj65i0/ubkvdRt7f2AYxpa7By9dXw+eeRRitc\nfrldbif80NYoxrPcQ+FfC3HnuVl41UKW3L3ErltWiPNRJ67hLiu4Q2gT4CHE/u4PsHMB7SzzcGhm\n+HieJzws/c1S1q9fH/e6Rm5I7c+RDxwP1EDB/QV4lnsiN9KYhjZ7gCDkDs5NaTMZRek3kvEZpeJF\nFvr8OyulHPYtJ4pK6VWqqtqaqoB9tWuwsj8kKq2w8KaFNtnK0XEOoOBLBRF/fEyj+S56EndWwTR8\nXaOjn6LPEVOfKF5dpE4yixWlv0CjfTKPtWvXMmHqBHadu8talJ+C4/cOclpycBe7afqsieuuuY4f\nnP4DNm/eDBA36iVpgsE2X314edw4a+3n5MDtt8e6eBJY+vtDTIz/Z9h+uINs7+Cq26o4puyYSMvJ\nsEssOsY/OpKpPTHXNYT7bjfiF9zD3QS2B6haHP8cCXsi31vN7Etmt0UspSJfQlFSgEb7ZCAxFqUz\nVP8+bAV30qYxZbS02OidcCG2lhYbtVNS0mbtQ6y130k9nv2hQw2dq5H8g/Klrq6u0/26E3ETr9ZR\nV5E83Rlvb1cTVZSegEb7ZA4+n4+1a9cCULW4Cuqxsy6nAcOxvu1nsDHvl4F/pp+KyooOjdp7TDBo\nX8cfD9/6FkyZApMnw4gRMGkSbNsGVVU2O7eqylr7gYDdN0mLP0yM7x1gN7TusXMZnVFcXNxlJFPM\nXMOyQlyPufAc4EkYydPd8UYmliFmQlhRMpmkxd8Yc4gx5iVjzAZjzNvGmMtC64caY543xvzdGFNn\njBmc/HAzl/YTvD6fj0HDB9mok3BI5AfYycgkxCohfj+MGWNDNW+5BTZtspO4e/bAZ5/BzJmwa1e3\nJnSTob1AR0+wpoLp06ZHJsLXrV1nI46SEO7eHq+i9BdJ+/yNMSOAESKy3hjjBd4AvgecB3wuIj83\nxvwEGCoi18TZX5IdQ7oT4+dOVGfmM+D3QCsd/MtvvP4GjY2NPc/SDUfyzJoFjzwCxx4LdXVt348d\nC++8AytXQnl50r+3O/RVDZ3aFbUxNYGuu+Y6Ki+q3O9zas0fJd1IO58/8CQwHngPODC0bgTwXoLt\nU+4LSzcS1YpfeNPCDlm8Z0w9Y7/aNHbK3r1tn6uqJNI0HURycmwUj8slUlhov4/O1M0gupoPiG5o\nk8r+v4rSn5BO0T7GmFKsJ/vfgc0iMjTqu89FpCjOPpLKMaQj8Sz/cMQIEKnfE47oCVuZXq+XY0cf\nG3e/Lq1Pvx+8XvjhD+Guu+CSS+Duu+13Tie0tNiM3WOOgTlzoKDAJnOl2M2TLPEs7uh1q1atipSG\nTlSXp7Prr1a8kqkka/mnLMM35PJ5DJgrIo3GmG4r+oIFCyKfy8vLKe8j10NfEfYbJ6oVP3HixA7b\nAzz77LPkDc3rfk3+6NBNjwcqK63g/+EP1scPcPTRtjRDZaV188yda5chNuwzDYiu+R8WdoTIuqbP\nmiKlnTvLwI3J7oX96m2gKOlCfX099fX1KTteSix/Y0weNmL7jyJyZ2jdu0C5iGwLzQusFpEj4+w7\n4C3/MN31G4dFL29wHru37u5ejHkwaCd0Fy2yfvtwPf1PPoGPPrLb3HYb/OhHbXH7DQ32JpGGJJon\nianR/xb2OfOytv0KlxWy6tFVMRU9k3qCUpQ0JVnLP1WhnvcDG8LCH+Jp7FQmwCzgqRSdK2MJhyuC\nTUiKF8IZXU9md8VuGEvc0gMdyM21wn/GGbBggX0fMcIKf0mJ3ebDD+17OJInTYUf2qz19p24cobk\ntK07nITRPNHRVceOPpaKWRUasaMo0SQzYRCy2I8HgsB6YB3wJjAJWytxFfB34AVgSIL9UzoJku7E\nayoeTbzJYe+hXlm2bFn3kpPmz7cTutdeayd158yx6+fMscvRk8BpTLxSDfH68jo8jpgJ8piG752U\nz9ifcWg5ZyUdQRu4pz9hAdmwYUPcBuXhujSJmpF3Oyt19WqR4cPtDWD4cJE//jH2+wwR/jCJOnG1\nX9deoBNFV+1vVm5XN2pF6U9U/NOY9iGGua7cuE1YFt60MEZkwuGd+9V+saWlrVyDiH3P0NDNaBIV\nh+sqtLPHN9AUHkNRepNkxV8Lu/UScZut/xIwxLRfdP/WjcmJbTTe48SueIXa0iyCp6+ITu7qrBhc\nIuIViYueTFaU/iZtQj2VNsKTtvsm7oP/pa1JyTDsDMkDwGDgc5hVMYsVz63oEIbY2Ni4/yLTXuiz\nVPjBlnkYP258j7NyY2oQtWvyoigDAS3s1gvEbWMYbiAyHLgE+C9wu9zMvWxubKEzFZmU0Z1icJ3t\nqzV9lIGMun16gUQ16x17HZgcg/sAd4wrIlkXhdJ7aE0fJV1J1u2j4t9LJCooBvEbiMSIzLBh6rtX\nFKVTVPzTiPZWYo+sxkSZuq++qjcARVEiqPinCfHq0PTYdVNfbzN058yxtXn6sNSyoiiZgYp/GtCh\nDs2H4HrMxbq16zjyyA7ljLrHggXw05/C/Pn2s7JfqK9eGeikS22frCamDs3/AY9Ck7uJsuPKqF1R\nG3+nYDDxcn29tfjnz7fvKazklw2075qW8N9AUbIYtfxTQMTy/74fHsWWseusemRnfn1Qn38SaO1+\nJVtQyz8NCMeEux5zwSC67sEbrwLnokV2fW6uFfqwj7+8XIV/P4hXDTRlfZAVZQCh4p8ipk+bzrq1\n63A1ueInbLV385xwgp3Q/elP7Xv0hK5m6vaYmMxc0KQ5RUmAin8KOfLII1l639KOWaHDhllXTth3\nX18PRx1l2yuqXz+laGauonQP9fn3Aj6fj3V//Svk5kb68vLiizBtWlv4ZnEx/PrX6tfvJTTaRxno\naKhnPxNPZFY8/DCHzTqbhcMH8ZK/ld/P/RHjXlgFEybAwoXW2r/hBs3iVRSlx6j49yPxErvGjxtP\nyRElHDfWz8oX4e4jYc6b4LlxId4779TELUVRUoKKfz+RKKTwyZVPMrVyKrvO3cX81bDgZbhlsJPZ\nBx/O1ssuo+gHP6D4nXfUzaMoSlJoqGc/EQ4pzDkgtCIUUkgwSGB7gLFvwJy1sOBYqNgVYPqH7zP6\n1mts0tHWT1X4FUXpV9Ty7yG+rVs5/Cul1OU3MW8SvOKCib918PTXj+J3l8+l9NxZ3Djcw0t7WhnT\n3MKNQ1oYUwmtDZp0pChK8qjbpz8IBuH449n+0Ufc/FkDN7QECQgU5eSQ8/zzMG4cvq1b2bh5Mzt2\n7GBq5VR2n7OL1tBzlrYDVBQlWdTt0x/k5vLihAnkfPopN7YE8QgUAzlnnw3jxgFQPGIEo0aNoqys\njMD2AK0NoX016UhRlDRAxb8H+Hw+vvvL27njWBgk4AT+lGNo/cMfOiRradKRoijpSK+7fYwxk4A7\nsDeaGhG5td33Gef2Wbt2LT/97rd5aMceCpohYCAg8OlFlRz+17+y/q67KD388MSdulT4FUVJkrT2\n+RtjcoB/AOOAT4C1wDQReS9qm4wTf9/Wrfxr5EF4B8Ol/w6tr8PdQdgtMM6Vh2NEfvINXRRFUToh\n3cV/NDBfRL4TWr4GkGjrPxPFH2DF8oepuKiCvYEmqICcfOAuaD0PLSWsKEqvk+4TviOBzVHLW0Lr\nMp5pM2byxO+eJn9EPoyA1i+gdRhaSlhRlIwgr5ePH++u1MHMXxDVprC8vJzyDCl7UFZWRuuuVls+\neAiwA/s5ZPlrVI+iKKmivr6e+hRW/+0Lt88CEZkUWh4wbp8wtStqqaiswDHMgX+bH2MM7gPcNG9v\nVp+/oii9Rrr7/HOBv2MnfD8F1gDTReTdqG0yWvwhNpIH0KgeRVF6nbQWf4iEet5JW6jnLe2+z3jx\nVxRF6WvSXvy7HICKv6Ioyn6T7tE+iqIoShqi4q8oipKFqPgriqJkISr+iqIoWYiKv6IoShai4q8o\nipKFqPgriqJkISr+iqIoWYiKv6IoShai4q8oipKFqPgriqJkISr+iqIoWYiKv6IoShai4q8oipKF\nqPgriqJkISr+iqIoWYiKv6IoShai4q8oipKFqPgriqJkISr+iqIoWYiKv6IoShai4q8oipKFJCX+\nxpifG2PeNcasN8Y8bowpjPpunjHmn6HvJyY/VEVRFCVVJGv5Pw98Q0S+BfwTmAdgjPk6MBU4EvgO\n8GtjjEnyXGlJfX19fw8hKXT8/Usmjz+Txw6ZP/5kSUr8RWSViLSGFl8HDgl9Pg1YISItIrIRe2M4\nLplzpSuZ/gek4+9fMnn8mTx2yPzxJ0sqff7nA8+GPo8ENkd993FonaIoipIG5HW1gTHmBeDA6FWA\nANeJyO9D21wHNItIbdQ27ZEkx6ooiqKkCCOSnCYbY2YBFwEniUhTaN01gIjIraHl54D5IvKXOPvr\nTUFRFKUHiEiP51KTEn9jzCTgduBEEfk8av3XgYeB/8C6e14AviLJ3mkURVGUlNCl26cLfgU4gRdC\nwTyvi8jFIrLBGPMosAFoBi5W4VcURUkfknb7KIqiKJlHv2X4DoQEMWPMJGPMe8aYfxhjftLf4+kM\nY8whxpiXjDEbjDFvG2MuC60faox53hjzd2NMnTFmcH+PtTOMMTnGmDeNMU+HlkuNMa+Hxl9rjEn2\nabbXMMYMNsasDP1dv2OM+Y9Muv7GmCuMMf9njHnLGPOwMcaZztffGFNjjNlmjHkral3C622M+WVI\nd9YbY77VP6NuI8H4U6ab/VneIaMTxIwxOcBdwMnAN4Dpxpiv9e+oOqUFuFJEvg78JzAnNN5rgFUi\n8lXgJUL/DmnMXKw7McytwO2h8e8EKvplVN3jTuBZETkSOBp4jwy5/saYg4FLgWNE5Cisy3g66X39\nl2L/f0YT93obY74DHC4iXwEqgSV9OdAExBt/ynSz38R/ACSIHQf8U0Q2iUgzsAL4Xj+PKSEislVE\n1oc+NwLvYq/594AHQps9AEzunxF2jTHmEOAU4DdRq08CHg99fgD4fl+PqzsYYwqAE0RkKUDo73sX\nGXT9gVwgP2Tde4BPgG+TptdfRF4FdrRb3f56fy9q/YOh/f4CDDbGHEg/Em/8qdTNdCnslokJYu3H\nuYX0HGcHjDGlwLewfzwHisg2sDcIoLj/RtYlVcDVhHJGjDFFwI6o/wxbgIP7aWxd8WXgM2PM0pDb\n6l5jzCAy5PqLyCfYyL6PsP8ndwFvAjsz5PqHOaDd9T4gtD5TdCeapHSzV8XfGPNCyD8Yfr0dev9u\n1DaZmiCWKeOMwRjjBR4D5oaeANJ+zADGmFOBbaGnl/C1N3T8d0jX35MHHAPcLSLHAHuwLoh0HW8M\nxpghWOu4BCvw+Vj3Qnsy4vfEIaP+P6dCN3t1ckZEJnT2fShB7BTso3uYLcChUcuHYB8v040twJei\nltN1nBFCj+uPAb8VkadCq7cZYw4UkW3GmBFAQ/+NsFOOB04zxpyCdTkUAHdgH89zQtZnOv8bbAE2\ni8hfQ8uPY8U/U67/eOBfIrIdwBjzBPBfwJAMuf5hEl3vTNGdlOlmf0b7TAJ+DJwWzgwO8TQwLRRJ\ncBhwBLCmP8bYBWuBI4wxJcYYJzANO/Z05n5gg4jcGbXuaeDc0OdZwFPtd0oHRORaEfmSiHwZe61f\nEpGzgNXAGaHN0nn824DNxph/C60aB7xDhlx/rLtntDHGHZpIDI8/3a9/+6fD6Ot9Lm3jfRo4B8AY\nMxrrztrWN0PslJjxp1Q3RaRfXtgJiU1Yv+GbwK+jvpsHvI+dlJzYX2Psxm+YBPw99Fuu6e/xdDHW\n41cTN4MAAACoSURBVIEgsB5YF7rmk4BhwKrQ73gBGNLfY+3GbxkLPB36fBjwF+AfwCOAo7/H18m4\nj8YaDeuB3wGDM+n6A/ND/yffwk6WOtL5+gPLsdZvE/bmdR4wNNH1xkbvvQ/8DRvVlI7jT5luapKX\noihKFpIu0T6KoihKH6LiryiKkoWo+CuKomQhKv6KoihZiIq/oihKFqLiryiKkoWo+CuKomQhKv6K\noihZyP8Hlbxmpg4K2/0AAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x1174b0050>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "opt = tf.train.AdamOptimizer().minimize(loss)\n",
    "\n",
    "with tf.Session() as sess:\n",
    "    sess.run(tf.global_variables_initializer())\n",
    "    for _ in range(5000):\n",
    "        # Select random minibatch\n",
    "        indices = np.random.choice(n_samples, batch_size)\n",
    "        X_batch, y_batch = X_data[indices], y_data[indices]\n",
    "        # Do gradient descent step\n",
    "        _, loss_val = sess.run([opt, loss], feed_dict={X: X_batch, y: y_batch})\n",
    "    # Define input data\n",
    "    indices = np.random.choice(n_samples, batch_size)\n",
    "    X_input = X_data[indices]\n",
    "    y_predict = sess.run(y_pred, feed_dict={X: X_input})    \n",
    "    # Plot input data\n",
    "    plt.scatter(X_data, y_data, c='g')\n",
    "    plt.scatter(X_input, y_predict, marker='x', c='r')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Binary Classification\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import random\n",
    "\n",
    "def generate_input():\n",
    "    batch_xs = []\n",
    "    batch_ys = []\n",
    "    for i in range(64):\n",
    "        batch_xs.append( [random.uniform(0, 100), random.uniform(0, 100)] )\n",
    "        batch_ys.append( [float(batch_xs[-1][0] > batch_xs[-1][1])] )\n",
    "    return batch_xs, batch_ys"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.collections.PathCollection at 0x118574890>"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX8AAAEACAYAAABbMHZzAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xd4FGXb9/HvmYQQmjTpVUCRpohURU1oUqSIFAUFFQXl\n9hG7yO1zG9RXsaCgKCDwICC9SBERgRCKgNQI0nsnhpAE0rPZ6/1jl70DgpDsJrObPT/HkcOZ2Z3Z\nX0Zy7uw518yKMQallFL+JcDqAEoppfKeFn+llPJDWvyVUsoPafFXSik/pMVfKaX8kBZ/pZTyQzdd\n/EVkkohEi8jOLMs+FZG9IhIlIvNF5JYsj70jIgedj7fzdHCllFI5l50j/8nAw1ct+xWoZ4xpCBwE\n3gEQkbpAL6AO0AH4VkTE/bhKKaU84aaLvzFmPRB31bKVxhi7c3YTUNk53QWYZYyxGWOO4XhjaOp+\nXKWUUp7gyZ7/s8DPzulKwMksj512LlNKKeUFPFL8ReTfQIYxZublRdd4mt5HQimlvESQuxsQkf5A\nR6BVlsWngCpZ5isDZ66zvr4pKKVUDhhjcnwuNbtH/kKWo3oRaQ+8BXQxxqRled5i4HERCRaR24Ba\nwObrbdQY47M/7733nuUZNL/1Ofwxvy9nzw/53XXTR/4iMgMIBUqLyAngPWAYEAyscA7m2WSMGWyM\n2SMic4A9QAYw2HgirVJKKY+46eJvjOlzjcWT/+H5HwMf5ySUUkqp3KVX+LopNDTU6ghu0fzW8uX8\nvpwdfD+/u8TqboyIaEdIKaWySUQweXjCVymlVD6gxV8ppfyQFn+llPJDWvyVUsoPafFXSik/pMVf\nKaX8kBZ/pZTyQ1r8lVLKD2nxV0opP6TFXyml/JAWf6WU8kNa/JVSyg9p8VdKKT+kxV8ppfyQFn+l\nlPJDWvyVUsoPafFXSik/pMVfKaX8kBZ/pZTyQ1r8lVLKD2nxV0opP3TTxV9EJolItIjszLKspIj8\nKiL7RWS5iBTP8thXInJQRKJEpKGngyullMq57Bz5TwYevmrZUGClMaY2EAG8AyAiHYCaxpjbgUHA\nOA9kVUop5SE3XfyNMeuBuKsWdwWmOKenOOcvL5/qXO93oLiIlHMvqlJKKU9xt+df1hgTDWCMOQeU\ndS6vBJzM8rzTzmUqB6KjExk/fqtrftOmUyxffsjCREopXxeUS9uVaywz13tyeHi4azo0NJTQ0FDP\nJ/Jhdrvhyy83cf58Mq1b16BLl5l8/303q2MppfJQZGQkkZGRHtueGHPdmvz3J4tUA5YYY+5yzu8F\nQo0x0SJSHlhtjKkjIuOc07Odz9sHPHT5U8JV2zTZyeCvzp69RMWKXwCwdGkfOna83eJEKqeMMcTH\np1KyZCEAkpLSKVAgkODgQIuTKV8iIhhjrnWgfVOy2/YRrjyqXww87Zx+GliUZXk/Z8DmQPy1Cr+6\necePJ7imd+w4a2ESzxg1ahMffbQOgJSUDLp1m5Uvfq+bERFxlGbNJnLmzCUSE9Pp2HEGEyZsszqW\n8jM33fYRkRlAKFBaRE4A7wEjgLki8ixwAugJYIz5WUQ6isghIAl4xtPB/cmePTF06TKTpUv7cM89\n5QkLm8IttxTkf/6nmdXRcqx373qEhU0hNdXG77+fpnTpQjRo4B9jAlq3rsGzz95DkyYTKFy4AA8+\nWJUXX2xidSzlZ7LV9smVANr2uaH09Eyios7RtKnjnPnZs5dITs6gZs1SFidzz5EjcdSs+RUAGRn/\nS1CQ/1xzmJiYTrFiHwNw8uSrVK58i8WJlK9xt+2jxV9ZwtHqmU16eiYnTybw7LP3MGzYA1bHyhOJ\niel06jSDWrVKUqtWKSZPjiIy8mkqVixmdTTlQ9wt/rk12kepf/TFFxspXboQU6c+SkxMEmFhU2jX\nriaNG1e0Olqu+/PPv6hfvwxff92RgAAhIEBYu/Y4jz9e3+poyo/okb+yRHp6JgEB4mr1pKRkUKhQ\nAYtT/d3mzaeZN28PI0a0JiAggG++2UypUoV44okGVkdTfk7bPkrlotjYZO64YwzVqhXn6afvZsSI\n38jMNERFDaJCBW3TKOvk9VBP5UPWrDnGoEFLsNsNxhjefTeCOXN2Wx3Lp5QuXZitW59nx45zDBmy\nnPT0TCZO7Oy3hT89PZPMTLtrPiUlw8I0yh1a/POxxo0rsn9/LAMGLGbYsFUsXryfVq1uszqWz1m6\n9CCFCjlOj2Vk2OnUyX8vsPvoo3U8++xiMjPtnD59kYYNxxMVdc7qWCoHtO2TzyUlpVO0qGNIYXT0\nG5QtW8TiRL7l118P8/TTC0lPz+SDD8J4440VNGtWiVWr+iGS40/cPis5OYNHHplBQIBw9Gg8gwbd\ny1tv3W91LL+kPX91XZdbPbNn7yYkJIimTSsxcWIXAgL8r2jlVGamnVdeWU67djXo3Lk2x47FM2TI\nMsaP70z58kWtjmeJgwdjueOOMQDYbP9LYKA2EKygxV9d17JlB3n77ZVERPSnUKEgOnWaQd++DXj+\n+XutjvaPMjPtZGTYCQlxtFqSktIpXLgA+/fHkp6eyV13Oa4EXrHiMI0bV3TdI0flvtOnLxIaOoW+\nfRuwdu1xqlQpzv/9Xxd9A7CAFn91XcYY55WkBQHHR/bg4ECvv5J2/PitLFiwj4ULe5OcnEGbNtMY\nMaI1yckZDB78M8uXP8nhwxd44YWlrFjxlOvNQOW+zz77DWPgrbfuJzk5g1695vLxx6395tYc3kSL\nvx8yxhAeHsngwU0oV64ocXEpjBy5kfDwUK8v7DfDZrPTr9+PHDgQS1JSBl263MGIEW0QEebM2U3v\n3vMA2LZtII0aVbA4rcpr8cePs3nMGNqMGEFAYCCHfvmFi6dO0ei556yOlqd0qKcfEhFEhFatprJv\n33natJlGSkoGgYH5o5cfFBTAqFHt2bbtLPv2nSc8PNR1crVAgYArnqf8T5EyZTi3fTuLnnmGA0uX\n8mO/fpSpV8/qWD5Hj/x92Cuv/MLo0b/Tt28Dpk17NN+MPomNTaZNm2m0alWds2cTiY1NYeHC3qxY\ncYTnn1/CsmV9OXToAkOG/MLKlU9Rr17ZG29U5SsZycl8VMQxcu2Z9euper//jTjSe/v4qbi4FNat\nOwHAjh3n+OuvJMqVyx+jT5YvP0y7djUYMaINmZmGAQMWs3nzaWrWLMmyZX1p1KgCjRpVoGjR4Hzz\nO6vsOb52LQWKFAFj2DZ+PJWbNycgUL8MJzv0yN8HGWNo2XIyzZtX4vPP2zF8+Bp+/HEf27YN1FaI\nyvfO79vH5Acf5PFFiyh/993M7NyZyi1a0OrDD62Olqf0hK+f2rMnhjp1bnW1enbv/kvbH8ovGGNI\nOH6cEtWrA44WUNqlSxQt518jjrT4K6WUH9LRPirfS07O4M8//3LNHz0aR0xMkoWJlPJ9WvyVR6Sm\n2v5x3h1bt56hdeupbN16hiNH4ggNncLKlUc8tn2l/JEWf+U2Ywxt2kxlypQoAFauPELDhuNIS/PM\nG8CDD1ZjwoTONGkygZo1v2Lo0Pt99stUdu6Mxmb77y2Rd+w4a2Ea/xJ/7Bhbx41zzR9esYIjq1ZZ\nmMhaOtRTXZcxhgsXUihdujAAly6lUaBAoOueO5eJCBMndqF166msXHmU5csPMX9+LwoW9Nw/r/r1\n/3syu0mTSh7bbl774IO1iMCMGY8xZsxmvv56M3/88QJFiwZbHS3fCyhQgI0jR5J28SLl77mHBX37\n0nvBAqtjWccYY+mPI4LyRitXHjY1a442J07Em4sXU819900y33yz+brPHzPmdwPhZsCARR7Ncfjw\nBVO16pfm2283m0WL9pmyZT8zW7ac9uhr5JWUlAzTocMPBsJN1apfmuPH462O5FcSTp0y4WDCwRxf\nt87qOG5x1s4c115t+6jrat26BoMHN6Fx4wnUrz+WBg3K8sILja/53JUrjzB8+BomTOjMsmWHXC0g\nT7DZ7AwfHsqLLzahS5faTJzYmfT0TI9tPy+FhATx0EPVAKhYsRgVK/rnN4JZJWbPHtf0yQ0bLEzi\nBdx557j8A7wK/AnsBKYDwUB1YBOwH5gJBF1n3dx8c1Ruungx1UC4gfDrHqXa7XbTrdsss3btMWOM\nMXv3xpiwsO9NampGXkb1CV9+udHUqDHaHDhw3nTo8IPp2XOOycjItDqWXzi9ZYv5tEwZc3zdOpNw\n6pT5qlYts/W776yOlWO4eeTv9jh/EakIrAfuNMaki8hs4GegIzDPGDNXRMYCUcaY8ddY37ibQeWO\nS5fSaN9+Og0alOX220sxduxWVq/uT5Uqxf/2XGPMFfcWunpeOYwatYnu3etQtWpxUlNtfPrpb7z1\n1v1/O4+iPM+Wlsb5ffsof/fdAFw8fRoRoVjFihYnyxlvGecfCBQRkSCgEHAGCAPmOx+fAjzqoddS\neWTPnhiaNKnIt9924vXX72Pw4CZs3Hjqms+9utBbUfiPHYvnq69+d82vWHGYZcsO5nmOf/LKK82p\nWtXx5hkSEsR//vPQDQu/3W7o3Hkmf/zh+K7cnTujeeSRGVd8kbq6saCCBV2FH+CWSpV8tvB7gtuH\nG8aYMyIyEjgBJAO/AtuBeGPM5X+dpwD/3cs+qlmzyjRrVtk1/9prLSxMc2MhIUGMG7eV+PhUWrSo\nTN++C1iwoLfVsdwWECD063cXDz/8A5991pY331zB6NHt9duzlFvcLv4iUgLoClQDEoC5QIdrPPW6\nvZ3w8HDXdGhoKKGhoe7GUn6ofPmiRET0p0KFkQCsW/cMLVtWtTiVZ/TsWY+dO6Pp128hw4a1pHfv\n+jne1scfr6NIkWBefrkZSUnpPP74fD79tA116pTxYGLlaZGRkURGRnpse55oNLYBjhhjLgCIyI/A\nfUAJEQlwHv1XxtEKuqasxV8pd+zaFe2ajog4mm+K/86d0UyYsJ0ePeoyadIOevWqx913l8/Rtvr0\naUBY2BSSktL59dcjVK9egjvuKO3hxMrTrj4wHj58uFvb80TxPwE0F5EQIA1oDWwBSgM9gdlAf2CR\nB15Lqevavv0sffsuYN26Z6hVqxStWk2hdOlC/OtfTa2O5ha73fDUUz8yenR7eveuz9y5u3nqqR/Z\nsWNQjlo/1aqVYOnSPtSt+y0ANtv/agvJD3nkrp4i8h7wOJAB7ACew3G0Pwso6Vz2pDEm4xrr6mgf\n5REZGZns23fe9WXi584lYrebfDGWPikpnSJFgq87n91tPfLITETgyJE4XnutBS+/3MxTUVUe0Vs6\nK6Wy5d//XsWZM4lMnNiZU6cu0qrVVBYtevyKW2go76fFXymVLampNgoUCHC1epKTMyhcuIDFqVR2\necs4f7+TmWnn1KmLrvm4uBQuXUqzMJFSNyckJOiKHr8Wfv+kxT+HNmw4SfPmE9m//zxxcSm0aTON\nqVP/uKl1s37S0U89/uXChRR+/fWwaz4q6hx798ZYmEj5Ky3+OfTAA9X48MNWNGr0HVWrjiI0tBqD\nBze54XoLFuzliSfmY7PZsdnsPPHEfBYs2JsHiZU3OHXqIv36/ciCBXvZseMs7dv/wIEDsVbHUn5I\ne/5uiItLoVSpTwHYu/df3HnnrTdcJzXVxqOPzqZIEcdH7aSkDH78sbfe28WP7NhxlkaNvgNg3rye\nPPZYXYsTKV/kbs9fK04OXW71vPZacxo0KEebNlNZtaoftWv/8xtASEgQc+f2pFixjwG4dOkdLfx+\nTG9+p6yiVSeHTpxIoFOn2xk+PNT1B7xnT8wNi7/NZmfAgMWEhVUHYMCAxUyf3p2gIO3A+YNdu6Lp\n0GE68+b1pEaNkrRvP53AQKFr1zutjqb8jLZ98ticObuZPDmKH3903HDs0Udn88wzDenVq57FybxH\nRMRREhPT6dKlNgBjx26hdesa+eIWBLGxyWzdeoaHH64FOFpABQsGUbeu3ldHZY+O8/cxxhgyM43r\nSN9msxMYKPrxP4utW8/QqdMMJkzozIEDsYwbt5XIyKepXPkWq6OpXGDsdpD//g3YMzMJCAy0OJX3\n0+Kv8qWtW8/QpMkEAE6efFULfz62fsQIEk6coOM335AaH8/09u3pMGYMlZrcePScP9OLvFS+FBl5\nzDW9fftZj247NjaZ5csPueZ37DirY+0t1Oi55zizdSsL+vZlWps2VLj3Xirce6/VsfI9Lf7K60ye\nvINx47Zy8uSrbNnyPM8/v4Tffjvhse2fPn2J/v0Xusbad+gwnYMHL3hs+yp7ds+Zg4jw58yZnN2+\nnaMRERyLiLA6Vr6nbR/ldWJikkhLy3S1evbsiaFmzZIULOi5wWlZx9rPn9+L7t3reGzbKnuSY2MZ\nU7s2dpsNW0oK5Rs2ZMCmTXoe7Aa07aPynTJlilzR469bt4xHC39+ZUtNZcvYsY4TqEDCyZPsnjvX\n4lQ3tmPSJOp0705aQgKZ6elIYCBnt22zOla+p8Vf+Z2dOx1j7efP78X27QMZPHgpixfvtzqW2zIz\nMtg5bRo/v/QSCSdOMCUsjIunTlkd64Ya9O3L8bVrCfvwQxo+8wwBBQpwax39JJbbtO2j/M61xtqH\nhATli++wTbt4kW/r1ePiqVO0/fxz7nv9dasj3dCfs2YRd/QoD7zzDsZu5+eXXuKuJ5+kyn33WR3N\nq+lQT6WUS8LJk3x3770kx8TQ+MUX6ThmDBKgH/DzI+35K6UASD5/nilhYbQcOpShCQmci4ri1zfe\nsDqW8lJ65K9UPmHsdo5GRFCjTRvA0QKK2buXys30+3nzI237KKWUH9K2j1JKqWzT4q+UUn5Ii79S\nSvkhjxR/ESkuInNFZK+I7BaRZiJSUkR+FZH9IrJcRIp74rWUUkq5z1NH/qOBn40xdYC7gX3AUGCl\nMaY2EAG846HXUkop5Sa3i7+IFAMeMMZMBjDG2IwxCUBXYIrzaVOAbu6+llK+ID09k3ffjeDSpTQA\nzpy5xIcfrkVHtSlv4okj/xrAeRGZLCLbReQ7ESkMlDPGRAMYY84Bvn/t/E3YtSuaoUNXuv7Qp079\ng9mz/7Q4lcpLQUEBnD+fTPv209m//zyhod8TFBSgd6nMIjE6moh338WemQnAyQ0b2D5pksWp/Isn\nbpUYBDQC/mWM2SoiX+Jo+dz0YU54eLhrOjQ0lNDQUA/EskbVqsVZs+Y4L730M02bVmLYsAhWrepn\ndawcs9sN3brNIjw8lEaNKrB791+88cYKlix5Qr90/joCAoRvv+1Et26zuPPOb/j3vx9g6NCWVsfy\nKsFFi3JywwYWDxjAPQMGMOexx3h02jSrY3m1yMhIIiMjPbY9ty/yEpFywEZjTA3nfEscxb8mEGqM\niRaR8sBq5zmBq9fPdxd5JSSkUqLEJwDs2TPY528YtnDhPgYN+onPP2/L22+v5PPP29GnTwOrY3m1\nM2cu8dBD33Po0AXuu68Kv/zSl2LFClody6ukJyXxaenSZKal0efnn7m9QwerI/kUyy/ycrZ2TorI\nHc5FrYHdwGLgaeey/sAid1/LVyxatJ+iRYMpUSKEMWM2+3yvt1u3O3n55ab067eQPn0aaOG/gbQ0\nG61bT2XAgHvIzPwPDRqUpXv3OT7/78DTov/4g4AgR/Nh9+zZrhaQyhseub2DiNwNTAQKAEeAZ4BA\nYA5QBTgB9DTGxF9j3Xx15L9x40l69JjLqlX9qFChKO3bT6dXr7q8+moLq6Pl2O7df9G27TRatqzK\nmjXHWbasL40aVbA6llf788+/qF+/LOBone3dG0O9emUtTuU94o4eZWKzZjw6bRpVW7ZkZufOVGzS\nhLaffGJ1NJ+h9/bxMna74ezZS1Sq5PgmqoSEVESEW27xzY/8druhSZMJvP56C/r0acDChft4990I\noqJe0J5/LjuzbRu31q5NcNGiABxfu5aqDzyQL04cG2OI3b+fW++8E3C0gFJiYyletarFyXyHFn/l\nEcYY3n03goED76VatRLExaXwwQdrGTGiDTabncKFC7iem5ycccW8yh3Lhgzh3Pbt9F22jKgpU9jw\n2Wc8v2ULRcr49jkk5RmW9/xV/iAilCtXlEaNvmPChG20bTuNzEw7r7yyjISE1Cueq4U/b7T/8ktK\n33knHxcrxprwcJ6OjNTCrzxGi79yefnlZnTpcgcDB/5ESEgQR47Ece5cErfeWtjqaH5JAgIod9dd\nAASFhFD41lstTqTyEy3+yiUuLoVdu/4iIED47beTrF17gtmze1CgQKDV0fzS5m++YePIkbx8+DA1\n27dneocOpCcmXvf5xhjsNptr3m6z6QgjdV1a/BXgKBzdu8/h/vur0LFjLQoXLkBiYjoREUetjua3\nCpUqxdORkZSsUYPO48dTp0ePf/w+3t1z5jCzSxdsqanYUlOZ8cgj7Jk3Lw8TK1+iJ3yVy+HDF/ji\ni42cPZvI7Nk9mDNnN6++upxt2wZSpYrelNXb2W025vfpQ/L582AMRcqVo/sPP7jG0qv8RUf7KI86\ndy6R0qULuVo9J08maOH3IemJiXxcrBgA7yQmElykiMWJVG7R4q+UAsCWmsqsbt0c1wUYQ0ZKCr0X\nLCAoJMTqaCoX6FBPpRQAB5YupVDJkvSYNYses2dTsFgxDi5bZnUs5aX0yF+pfMQY47oCOOu0yn/0\nyF8p5ZK12GvhV/9Ei79SSvkhLf5KKeWHtPgr5UdWvP02+xcvBiA1Pp5Z3bqReO6cxamUFfTqD6X8\nSL2ePZnRqROpCQls/vprKrdoQZFy5ayOpSygo32U8jNHVq5kWtu2lLjtNl4+fFhPDPsoHe2jlLpp\nqfHxrBo2jCr33Ud6YiIHliyxOpKyiBZ/pfzIL6+8QuUWLXhm/Xr6/vwzP73wgvb8/ZS2fZTyI2kX\nLxJcrJir1ZOakEBIcb13ky/Se/sopZQf0p6/UkqpbNPir5RSfkiLv1JK+SGPFX8RCRCR7SKy2Dlf\nXUQ2ich+EZkpInpBmVJKeQlPHvkPAfZkmf8EGGmMqQ3EAwM8+FpKKaXc4JHiLyKVgY7AxCyLWwHz\nndNTgEc98VpKKaXc56kj/y+BNwEDICKlgThjjN35+CmgoodeSymllJvc7sOLSCcg2hgTJSKhlxc7\nf7K67mD+8PBw13RoaCihoaHXe6pSCjgXFUXU99/z8BdfIAEBbB0/nuCiRbmrb1+ro6lcEhkZSWRk\npMe25/ZFXiLyEfAkYAMKAcWAhUA7oLwxxi4izYH3jDEdrrG+XuSlVDalJyYyvWNHSt9xBxXuvZf1\nH39M/9WrKVWzptXRVB7xqit8ReQh4HVjTBcRmQ0sMMbMFpGxwB/GmHHXWEeLv1I5kJ6YyMfFigHw\n0oEDlL79dosTqbzkzVf4DgVeE5EDQClgUi6+llJ+Z+f06RQqVYpilSrx2yefYOz2G6+Uy1ITEpjb\nqxfJsbEAnFi/nl9eecXiVOpaPFr8jTFrjDFdnNNHjTHNjDF3GGN6G2MyPPlaSvmzE7/9xrr/9/94\nbvNmXtq3j9gDB9g0apTVsSh4yy2UuO02prVpw75Fi5jdvTu3d+pkdSx1DXpjN6V8kDGG5JgYipQt\nCzhaQIgQXKSIxckc2Sa3bMnJDRt4bNYs6vfubXWkfMmrev45CqDFX6l85cT69cx+9FFuqVIFjOGp\nlSspXLq01bHyHW/u+Sul/ExqfDzzevem+4wZDNy2jRrt2rF4gF7c7430yF8p5VHJsbGuI31jDKlx\ncRQqVcriVPmPtn2UUsoPadtHKXVT7DYbu2bO5PLBVtJff3Fo+XKLUymraPFXyk+kXbrEbyNGEPHv\nf5MYHc2UsDBO//671bGURbTto1QuMHY7iLi+KN3Y7UiA9cdayefPM7ZBAxLPnePB//yHsOHD8zxD\n1n3hLfvFF2nbx0elptpYv/6Ea/748XgOHIi1MJHypN8++4zlr73mOOGZkMDkBx7gzLZtVsfC2O2u\nto89I4O8PvD6Y9o0Fj79NPbMTGxpaczq2pWDy5blaQbloMXfTZcupXHo0AXX/OHDF7h4Me2G6x0+\nfIHHHpvDkiX7OX48nrCwKaxZcywXk6q8dO/AgZxcv57FAwbwQ7t2lG/UiAqNGlmaKeXCBaaEhdH4\nhRd4MyaGg0uXEvnee3maoe5jj3Hp9GkW9OnD7EcfJbBgQWq0aZOnGZSTMcbSH0cE3/XTT/tNxYoj\nzd69MWb//vOmUqWRZtGifTe17vr1xw2EGwg3o0ZtNJs3nzJ9+843drvdrUxjxvxujh2LM8YYk5Zm\nMx9+uMakpGS4tU2VfXHHjplwMOFg7JmZVscxmTab2T1vnms+KSbGHF65Ms9zJF+44NovGSkpef76\n+YWzdua49uqRv5s6dbqDjz9uTZ0631C79hjefz+MLl1q39S6lSvf4po+c+YSnTrNoFeveq4+cU7Z\nbHbCwqZw6NAFevacy9atZwkIcG+bKntSExKY16sXdz35JBXuvZflr7+e5y2WqwUEBlL3scdc84Vv\nvZUarVvnaQZbWho/PvUUtTp0oNqDD7Jk4EDsmZl5mkE5aPH3gObNK7um4+NTAccnquHDI9m7N+aa\n65w4kUBY2BRGjXqYDh1q8emnG+jW7c6bfuP4J0OGNOfFFxtz++1fc+rURWbP7kFwcKDb21U3b+vY\nsVRs2pRuU6fy1IoVnNq4kbNe0PO32p8zZxIUEsLjixbRd9kyLp0+zaFffrmpda9+87T6zdTX6Wgf\nNx04EEurVlN4//0wzpy5xHvvRfLZZ205c+YSq1cfY8WKpyhVqtDf1vvrryR++eUQdercSqdOM6he\nvQQnT15k48YBVK9ewq1M6emZ9Ow5l8WL91O58i2sX/8M1aq5t02VPVeP9rFnZhIQqG/AxhhMZiYB\nQY4vEbTbbK7pG60385FHaPbKK9Rs25boXbv4aeBA+kdGElSwYG7H9krujvbRnr+btm8/Y6ZMiXLN\nf/TRWlcfPzo60YwcucGkpdmMMcZERyeaceO2uJ6bmpphatQY7TpH8NVXm8z9909yu+ffp898063b\nLJOWZjOjRm00t902yiQkpLq1TaWsdnzdOvNpmTJmwxdfmM/Llze7Zs2yOpKlcLPnr0f+HmSM4c03\nV/DVV7+teMnfAAARR0lEQVQTECBMmtSF+fP3YrPZ+fbbTjz88A/06FGH4cPDXOtcvJjGLbcUvO58\nTvzxxznq1CnjavVs3XqGxo0rurVNpbzBlrFj+XnwYO4ZMIAuEydaHcdSOs7fi3z99WZWrz7GuXNv\nsHnz87zxxgpefbU5Fy6kUKXKl3TtWvuKwg/8rdC7W/gB7r67/BU9fi38Kj+I3rWLte+/T5OXXmL/\n4sUcXrHC6kg+TY/8PSguLgVjcPX4jx2Lp2DBQB566HsOHrxA5853MG9eLz35qlQ2GWMc1yi8+CL1\ne/fmxPr1/DRoEAO3b9eef07Xt7rw5qfif7WUlAyaNJlAjx51GTbsAR5/fB6BgQHMndvT6mhK+ZzM\njAwCCxS47ry/0eLv5davP0HLllUByMjIZPv2szRrVvkGaymlAFa/9x539+tHqZo1SYqJYf3HH9P6\no48ICgnBlpbmt0f9oD1/r3e58AMUKBCohV+pbChWsaLj7qObNzOqWjUuHDpEYHAwO3/4gamtW+tY\nfzdo8VdK5djGL790XaSVmZ7OTy+8wMVTpzy2/caDBtHs5ZeZ2KwZZevX5+TGjSx+7jlWvPUWnb/7\nzu2r4f3Zja+uUEqp66jcvDmzunaly8SJ7Jg0CQkIoEjZsh7bftrFi+xdsAAJCCDx7FlqtG5N1OTJ\ndPj6a8rUreux1/FHbh/5i0hlEYkQkT0isktEXnYuLykiv4rIfhFZLiLF3Y+rlLqW8/v3u6ZtqanE\nHzuWJ69bpUULes6dy6yuXTn866/0mD2bwODgHG0r9sABtk+a5Jrfv2QJ09q1o3zDhvxvRgY12rZl\nz7x5tB89mjXvv8+RVas89Wv4JU+0fWzAa8aYukAL4F8icicwFFhpjKkNRADveOC1lBdLjI6+4g/y\n9ObNXDh0yMJE/uHi6dN8/+CD7F+yBFtqKnMee4wNn3+eJ6+dmZ7Opi++oEi5cgSFhHA0IiLH2woM\nDmbdhx+yecwY9i9ezJLnnqPl0KF0HDMGu81G3OHD9FmyhGYvv0yvefP4ffRo7fm7weOjfURkITDG\n+fOQMSZaRMoDkcaYO6/x/Hw92sefnNm6lekdO9Lt++8pVKoUM7t0ofsPP1CzXTuro+V7p7dsYVrb\ntqQlJFC3Z0+6T5+eJ8Mgf3n1VRKOHaPH7Nmc2baNWV278tymTZSsUSNH24s/dozRt90GwPNbtlCx\ncWPXY8aYK3r8V8/7G3dH+3i05y8i1YGGwCagnDEmGsAYc05EynjytZT3qdi4MU8sXsykFi0A6LN0\nqRb+PFKuQQNXu6X+E0/k2fj3B4YNI6R4cQKDg6nSogUvREVRrGLOryiP3rnTNX1q06Yriv/Vhd6f\nC78neKz4i0hRYB4wxBiTKCI3fTgfHh7umg4NDSU0NNRTsZTK9y63eqqHhtL81VeZ3a0bAUFB1O7c\nOddfu0iZK4/p3Cn8x9etY8nzz/P8li0UvvVWpoSFEVKyJHf17etuzHwhMjKSyMhIj23PI20fEQkC\nfgKWGWNGO5ftBUKztH1WG2PqXGNdbfvkE9r2sUby+fNs+Pxzwj74gMACBTi9ZQunN2+m6b/+ZXW0\nbMlISSHuyBHK1qsHQPzx4xQoXPhvbzDKwSuu8BWRqcB5Y8xrWZZ9AlwwxnwiIm8DJY0xQ6+xrhb/\nfCIxOpq//vzT9e1QpzdvplCpUpSqVcviZHkv8dw5ipYvDzju7Z8UE0PRcuUsTqXyE8uLv4jcD6wF\ndgHG+TMM2AzMAaoAJ4Cexpj4a6yvxV/lK8nnz/Nt/fo8OnUqNdq0YcmgQdiSk+k+fbrV0VQ+Ynnx\nd5cWf5UfnVi/nlnduhFctCjFq1Sh77JlBBctetPrH/rlF4pVqkS5Bg0A2D5xInUee4xCJUvmVmTl\nY/TePkp5oSr33UeRsmVJOH6c5q+9lq3CD44vgP+hXTuid+1izfvvs/GLL8hMT8+ltMof6e0dlPIw\nY7ezZNAgCpcuTd9ly/ixXz+CixTJ1onv+r17AzDurrsAeP3sWT1noDxKj/yV8rCUuDiMzUbfZcuo\n1b49vRcs4MT69dneTuzlWzaIkBQT4+GUyt9pz18pL7Rh5Eh2TJpE/4gIjq1Zw/JXXuGZ9espVbOm\n1dGUl9ATvkrlQ3/t3k3h0qVdw0WPr11L5RYt/Pqbq9SVtPgrpZQf0tE+Simlsk2Lv1JK+SEt/kop\n5Ye0+CtlgUUDBnDMeYfGi6dPM71jR9ITE60NpfyKFn8LJSamc/BgrGv+8OELJCSkWphI5ZW7nnyS\nuT17smvGDKaEhVHtoYeyfRWwUu7Q4m+hdeuOExo6hT17YjhwIJbQ0CmsXn3M6lgAf7uVgN5awLNu\nCwvj4VGjWNC3L6Vq1aLl229bHUn5GS3+FurQ4XY++aQN9ep9S+3aYwgPf4hu3f72TZd5LjMjg/GN\nGhH1/ffEHTnC7rlz+f6hhzi2Zg0ZyclWx8sXLp4+zZrhw6nVvj1ntmxxtYB8zdZx41j30UeA4wBh\n/hNPcGbrVotTqZuh9/axWNOmlVzTLVpUsTDJfwUWKECnsWOZ0akTGENQSAgPvPsu83r1ot+qVZSt\nX9/qiD5vyXPPcc+AAbR8+22Orl7Nwv79Gbx7t8+1fmp37cqUsDAyMzI4t2MHAOWc9yNS3k0v8rLQ\ngQOxtG49lfDwhyhYMIi3317JihVPUbeud3xz0bIhQ9j81Veu+au/UFvlXNqlSxQsVuy6874k/vhx\nRlevDsC7aWmu7xJWucurvsBdZU9ycgYffhhG//4NAQgIEJKSvKO3vnvuXHbPns29Awey7bvvAChU\nqpTFqfKPqwu9rxb+zPR0fhkyhEpNm5IUE8OGzz/ngWHDrI6lboL2/C3UsGF5V+EH6NOnAU2aVPqH\nNdxnt9mY/8QTxB87BkDM3r0s7N+frJ++MjMy2DJmDC2HDWPfwoU88t13lKlbl+9DQ4k7ejRX8ynf\nsmn0aACeWbeOZ3/7jT+mTuXkhg0e2fa+RYtYMmgQxm7H2O0sfv559i9Z4pFtK237+KXfv/6aTV98\nQaexY1k8YACtR4zg7qeeuuI5xhhObdpEYIECVGzcGGMMf86aRY02bbziC7VP/PYbZerUcX0aOfjz\nz9R8+GECAgMtTuZfLo8Cu9zqSU9KIrhIEY9sOz0xkekdOlDqjjvAGOIOH6bP0qU+d14kt+iN3VSO\nLB08mK1jxxL2wQc8+O67VsfJtlXDhnF4+XKeWrGC7ZMmsf2773h2wwaveGNSnpN28SIjihcHYOjF\niz7bHssNWvxVtsXs3cu0Nm0oVqkSyTEx9F+9mhLOE3a+whjDijffZOPIkRQqXZoXoqK4pXJlq2Mp\nD7rc6omOisIYQ4V776Xz+PFIgHarQe/qqbLJbrMxu1s3Wo8YwfObN9P8tdeY06MHvvYGLCIUcX6t\nYUBQEAUKF7Y4kfK03XPmEHfoEE+vWcMza9cSu28fe+bNszpWvqFH/n4oOTaWwqVLX3feF2z4/HO2\njR9P/9Wr2TRqFMdWr6bfqlWElChhdTTlIcYYMtPSCAoJASAjJYWgkBBEcnywm69o20f5pf1LllDh\nnnu4pXJljDFEff89Dfr0IahgQaujKZUnvL74i0h7YBSOFtMkY8wnVz2uxV8ppbLJq4u/iAQAB4DW\nwBlgC/C4MWZfludo8VdKqWzy9hO+TYGDxpjjxpgMYBbQNZdfUyml1A3kdvGvBJzMMn/KuUwppZSF\ncvvePtf6SPK3Hk94eLhrOjQ0lNDQ0NxLpJRSPigyMpJID976O7d7/s2BcGNMe+f8UMBkPemrPX+l\nlMo+b+/5bwFqiUg1EQkGHgcW5/JrKqWUuoFcbfsYYzJF5CXgV/471HNvbr6mUkqpG9OLvJRSygd5\ne9tHKaWUF9Lir5RSfkiLv1JK+SEt/kop5Ye0+CullB/S4q+UUn5Ii79SSvkhLf5KKeWHtPgrpZQf\n0uKvlFJ+SIu/Ukr5IS3+Sinlh7T4K6WUH9Lir5RSfkiLv1JK+SEt/kop5Ye0+CullB/S4q+UUn5I\ni79SSvkhLf5KKeWHtPgrpZQf0uKvlFJ+yK3iLyKfisheEYkSkfkickuWx94RkYPOx9u5H1UppZSn\nuHvk/ytQzxjTEDgIvAMgInWBXkAdoAPwrYiIm6/llSIjI62O4BbNby1fzu/L2cH387vLreJvjFlp\njLE7ZzcBlZ3TXYBZxhibMeYYjjeGpu68lrfy9X9Amt9avpzfl7OD7+d3lyd7/s8CPzunKwEnszx2\n2rlMKaWUFwi60RNEZAVQLusiwAD/NsYscT7n30CGMWZmludczbiZVSmllIeIMe7VZBHpDwwEWhlj\n0pzLhgLGGPOJc/4X4D1jzO/XWF/fFJRSKgeMMTk+l+pW8ReR9sBI4EFjTGyW5XWB6UAzHO2eFcDt\nxt13GqWUUh5xw7bPDXwNBAMrnIN5NhljBhtj9ojIHGAPkAEM1sKvlFLew+22j1JKKd9j2RW++eEC\nMRFpLyL7ROSAiLxtdZ5/IiKVRSRCRPaIyC4Redm5vKSI/Coi+0VkuYgUtzrrPxGRABHZLiKLnfPV\nRWSTM/9MEXH302yuEZHiIjLX+e96t4g086X9LyKvisifIrJTRKaLSLA3738RmSQi0SKyM8uy6+5v\nEfnKWXeiRKShNan/6zr5PVY3rby9g09fICYiAcAY4GGgHvCEiNxpbap/ZANeM8bUBVoA/3LmHQqs\nNMbUBiJw/n/wYkNwtBMv+wQY6cwfDwywJNXNGQ38bIypA9wN7MNH9r+IVAT+B2hkjLkLR8v4Cbx7\n/0/G8feZ1TX3t4h0AGoaY24HBgHj8jLodVwrv8fqpmXFPx9cINYUOGiMOW6MyQBmAV0tznRdxphz\nxpgo53QisBfHPu8KTHE+bQrQzZqENyYilYGOwMQsi1sB853TU4BH8zrXzRCRYsADxpjJAM5/3wn4\n0P4HAoEizqP7QsAZIAwv3f/GmPVA3FWLr97fXbMsn+pc73eguIiUw0LXyu/JuuktN3bzxQvErs55\nCu/M+TciUh1oiOMfTzljTDQ43iCAMtYlu6EvgTdxXjMiIqWBuCx/DKeAihZlu5EawHkRmexsW30n\nIoXxkf1vjDmDY2TfCRx/kwnAdiDeR/b/ZWWv2t9lnct9pe5k5VbdzNXiLyIrnP3Byz+7nP/tnOU5\nvnqBmK/kvIKIFAXmAUOcnwC8PjOAiHQCop2fXi7ve+Hv/x+89fcJAhoB3xhjGgFJOFoQ3pr3CiJS\nAsfRcTUcBb4IjvbC1Xzi97kGn/p79kTdzNWTM8aYtv/0uPMCsY44PrpfdgqokmW+Mo6Pl97mFFA1\ny7y35nRxflyfB0wzxixyLo4WkXLGmGgRKQ/8ZV3Cf3Q/0EVEOuJoORQDRuH4eB7gPPr05v8Hp4CT\nxpitzvn5OIq/r+z/NsARY8wFABH5EbgPKOEj+/+y6+1vX6k7HqubVo72aQ+8BXS5fGWw02LgcedI\ngtuAWsBmKzLewBaglohUE5Fg4HEc2b3Z/wF7jDGjsyxbDDztnO4PLLp6JW9gjBlmjKlqjKmBY19H\nGGOeBFYDPZ1P8+b80cBJEbnDuag1sBsf2f842j3NRSTEeSLxcn5v3/9XfzrMur+f5r95FwP9AESk\nOY52VnTeRPxHV+T3aN00xljyg+OExHEcfcPtwLdZHnsHOITjpGQ7qzLexO/QHtjv/F2GWp3nBlnv\nBzKBKGCHc5+3B0oBK52/xwqghNVZb+J3eQhY7Jy+DfgdOADMBgpYne8fct+N46AhClgAFPel/Q+8\n5/yb3InjZGkBb97/wAwcR79pON68ngFKXm9/4xi9dwj4A8eoJm/M77G6qRd5KaWUH/KW0T5KKaXy\nkBZ/pZTyQ1r8lVLKD2nxV0opP6TFXyml/JAWf6WU8kNa/JVSyg9p8VdKKT/0/wHXDwiJZczecwAA\nAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x11814c190>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "# Define input data\n",
    "X_data, y_data = generate_input()\n",
    "\n",
    "X_data = np.asarray(X_data)\n",
    "\n",
    "# Plot input data\n",
    "plt.scatter(X_data[:, 0], X_data[:, 1], marker='x', c=y_data)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[ 0.43630835  0.56369162]]\n",
      "[[ 0.56700599  0.43299398]]\n",
      "[[ 0.21740478  0.78259516]]\n"
     ]
    }
   ],
   "source": [
    "import random\n",
    "\n",
    "# Build a graph.\n",
    "x = tf.placeholder(tf.float32, [None, 2])\n",
    "y_ = tf.placeholder(tf.float32, [None, 2])\n",
    "\n",
    "W = tf.Variable(tf.zeros([2, 2]))\n",
    "b = tf.Variable(tf.zeros([2]))\n",
    "y = tf.nn.softmax(tf.matmul(x, W) + b)\n",
    "cross_entropy = tf.nn.softmax_cross_entropy_with_logits(y, y_)\n",
    "\n",
    "train_step = tf.train.GradientDescentOptimizer(0.0001).minimize(cross_entropy)    \n",
    "\n",
    "def generate_input():\n",
    "    batch_xs = []\n",
    "    batch_ys = []\n",
    "    for i in range(64):\n",
    "        batch_xs.append( [random.uniform(0, 100), random.uniform(0, 100)] )\n",
    "        batch_ys.append( [float(batch_xs[-1][0] > batch_xs[-1][1]) , float(batch_xs[-1][0] < batch_xs[-1][1])] )\n",
    "    return batch_xs, batch_ys\n",
    "    \n",
    "\n",
    "# Launch the graph in a session.\n",
    "with tf.Session() as sess:\n",
    "    tf.global_variables_initializer().run()\n",
    "    for i in range(100):\n",
    "        batch_xs, batch_ys = generate_input()\n",
    "        sess.run(train_step, feed_dict={x: batch_xs, y_: batch_ys})\n",
    "    print(sess.run(y, feed_dict={x: [[1, 2]]}))\n",
    "    print(sess.run(y, feed_dict={x: [[2, 1]]}))\n",
    "    print(sess.run(y, feed_dict={x: [[5, 10]]}))\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "## Number Recognition with TensorFlow\n",
    "\n",
    "Builds the MNIST network.\n",
    "\n",
    "Implements the inference/loss/training pattern for model building.\n",
    "\n",
    "1. inference() - Builds the model as far as is required for running the network\n",
    "forward to make predictions.\n",
    "2. loss() - Adds to the inference model the layers required to generate loss.\n",
    "3. training() - Adds to the loss model the Ops required to generate and\n",
    "apply gradients.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Input"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Extracting /tmp/tensorflow/mnist/input_data/train-images-idx3-ubyte.gz\n",
      "Extracting /tmp/tensorflow/mnist/input_data/train-labels-idx1-ubyte.gz\n",
      "Extracting /tmp/tensorflow/mnist/input_data/t10k-images-idx3-ubyte.gz\n",
      "Extracting /tmp/tensorflow/mnist/input_data/t10k-labels-idx1-ubyte.gz\n",
      "3\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<matplotlib.image.AxesImage at 0x10e26aa50>"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAP4AAAD8CAYAAABXXhlaAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAADURJREFUeJzt3W+IXOUVx/HfkdAXjSCSXbNgmk2pNKGCBGuEagyGtlFL\nIYlgK4qoxUTEtmreNFExL2JiW1BMXvi3qUQxqF3dJi1abRFSpVQXTVpld6NQYhJKNhux0uAb2z19\nsdd1TWefZ7J37szdnO8HFmfv2dk5Gfc3d2bO3PuYuwtALKd1ugEA7UfwgYAIPhAQwQcCIvhAQAQf\nCKhU8M3scjMbNrP3zOxnrWoKQLVsunN8MztN0nuSvi3pn5IGJF3t7sMn/BwfFAA6xN2t0fYye/wL\nJb3v7h+4+6eSnpG0ssTvA9AmZYJ/tqRDk74/XGwDUHNlgt/oKQRP64EZoEzwD0uaP+n7eRp/rQ+g\n5soEf0DSOWbWa2ZfknS1pN2taQtAlWZN94ru/l8z+7GkVzT+ALLd3Yda1hmAykx7nNf0DTDOAzqm\ninEegBmK4AMBEXwgIIIPBETwgYAIPhAQwQcCIvhAQAQfCIjgAwERfCAggg8ERPCBgAg+EBDBBwIi\n+EBABB8IiOADARF8ICCCDwRE8IGACD4QEMEHAiL4QEAEHwiI4AMBEXwgIIIPBETwgYAIPhAQwQcC\nmlXmymZ2QNLHksYkferuF7aiKbRPb29vsn7TTTcl63fddVey7u7JulnD5dsnDA0NJet33313st7f\n35+sR1Uq+BoP/KXu/lErmgHQHmWf6lsLfgeANisbWpf0spkNmNmaVjQEoHpln+pf5O5HzKxb0h/N\nbMjdX29FYwCqU2qP7+5Hiv+OSuqXxJt7wAww7eCb2ZfN7PTi8mxJKyS926rGAFSnzFP9uZL6zcyL\n3/O0u7/SmrYAVMlyc9bSNzD+wICKdHd3J+sbNmxI1q+99tpkfc6cOcl6bg5fdo6fu/6hQ4eS9SVL\nliTrx44dS9ZnOndveAczigMCIvhAQAQfCIjgAwERfCAggg8ERPCBgJjj11zuePdNmzYl652eo4+O\njibrOV1dXcn6ggULkvXBwcFk/dxzzz3ZlmYU5vgAJhB8ICCCDwRE8IGACD4QEMEHAiL4QEDM8Wtu\nYGAgWT///POT9bJz/NwcfPny5cl62ePdly5dmqzv2bMnWc/9+2fNKnvayXpjjg9gAsEHAiL4QEAE\nHwiI4AMBEXwgIIIPBMQcv8MWLVqUrOfm+B9++GGynjsePjdnv+OOO5L122+/PVnfsmVLsn7w4MFk\nPSf39zs2Npas33LLLcn6Y489dtI91QlzfAATCD4QEMEHAiL4QEAEHwiI4AMBEXwgoOwc38y2S/q+\npBF3P6/YdqakZyX1Sjog6Qfu/vEU12eOX0Juzp+bw5c9Hn7t2rXJ+sMPP5ys59anf/vtt5P11atX\nJ+t9fX3Jeu7vu6enJ1kve/91Wpk5/hOSLjth23pJf3L3hZJelbShXHsA2ikbfHd/XdJHJ2xeKWlH\ncXmHpFUt7gtAhab7Gv8sdx+RJHc/Iqm7dS0BqBpv7gEBTTf4I2Y2V5LMrEfS0da1BKBqzQbfiq/P\n7JZ0Q3H5ekm7WtgTgIplg29mOyX9RdLXzeygmd0o6eeSvmtm+yV9p/gewAyRPam4u18zRek7Le4F\nDQwPD3f09nPH8+/fvz9Zz50vIHe8//r165P13LoAVX/OYabizT0gIIIPBETwgYAIPhAQwQcCIvhA\nQAQfCOjUXhw8gGXLliXrueP5c3P6oaGhZH3hwoXJ+htvvJGsd3enj+/KHU+f6/+KK65I1qNijw8E\nRPCBgAg+EBDBBwIi+EBABB8IiOADATHHn+GuuWaq0yWMW7NmTbKeO569iXUXkvXcnL7s8fTbtm1L\n1nPn7Y+KPT4QEMEHAiL4QEAEHwiI4AMBEXwgIIIPBMQc/xSXm8N3+vqvvfZasr5u3bpknTn99LDH\nBwIi+EBABB8IiOADARF8ICCCDwRE8IGAsnN8M9su6fuSRtz9vGLbRklrJB0tfuxOd/9DZV1iSjt3\n7kzWe3t7k/Wurq5kPXde/tmzZyfrOffcc0+yzpy+Gs3s8Z+QdFmD7Q+4+/nFF6EHZpBs8N39dUkf\nNSilT50CoLbKvMa/1cz2mdmvzOyMlnUEoHLTDf5Dkr7m7oslHZH0QOtaAlC1aQXf3Uf986MvHpe0\npHUtAahas8E3TXpNb2Y9k2pXSnq3lU0BqFYz47ydki6VNMfMDkraKGm5mS2WNCbpgKSbK+wRQItZ\n2eOtszdgVu0NoFK5Of69996brK9atSpZ37t3b7KeW98+d9796Ny94fSNT+4BARF8ICCCDwRE8IGA\nCD4QEMEHAiL4QEDM8TNy67uPjo62qZOZ6aWXXkrWL7us0RHfn8udV//BBx886Z4iYY4PYALBBwIi\n+EBABB8IiOADARF8ICCCDwSUPRHHqW7ZsmXJ+v3335+sDw8PJ+vXXXfdSfd0Ktm8eXOyvmLFimR9\n4cKFrWwHBfb4QEAEHwiI4AMBEXwgIIIPBETwgYAIPhDQKT/Hzx1P/8gjjyTrR48eTdajz+lnz56d\nrD/66KPJuhmLLncCe3wgIIIPBETwgYAIPhAQwQcCIvhAQAQfCCg7xzezeZKelNQj6b+SHnf3bWZ2\npqRnJfVKOiDpB+7+cYW9Tsvq1auT9dzx3nv27GllOzPOokWLkvXnn38+Wc/dv7l1HXLnO8D0NLPH\n/4+kde7+DUnfknSrmS2StF7Sn9x9oaRXJW2ork0ArZQNvrsfcfd9xeXjkoYkzZO0UtKO4sd2SFpV\nVZMAWuukXuOb2QJJiyX9VdJcdx+Rxh8cJKU/GwugNpoOvpmdLqlP0m3Fnn9Gr4kHRNZU8M1slsZD\n/5S77yo2j5jZ3KLeIyl9NAuA2mh2j/9rSYPuvnXStt2SbiguXy9p14lXAlBPzYzzLpZ0raR3zGyv\nxp/i3ynpF5KeM7MfSToo6aoqGwXQOpabo5a+AbOOvheQm0MPDQ0l64ODg8n6fffdV+r3v/XWW8l6\nTm9vb7J+ySWXJOu5zzmsWpUe1uSOp8/9fW3dujVZX7duXbKONHdv+D+IT+4BARF8ICCCDwRE8IGA\nCD4QEMEHAiL4QECn/Bw/p6+vL1mveo69d+/eZD1n/vz5yfqcOXOS9bL9566/efPmZH3btm3J+rFj\nx5J1pDHHBzCB4AMBEXwgIIIPBETwgYAIPhAQwQcCCj/H7+5OnyP0xRdfTNYvuOCCZH1sbCxZr3qO\nnrv+J598kqznzmu/ZcuWZL2/vz9ZR7WY4wOYQPCBgAg+EBDBBwIi+EBABB8IiOADAYWf4+d0dXUl\n65s2bSr1+9euXZusv/DCC8l62ePVc+e1Z336mY05PoAJBB8IiOADARF8ICCCDwRE8IGAssE3s3lm\n9qqZDZrZO2b2k2L7RjM7bGZvF1+XV98ugFbIzvHNrEdSj7vvM7PTJb0laaWkH0r6t7s/kLn+jJ7j\nAzPZVHP8WU1c8YikI8Xl42Y2JOnsopw+CwSAWjqp1/hmtkDSYklvFJtuNbN9ZvYrMzujxb0BqEjT\nwS+e5vdJus3dj0t6SNLX3H2xxp8RJJ/yA6iPpj6rb2azJP1e0kvu/n8f7jazXkm/c/fzGtR4jQ90\nSNnP6v9a0uDk0Bdv+n3mSknvTr89AO3UzLv6F0v6s6R3JHnxdaekazT+en9M0gFJN7v7SIPrs8cH\nOmSqPT6H5QKnMA7LBTCB4AMBEXwgIIIPBETwgYAIPhAQwQcCIvhAQAQfCIjgAwERfCAggg8ERPCB\ngAg+EBDBBwIi+EBABB8IqPIz8ACoH/b4QEAEHwiobcE3s8vNbNjM3jOzn7XrdptlZgfM7G9mttfM\n3qxBP9vNbMTM/j5p25lm9oqZ7Tezlzu5etEU/dVmIdUGi73+tNhei/uw04vRtuU1vpmdJuk9Sd+W\n9E9JA5Kudvfhym+8SWb2D0nfdPePOt2LJJnZUknHJT352UIlZvYLSR+6+y+LB88z3X19jfrbqCYW\nUm2HxGKvN6oG92HZxWjLatce/0JJ77v7B+7+qaRnNP6PrBNTjV76uPvrkk58EFopaUdxeYekVW1t\napIp+pNqspCqux9x933F5eOShiTNU03uwyn6a9titO36Qz9b0qFJ3x/W5//IunBJL5vZgJmt6XQz\nUzjrs0VLilWMuzvcTyO1W0h10mKvf5U0t273YScWo21X8Bs9gtVtjniRu18g6Xsav+OXdrqhGah2\nC6k2WOy1Vn93nVqMtl3BPyxp/qTv52n8tX5tFI/+cvdRSf0af3lSNyNmNleaeI14tMP9fIG7j/rn\nbxo9LmlJJ/spFnvtk/SUu+8qNtfmPmzUX7vuw3YFf0DSOWbWa2ZfknS1pN1tuu0sM/ty8cgrM5st\naYXqsQio6YvPlnZLuqG4fL2kXSdeoc2+0F8NF1L9v8VeVa/7sGOL0bbtk3vFWGKrxh9strv7z9ty\nw00ws69qfC/vkmZJerrT/ZnZTkmXSpojaUTSRkm/lfQbSV+RdFDSVe7+rxr1t1xNLKTapv6mWuz1\nTUnPqcP3YdnFaEvfPh/ZBeKpzfgKQPsQfCAggg8ERPCBgAg+EBDBBwIi+EBABB8I6H/KvBnbMkMr\nlgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x11852b590>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "from tensorflow.examples.tutorials.mnist import input_data\n",
    "\n",
    "data_sets = input_data.read_data_sets('/tmp/tensorflow/mnist/input_data', False)\n",
    "\n",
    "img = data_sets.train.images[1].reshape(28, 28)\n",
    "print(data_sets.train.labels[1])\n",
    "\n",
    "plt.imshow(img, interpolation=\"nearest\", cmap='gray')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Inference"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from __future__ import absolute_import\n",
    "from __future__ import division\n",
    "from __future__ import print_function\n",
    "\n",
    "import math\n",
    "\n",
    "import tensorflow as tf\n",
    "\n",
    "# The MNIST dataset has 10 classes, representing the digits 0 through 9.\n",
    "NUM_CLASSES = 10\n",
    "\n",
    "# The MNIST images are always 28x28 pixels.\n",
    "IMAGE_SIZE = 28\n",
    "IMAGE_PIXELS = IMAGE_SIZE * IMAGE_SIZE\n",
    "\n",
    "def inference(images, hidden1_units, hidden2_units):\n",
    "    \"\"\"Build the MNIST model up to where it may be used for inference.\n",
    "    Args:\n",
    "        images: Images placeholder, from inputs().\n",
    "        hidden1_units: Size of the first hidden layer.\n",
    "        hidden2_units: Size of the second hidden layer.\n",
    "    Returns:\n",
    "        softmax_linear: Output tensor with the computed logits.\n",
    "    \"\"\"\n",
    "    # Hidden 1\n",
    "    with tf.name_scope('hidden1'):\n",
    "        weights = tf.Variable(tf.truncated_normal([IMAGE_PIXELS, hidden1_units], \n",
    "                                                  stddev=1.0 / math.sqrt(float(IMAGE_PIXELS))),\n",
    "                                                  name='weights')\n",
    "        biases = tf.Variable(tf.zeros([hidden1_units]), name='biases')\n",
    "        hidden1 = tf.nn.relu(tf.matmul(images, weights) + biases)\n",
    "    # Hidden 2\n",
    "    with tf.name_scope('hidden2'):\n",
    "        weights = tf.Variable(tf.truncated_normal([hidden1_units, hidden2_units],\n",
    "                                                  stddev=1.0 / math.sqrt(float(hidden1_units))),\n",
    "                                                  name='weights')\n",
    "        biases = tf.Variable(tf.zeros([hidden2_units]), name='biases')\n",
    "        hidden2 = tf.nn.relu(tf.matmul(hidden1, weights) + biases)\n",
    "    # Linear\n",
    "    with tf.name_scope('softmax_linear'):\n",
    "        weights = tf.Variable(tf.truncated_normal([hidden2_units, NUM_CLASSES], \n",
    "                                                  stddev=1.0 / math.sqrt(float(hidden2_units))),\n",
    "                                                  name='weights')\n",
    "        biases = tf.Variable(tf.zeros([NUM_CLASSES]), name='biases')\n",
    "        logits = tf.matmul(hidden2, weights) + biases\n",
    "    return logits\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Loss"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def loss(logits, labels):\n",
    "    \"\"\"Calculates the loss from the logits and the labels.\n",
    "    Args:\n",
    "        logits: Logits tensor, float - [batch_size, NUM_CLASSES].\n",
    "        labels: Labels tensor, int32 - [batch_size].\n",
    "    Returns:\n",
    "        loss: Loss tensor of type float.\n",
    "    \"\"\"\n",
    "    labels = tf.to_int64(labels)\n",
    "    cross_entropy = tf.nn.sparse_softmax_cross_entropy_with_logits(logits, labels, name='xentropy')\n",
    "    loss = tf.reduce_mean(cross_entropy, name='xentropy_mean')\n",
    "    return loss"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Training"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def training(loss, learning_rate):\n",
    "    \"\"\"Sets up the training Ops.\n",
    "\n",
    "    Creates a summarizer to track the loss over time in TensorBoard.\n",
    "\n",
    "    Creates an optimizer and applies the gradients to all trainable variables.\n",
    "\n",
    "    The Op returned by this function is what must be passed to the\n",
    "    `sess.run()` call to cause the model to train.\n",
    "\n",
    "    Args:\n",
    "        loss: Loss tensor, from loss().\n",
    "        learning_rate: The learning rate to use for gradient descent.\n",
    "\n",
    "    Returns:\n",
    "        train_op: The Op for training.\n",
    "    \"\"\"\n",
    "    # Add a scalar summary for the snapshot loss.\n",
    "    tf.summary.scalar('loss', loss)\n",
    "    # Create the gradient descent optimizer with the given learning rate.\n",
    "    optimizer = tf.train.GradientDescentOptimizer(learning_rate)\n",
    "    # Create a variable to track the global step.\n",
    "    global_step = tf.Variable(0, name='global_step', trainable=False)\n",
    "    # Use the optimizer to apply the gradients that minimize the loss\n",
    "    # (and also increment the global step counter) as a single training step.\n",
    "    train_op = optimizer.minimize(loss, global_step=global_step)\n",
    "    return train_op"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Put it All Together"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Extracting /tmp/tensorflow/mnist/input_data/train-images-idx3-ubyte.gz\n",
      "Extracting /tmp/tensorflow/mnist/input_data/train-labels-idx1-ubyte.gz\n",
      "Extracting /tmp/tensorflow/mnist/input_data/t10k-images-idx3-ubyte.gz\n",
      "Extracting /tmp/tensorflow/mnist/input_data/t10k-labels-idx1-ubyte.gz\n"
     ]
    }
   ],
   "source": [
    "from tensorflow.examples.tutorials.mnist import input_data\n",
    "from tensorflow.examples.tutorials.mnist import mnist\n",
    "import time\n",
    "\n",
    "def placeholder_inputs(batch_size):\n",
    "    images_placeholder = tf.placeholder(tf.float32, shape=(batch_size, mnist.IMAGE_PIXELS))\n",
    "    labels_placeholder = tf.placeholder(tf.int32, shape=(batch_size))\n",
    "    return images_placeholder, labels_placeholder\n",
    "\n",
    "def fill_feed_dict(data_set, images_pl, labels_pl):\n",
    "    images_feed, labels_feed = data_set.next_batch(100, False)\n",
    "    feed_dict = {\n",
    "        images_pl: images_feed,\n",
    "        labels_pl: labels_feed,\n",
    "    }\n",
    "    return feed_dict\n",
    "\n",
    "def run_training():\n",
    "    \"\"\"Train MNIST for a number of steps.\"\"\"\n",
    "    # Get the sets of images and labels for training, validation, and\n",
    "    # test on MNIST.\n",
    "    data_sets = input_data.read_data_sets('/tmp/tensorflow/mnist/input_data', False)\n",
    "\n",
    "    # Tell TensorFlow that the model will be built into the default Graph.\n",
    "    with tf.Graph().as_default():\n",
    "        # Generate placeholders for the images and labels.\n",
    "        images_placeholder, labels_placeholder = placeholder_inputs(100)\n",
    "\n",
    "        # Build a Graph that computes predictions from the inference model.\n",
    "        logits = mnist.inference(images_placeholder, 128, 32)\n",
    "\n",
    "        # Add to the Graph the Ops for loss calculation.\n",
    "        loss = mnist.loss(logits, labels_placeholder)\n",
    "\n",
    "        # Add to the Graph the Ops that calculate and apply gradients.\n",
    "        train_op = mnist.training(loss, 0.01)\n",
    "\n",
    "        # Add the Op to compare the logits to the labels during evaluation.\n",
    "        eval_correct = mnist.evaluation(logits, labels_placeholder)\n",
    "\n",
    "        # Build the summary Tensor based on the TF collection of Summaries.\n",
    "        summary = tf.summary.merge_all()\n",
    "\n",
    "        # Add the variable initializer Op.\n",
    "        init = tf.global_variables_initializer()\n",
    "\n",
    "        # Create a saver for writing training checkpoints.\n",
    "        saver = tf.train.Saver()\n",
    "\n",
    "        # Create a session for running Ops on the Graph.\n",
    "        sess = tf.Session()\n",
    "\n",
    "        # Instantiate a SummaryWriter to output summaries and the Graph.\n",
    "        summary_writer = tf.summary.FileWriter('/tmp/tensorflow/mnist/logs/fully_connected_feed', sess.graph)\n",
    "\n",
    "        # And then after everything is built:\n",
    "\n",
    "        # Run the Op to initialize the variables.\n",
    "        sess.run(init)\n",
    "\n",
    "        # Start the training loop.\n",
    "        for step in xrange(2000):\n",
    "            start_time = time.time()\n",
    "\n",
    "            # Fill a feed dictionary with the actual set of images and labels\n",
    "            # for this particular training step.\n",
    "            feed_dict = fill_feed_dict(data_sets.train, images_placeholder, labels_placeholder)\n",
    "\n",
    "            # Run one step of the model.  The return values are the activations\n",
    "            # from the `train_op` (which is discarded) and the `loss` Op.  To\n",
    "            # inspect the values of your Ops or variables, you may include them\n",
    "            # in the list passed to sess.run() and the value tensors will be\n",
    "            # returned in the tuple from the call.\n",
    "            _, loss_value = sess.run([train_op, loss], feed_dict=feed_dict)\n",
    "\n",
    "run_training()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.10"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}