SyPy presentation on __slots__ using iPython notebook

__slots__.ipynb

{
 "metadata": {
  "name": "__slots__"
 },
 "nbformat": 3,
 "nbformat_minor": 0,
 "worksheets": [
  {
   "cells": [
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "# \\__slots__ flyweight pattern made easy\n",
      "\n",
      "A flyweight is an object that minimizes memory use by sharing as much data as\n",
      "possible with other similar objects; it is a way to use objects in large\n",
      "numbers when a simple repeated representation would use an unacceptable amount\n",
      "of memory"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "import random\n",
      "\n",
      "class Pokies(object):\n",
      "    \"\"\"\n",
      "    The Aussie one arm bandit\n",
      "    \"\"\"\n",
      "\n",
      "    def __init__(self, reels, symbols):\n",
      "        self.reels = reels\n",
      "        self.symbols = symbols\n",
      "\n",
      "    def draw(self):\n",
      "        \"\"\"\n",
      "        Random draw of reels * symbols\n",
      "        \"\"\"\n",
      "        return tuple(random.randint(1, self.symbols) for _ in range(self.reels))\n",
      "    \n",
      "pokies = Pokies(3, 10)\n",
      "pokies.draw()"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "pyout",
       "prompt_number": 2,
       "text": [
        "(4, 5, 2)"
       ]
      }
     ],
     "prompt_number": 2
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "dir(pokies)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "pyout",
       "prompt_number": 3,
       "text": [
        "['__class__',\n",
        " '__delattr__',\n",
        " '__dict__',\n",
        " '__doc__',\n",
        " '__format__',\n",
        " '__getattribute__',\n",
        " '__hash__',\n",
        " '__init__',\n",
        " '__module__',\n",
        " '__new__',\n",
        " '__reduce__',\n",
        " '__reduce_ex__',\n",
        " '__repr__',\n",
        " '__setattr__',\n",
        " '__sizeof__',\n",
        " '__str__',\n",
        " '__subclasshook__',\n",
        " '__weakref__',\n",
        " 'draw',\n",
        " 'reels',\n",
        " 'symbols']"
       ]
      }
     ],
     "prompt_number": 3
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "# \\__dict__\n",
      "\n",
      "The attribute dictionary of an instance"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "pokies.__dict__"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "pyout",
       "prompt_number": 4,
       "text": [
        "{'reels': 3, 'symbols': 10}"
       ]
      }
     ],
     "prompt_number": 4
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "pokies.owner = 'Aristocrat'\n",
      "pokies.__dict__"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "pyout",
       "prompt_number": 5,
       "text": [
        "{'owner': 'Aristocrat', 'reels': 3, 'symbols': 10}"
       ]
      }
     ],
     "prompt_number": 5
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "del pokies.owner\n",
      "pokies.__dict__"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "pyout",
       "prompt_number": 6,
       "text": [
        "{'reels': 3, 'symbols': 10}"
       ]
      }
     ],
     "prompt_number": 6
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "# \\__weakref__\n",
      "\n",
      "A weak reference to an object is not enough to keep the object alive: when the only remaining references to a referent are weak references, garbage collection is free to destroy the referent and reuse its memory for something else. A primary use for weak references is to implement caches or mappings holding large objects, where it\u2019s desired that a large object not be kept alive solely because it appears in a cache or mapping."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "import weakref\n",
      "ref = weakref.ref(pokies)\n",
      "ref"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "pyout",
       "prompt_number": 7,
       "text": [
        "<weakref at 0x102daff70; to 'Pokies' at 0x102db5cd0>"
       ]
      }
     ],
     "prompt_number": 7
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "One accesses the referent by calling the weak reference:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "ref().draw()"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "pyout",
       "prompt_number": 8,
       "text": [
        "(6, 4, 6)"
       ]
      }
     ],
     "prompt_number": 8
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "pokies.__weakref__"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "pyout",
       "prompt_number": 9,
       "text": [
        "<weakref at 0x102daff70; to 'Pokies' at 0x102db5cd0>"
       ]
      }
     ],
     "prompt_number": 9
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "# A flyweight minimizes memory use\n",
      "\n",
      "The __slots__ declaration takes a sequence of instance variables and reserves just enough space in each instance to hold a value for each variable. Space is saved because __dict__ is not created for each instance."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "class FlyweightPokies(object):\n",
      "    \"\"\"\n",
      "    One arm bandit with smaller memory footprint\n",
      "    \"\"\"\n",
      "\n",
      "    __slots__ = ['reels', 'symbols']\n",
      "\n",
      "    def __init__(self, reels, symbols):\n",
      "        self.reels = reels\n",
      "        self.symbols = symbols\n",
      "\n",
      "    def draw(self):\n",
      "        \"\"\"\n",
      "        Random draw of reels x symbols\n",
      "        \"\"\"\n",
      "        return tuple(random.randint(1, self.symbols) for _ in range(self.reels))\n",
      "    \n",
      "flyweight = FlyweightPokies(3, 10)\n",
      "flyweight.draw()"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "pyout",
       "prompt_number": 10,
       "text": [
        "(9, 5, 5)"
       ]
      }
     ],
     "prompt_number": 10
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "# What is the difference?"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "set(dir(pokies)) - set(dir(flyweight))"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "pyout",
       "prompt_number": 11,
       "text": [
        "set(['__dict__', '__weakref__'])"
       ]
      }
     ],
     "prompt_number": 11
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "There is no \\__dict__ so no way to add attribute to the instance:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "flyweight.owner = 'Aristocrat'"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "ename": "AttributeError",
       "evalue": "'FlyweightPokies' object has no attribute 'owner'",
       "output_type": "pyerr",
       "traceback": [
        "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m\n\u001b[0;31mAttributeError\u001b[0m                            Traceback (most recent call last)",
        "\u001b[0;32m<ipython-input-12-a600c6f9aae6>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m()\u001b[0m\n\u001b[0;32m----> 1\u001b[0;31m \u001b[0mflyweight\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mowner\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;34m'Aristocrat'\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
        "\u001b[0;31mAttributeError\u001b[0m: 'FlyweightPokies' object has no attribute 'owner'"
       ]
      }
     ],
     "prompt_number": 12
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "There is no \\__weakref__ so no way to create a weak reference to the instance:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "weakref.ref(flyweight)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "ename": "TypeError",
       "evalue": "cannot create weak reference to 'FlyweightPokies' object",
       "output_type": "pyerr",
       "traceback": [
        "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m\n\u001b[0;31mTypeError\u001b[0m                                 Traceback (most recent call last)",
        "\u001b[0;32m<ipython-input-13-e09859bf60e5>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m()\u001b[0m\n\u001b[0;32m----> 1\u001b[0;31m \u001b[0mweakref\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mref\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mflyweight\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
        "\u001b[0;31mTypeError\u001b[0m: cannot create weak reference to 'FlyweightPokies' object"
       ]
      }
     ],
     "prompt_number": 13
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "We could make a *weakreferencable* pokies class though, by adding \\__weakref__ to list of attributes in \\__slots__:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "class WeakPokies(object):\n",
      "    \"\"\"\n",
      "    A weakrefencable one arm bandit with smallish memory footprint \n",
      "    \"\"\"\n",
      "\n",
      "    __slots__ = ['reels', 'symbols', '__weakref__']\n",
      "\n",
      "    def __init__(self, reels, symbols):\n",
      "        self.reels = reels\n",
      "        self.symbols = symbols\n",
      "\n",
      "    def draw(self):\n",
      "        \"\"\"\n",
      "        Random draw of reels x symbols\n",
      "        \"\"\"\n",
      "        return tuple(random.randint(1, self.symbols) for _ in range(self.reels))\n",
      "    \n",
      "weak = WeakPokies(3, 10)\n",
      "weak.draw()"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "pyout",
       "prompt_number": 20,
       "text": [
        "(10, 10, 7)"
       ]
      }
     ],
     "prompt_number": 20
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "weakref.ref(weak)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "pyout",
       "prompt_number": 21,
       "text": [
        "<weakref at 0x103864158; to 'WeakPokies' at 0x10385f7a0>"
       ]
      }
     ],
     "prompt_number": 21
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "# ~25% memory usage\n",
      "\n",
      "The dramatic change is the memory footprint of the flyweight instance compare to a run of the mill object:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "%load_ext memory_profiler"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 15
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "%memit [Pokies(3, 10) for _ in xrange(int(1e6))]"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "maximum of 1: 360.085938 MB per loop\n"
       ]
      }
     ],
     "prompt_number": 16
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "%memit [FlyweightPokies(3, 10) for _ in xrange(int(1e6))]"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "maximum of 1: 91.593750 MB per loop\n"
       ]
      }
     ],
     "prompt_number": 17
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "%memit [WeakPokies(3, 10) for _ in xrange(int(1e6))]"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "maximum of 1: 97.730469 MB per loop\n"
       ]
      }
     ],
     "prompt_number": 22
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "# Do it last, if ever\n",
      "\n",
      "\\__slots__ is an optimization that is only relevant if you are going to create **MANY** instances of an object and which you should do last if ever!"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "class BrokenPokies(object):\n",
      "    \"\"\"\n",
      "    Getting it wrong raises an exception on instantiation\n",
      "    \"\"\"\n",
      "\n",
      "    __slots__ = ()\n",
      "\n",
      "    def __init__(self, reels, symbols):\n",
      "        self.reels = reels\n",
      "        self.symbols = symbols\n",
      "        \n",
      "broken = BrokenPokies(3, 10)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "ename": "AttributeError",
       "evalue": "'BrokenPokies' object has no attribute 'reels'",
       "output_type": "pyerr",
       "traceback": [
        "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m\n\u001b[0;31mAttributeError\u001b[0m                            Traceback (most recent call last)",
        "\u001b[0;32m<ipython-input-23-564a4def7260>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m()\u001b[0m\n\u001b[1;32m     10\u001b[0m         \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0msymbols\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0msymbols\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     11\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 12\u001b[0;31m \u001b[0mbroken\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mBrokenPokies\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;36m3\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;36m10\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
        "\u001b[0;32m<ipython-input-23-564a4def7260>\u001b[0m in \u001b[0;36m__init__\u001b[0;34m(self, reels, symbols)\u001b[0m\n\u001b[1;32m      7\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m      8\u001b[0m     \u001b[0;32mdef\u001b[0m \u001b[0m__init__\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mreels\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0msymbols\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 9\u001b[0;31m         \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mreels\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mreels\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m     10\u001b[0m         \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0msymbols\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0msymbols\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     11\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n",
        "\u001b[0;31mAttributeError\u001b[0m: 'BrokenPokies' object has no attribute 'reels'"
       ]
      }
     ],
     "prompt_number": 23
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [],
     "language": "python",
     "metadata": {},
     "outputs": []
    }
   ],
   "metadata": {}
  }
 ]
}