Skip to content

Instantly share code, notes, and snippets.

@dutc
Last active March 11, 2025 09:16

Revisions

  1. dutc revised this gist Feb 21, 2014. 1 changed file with 0 additions and 9 deletions.
    9 changes: 0 additions & 9 deletions notes.md
    Original file line number Diff line number Diff line change
    @@ -1,12 +1,3 @@
    # schedule

    6:30 - 6:40: settling in
    6:40 - 7:00: Julian, quick introduction to PyPy
    7:00 - 7:10: Andy, quick introduction to CLI
    7:10 - 7:15: James, NYC Python announcements
    7:15 - 8:30: James, CPython workshop
    8:30 - 9:00: mingling + sponsor announcements

    # themes

    1. CPython for greater understanding of the Python programming language (but "reference implementations always overspecify")
  2. dutc revised this gist Feb 16, 2014. 1 changed file with 1 addition and 1 deletion.
    2 changes: 1 addition & 1 deletion notes.md
    Original file line number Diff line number Diff line change
    @@ -74,7 +74,7 @@ cd Python-3.3.3
    # e.g., we get the ability to view macro definitions
    # --with-pydebug enables basic CPython interpreter debugging features (e.g., ref counts)
    # --prefix=$PWD-build lets us install this locally (not system-wide)
    CFLAGS="-g4 -ggdb -gdwarf-4" ./configure --with-pydebug --prefix=$PWD-build
    CFLAGS="-g3 -ggdb -gdwarf-4" ./configure --with-pydebug --prefix=$PWD-build
    # make -j9 for parallel make; make install to install into $PWD-build
    # $PWD should be something like /home/nycpython/Python-3.3.3
  3. dutc revised this gist Nov 20, 2013. 1 changed file with 1 addition and 1 deletion.
    2 changes: 1 addition & 1 deletion notes.md
    Original file line number Diff line number Diff line change
    @@ -516,6 +516,6 @@ def f():
    [1] http://mail.python.org/pipermail/python-dev/2004-August/046711.html
    ```

    Here's the issue I created (http://bugs.python.org/issue19660)[http://bugs.python.org/issue19660].
    Here's the issue I created http://bugs.python.org/issue19660. Let's see if it gets accepted (or if it needs more work!)

    Awesome!
  4. dutc revised this gist Nov 20, 2013. 1 changed file with 324 additions and 67 deletions.
    391 changes: 324 additions & 67 deletions notes.md
    Original file line number Diff line number Diff line change
    @@ -9,9 +9,9 @@

    # themes

    CPython for greater understanding of the Python programming language (but "reference implementations always overspecify")
    1. CPython for greater understanding of the Python programming language (but "reference implementations always overspecify")
    Reading source to solve problems
    Getting involved, contributing to the project
    2. getting involved, contributing to the project

    # introduction

    @@ -35,74 +35,147 @@ These tools are extremely rich, and we could do a workshop on each of them indiv

    We're going to start this workshop by downloading the C source code for CPython 3.3.3. We're going to build it, install it, and run it under `gdb`.


    ```
    http://python.org/download/
    # first, go to Python.org/download
    # here, we see links to "Python 3.3.3 xzipped source tarball (for Linux, Unix or Mac OS X, better compression)"
    lynx http://python.org/download/
    # this is the source code for the CPython interpreter
    # let's download & extract it
    wget 'http://python.org/ftp/python/3.3.3/Python-3.3.3.tar.xz' # get the source code
    tar xJvf Python-3.3.3.tar.xz # extract
    # for convenience
    sudo apt-get install build-essential
    # a quick, incomplete, and imprecise tour of the contents
    ls Doc/ # reStructuredText documentation
    less Grammar/Grammar # Python 'grammar' EBNF
    ls Include/ # header files
    ls Lib/ # the Python standard library, Python code, e.g., the source for collections.namedtuple
    ls Modules/ # Python modules written in C, e.g., the source for math.pow
    ls Objects/ # Python basic types
    ls Parser/ # language parser
    ls Python/ # interpreter
    # the difficulty of building software from source is often getting all the dependent parts
    # we can solve this on Ubuntu
    sudo apt-get install build-essential # first, get all the tools necessary to build anything
    sudo apt-get build-dep python3.3 # gives you all the other software Python depends on
    # configure, build, install
    # apt-get build-dep is a GREAT tool; it will install all the dependencies necessary for building a given package!
    # Python uses autoconf; the standard procedure for building and installing software is ./configure; make; make install
    cd Python-3.3.3
    # configure
    # the CFLAGS above give us some extra debugging information in `gdb`: very, very handy!
    # e.g., we get the ability to view macro definitions
    # --with-pydebug enables basic CPython interpreter debugging features (e.g., ref counts)
    # --prefix=$PWD-build lets us install this locally (not system-wide)
    CFLAGS="-g4 -ggdb -gdwarf-4" ./configure --with-pydebug --prefix=$PWD-build
    # make -j9 for parallel make; make install to install into $PWD-build
    # $PWD should be something like /home/nycpython/Python-3.3.3
    # our Python will be installed into /home/nycpython/Python3.3.3-build
    make -j9
    make install
    Doc # reStructuredText documentation
    Grammar/Grammar # Python 'grammar' EBNF
    Include # header files
    Lib # the Python standard library, Python code, e.g., collections.namedtuple
    Modules # Python modules written in C, math.pow
    Objects # Python basic types
    Parser # language parser
    Python # interpreter
    # run Python, see if it works!
    $PWD-build/bin/python3
    ```

    Let's try some sample code.

    We have our neat swapping syntax in Python. Let's try to figure out what it does. Does it allocate a temporary variable behind the scenes?

    ```
    def f(x, y):
    x, y = y, x
    return x, y
    from dis import dis
    dis(f)
    ```

    We get:

    ```
    2 0 LOAD_FAST 1 (y)
    3 LOAD_FAST 0 (x)
    6 ROT_TWO
    7 STORE_FAST 0 (x)
    10 STORE_FAST 1 (y)
    3 13 LOAD_FAST 0 (x)
    16 LOAD_FAST 1 (y)
    19 BUILD_TUPLE 2
    22 RETURN_VALUE
    ```

    1. First column is the line number (formal args and `def` are line 1.)
    2. Second column is the bytecode index. (Notice that many bytecodes are 3 bytes long.)
    3. Third column is the bytecode. (Notice ROT_TWO!)
    4. Fourth column is the argument to that bytecode.
    5. Fifth column is an interpretation of that argument.

    CPython is a stack-based VM. Values are pushed and popped from the stack. Notice in this example, we load two local variables, x and y onto the stack. We rotate their order. Then we store the top elements of the stack back to x and y. No temporary storage needed! Then we load x and y back up, make a tuple out of them, and then return that.

    `dis.dis` is a fantastic tool for figuring out what is going on behind the scenes. We will use it throughout this workshop.

    I learnt my way around CPython using exactly this tool. I saw bytecodes, then I `grep`ed for them. This lead me to `ceval.c`

    ```
    find -iname '*.c' -print0 | xargs -0 grep -n ROT_TWO
    ```

    We see:
    ```
    ./Python/compile.c:793: case ROT_TWO:
    ./Python/compile.c:2877: ADDOP(c, ROT_TWO);
    ./Python/compile.c:3393: ADDOP(c, ROT_TWO);
    ./Python/peephole.c:542: codestr[i] = ROT_TWO;
    ./Python/peephole.c:547: codestr[i+1] = ROT_TWO;
    ./Python/ceval.c:1389: TARGET(ROT_TWO)
    ```

    Let's look at ceval.c on line 1389.

    >>> from dis import dis
    >>> dis(f)
    2 0 LOAD_FAST 1 (y)
    3 LOAD_FAST 0 (x)
    6 ROT_TWO
    7 STORE_FAST 0 (x)
    10 STORE_FAST 1 (y)
    13 LOAD_CONST 0 (None)
    16 RETURN_VALUE
    $ find -iname '*.c' -print0 | xargs -0 grep ROT_TWO
    ./Python/compile.c: case ROT_TWO:
    ./Python/compile.c: ADDOP(c, ROT_TWO);
    ./Python/compile.c: ADDOP(c, ROT_TWO);
    ./Python/peephole.c: codestr[i] = ROT_TWO;
    ./Python/peephole.c: codestr[i+1] = ROT_TWO;
    ./Python/ceval.c: TARGET(ROT_TWO)
    Oh, my! This is the file that implements all the bytecodes! It's the interpreter loop! (If we look up, there's even a switch(opcode) up there... though the situation here is a bit more complicated, since Python3.3.2 is a "direct threaded interpreter.") Let's use gdb to put a breakpoint on that line and see what happens!

    Run your Python under `gdb`:
    ```
    gdb --args $PWD-build/bin/python3
    (gdb) source Tools/gdb/libpython.py
    (gdb) r
    ```

    Type... (note: ^C means hit control-c)
    ```
    (gdb) source Tools/gdb/libpython.py # nice helpers!
    (gdb) run # start the program
    >>> def f(x, y):
    ... x, y = y, x
    ^C
    (gdb) tbreak ceval.c:1389
    (gdb) c
    >>> f(10, 20)
    (gdb) list
    (gdb) info macro TARGET
    (gdb) info function PyObject_GetAttr
    (gdb) backtrace
    (gdb) next
    (gdb) step
    (gdb) finish
    ... return x, y
    ...
    >>>
    ^C
    (gdb) tbreak ceval.c:1389 # put a breakpoint on that line
    (gdb) continue
    >>> f(10, 20) # evaluate our code
    (gdb) list # show the source code around where we broke
    (gdb) info macro TARGET # get information about a C macro
    (gdb) info function PyObject_GetAttr # get information about a C function
    (gdb) backtrace # see all the C function calls leading up to this point
    (gdb) next # step to the next line of C-code (stepping over functions)
    (gdb) step # step to the next line of C-code (stepping into functions)
    (gdb) finish # run the current C function to completion
    ```

    # which `__mul__`?

    This is our first exercise. Let's start with some Python code.

    ```
    class Foo(int):
    def __mul__(self, other):
    @@ -111,13 +184,23 @@ class Foo(int):
    class Bar(int):
    def __mul__(self, other):
    print('Bar.__mul__({}, {})'.format(self, other))
    ```

    We have a custom object with `__mul__` implemented. Let's try it out!

    ```
    foo, bar = Foo(10), Bar(20)
    foo * bar
    bar * foo
    10 * foo
    bar * 10
    ```

    This makes sense. In some cases we see `Foo.__mul__` being used; in others, `Bar.__mul__`

    Let's make this example a bit richer and implement `__rmul__`.

    ```
    class Foo(int):
    def __mul__(self, other):
    print('Foo.__mul__({}, {})'.format(self, other))
    @@ -129,13 +212,22 @@ class Bar(int):
    print('Bar.__mul__({}, {})'.format(self, other))
    def __rmul__(self, other):
    print('Bar.__rmul__({}, {})'.format(self, other))
    ```

    Try it out:
    ```
    foo, bar = Foo(10), Bar(20)
    foo * bar
    bar * foo
    10 * foo
    bar * 10
    ```

    This behaviour makes sense, too!

    What about:

    ```
    class Foo(int):
    def __mul__(self, other):
    print('Foo.__mul__({}, {})'.format(self, other))
    @@ -147,59 +239,135 @@ class Bar(Foo):
    print('Bar.__mul__({}, {})'.format(self, other))
    def __rmul__(self, other):
    print('Bar.__rmul__({}, {})'.format(self, other))
    ```

    ```
    foo, bar = Foo(10), Bar(20)
    foo * bar
    bar * foo
    10 * foo
    bar * 10
    ```
    That's interesting! `Bar.__rmul__` seems to be preferred! Why?

    Here is our standard procedure. Use `dis` to find where to put a breakpoint, then step through the code.
    ```
    from dis import dis
    def f(x, y):
    return x * y
    dis(f)
    ```

    ```
    [insert live coding stepping into PyNumber_Multiply]
    ```

    Here's what the documentation says.

    *Note If the right operand’s type is a subclass of the left operand’s type and that subclass provides the reflected method for the operation, this method will be called before the left operand’s non-reflected method. This behavior allows subclasses to override their ancestors’ operations.*

    But in this last example, we were able to determine this behaviour conclusively by ourselves by just *reading (& debugging) the source*!

    # why hash(-1) == -2?

    Here's our next example.

    Run `python3`:
    ```
    $ python3
    hash(100)
    hash(10)
    hash(2)
    hash(1)
    hash(-100)
    hash(-10)
    hash(-2)
    hash(-1)
    hash(-1) # weird!!
    ```

    $ pypy
    hash(-1)
    But look at `pypy`:
    ```
    hash(-1) # different answer than above
    ```
    tb ceval.c:2671
    p PyCFunction_Check(func)

    What is going on?! Why does `hash(-1) == -2` only in CPython?!

    Let's do the same thing. Use `dis` to find an entry point, then step through the code.

    ```
    from dis import dis
    def f():
    hash(-1)
    dis(f)
    ```

    Let's put a breakpoint on CALL_FUNCTION.

    ```
    (gdb) tb ceval.c:2671
    ```

    ```
    [live coding stepping through CALL_FUNCTION to builtin_hash in Python/bltinmodule.c; end up at Objects/longobject.c:2785:long_hash where we see this behaviour hard-coded]
    if (x == (Py_uhash_t)-1)
    x = (Py_uhash_t)-2;
    ```

    Side note: `gdb` is powerful enough to let us evaluate code live. This helps us step through code and inspect things.
    ```
    (gdb) print PyCFunction_Check(func)
    (gdb) print ((PyFunctionObject*)(((PyMethodObject*)(func))->im_func))->func_name
    ```

    What about a custom object?
    ```
    class Foo(object):
    def __hash__(self):
    return -1
    foo = Foo()
    hash(foo)
    ```

    Even custom-objects do this! We can't ever get a -1 return value from `hash`!

    tb builtin_hash
    Add a breakpoint.
    ```
    (gdb) tb builtin_hash
    ```

    ```
    [live-coding; step through ending up at Objects/typeobject.c:5309:slot_tp_hash; slot_tp_hash is the slot wrapper for __hash__; also hard-codes this behaviour but gives us a hint]
    /* -1 is reserved for errors. */
    if (h == -1)
    h = -2;
    ```

    -1 is a common convention for error values in C. If we look at `PyObject_Hash` we can see this in use. This is why PyPy shows different behaviour (it's not written in C)

    We see this in `Python/bltinmodule.c:1238:builtin_hash`:
    ```
    if (x == -1)
    return NULL;
    ```

    ```
    [discussion on difference between the contents of these files]
    Objects/abstract.c vs Objects/object.c vs Objects/typeobject.c
    p ((PyFunctionObject*)(((PyMethodObject*)(func))->im_func))->func_name
    ```

    # lambda decorators

    docs.python.org/devguide
    pythonmentors.com
    First, Nick Coghlan wanted me to mention docs.python.org/devguide and pythonmentors.com

    docs.python.org/devguide is the comprehensive guide to contributing to the CPython project. It contains every resource you need.

    pythonmentors.com contains a lot of resources for mentorship around contributing to the CPython project.

    Briefly: check out the #python-dev and #python-ideas and #core-mentorship mailing lists. Check out bugs.python.org

    Okay, so we know about decorators in Python. The follow decorators don't do anything interesting.

    ```
    def dec(func):
    @@ -208,37 +376,122 @@ def dec(func):
    @dec
    def foo(x, y):
    return x * y
    ```

    We know we can write `dec` as follows, too:
    ```
    dec = lambda func: func
    ```

    But notice we can't write:
    ```
    @(lambda func:func)
    def foo(x, y):
    return x * y
    http://mail.python.org/pipermail/python-dev/2004-August/046711.html
    ```

    Why not? Guido made a "gut feeling" decision about this: http://mail.python.org/pipermail/python-dev/2004-August/046711.html

    Let's put our knowledge to use to figure out how to lift this restriction.

    First, let's look at `Grammar/Grammar:22`:
    ```
    Grammar/Grammar
    decorator: '@' dotted_name [ '(' [arglist] ')' ] NEWLINE
    ```

    So, after the decorator, we can only have a `dotted_name` (the same thing we can have in `from ... import ...`): just a bunch of names with dots between them like `a.b.c` We are allowed one function call at the very end. This means that `f().g()` is invalid as a decorator.

    Let's switch this to a `testlist` which is the term used for arbitrary expressions.

    ```
    diff -r 177e0254fdee Grammar/Grammar
    --- a/Grammar/Grammar Tue Nov 19 11:06:44 2013 -0500
    +++ b/Grammar/Grammar Tue Nov 19 15:15:33 2013 -0500
    @@ -19,7 +19,7 @@
    file_input: (NEWLINE | stmt)* ENDMARKER
    eval_input: testlist NEWLINE* ENDMARKER
    -decorator: '@' dotted_name [ '(' [arglist] ')' ] NEWLINE
    +decorator: '@' testlist NEWLINE
    decorators: decorator+
    decorated: decorators (classdef | funcdef)
    funcdef: 'def' NAME parameters ['->' test] ':' suite
    ```

    We need to make one more change to get this to work. In `Python/ast.c` we have some code that helps create the AST. Where we used to expect a `dotted_name` for a decorator, we now expect a `testlist`.

    -name_expr = ast_for_dotted_name(c, CHILD(n, 1));
    +name_expr = ast_for_testlist(c, CHILD(n, 1));
    ```
    diff -r 177e0254fdee Python/ast.c
    --- a/Python/ast.c Tue Nov 19 11:06:44 2013 -0500
    +++ b/Python/ast.c Tue Nov 19 15:15:33 2013 -0500
    @@ -1429,7 +1429,7 @@
    REQ(CHILD(n, 0), AT);
    REQ(RCHILD(n, -1), NEWLINE);
    - name_expr = ast_for_dotted_name(c, CHILD(n, 1));
    + name_expr = ast_for_testlist(c, CHILD(n, 1));
    if (!name_expr)
    return NULL;
    ```

    Now, everything just works! Isn't that amazing! Fundamentally changing the language with just two changes to two files.

    But our job isn't over. If we want to submit a patch, we need to make sure all tests pass with `make test` or `$PWD-build/bin/python3 -m test -j3`.

    We see two tests that fail: `Lib/test/test_decorators.py` and `Lib/test/test_parser.py`

    Let's fix those.

    The first is easy. There is a test-case that is no longer appropriate (checking for invalid forms that we now consider valid.)

    Just remove the test.

    ```
    diff -r 177e0254fdee Lib/test/test_decorators.py
    --- a/Lib/test/test_decorators.py Tue Nov 19 11:06:44 2013 -0500
    +++ b/Lib/test/test_decorators.py Tue Nov 19 15:15:33 2013 -0500
    @@ -152,15 +152,6 @@
    self.assertEqual(counts['double'], 4)
    def test_errors(self):
    - # Test syntax restrictions - these are all compile-time errors:
    - #
    - for expr in [ "1+2", "x[3]", "(1, 2)" ]:
    - # Sanity check: is expr is a valid expression by itself?
    - compile(expr, "testexpr", "exec")
    -
    - codestr = "@%s\ndef f(): pass" % expr
    - self.assertRaises(SyntaxError, compile, codestr, "test", "exec")
    -
    # You can't put multiple decorators on a single line:
    #
    self.assertRaises(SyntaxError, compile,
    ```
    make test
    Lib/test/test_decorators.py
    Lib/test/test_parser.py

    - validate_dotted_name(CHILD(tree, 1)) &&
    + validate_testlist(CHILD(tree, 1)) &&
    The second is also easy. We have some validation of input to validate syntax. Instead of expecting a `dotted_name` we now expect a `testlist`:

    ```
    diff -r 177e0254fdee Modules/parsermodule.c
    --- a/Modules/parsermodule.c Tue Nov 19 11:06:44 2013 -0500
    +++ b/Modules/parsermodule.c Tue Nov 19 15:15:33 2013 -0500
    @@ -2541,7 +2541,7 @@
    ok = (validate_ntype(tree, decorator) &&
    (nch == 3 || nch == 5 || nch == 6) &&
    validate_at(CHILD(tree, 0)) &&
    - validate_dotted_name(CHILD(tree, 1)) &&
    + validate_testlist(CHILD(tree, 1)) &&
    validate_newline(RCHILD(tree, -1)));
    if (ok && nch != 3) {
    ```

    bugs.python.org
    All done!

    `make && make install` then `$PWD-build/bin/python3 -m test -j3` and all tests pass!

    Let's submit this as a patch to bugs.python.org/

    Here's my text:
    ```
    Decorator syntax currently allows only a dotted_name after the @. As far as I can tell, this was a gut-feeling decision made by Guido. [1]
    @@ -261,4 +514,8 @@ def f():
    pass
    [1] http://mail.python.org/pipermail/python-dev/2004-August/046711.html
    ```
    ```

    Here's the issue I created (http://bugs.python.org/issue19660)[http://bugs.python.org/issue19660].

    Awesome!
  5. dutc revised this gist Nov 19, 2013. 1 changed file with 42 additions and 0 deletions.
    42 changes: 42 additions & 0 deletions setup.sh
    Original file line number Diff line number Diff line change
    @@ -0,0 +1,42 @@
    #!/bin/bash

    apt-get install build-essential python{2.7,3.3} python{2.7,3.3}-dev
    apt-get build-dep python{2.7,3.3}

    # some extras
    apt-get install bpython{,3} cython{,3} emacs exuberant-ctags gdb git htop ipython{,3} mercurial moreutils pypy python-virtualenv python{,3}-pip qalc screen tmux tree vim zsh

    mkdir built
    pushd built

    wget 'http://python.org/ftp/python/2.7.6/Python-2.7.6.tar.xz'
    wget 'http://python.org/ftp/python/3.3.3/Python-3.3.3.tar.xz'
    hg clone http://hg.python.org/cpython


    if [[ ! -d Python-2.7.6 ]]; then
    tar xJvf Python-2.7.6.tar.xz

    pushd Python-2.7.6
    find \( -iname '*.py' -or -iname '*.c' -or -iname '*.h' \) -print0 | xargs -0 ctags-exuberant
    CFLAGS="-g3 -ggdb -gdwarf-4" ./configure --with-pydebug --prefix=$PWD-build && make -j9 && make install
    popd
    fi

    if [[ ! -d Python-3.3.3 ]]; then
    tar xJvf Python-3.3.3.tar.xz

    pushd Python-3.3.3
    find \( -iname '*.py' -or -iname '*.c' -or -iname '*.h' \) -print0 | xargs -0 ctags-exuberant
    CFLAGS="-g3 -ggdb -gdwarf-4" ./configure --with-pydebug --prefix=$PWD-build && make -j9 && make install
    popd
    fi

    if [[ -d cpython ]]; then
    pushd cpython
    find \( -iname '*.py' -or -iname '*.c' -or -iname '*.h' \) -print0 | xargs -0 ctags-exuberant
    CFLAGS="-g3 -ggdb -gdwarf-4" ./configure --with-pydebug --prefix=$PWD-build && make -j9 && make install
    popd
    fi

    chown -R nycpython:nycpython ~nycpython
  6. dutc revised this gist Nov 19, 2013. 1 changed file with 11 additions and 7 deletions.
    18 changes: 11 additions & 7 deletions notes.md
    Original file line number Diff line number Diff line change
    @@ -1,11 +1,11 @@
    # schedule

    6:30 - 6:40: settling in
    6:40 - 7:00: Julian, quick introduction to PyPy
    7:00 - 7:10: Andy, quick introduction to CLI
    7:10 - 7:15: James, NYC Python announcements
    7:15 - 8:30: James, CPython workshop
    8:30 - 9:00: mingling + sponsor announcements
    6:30 - 6:40: settling in
    6:40 - 7:00: Julian, quick introduction to PyPy
    7:00 - 7:10: Andy, quick introduction to CLI
    7:10 - 7:15: James, NYC Python announcements
    7:15 - 8:30: James, CPython workshop
    8:30 - 9:00: mingling + sponsor announcements

    # themes

    @@ -237,6 +237,9 @@ Lib/test/test_parser.py
    + validate_testlist(CHILD(tree, 1)) &&
    ```

    bugs.python.org

    ```
    Decorator syntax currently allows only a dotted_name after the @. As far as I can tell, this was a gut-feeling decision made by Guido. [1]
    I spoke with Nick Coghlan at PyTexas about this, and he suggested that if someone did the work, there might be interest in revisiting this restriction.
    @@ -257,4 +260,5 @@ def f():
    def f():
    pass
    [1] http://mail.python.org/pipermail/python-dev/2004-August/046711.html
    [1] http://mail.python.org/pipermail/python-dev/2004-August/046711.html
    ```
  7. dutc revised this gist Nov 19, 2013. 1 changed file with 42 additions and 9 deletions.
    51 changes: 42 additions & 9 deletions notes.md
    Original file line number Diff line number Diff line change
    @@ -1,9 +1,17 @@
    # schedule

    6:30 - 6:40 // settling in
    6:40 - 7:00 // Julian, quick introduction to PyPy
    7:00 - 7:10 // Andy, quick introduction to CLI
    7:10 - 8:30 // CPython workshop
    6:30 - 6:40: settling in
    6:40 - 7:00: Julian, quick introduction to PyPy
    7:00 - 7:10: Andy, quick introduction to CLI
    7:10 - 7:15: James, NYC Python announcements
    7:15 - 8:30: James, CPython workshop
    8:30 - 9:00: mingling + sponsor announcements

    # themes

    CPython for greater understanding of the Python programming language (but "reference implementations always overspecify")
    Reading source to solve problems
    Getting involved, contributing to the project

    # introduction

    @@ -25,20 +33,20 @@ We recommend using our Ubuntu virtual machine (locally or remotely,) since it re

    These tools are extremely rich, and we could do a workshop on each of them individually. We're going to use them in this workshop, but going into depth on anything outside of basic `gdb` is out of our scope.

    We're going to start this workshop by downloading the C source code for CPython 3.3.2. We're going to build it, install it, and run it under `gdb`.
    We're going to start this workshop by downloading the C source code for CPython 3.3.3. We're going to build it, install it, and run it under `gdb`.

    ```
    http://python.org/download/
    wget 'http://python.org/ftp/python/3.3.2/Python-3.3.2.tar.xz' # get the source code
    tar xJvf Python-3.3.2.tar.xz # extract
    wget 'http://python.org/ftp/python/3.3.3/Python-3.3.3.tar.xz' # get the source code
    tar xJvf Python-3.3.3.tar.xz # extract
    # for convenience
    sudo apt-get install build-essential
    sudo apt-get build-dep python3.3 # gives you all the other software Python depends on
    # configure, build, install
    cd Python-3.3.2
    cd Python-3.3.3
    CFLAGS="-g4 -ggdb -gdwarf-4" ./configure --with-pydebug --prefix=$PWD-build
    make -j9
    make install
    @@ -190,6 +198,9 @@ p ((PyFunctionObject*)(((PyMethodObject*)(func))->im_func))->func_name

    # lambda decorators

    docs.python.org/devguide
    pythonmentors.com

    ```
    def dec(func):
    return func
    @@ -224,4 +235,26 @@ Lib/test/test_parser.py
    - validate_dotted_name(CHILD(tree, 1)) &&
    + validate_testlist(CHILD(tree, 1)) &&
    ```
    ```

    Decorator syntax currently allows only a dotted_name after the @. As far as I can tell, this was a gut-feeling decision made by Guido. [1]

    I spoke with Nick Coghlan at PyTexas about this, and he suggested that if someone did the work, there might be interest in revisiting this restriction.

    The attached patch allows any testlist to follow the @.

    The following are now valid:

    @(lambda x:x)
    def f():
    pass

    @(spam if p else eggs)
    def f():
    pass

    @spam().ham().eggs()
    def f():
    pass

    [1] http://mail.python.org/pipermail/python-dev/2004-August/046711.html
  8. dutc revised this gist Nov 19, 2013. 1 changed file with 2 additions and 1 deletion.
    3 changes: 2 additions & 1 deletion notes.md
    Original file line number Diff line number Diff line change
    @@ -2,7 +2,8 @@

    6:30 - 6:40 // settling in
    6:40 - 7:00 // Julian, quick introduction to PyPy
    7:00 - 8:30 // CPython workshop
    7:00 - 7:10 // Andy, quick introduction to CLI
    7:10 - 8:30 // CPython workshop

    # introduction

  9. dutc revised this gist Nov 19, 2013. 1 changed file with 40 additions and 4 deletions.
    44 changes: 40 additions & 4 deletions notes.md
    Original file line number Diff line number Diff line change
    @@ -4,7 +4,7 @@
    6:40 - 7:00 // Julian, quick introduction to PyPy
    7:00 - 8:30 // CPython workshop

    1. introduction
    # introduction

    This workshop will cover the basics of the CPython runtime and interpreter. There is an enormous amount of material to cover, and I'll try to to rush through as much as I can.

    @@ -92,7 +92,7 @@ gdb --args $PWD-build/bin/python3
    (gdb) finish
    ```

    2. which `__mul__`?
    # which `__mul__`?

    ```
    class Foo(int):
    @@ -153,7 +153,7 @@ dis(f)

    *Note If the right operand’s type is a subclass of the left operand’s type and that subclass provides the reflected method for the operation, this method will be called before the left operand’s non-reflected method. This behavior allows subclasses to override their ancestors’ operations.*

    3. why hash(-1) == -2?
    # why hash(-1) == -2?

    ```
    $ python3
    @@ -187,4 +187,40 @@ Objects/abstract.c vs Objects/object.c vs Objects/typeobject.c
    p ((PyFunctionObject*)(((PyMethodObject*)(func))->im_func))->func_name
    ```

    4. lambda decorators
    # lambda decorators

    ```
    def dec(func):
    return func
    @dec
    def foo(x, y):
    return x * y
    dec = lambda func: func
    @(lambda func:func)
    def foo(x, y):
    return x * y
    http://mail.python.org/pipermail/python-dev/2004-August/046711.html
    ```

    ```
    Grammar/Grammar
    -decorator: '@' dotted_name [ '(' [arglist] ')' ] NEWLINE
    +decorator: '@' testlist NEWLINE
    -name_expr = ast_for_dotted_name(c, CHILD(n, 1));
    +name_expr = ast_for_testlist(c, CHILD(n, 1));
    ```

    ```
    make test
    Lib/test/test_decorators.py
    Lib/test/test_parser.py
    - validate_dotted_name(CHILD(tree, 1)) &&
    + validate_testlist(CHILD(tree, 1)) &&
    ```
  10. dutc revised this gist Nov 19, 2013. 1 changed file with 38 additions and 0 deletions.
    38 changes: 38 additions & 0 deletions notes.md
    Original file line number Diff line number Diff line change
    @@ -144,9 +144,47 @@ foo * bar
    bar * foo
    10 * foo
    bar * 10
    from dis import dis
    def f(x, y):
    return x * y
    dis(f)
    ```

    *Note If the right operand’s type is a subclass of the left operand’s type and that subclass provides the reflected method for the operation, this method will be called before the left operand’s non-reflected method. This behavior allows subclasses to override their ancestors’ operations.*

    3. why hash(-1) == -2?

    ```
    $ python3
    hash(100)
    hash(10)
    hash(2)
    hash(1)
    hash(-100)
    hash(-10)
    hash(-2)
    hash(-1)
    $ pypy
    hash(-1)
    ```

    ```
    tb ceval.c:2671
    p PyCFunction_Check(func)
    ```

    ```
    class Foo(object):
    def __hash__(self):
    return -1
    tb builtin_hash
    Objects/abstract.c vs Objects/object.c vs Objects/typeobject.c
    p ((PyFunctionObject*)(((PyMethodObject*)(func))->im_func))->func_name
    ```

    4. lambda decorators
  11. dutc revised this gist Nov 19, 2013. 1 changed file with 116 additions and 113 deletions.
    229 changes: 116 additions & 113 deletions notes.md
    Original file line number Diff line number Diff line change
    @@ -26,122 +26,125 @@ These tools are extremely rich, and we could do a workshop on each of them indiv

    We're going to start this workshop by downloading the C source code for CPython 3.3.2. We're going to build it, install it, and run it under `gdb`.

    http://python.org/download/

    wget 'http://python.org/ftp/python/3.3.2/Python-3.3.2.tar.xz' # get the source code
    tar xJvf Python-3.3.2.tar.xz # extract

    # for convenience
    sudo apt-get install build-essential
    sudo apt-get build-dep python3.3 # gives you all the other software Python depends on

    # configure, build, install
    cd Python-3.3.2
    CFLAGS="-g4 -ggdb -gdwarf-4" ./configure --with-pydebug --prefix=$PWD-build
    make -j9
    make install

    Doc # reStructuredText documentation
    Grammar/Grammar # Python 'grammar' EBNF
    Include # header files
    Lib # the Python standard library, Python code, e.g., collections.namedtuple
    Modules # Python modules written in C, math.pow
    Objects # Python basic types
    Parser # language parser
    Python # interpreter

    $PWD-build/bin/python3

    def f(x, y):
    x, y = y, x

    >>> from dis import dis
    >>> dis(f)
    2 0 LOAD_FAST 1 (y)
    3 LOAD_FAST 0 (x)
    6 ROT_TWO
    7 STORE_FAST 0 (x)
    10 STORE_FAST 1 (y)
    13 LOAD_CONST 0 (None)
    16 RETURN_VALUE

    $ find -iname '*.c' -print0 | xargs -0 grep ROT_TWO
    ./Python/compile.c: case ROT_TWO:
    ./Python/compile.c: ADDOP(c, ROT_TWO);
    ./Python/compile.c: ADDOP(c, ROT_TWO);
    ./Python/peephole.c: codestr[i] = ROT_TWO;
    ./Python/peephole.c: codestr[i+1] = ROT_TWO;
    ./Python/ceval.c: TARGET(ROT_TWO)

    gdb --args $PWD-build/bin/python3
    (gdb) source Tools/gdb/libpython.py
    (gdb) r
    >>> def f(x, y):
    ... x, y = y, x
    ^C
    (gdb) tbreak ceval.c:1389
    (gdb) c
    >>> f(10, 20)
    (gdb) list
    (gdb) info macro TARGET
    (gdb) info function PyObject_GetAttr
    (gdb) backtrace
    (gdb) next
    (gdb) step
    (gdb) finish
    ```
    http://python.org/download/
    wget 'http://python.org/ftp/python/3.3.2/Python-3.3.2.tar.xz' # get the source code
    tar xJvf Python-3.3.2.tar.xz # extract
    # for convenience
    sudo apt-get install build-essential
    sudo apt-get build-dep python3.3 # gives you all the other software Python depends on
    # configure, build, install
    cd Python-3.3.2
    CFLAGS="-g4 -ggdb -gdwarf-4" ./configure --with-pydebug --prefix=$PWD-build
    make -j9
    make install
    Doc # reStructuredText documentation
    Grammar/Grammar # Python 'grammar' EBNF
    Include # header files
    Lib # the Python standard library, Python code, e.g., collections.namedtuple
    Modules # Python modules written in C, math.pow
    Objects # Python basic types
    Parser # language parser
    Python # interpreter
    $PWD-build/bin/python3
    def f(x, y):
    x, y = y, x
    >>> from dis import dis
    >>> dis(f)
    2 0 LOAD_FAST 1 (y)
    3 LOAD_FAST 0 (x)
    6 ROT_TWO
    7 STORE_FAST 0 (x)
    10 STORE_FAST 1 (y)
    13 LOAD_CONST 0 (None)
    16 RETURN_VALUE
    $ find -iname '*.c' -print0 | xargs -0 grep ROT_TWO
    ./Python/compile.c: case ROT_TWO:
    ./Python/compile.c: ADDOP(c, ROT_TWO);
    ./Python/compile.c: ADDOP(c, ROT_TWO);
    ./Python/peephole.c: codestr[i] = ROT_TWO;
    ./Python/peephole.c: codestr[i+1] = ROT_TWO;
    ./Python/ceval.c: TARGET(ROT_TWO)
    gdb --args $PWD-build/bin/python3
    (gdb) source Tools/gdb/libpython.py
    (gdb) r
    >>> def f(x, y):
    ... x, y = y, x
    ^C
    (gdb) tbreak ceval.c:1389
    (gdb) c
    >>> f(10, 20)
    (gdb) list
    (gdb) info macro TARGET
    (gdb) info function PyObject_GetAttr
    (gdb) backtrace
    (gdb) next
    (gdb) step
    (gdb) finish
    ```

    2. which `__mul__`?

    class Foo(int):

    def __mul__(self, other):
    print('Foo.__mul__({}, {})'.format(self, other))

    class Bar(int):
    def __mul__(self, other):
    print('Bar.__mul__({}, {})'.format(self, other))

    foo, bar = Foo(10), Bar(20)
    foo * bar
    bar * foo
    10 * foo
    bar * 10

    class Foo(int):
    def __mul__(self, other):
    print('Foo.__mul__({}, {})'.format(self, other))
    def __rmul__(self, other):
    print('Foo.__rmul__({}, {})'.format(self, other))

    class Bar(int):
    def __mul__(self, other):
    print('Bar.__mul__({}, {})'.format(self, other))
    def __rmul__(self, other):
    print('Bar.__rmul__({}, {})'.format(self, other))

    foo, bar = Foo(10), Bar(20)
    foo * bar
    bar * foo
    10 * foo
    bar * 10

    class Foo(int):
    def __mul__(self, other):
    print('Foo.__mul__({}, {})'.format(self, other))
    def __rmul__(self, other):
    print('Foo.__rmul__({}, {})'.format(self, other))

    class Bar(Foo):
    def __mul__(self, other):
    print('Bar.__mul__({}, {})'.format(self, other))
    def __rmul__(self, other):
    print('Bar.__rmul__({}, {})'.format(self, other))

    foo, bar = Foo(10), Bar(20)
    foo * bar
    bar * foo
    10 * foo
    bar * 10
    ```
    class Foo(int):
    def __mul__(self, other):
    print('Foo.__mul__({}, {})'.format(self, other))
    class Bar(int):
    def __mul__(self, other):
    print('Bar.__mul__({}, {})'.format(self, other))
    foo, bar = Foo(10), Bar(20)
    foo * bar
    bar * foo
    10 * foo
    bar * 10
    class Foo(int):
    def __mul__(self, other):
    print('Foo.__mul__({}, {})'.format(self, other))
    def __rmul__(self, other):
    print('Foo.__rmul__({}, {})'.format(self, other))
    class Bar(int):
    def __mul__(self, other):
    print('Bar.__mul__({}, {})'.format(self, other))
    def __rmul__(self, other):
    print('Bar.__rmul__({}, {})'.format(self, other))
    foo, bar = Foo(10), Bar(20)
    foo * bar
    bar * foo
    10 * foo
    bar * 10
    class Foo(int):
    def __mul__(self, other):
    print('Foo.__mul__({}, {})'.format(self, other))
    def __rmul__(self, other):
    print('Foo.__rmul__({}, {})'.format(self, other))
    class Bar(Foo):
    def __mul__(self, other):
    print('Bar.__mul__({}, {})'.format(self, other))
    def __rmul__(self, other):
    print('Bar.__rmul__({}, {})'.format(self, other))
    foo, bar = Foo(10), Bar(20)
    foo * bar
    bar * foo
    10 * foo
    bar * 10
    ```

    *Note If the right operand’s type is a subclass of the left operand’s type and that subclass provides the reflected method for the operation, this method will be called before the left operand’s non-reflected method. This behavior allows subclasses to override their ancestors’ operations.*

  12. dutc revised this gist Nov 19, 2013. 1 changed file with 1 addition and 0 deletions.
    1 change: 1 addition & 0 deletions notes.md
    Original file line number Diff line number Diff line change
    @@ -93,6 +93,7 @@ We're going to start this workshop by downloading the C source code for CPython
    2. which `__mul__`?

    class Foo(int):

    def __mul__(self, other):
    print('Foo.__mul__({}, {})'.format(self, other))

  13. dutc revised this gist Nov 19, 2013. 1 changed file with 23 additions and 23 deletions.
    46 changes: 23 additions & 23 deletions notes.md
    Original file line number Diff line number Diff line change
    @@ -93,30 +93,30 @@ We're going to start this workshop by downloading the C source code for CPython
    2. which `__mul__`?

    class Foo(int):
    def __mul__(self, other):
    print('Foo.__mul__({}, {})'.format(self, other))
    def __mul__(self, other):
    print('Foo.__mul__({}, {})'.format(self, other))

    class Bar(int):
    def __mul__(self, other):
    print('Bar.__mul__({}, {})'.format(self, other))
    def __mul__(self, other):
    print('Bar.__mul__({}, {})'.format(self, other))

    foo, bar = Foo(10), Bar(20)
    foo * bar
    bar * foo
    10 * foo
    bar * 10

    class Foo(int):
    def __mul__(self, other):
    print('Foo.__mul__({}, {})'.format(self, other))
    def __rmul__(self, other):
    print('Foo.__rmul__({}, {})'.format(self, other))
    def __mul__(self, other):
    print('Foo.__mul__({}, {})'.format(self, other))
    def __rmul__(self, other):
    print('Foo.__rmul__({}, {})'.format(self, other))

    class Bar(int):
    def __mul__(self, other):
    print('Bar.__mul__({}, {})'.format(self, other))
    def __rmul__(self, other):
    print('Bar.__rmul__({}, {})'.format(self, other))
    def __mul__(self, other):
    print('Bar.__mul__({}, {})'.format(self, other))
    def __rmul__(self, other):
    print('Bar.__rmul__({}, {})'.format(self, other))

    foo, bar = Foo(10), Bar(20)
    foo * bar
    @@ -125,16 +125,16 @@ We're going to start this workshop by downloading the C source code for CPython
    bar * 10

    class Foo(int):
    def __mul__(self, other):
    print('Foo.__mul__({}, {})'.format(self, other))
    def __rmul__(self, other):
    print('Foo.__rmul__({}, {})'.format(self, other))
    def __mul__(self, other):
    print('Foo.__mul__({}, {})'.format(self, other))
    def __rmul__(self, other):
    print('Foo.__rmul__({}, {})'.format(self, other))

    class Bar(Foo):
    def __mul__(self, other):
    print('Bar.__mul__({}, {})'.format(self, other))
    def __rmul__(self, other):
    print('Bar.__rmul__({}, {})'.format(self, other))
    def __mul__(self, other):
    print('Bar.__mul__({}, {})'.format(self, other))
    def __rmul__(self, other):
    print('Bar.__rmul__({}, {})'.format(self, other))

    foo, bar = Foo(10), Bar(20)
    foo * bar
  14. dutc revised this gist Nov 19, 2013. 1 changed file with 55 additions and 3 deletions.
    58 changes: 55 additions & 3 deletions notes.md
    Original file line number Diff line number Diff line change
    @@ -52,8 +52,9 @@ We're going to start this workshop by downloading the C source code for CPython

    $PWD-build/bin/python3

    >>> def f(x, y):
    >>> x, y = y, x
    def f(x, y):
    x, y = y, x

    >>> from dis import dis
    >>> dis(f)
    2 0 LOAD_FAST 1 (y)
    @@ -89,8 +90,59 @@ We're going to start this workshop by downloading the C source code for CPython
    (gdb) step
    (gdb) finish

    2. which __mul__?
    2. which `__mul__`?

    class Foo(int):
    def __mul__(self, other):
    print('Foo.__mul__({}, {})'.format(self, other))

    class Bar(int):
    def __mul__(self, other):
    print('Bar.__mul__({}, {})'.format(self, other))

    foo, bar = Foo(10), Bar(20)
    foo * bar
    bar * foo
    10 * foo
    bar * 10

    class Foo(int):
    def __mul__(self, other):
    print('Foo.__mul__({}, {})'.format(self, other))
    def __rmul__(self, other):
    print('Foo.__rmul__({}, {})'.format(self, other))

    class Bar(int):
    def __mul__(self, other):
    print('Bar.__mul__({}, {})'.format(self, other))
    def __rmul__(self, other):
    print('Bar.__rmul__({}, {})'.format(self, other))

    foo, bar = Foo(10), Bar(20)
    foo * bar
    bar * foo
    10 * foo
    bar * 10

    class Foo(int):
    def __mul__(self, other):
    print('Foo.__mul__({}, {})'.format(self, other))
    def __rmul__(self, other):
    print('Foo.__rmul__({}, {})'.format(self, other))

    class Bar(Foo):
    def __mul__(self, other):
    print('Bar.__mul__({}, {})'.format(self, other))
    def __rmul__(self, other):
    print('Bar.__rmul__({}, {})'.format(self, other))

    foo, bar = Foo(10), Bar(20)
    foo * bar
    bar * foo
    10 * foo
    bar * 10

    *Note If the right operand’s type is a subclass of the left operand’s type and that subclass provides the reflected method for the operation, this method will be called before the left operand’s non-reflected method. This behavior allows subclasses to override their ancestors’ operations.*

    3. why hash(-1) == -2?
    4. lambda decorators
  15. dutc revised this gist Nov 19, 2013. 1 changed file with 1 addition and 1 deletion.
    2 changes: 1 addition & 1 deletion notes.md
    Original file line number Diff line number Diff line change
    @@ -45,7 +45,7 @@ We're going to start this workshop by downloading the C source code for CPython
    Grammar/Grammar # Python 'grammar' EBNF
    Include # header files
    Lib # the Python standard library, Python code, e.g., collections.namedtuple
    Modules # Python modules written in C, mathmodule.c
    Modules # Python modules written in C, math.pow
    Objects # Python basic types
    Parser # language parser
    Python # interpreter
  16. dutc revised this gist Nov 19, 2013. 1 changed file with 4 additions and 4 deletions.
    8 changes: 4 additions & 4 deletions notes.md
    Original file line number Diff line number Diff line change
    @@ -1,8 +1,8 @@
    # schedule

    6:30 - 6:40 // settling in
    6:40 - 7:00 // Julian, quick introduction to PyPy
    7:00 - 8:30 // CPython workshop
    6:30 - 6:40 // settling in
    6:40 - 7:00 // Julian, quick introduction to PyPy
    7:00 - 8:30 // CPython workshop

    1. introduction

    @@ -29,7 +29,7 @@ We're going to start this workshop by downloading the C source code for CPython
    http://python.org/download/

    wget 'http://python.org/ftp/python/3.3.2/Python-3.3.2.tar.xz' # get the source code
    tar xJvf Python-3.3.2.tar.xz # extract
    tar xJvf Python-3.3.2.tar.xz # extract

    # for convenience
    sudo apt-get install build-essential
  17. dutc revised this gist Nov 19, 2013. 1 changed file with 64 additions and 62 deletions.
    126 changes: 64 additions & 62 deletions notes.md
    Original file line number Diff line number Diff line change
    @@ -1,3 +1,5 @@
    # schedule

    6:30 - 6:40 // settling in
    6:40 - 7:00 // Julian, quick introduction to PyPy
    7:00 - 8:30 // CPython workshop
    @@ -24,68 +26,68 @@ These tools are extremely rich, and we could do a workshop on each of them indiv

    We're going to start this workshop by downloading the C source code for CPython 3.3.2. We're going to build it, install it, and run it under `gdb`.

    http://python.org/download/

    wget 'http://python.org/ftp/python/3.3.2/Python-3.3.2.tar.xz' # get the source code
    tar xJvf Python-3.3.2.tar.xz # extract

    # for convenience
    sudo apt-get install build-essential
    sudo apt-get build-dep python3.3 # gives you all the other software Python depends on

    # configure, build, install
    cd Python-3.3.2
    CFLAGS="-g4 -ggdb -gdwarf-4" ./configure --with-pydebug --prefix=$PWD-build
    make -j9
    make install

    Doc # reStructuredText documentation
    Grammar/Grammar # Python 'grammar' EBNF
    Include # header files
    Lib # the Python standard library, Python code, e.g., collections.namedtuple
    Modules # Python modules written in C, mathmodule.c
    Objects # Python basic types
    Parser # language parser
    Python # interpreter

    $PWD-build/bin/python3

    >>> def f(x, y):
    >>> x, y = y, x
    >>> from dis import dis
    >>> dis(f)
    2 0 LOAD_FAST 1 (y)
    3 LOAD_FAST 0 (x)
    6 ROT_TWO
    7 STORE_FAST 0 (x)
    10 STORE_FAST 1 (y)
    13 LOAD_CONST 0 (None)
    16 RETURN_VALUE

    $ find -iname '*.c' -print0 | xargs -0 grep ROT_TWO
    ./Python/compile.c: case ROT_TWO:
    ./Python/compile.c: ADDOP(c, ROT_TWO);
    ./Python/compile.c: ADDOP(c, ROT_TWO);
    ./Python/peephole.c: codestr[i] = ROT_TWO;
    ./Python/peephole.c: codestr[i+1] = ROT_TWO;
    ./Python/ceval.c: TARGET(ROT_TWO)

    gdb --args $PWD-build/bin/python3
    (gdb) source Tools/gdb/libpython.py
    (gdb) r
    >>> def f(x, y):
    ... x, y = y, x
    ^C
    (gdb) tbreak ceval.c:1389
    (gdb) c
    >>> f(10, 20)
    (gdb) list
    (gdb) info macro TARGET
    (gdb) info function PyObject_GetAttr
    (gdb) backtrace
    (gdb) next
    (gdb) step
    (gdb) finish
    http://python.org/download/

    wget 'http://python.org/ftp/python/3.3.2/Python-3.3.2.tar.xz' # get the source code
    tar xJvf Python-3.3.2.tar.xz # extract

    # for convenience
    sudo apt-get install build-essential
    sudo apt-get build-dep python3.3 # gives you all the other software Python depends on

    # configure, build, install
    cd Python-3.3.2
    CFLAGS="-g4 -ggdb -gdwarf-4" ./configure --with-pydebug --prefix=$PWD-build
    make -j9
    make install

    Doc # reStructuredText documentation
    Grammar/Grammar # Python 'grammar' EBNF
    Include # header files
    Lib # the Python standard library, Python code, e.g., collections.namedtuple
    Modules # Python modules written in C, mathmodule.c
    Objects # Python basic types
    Parser # language parser
    Python # interpreter

    $PWD-build/bin/python3
    >>> def f(x, y):
    >>> x, y = y, x
    >>> from dis import dis
    >>> dis(f)
    2 0 LOAD_FAST 1 (y)
    3 LOAD_FAST 0 (x)
    6 ROT_TWO
    7 STORE_FAST 0 (x)
    10 STORE_FAST 1 (y)
    13 LOAD_CONST 0 (None)
    16 RETURN_VALUE

    $ find -iname '*.c' -print0 | xargs -0 grep ROT_TWO
    ./Python/compile.c: case ROT_TWO:
    ./Python/compile.c: ADDOP(c, ROT_TWO);
    ./Python/compile.c: ADDOP(c, ROT_TWO);
    ./Python/peephole.c: codestr[i] = ROT_TWO;
    ./Python/peephole.c: codestr[i+1] = ROT_TWO;
    ./Python/ceval.c: TARGET(ROT_TWO)

    gdb --args $PWD-build/bin/python3
    (gdb) source Tools/gdb/libpython.py
    (gdb) r
    >>> def f(x, y):
    ... x, y = y, x
    ^C
    (gdb) tbreak ceval.c:1389
    (gdb) c
    >>> f(10, 20)
    (gdb) list
    (gdb) info macro TARGET
    (gdb) info function PyObject_GetAttr
    (gdb) backtrace
    (gdb) next
    (gdb) step
    (gdb) finish

    2. which __mul__?

  18. dutc renamed this gist Nov 19, 2013. 1 changed file with 0 additions and 0 deletions.
    File renamed without changes.
  19. dutc created this gist Nov 19, 2013.
    94 changes: 94 additions & 0 deletions notes.txt
    Original file line number Diff line number Diff line change
    @@ -0,0 +1,94 @@
    6:30 - 6:40 // settling in
    6:40 - 7:00 // Julian, quick introduction to PyPy
    7:00 - 8:30 // CPython workshop

    1. introduction

    This workshop will cover the basics of the CPython runtime and interpreter. There is an enormous amount of material to cover, and I'll try to to rush through as much as I can.

    This introduction will be rushed, so it may not be perfectly accurate.

    CPython is the reference implementation of the Python programming language. The Python programming language exists separate from this implementation, and, in fact, alternate implementations of the language exist. Julian spoke about PyPy. There are also implementations built on top of the .NET runtime (IronPython,) the JVM (Jython,) &c.

    CPython is still the dominant, most commonly used implementation of the Python language. (Whether this should or should not be the case is an independent question, one which we can see Julian feels passionately about.) CPython is probably what you are using when you type `python` at the command line.

    CPython is written in the C programming language. It should compile without errors on a C89 or C99 compliant compiler.

    If you want to learn more about the C programming language, stop by office hours. There are many C/C++/Java programmers who might be able to help you out. There are also on-line resources such as c.learncodethehardway.org/book. You may find the syntax of C to be not completely alien to you as a Python programmer. The CPython interpreter is written to be fairly straightforward to understand and to contribute to.

    We are going to be using a couple of other tools, too. autoconf, gcc, gdb, coreutils.

    We recommend using our Ubuntu virtual machine (locally or remotely,) since it removes a lot of the difficulty from this process.

    These tools are extremely rich, and we could do a workshop on each of them individually. We're going to use them in this workshop, but going into depth on anything outside of basic `gdb` is out of our scope.

    We're going to start this workshop by downloading the C source code for CPython 3.3.2. We're going to build it, install it, and run it under `gdb`.

    http://python.org/download/

    wget 'http://python.org/ftp/python/3.3.2/Python-3.3.2.tar.xz' # get the source code
    tar xJvf Python-3.3.2.tar.xz # extract

    # for convenience
    sudo apt-get install build-essential
    sudo apt-get build-dep python3.3 # gives you all the other software Python depends on

    # configure, build, install
    cd Python-3.3.2
    CFLAGS="-g4 -ggdb -gdwarf-4" ./configure --with-pydebug --prefix=$PWD-build
    make -j9
    make install

    Doc # reStructuredText documentation
    Grammar/Grammar # Python 'grammar' EBNF
    Include # header files
    Lib # the Python standard library, Python code, e.g., collections.namedtuple
    Modules # Python modules written in C, mathmodule.c
    Objects # Python basic types
    Parser # language parser
    Python # interpreter

    $PWD-build/bin/python3

    >>> def f(x, y):
    >>> x, y = y, x
    >>> from dis import dis
    >>> dis(f)
    2 0 LOAD_FAST 1 (y)
    3 LOAD_FAST 0 (x)
    6 ROT_TWO
    7 STORE_FAST 0 (x)
    10 STORE_FAST 1 (y)
    13 LOAD_CONST 0 (None)
    16 RETURN_VALUE

    $ find -iname '*.c' -print0 | xargs -0 grep ROT_TWO
    ./Python/compile.c: case ROT_TWO:
    ./Python/compile.c: ADDOP(c, ROT_TWO);
    ./Python/compile.c: ADDOP(c, ROT_TWO);
    ./Python/peephole.c: codestr[i] = ROT_TWO;
    ./Python/peephole.c: codestr[i+1] = ROT_TWO;
    ./Python/ceval.c: TARGET(ROT_TWO)

    gdb --args $PWD-build/bin/python3
    (gdb) source Tools/gdb/libpython.py
    (gdb) r
    >>> def f(x, y):
    ... x, y = y, x
    ^C
    (gdb) tbreak ceval.c:1389
    (gdb) c
    >>> f(10, 20)
    (gdb) list
    (gdb) info macro TARGET
    (gdb) info function PyObject_GetAttr
    (gdb) backtrace
    (gdb) next
    (gdb) step
    (gdb) finish

    2. which __mul__?


    3. why hash(-1) == -2?
    4. lambda decorators