Django, Django forms and Django rest framework: Payload validation

Doc.rst

Django request flow

-------------------------------                         ------------------ Django --------------------
| Browser: GET /udo/contact/2 |    === wsgi/fcgi ===>   | 1. Asks OS for DJANGO_SETTINGS_MODULE      |
-------------------------------                         | 2. Build Request (from wsgi/fcgi callback) |
                                                        | 3. Get settings.ROOT_URLCONF module        |
                                                        | 4. Resolve URL/view from request.path      | # url(r'^udo/contact/(?P<id>\w+)', view, name='url-identifier')
                                                        | 5. Apply request middlewares               | # settings.MIDDLEWARE_CLASSES
                                                        | 6. Run VIEW                                | # def view(request, id):
---------------------------------------------------------                                            -------------------------------------------------------------------------------------

# Clasic views (<= django 1.2)
from django import http

def view(request, id):
    if request.method == 'GET':
        return http.HttpResponse("some content")
    elif request.method == 'POST':
        return http.HttpResponseBadRequest("no way")

# Class views (>= django 1.3)

[urls.py]
url(r'^udo/contact/(?P<id>\w+)', ClassView.as_view(), name='url-identifier')

[views.py]
from django.views.generic import View

class ClassView(View):

    def get(request, id):
        return http.HttpResponse("some content")

    def post(request, id):
        return http.HttpResponseBadRequest("no way")

---------------------------------------------------------                                            --------------------------------------------------------------------------------------
                                                        | 7. Run response middlewares                |
                                                        |                                            |
                                                        |                                            |
                                                        ----------------------------------------------
                                                                          ||
                                                                          ||
                                                                          ||
                                                                          VV
                                                              --------------------------
                                                              | Browser: Read Response |
                                                              --------------------------

Django & Django rest framework flow

If you understand the previous section, I focus now on step 6

[urls.py]
url(r'^udo/contact/(?P<id>\w+)', Drf.as_view(), name='url-identifier')

[views.py]
from djangorestframework.views import View

# I'm exposing some default attrs on DRF View (in pseudo code)

class Drf(View):
    # parsers = (Json, form, multipart, xml)
    # renderers = (Json, jsonp, html, xhtml, plain, xml)
    # authentication = (Userlogged, BasicAuth)
    # permissions = (FullAnonAccess, )

    def post(request, id):
        validated = self.CONTENT
        raw = self.DATA
        user_regarding_of_permissions = self.user
        # ...

6. Run view

    * Run Drf.as_view() CLASS function (django internal, because need a callable and we cant have an instance of Drf in urls) return a callable

** [Spanish: Aquí surge una pregunta, quien la adivine gana un caramelo (le invito a un café) :P | Hint: View]**

6.1 Run that callable that in fact run Drf.dispatch bound method
 ||
 ||
 VV
 ------ This is DRF behaviour -------------------------------------------------------------------------------------------

    6.2 Read permissions attr (list of permissions handlers with ``check_permission`` methods)
        6.2.1 Get user

            6.2.1.1 Read authentication attr (list of authenticators with ``authenticate`` methods)
            6.2.1.2 Run through this authenticators IN ORDER to attempt to authenticate the request
            6.2.1.3 Authenticators could return an User or None

                    UserLogged => Read from request.session
                    BasicAuth  => Read from request.headers

        6.2.2 Run through this permissions IN ORDER to check regarding of ``user`` obtained before
        6.2.3 This permissions could pass (continue request) or raise some 403 response (by default)

                FullAnonAccess are as you can imagine a dummy. always pass
                djangorestframework.permissions.IsAuthenticated on the other hand check if user.is_authenticated() [OMG, rly? :P]

    6.3 Set ``method`` from request
    6.4 Set ``content_type`` from request (so important to parsers)
    6.5 Run ``method`` method from view (GET => def get(request...)

        NOTE: If it can't obtain that method from view, raise a METHOD_NOT_ALLOWED response exception
        ||
        ||
        VV
        ------------------- Ok, now we are in our code... but with some magic --------------------------------------------
        6.5.1 Django raw request are in self.request
        6.5.2 If flow pass permissions layer, and regarding of that permissions, we got User or AnonymousUser in self.user
        6.5.3 self.DATA is the payload RAW (similar to request.POST) [Detail explained bellow]
        6.5.4 self.FILES is the request files (similar to request.FILES). Only multipart requests for example [Detail explained bellow]
        6.5.5 self.CONTENT is validated self.DATA (though django.forms) [Detail explained bellow]
        6.5.6 self.PARAMS is validated self.request.GET (though django.forms) [Detail explained bellow]

Django rest framework and Django forms VALIDATION

I need to explain some concepts in order to understand the big picture

Django forms

Full doc: https://docs.djangoproject.com/en/dev/topics/forms/

Explained as a black box: payload -> form -> payload_cleaned or raise Invalid

Easy example:

[forms.py]
from django import forms

class Contact(forms.Form):

    name = forms.CharField(required=True, max_length=50)
    email = forms.EmailField()
    language = forms.CharField(required=True)
    age = forms.IntegerField()

    def clean_language(self):
        language = self.cleaned_data['language']
        if language.lower() != u'python':
            raise forms.ValidationError("Are you crazy? Python rocks!")
        return language

[views.py]
mock_payload = {
    'name': 'copitux',
    'email': '[email protected]',
    'language': 'python',
    'age': 22,
}
# Valid
form = forms.Contact(mock_payload)
assert form.is_valid()

# Required fields
del mock_payload['name']
form = forms.Contact(mock_payload)
assert not form.is_valid()
assert form.errors = [{'name': 'This field is required'}]

# clean validation
mock_payload.update({'language': 'javascript', 'name': 'copitux'})
form = forms.Contact(mock_payload)
assert not form.is_valid()
assert form.errors = [{'language': 'Are you crazy? Python rocks!'}]

Validation/Sanitize flow

1. Each Field
2. clean_<field> methods
3. clean

Django rest framework PARSERS

Why if jQuery.ajax send payload as string we retrieve in our view.{post/put...} a python object? DRF.parsers!

Let examine a jQuery request

$.ajax({
    url: 'udo/contacts/2',
    type: 'POST',
    data: '{"name": "copitux"}', // or $.toJson({name: "copitux"})
    dataType: 'json',
    contentType: 'application/json'
});

With that code it build the request with some headers different to browser. To this context there are two important headers

#Ajax_request.send => Django request flow / Build Request

assert request['HTTP_ACCEPT'] in 'application/json, text/javascript, */*; q=0.01'
assert request['CONTENT_TYPE'] in 'application/json' # It maybe come as HTTP_CONTENT_TYPE
# It maybe differs, but I only want the concept

## Extra info! (for free :P)
Also jQuery send this header ('HTTP_X_REQUESTED_WITH': 'XMLHttpRequest')
and with that ``request.is_ajax()`` works

Only CONTENT_TYPE is important for parsers [Spanish: Caramelol!! - Para que puede ser importante HTTP_ACCEPT? | Hint: See attrs of Drf class View]

Did you remember this line? parsers = (Json, xml, form ...)

If fact it's a tuple of drf.parsers: parsers = (djangorestframework.parsers.JSONParser, ...)

Each parser has a content_type associated, so DRF'll try to parse with each parser until one of them has the content_type sended. In our example, it's obvious: JSONParser will parse the payload

But, when?

NOTE: self.DATA, self.FILES and so on are cached properties

When we call self.DATA the first time in that request so

Build request => Run view (wrapped by DRF permissions behaviour) => access self.DATA in view => DRF see the content_type => DRF identify the parser (JSONparser in the example) => and run it

[DRF.JSONparser internal - psecudo code]
def parse(raw_payload):
    return json.load(raw_payload)

In fact, when we call self.DATA; self.FILES will be cached too but it's beyond the scope of this tutorial

Django forms + Django rest framework parsers = VALIDATION glue

Yes, sorry. Another time I need to explain one new concept before glue all

Django rest framework RESOURCES

We could set a resource attribute which runs the validation and prepare response if all go fine or not Also we could set a <method>_form form attribute to handle that validation.

DRF has a lot of magic here, it's beyond the scope of this tutorial explain that so I explain the most used method with REST (IMHO)

class Drf(View):
    # Avoid another attrs yet described

    post_form = forms.NewContact
    get_form = forms.Contact

# For anybody interested in another ways

class Contact(FormResource):

    post_form = forms.NewContact
    get_form = forms.Contact

class Drf(View):

    resource = Contact

Glue all: VALIDATION

The validation process begin when we access to self.CONTENT the first time in the request

class Drf(View):
    # ...
    post_form = forms.Contact

    def post(self, request):
        payload_validated = self.CONTENT

This occurs internally [psecudo code]:

    form = forms.Contact(self.DATA)
    if form.is_valid():
        return form.cleaned_data
    else:
        raise ErrorResponse(status_code=BADREQUEST_400, content=friendly(form.errors))

Conclusion

If we define a protocol between Backend and Frontend to only send JSON dicts

payload = {
    'key1': 'value',
    'key2': 'value2', # ...
}

We could handle the validation cleanly and encapsulate our API rules in forms (per resource, per API. Whatever we want)

Advantages: All

• Clean code
• Smaller views
• Handle validation errors cleanly
• Decouple validation from another tasks => Change algorithms more easily
• Django forms are 100% flexible in an easy way (We can subclass forms, fields, inyect some specific validation to one specific field ...)
• Django rest framework Resources are also 70% flexible. (Por example; we can create a new Resource to handle JSON lists ['value1', 'value2'] or JSON list of dicts [{'key1': 'value1'}, ...]

Disadvantages: Only one but we shouldn't exploit it. If we define a workaround of REST stantard... we could have troubles (and hack DRF validation flow). Corollary: Follow rest standards (if all frameworks in the world; backend and frontend follow that... why we do not?

example.py

# -*- coding: utf-8 -*-

[Some JS | Pseudo code]
    workorder = read_form_wizard_css_selectors()
    // workoder = {status: status_value, name: name_value ...}
$.ajax({
    url: url,
    dataType: 'json',
    type: 'POST',
    contentType: 'application/json',
    data: $.toJson(workorder)
}).success(function(*args) {
    draw()
}).error(function(*args) {
    friendly_notify_user or parse response.errors and try again
});

[workorder.forms.py]
class NewWorkOrder(forms.Form):

    status = forms.CharField(required=True)
    responsible_name = forms.CharField(required=True)
    responsible_email = forms.EmailField()

[workorder.views.py]

# Instead of...
class List(View):

    def post(request, id):

        content = self.DATA
        new_workorder = {}
        if 'status' in self.DATA:
            if self.DATA['status'] is None:
                raise BadRequest('status cant be empty')
            new_workorder['status'] = self.DATA['status']

        # same for name, email and so on
        # two nested, 4 hardcoding

# that
class List(View):

    post_form = forms.NewWorkOrder

    def post(request, id):
        content = self.CONTENT
        new_workorder = {
            'status': content.get('status')
            # ...
        }

        # In fact... If we are strict in the validation, we could do
        # things like
        new_workoder = self.CONTENT
        self.save(new_workorder)


DRF handle the validation errors returning a 400 Response with field errors in
content