Simple Restish with Added Persistence and Content Negotiation

Well comparing rest web frameworks has become something of a theme recently with Martijn Faasen’s recent post about Grok. Adding persistence is something that we will typically all have to do at sometime in most real world web applications and will definitely slow things down a little; so I thought I’d approach the problem using my latest favourite storage engine, CouchDB - but with a library I’ve been writing that makes using CouchDB under restish a doddle.

First of all I set up the restish project including the couchish module and configuration (the code for which is a right mess at the moment but it works - just). I thought it would be good to show some of the content negotiation handling in restish too.. Here are the new resources..

class Root(resource.Resource):

    def get_counts(self, request):
        cdb = collection.CouchishDB(request)
        return cdb.get_by('song','song',key=self.id,include_docs=True)

    @resource.GET(accept='text/plain')
    def get_text(self, request):
        counts = self.get_counts(request)
        res = ','.join(['%s=%s' % (k, v) for k, v in counts.iteritems()])
        return http.ok([], res)

    @resource.GET(accept='application/json')
    def get_json(self, request):
        counts = self.get_counts(request)
        return http.ok([], simplejson.dumps(counts))

    @resource.GET(accept='text/html')
    @templating.page('songs.html')
    def get_html(self, request):
        counts = self.get_counts(request)
        return {'counts':counts}

    @resource.DELETE(accept='text')
    def delete(self, request):
        cdb.delete_all('song')
        return http.ok([], 'OK')

    @resource.child('song')
    def song(self, request, segments):
        return Song(segments[0]), ()

The code is similar to my previous examples but this time the count is being returned from CouchDB. I’ve also included accept headers for html, json and plain text. (I could have used ‘text’,’html’ and ‘json’ for the accept string too). (I also setup a template called song.html)

CouchishDB uses schemaish to convert json into python types (which includes file types stored as attachments). It also registers rdf like relationships between types and handles primitive document local caches of referred data (an example at the bottom of the post). In the yaml schema I included, I’ve also asked CouchDB to create a view indexed by song, which I’ve then used to get the songs back out by passing the type (song) and the view by attr (song).

CouchDB configuration

To setup the database, I created a yaml config file for couchish in the model directory and called it song.yaml

- name: song
  required: True
  viewby: True

- name: count
  type: int
  required: True

The Song resource

This will setup the CouchDB ‘song’ database and configure our song schema and it also tells couchish to generate an index on the song attribute. You can setup lots more things in the yaml file but more about that later (and in later posts).

Finally, here is the Song resource.

class Song(resource.Resource):

    def __init__(self, id):
        self.id = id

    def get_song(request, id):
        cdb = collection.CouchishDB(request)
        return cdb.get_by('song','song',key=self.id,include_docs=True)

    @resource.GET(accept='text')
    def text(self, request):
        songs = self.get_songs(request, id)
        if len(songs) >0:
            count = songs[0]['count']
            return http.ok([], str(count))
        return http.not_found([], '')

    @resource.POST(accept='text')
    def post(self, request):
        songs = self.get_songs(request, id)
        if len(songs) == 0:
            song = {'song': self.id, 'count':1}
            cdb.create('song', song)
        else:
            song = songs[0]
            song['count'] += 1
            cdb.set('song', song)
        return http.ok([], str(song['count']))

Couchish always returns a list, even when given a unique id, hence gettings songs out even when I want just one. The increment part is slightly different as couch only gets and sets whole documents, hence you must get the document out (as a dict), modify it and then put it back.

Nowhere near as succint as implicit python object persistance that Grok does!

Let’s try out content negotiation

This app now return these results ..

$ curl -X GET -H  "Accept: application/json" http://localhost:8080

[{"count": 4, "song": "1"}, {"count": 10, "song": "rock"}]

$ curl -X GET -H  "Accept: text/html" http://localhost:8080

<dl>
<dt>1</dt><dd>4</dd>
<dt>rock</dt><dd>10</dd>
</dl>

$ curl -X GET -H  "Accept: text/plain" http://localhost:8080

1=4,rock=10

This returns about 400 reqs/sec for me (down from 750 reqs/sec with the in memory storage) which is still pretty good. As an aside, CouchDB will return about 800 reqs/sec via straight http.

What else do we get

Well... we can also mount an admin application off the root resource that looks like this..

class Admin(base.BasePage):

    @resource.GET()
    @templating.page('root.html')
    def GET(self, request):
        model_metadata =  request.environ['service.model_metadata']
        return {'model_metadata':model_metadata}

    @resource.child('{type}')
    def items(self, request, segments, type=None):
        return collection.CollectionPage(type=type)

    @resource.child('{type}/{id}')
    def item(self, request, segments, type=None, id=None):
        return collection.CollectionItemPage(id, type=type)

This collection page is now an auto configured administration system based on formish. Lets have a look at the default index and songs page..

Well the songs list looks a bit arduous. Lets customise it.. If we add the following to the yaml files

metadata:
  labels:
    singular: Song
    plural: Songs
  templates:
    items-table:
       - label: song
         value: ${item['song']}
       - label: count
         value: <strong>${item['count']}</strong>

We get a better looking admin list..

OK, lets look at the song editor page..

So this does the job... but it’s a little simple. We can define any form we like using this system though (take a look at http://ish.io:8891 for examples of the types of forms.). As an example, here is a simple page editor I built to power a friends website.

- name: url
  required: True

- name: title
  required: True

- name: content
  required: True
  widget:
    type: text-area

- name: photos
  type: sequence/group

- name: photos.*.title

- name: photos.*.photo
  type: file
  widget:
    type: file-upload-couchdb
    options:
      resource_root: /filehandler
      originalurl: /images/blank.gif

Here we’ve got a page url, some content and a sequence of photos with titles. Here is the admin form for this definition (it uses jquery to create new form components for additional sequence fields).

And finally, here is how to create a relation between two couchish objects. The following is a tour.yaml file, used to store photography tours, each of which has a leader.

- name: leader
  type: tuple/(string,string,string,string)
  widget:
    type: select-choice-couchdb
    options:
      label: "%(surname)s, %(firstname)s"
      reference: leader
      keys: [firstname,surname]
      datakeys: [_id,firstname,surname,phone]

This uses creates a reference to the leader object and adds a byleader view which is retrived when the select-choice is shown. The data in the datakeys list is then cached on the tour object (CouchDB doesn’t have joins so simple cacheing of useful data is a solution to reduce the number of queries).

I hope this hasn’t been too rambling and if anyone wants to know more about couchish, please let me know and I’ll try to get it tidied up for external use.