Posts Tagged ‘Programming’

Django, AJAX, Scriptaculous

January 7, 2008

I looked a little at Scriptaculous this weekend.  I like it–it seems easy to use, and nice and concise.  It’s more focussed on glitzy effects (and I mean that in a good way) and less on widgets than the much larger DOJO; it also struck me (quite possibly wrongly, I’m just at the poking around stage) as easier to figure out and use.  Both effects and AJAX requests are wrapped pretty nicely; you have to write very little code to use them.

Unlike DOJO it does pollute the global js namespace a bit (as mentioned here); that might be annoying for people planning to use more of their own js than I’m ever likely to.

Here‘s what appears to be a nice scriptaculous-in-django tutorial, not that I’ve gone through it in any detail.

There seems to have been much debate in Django circles about whether to “include AJAX” in Django, which would (I assume) mean bundling a toolkit and providing a set of tags that wrap it, a la Rails and Scriptaculous.  The Django developers are reluctant to do that, not wanting to commit to a toolkit, and pointing out that there’s really not that much to wrap.  Now that I at least know what they’re talking about, I can sorta see their point.  Which doesn’t mean a some rails-like toolkit-wrapping tags wouldn’t be nice.
I’m working (in a slow and desultory fashion) on something where I might actually make some use of this stuff; so maybe I’ll be able to form an informed opinion in a few weeks.


Django and AJAX

January 3, 2008

The other day I fiddled a tiny bit with AJAX in Django (is that a nerdly way to spend New Year’s Day, or what?). I have no use at all for it now, but (i) I might someday, and (ii) it’s cool.

Brief summary, for n00bs like me. The “usual” way a browser communicates with a server is to send HttpRequests, to which the server responds with some response, generally a whole page of HTML which the browser loads. Loading a whole page, though, is inefficient if the browser only wants to change its existing page a little, and doesn’t need much new data to do it. So the point of AJAX is to ask only for the new data, and update the page in place. Also, the browser doesn’t need to wait for the data to come back. Instead, it can supply a callback function to do the updating whenever the data is available.

For a better example than mine of how this works in Django, see this.

It turns out to be a lot simpler than I had expected–I think I’ve spent more time writing about it than actually doing it. By using a javascript toolkit you can avoid the more disgusting details. I used Dojo, although really for my quick test it didn’t matter at all; I just needed the “xhrGET” function, an equivalent of which would be in any toolkit.

[What, by the way, should one look for in a js toolkit? I have very little idea.]

As an exercise, I rewrote the search results page from my music website, so that the search results would be updated in place in the “onchange” method of the search box. That’s a pretty silly use of AJAX, actually, and I won’t be putting it in the real site, but it served as practice.

So what to do? The existing page is is rendered by a search_results view, which finds all the objects (of models Work, Author, and Collection) that match the search criteria (passed in GET data), and renders them to a template. To AJAXify that, I added an onchange handler to the search input, which sends and XML request to retrieve the new search results, supplying a callback to update the page. The URL those requests go to is just the existing search page’s URL, with “xhr” added to the GET data. I could have created a new URL/view for that, but the existing one already did almost exactly what this needs; see below. In a more typical case, I think the same strategy would also work, but more logic in the view would be dependent on the type of the request.

I had to do very little to the view. The AJAX request is expected to return just the “guts” of the page–the lists of matches–which can be done just by using a different template.

It may be more typical, and in some ways is preferable, for AJAX requests to return not html but data in json format. In this case, that would require javascript code to attach the data to the html, which would duplicate the existing template code in a nasty way. Maybe there is some clever way to avoid that code duplication…

Anyway, here’s the code:

def search_results(request):
    works = []
    people = []
    collections = []
    searchwords = ""
        searchwords = request.GET['searchwords']
        works = utils.searchfields(Work, searchwords)
        people = utils.searchfields(Author, searchwords)
        collections = utils.searchfields(Collection, searchwords)
    except KeyError:

    if request.has_key('xhr'):
        templatename = 'music/searchresults.html'
        templatename = 'music/search.html'

    return render_to_response(
            { 'searchwords' : searchwords,
              'works' : works,
              'people' : people,
              'collections' : collections })

The template needed more work. First, of course, it needs a js function to respond to fire off the AJAX request. That I cribbed from the DOJO documentation. Then, to avoid code duplication, I refactored the page a bit. I abstracted the html code that displays the search results out into its own tag, used in the view function above, and put the tag inside a new div. The tag is just a simple inclusion/context tag, using the same template as the view above used for xhr requests. On a normal HTTP request the tag is rendered using the data passed in from the view function; on an AJAX/xhr request the view function generates exactly the same html as the tag does initially (if given the same searchwords), and the callback shoves it into the div. And that’s pretty much it. The code:

{% extends "music/base.html" %}
{% block scripts %}
<script type="text/javascript"
<script type="text/javascript">
    function updateajax() {
      dojo.xhrGet( {
        url: "?searchwords=" + dojo.byId("searchwords").value + ";xhr",
        handleAs: "text",

        timeout: 5000, // Time in milliseconds

        load: function(response, ioArgs) {
          dojo.byId("searchresults").innerHTML = response;
          return response;

        error: function(response, ioArgs) {
          console.error("HTTP status code: ", ioArgs.xhr.status);
          return response;
{% endblock %}

{% block content_search %}
{% simple_search '/music/search/' searchwords %}
{% endblock %}

{% block content %}
<h2>Search Results</h2>
    {% searchresults %}
<script type="text/javascript">
    dojo.byId("searchwords").onchange = updateajax
{% endblock %}

Some thoughts on What I Learned From This to follow.

Python and the clipboard

December 31, 2007

I liked the idea (found via Alex) of modifying clipboard data in place, and decided to steal it for my own use, in python, on win32 and ubuntu. Since it was surprisingly annoying to figure out how to use the clipboard in either case, I’ll post what I ended up with, so that I can find it again myself.

This works with gtk or win32; the latter requires pywin32:

#! /usr/bin/env python2.5

from __future__ import with_statement
from contextlib import contextmanager

    import win32clipboard as wcb
    import win32con

    def WinClipboard():
        A context manager for using the windows clipboard safely.

    def getcbtext():
        with WinClipboard():
            return wcb.GetClipboardData(win32con.CF_TEXT)

    def setcbtext(text):
        with WinClipboard():

except ImportError, e:
    # try gtk.  If that doesn't work, just let the exception go
    import gtk

    def getcbtext():
        return gtk.Clipboard().wait_for_text()

    def setcbtext(text):
        cb = gtk.Clipboard()

def replaceclipboard(fn):
    Modify text on the clipboard.

    fn: a callable object that maps strings to strings.

    >>> setcbtext("This is some text.")
    >>> replaceclipboard(lambda s : s.upper())
    >>> getcbtext()
    text = getcbtext()
    newtext = fn(text)

def _test():
    import doctest

if __name__ == '__main__':

Django serialization

December 29, 2007

I figured out the django serialization problem I mentioned in the previous post, more or less. With unicode, json serialization needs a utf8-aware stream. XML serialization, oddly, needs to NOT have one (that part I don’t get). The easy workaround is to wrap the output stream in a StreamWriter. Here’s the relevant part of my dump script:

def dump_data(strm, format='json', indent=None):
    Output the current contents of the database as a fixture of the given format
    from django.db.models import get_apps, get_models
    from django.core import serializers

    app_list = get_apps()

    # Check that the serialization format exists; this is a shortcut to
    # avoid collating all the objects and _then_ failing.
    except KeyError:
        sys.stderr.write("Unknown serialization format: %s\n" % format)

    objects = (o for app in app_list \
                    for model in get_models(app) \
                        for o in model.objects.all())
        # Bah!
        # json barfs without a codec
        # xml barfs WITH a codec
        if format == 'json':
            strm = codecs.getwriter('utf-8')(strm)
        serializers.serialize(format, objects,
                stream = strm,
                indent=indent, ensure_ascii=False)
    except Exception, e:
        sys.stderr.write("Unable to serialize database: %s\n" % e)

Upgrading django

December 28, 2007

I just upgraded my website to the development version of django. It has a bunch of new little features I’d like to use; and the next time I change the database I want to try django-evolution, which doesn’t work on 0.96.

It was easier than I had feared. For the website itself, I didn’t run into anything unexpected, just these, all documented:

  • In all the models I changed __str__ to __unicode__; that was a simple substitution.
  • Also in all the models’ fields maxlength went to max_length.
  • I had to add a call to smart_string to my csv view.
  • I’ve found one place I that needed protection from html escaping. The problem was a string containing   in a python file; I wrapped it with mark_safe there, rather than adding a more general but less safe fix to the tag that eventually uses it.

I have a couple of utility scripts that were more of a problem. One needed to be rewritten because of the massive changes to django’s module; the new version is a bit cleaner than the old, so that pleases me. The other is a problem with json serialization, which (at least as I’m (mis?)using it) now seems to have trouble with non-ascii unicode strings. But that I don’t really need (and if I did, xml seems to work fine), so figuring it out isn’t a high priority.

The big change here was obviously unicode. I see the advantage of that in the abstract, but I do have to wonder if it is really a win in my particular case. I have non-ascii (and non-latin1) data, but utf8-encoded strings worked just fine for me. Probably though that’s a reflection of the triviality of the text-processing I do…

UPDATE: Just found another gotcha: urllib.quote doesn’t like non-ascii unicode. Fixed with smart_str.

UPDATE 2: I should RTFM; the correct fix for the first update is to use django’s own url functions. Also, I forgot to mention one thing I had to do originally: fix a typo in a template! 0.96 let this get by:

	{% extends "foo.html" }

Syntax coloring

December 26, 2007

In case anyone wants to know: I did the syntax coloring in the previous post using Pygments.

Some django stuff

December 26, 2007

I generally love Django, but it does have some real holes. The worst problem for me is its complete lack of support for database changes. The Django devolppers argue, maybe convincingly, that database evolution is delicate enough that it’s dangerous to automate. But I feel like it’s dangerous enough it shouldn’t all be left to me either. I’m not particularly scared of SQL per se, but I know I could easily get something wrong. Do Rails’ schema evolution things make that easier, or at least less scary?

I just added “created” and “modified” date fields to one of my models, both using Django’s automated date fields, that set themselves to the current date when an object is added/modified. That led to an extra complication: I wanted the real dates for all my already-existing data. Fortunately Django’s admin keeps a log from which all the modification dates can be extracted. Once gotten, they can’t be saved to the database through Django’s normal model inerface–as that would be a modification that would auto-set the modification date, exactly what I don’t want to do.  So that required a bit more SQL.

OK, despite my whingeing the SQL for that was pretty trivial.  I’ll suck it up now. The interesting bit was the python function for extracting dates from logs:


def getdate(obj, addition):
    Look at the logs to find creation/modfication dates.
    model = type(obj)
    # we need the contenttype
    ct = ContentType.objects.get_for_model(model)
    # now we can look up the logs for this object
    logs = LogEntry.objects.filter(content_type=ct,
    dates = [ for l in logs \
                if not addition or l.is_addition() ]
        return dates[0]
    except IndexError:
        return None