Widgets are everywhere! It seems that every blog these days has at least a couple of widgets on their sidebar. Yesterday I realized that I had never written one, and probably more importantly, I really had no concrete idea how they worked. Using Django and a bit of Javascript, it turned out to be quite easy! I'm going to show you the basics of how to write a widget yourself, but if enough people would like the source code to mine, I'll happily open source it.

Before starting on creating a widget, we've got to make sure that it has an API of some sort. In the case of Pownce, there some Community Documentation about the various things that can be done with its API. For a simple widget, all we really need is to be able to fetch some public notes.

Our API call will be: http://api.pownce.com/1.1/public_note_lists.json

Now that we've determined where to get our data, let's parse it and do something with it.

from urllib2 import urlopen
from django.utils import simplejson
from django.template import loader, Context
from django.http import HttpResponse
def pownce_widget(request):
    api_call = "http://api.pownce.com/1.1/public_note_lists.json"
    notes = simplejson.loads(urlopen(api_call).read())['notes']
    t = loader.get_template('powncewidget/widget_content.html')
    c = Context({'notes' : notes})
    return HttpResponse(t.render(c))

Now you may be noticing that I'm not using the simpler django.shortcuts.render_to_response. That's because we're going to use the content that we've rendered here as context for another template which we'll use in a moment. Also note that if you were to do this in a production environment, using a simple urlopen is not considered good practice. See chapter 11 in Mark Pilgrim's excellent Dive Into Python for more information about what to do instead.

Now let's create a template:

<ul id="pownce_widget">
    {% for note in notes %}
        <li><a href="{{ note.permalink }}">{{ note.body|slice:":30" }}</a></li>
    {% endfor %}
</ul>

What we've done here is iterated over the public notes, and simply created an unordered list with the first 30 or less words from the note. Also, we've provided a link to Pownce for each note listed.

If browsers could make cross-domain requests, we'd be done now: you could embed a small Javascript file which would asynchronously request this page and update the DOM accordingly. However, this is not possible, so I've come up with a way to do it instead. I'm not sure if it's a best practice--it sure seems to me like it's not--but it works, which is probably more important anyway.

So let's modify our view:

from urllib2 import urlopen
from django.utils import simplejson
from django.template import loader, Context, RequestContext
from django.http import HttpResponse
from django.shortcuts import render_to_response
def pownce_widget(request):
    api_call = "http://api.pownce.com/1.1/public_note_lists"
    notes = simplejson.loads(urlopen(api_call).read())['notes']
    t = loader.get_template('powncewidget/widget_content.html')
    c = Context({'notes' : notes})
    context = {"widget_content" : t.render(c))}
    return render_to_response("powncewidget/widget.html", context, context_instance=RequestContext(request))

We've taken the rendered output from widget_content.html and used it as context for another template, widget.html, which we return as an HttpResponse to the browser. This doesn't make much sense until we've seen what that widget.html template contains. Here it is:

{% load stripwhitespace %}

var _cssNode = document.createElement('link');
_cssNode.type = 'text/css';
_cssNode.rel = 'stylesheet';
_cssNode.href = '{{ MEDIA_URL }}css/powncewidgetstyle.css';
_cssNode.media = 'screen';
document.getElementsByTagName("head")[0].appendChild(cssNode);
document.write('{{ widget_content|stripwhitespace|safe }}');

What we're doing here is dynamically creating Javascript using Django's templating system which injects your rendered HTML into the page. The first few lines add a new stylesheet to the page, and the last one writes your content to the page. But what is this stripwhitespace filter that we see? Javascript does not like multi-line string declarations such as what widget_content produces. With Django, it's easy to write a simple filter to make it all exist on one line:

from django import template
import re
inbetween = re.compile('>[ \r\n]+<')
newlines = re.compile('\r|\n')
register = template.Library()
def stripwhitespace(value):
    return newlines.sub('', inbetween.sub('><', value))
register.filter('stripwhitespace', stripwhitespace)

With that, we've pretty much finished up on creating our widget. There are lots of customization options from here: GET arguments, different API endpoints, etc. Not only that, but there's lots of room for visual customization using CSS. What we're doing is effectively generating Javascript, so anything that you'd like to do using Javascript is fair game as well.

The result after tweaking for a bit is what you see at the right under my "Widgets" links. Here is a picture in case it stops working for some reason:

If you'd like to create a pownce widget of your own by simply adding a snippet to your site, I've provided a Pownce Widget Creator for your convenience.

I think that it was fairly easy to do this using Django and Python, and if you've got any tips on best practices, please let me know so that I can code a better widget!

I was lucky enough to be able to attend PyCon last year, and hope to do so again, but some great resources are starting to show up online for those who couldn't. One example is Jacob Kaplan-Moss's talk named Becoming an Open Source Developer: Lessons from the Django Project.

Even if you've read James Bennett's excellent writeup from the conference, some audio from the event itself can always be fun to listen to. Right now that's all I could find, but there's new audio showing up from PyCon every week. The only way to stay updated is to subscribe to the PyCon 2007 Podcast RSS Feed.

P.S. Hello Django community aggregator!

1. Newforms-Admin

   Started soon after the newforms package started to become stable, the newforms-admin branch allows for a much more modular and flexible admin application. The amount of flexibility allowed in this branch extends to permissions, allowing for user-defined permissions schemes--can you say 'row-level permissions'? It's even flexible enough to require no permissions at all: I needed a quick display interface for work this past summer, and I used some of the hooks in the newforms-admin branch to disable login and permissions and simply display some data publicly.

   Also, multiple different admin interfaces are easy with newforms-admin. Group your apps into functional sets and give each functional set its own admin interface. Those who are using this branch know that once you've gone newforms-admin, you'll never go back!

2. GeoDjango

   This one flew under my radar until very recently. Their stated goal is "to make it as easy as possible to build GIS web applications and harness the power of spatially enabled data." These guys have done some really amazing things with this branch, including linking of the geographic data to cool widgets that can be displayed in the admin interface (think google maps, openlayers, and more to come as time goes on). I don't personally have a need for any of the capabilities being added in this project, but the scope and quality of what is being produced is impressive to say the least.

3. Queryset-Refactoring

   As code grows and evolves, it's difficult to stop at a certain point and say, "this code doesn't fit the bill anymore." Malcolm Tredinnick was the one who stopped and said it. Already at this stage, many many bugs have been fixed by this branch. Queryset-refactoring is not just for fixing bugs. Once it's completed, database backends will have a much easier time of customizing their generated SQL. With each change like this, Django becomes more modular.

4. Manually-Specified Intermediary Model Support

   OK I'm biased on this one. This is the patch that I've been working on lately. It's because every time I create an Django application, I start out with a ManyToManyField and end up breaking that off into a separate model. Once that's done, I have to change all of my code so that it doesn't use the convenient ManyToManyField helpers. Once this patch is complete, you'll be able to specify an intermediary model which will act as the storage for your m2m data. In this way, extra information can be attached to each relationship.

5. Fixing app_label

   We're entering into 2008 soon, and that will mean that Django has been public for over two years. Some of the core developers have been using it internally for even longer than that. So one of the things which people have started running into, is that app_labels have started clashing. There hasn't been any definitive decision on how to avoid these clashes, but there have been several promising proposals. In any case, expect this functionality to change somewhat in the next few months, so that 'ellington.search' can coexist alongside 'chicagocrime.search'. No longer will you have to worry about naming everything differently!

These are just a few of the developments that are going on which seem interesting and promising. I hope that each and every one of these developing features eventually makes it into trunk, so that everyone can benefit from the hard work that people have put into the project. Also, if you've got a feature that you're excited about, let me know through the comments.

Getting involved in an open source project is hard. I've been working with Django since early 2006 and doing so makes me want to contribute back to the project. Several times now I've made attempts to do so, but all of them have failed. Looking back it's easy to see where I, as a newcomer to the open source world, went wrong. So let's start out with what not to do, by example. After that I'll give some suggestions on what I think is the right way to go about it.

I've just relaunched the redesign of this site. Nothing major is different, but one fun thing that I'd like to highlight is the way the blogroll is displayed. I got the idea from Motel de Moka, which has more links to work with than I do.

First, I set up a very simple model:

class BlogRollLink(models.Model):
    name = models.CharField(max_length=128)
    date_added = models.DateTimeField(default=datetime.now)
    url = models.URLField()

But we've got a problem at this point: we can't order this by the number of characters in the name. So we must modify our model to have an integer called name_size, and then override BlogRollLink's save function to fill in that field any time the model is saved. Our final model is below:

class BlogRollLink(models.Model):
    name = models.CharField(max_length=128)
    name_size = models.IntegerField(editable=False)
    date_added = models.DateTimeField(default=datetime.now)
    url = models.URLField()

    def save(self):
        self.name_size = len(self.name)
        super(BlogRollLink, self).save()

    def __unicode__(self):
        return self.name

    def get_absolute_url(self):
        return self.url

    class Admin:
        pass

Now to display this on every page, I've created a context processor named blogroll_processor. It looks like this:

def blogroll_processor(request):
    blogrolls = cache.get('blogrolls', None)
    if blogrolls == None:
        blogrolls = list(BlogRollLink.objects.all().order_by('name_size'))
        cache.set('blogrolls', blogrolls)
    return {'blogrolls' : blogrolls}

And we're done! A nifty "waterfall" of blogs. Let me know what you think about this technique. Is it stupid?