tkbe

November 24, 2006

Django :: i18n

Filed under: django — tb @ 3:25 am

What's up with so many languages? E.g. in Switzerland there are four official languages, and in Norway there are two: bokmål and nynorsk (details). I'll be describing what I had to do to get a good user experience with these two. The reason I care is because government contracts usually have a hard requirement that you support all official languages.

My first problem was to figure out which language codes I should use, you know like en-us for English as spoken in the US. After consulting five different browsers on two platforms (I need a Mac!) I ended up with no for bokmål and nn for nynorsk as the only sane choices.

The Django documentation does a great job of describing the process of marking up your code and templates. Basically, wherever you've got a string that'll be presented to the user, change it from

PLAIN TEXT
PYTHON:
  1. v = 'Some string'

to

PLAIN TEXT
PYTHON:
  1. v = _('Some string')

and in your templates use either of:

PLAIN TEXT
HTML:
  1. {% trans "Some string" %}
  2. {% blocktrans %}
  3. Some string
  4. {% endblocktrans %}

Then you'll have to find a linux machine -- if you install the right packages this can be done on Windows too, but it was just easier to turn to my left and use the other keyboard ;-)

Put site-packages/django/bin on your path if you haven't allready done so, that's where make-messages.py and compile-messages.py live.

Go into your project directory and issue the following commands:

mkdir locale
make-messages.py -l no
make-messages.py -l nn
make-messages.py -l en

why did I add en (for English) as an option? Well, turns out that things go sour if you try to pass unicode outside the ascii range to _(). So

PLAIN TEXT
PYTHON:
  1. v = _(u'Født')  # doesn't work
  2. v = _('Fodt')  # will work, then put Født as a translation.
  3. v = _('Date of birth')  # works and provides English support.

the make-messages commands above created django.po files somewhere under the locale directory. Edit them with emacs:

find locale -name "*.po" -print | xargs emacs

since Emacs has a po-mode that makes this very easy (or send them off to get translated, which is even easier). When you're finished translating, compile your translations:

compile-messages.py

if you want to update your .po files use make-messages.py -a before compile-messages.py. Now, the site will display Norwegian bokmål to users that have set this in their browser and English to users that have set en as their language. It didn't show nynorsk to users with nn set as their language though... Turns out Django doesn't allow you to use a language unless it's part of Django's set of core languages (found in django/conf/locale/...). Sigh. A quick copy of django/conf/locale/no/* to django/conf/locale/nn and all was fine. (I'll submit nynorsk translations when I get them back from our translator ;-) ).

I'm not sure whether it was necessary to change the LANGUAGES in settings.py, but I did:

PLAIN TEXT
PYTHON:
  1. LANGUAGES = ( # IMO, these shouldn't be translated
  2.     ('no', 'Norsk'),
  3.     ('nn', 'Nynorsk'),
  4.     ('en', 'English'),
  5.     )

Now for the final touch... While all this automatic translation is cool, it assumes that users have set up their browser correctly -- yeah, right! What we need is a little 'gadget' listing the languages available, so that you can click on what you want. Turns out this is pretty easy with Django, although perhaps a little spread out.

First, in your base template, add:

{{ lang_switch }}

then in urls.py add:

PLAIN TEXT
PYTHON:
  1. (r'^i18n/', include('django.conf.urls.i18n')),

then define a function to create the necessary html:

PLAIN TEXT
PYTHON:
  1. def switch_language(lang):
  2.     languages = (
  3.         ('no', u'Bokmål'.encode('utf-8')), # django doesn't always like unicode..
  4.         ('nn', 'Nynorsk'),
  5.         ('en', 'English'),
  6.         )
  7.     res = []
  8.     for lc, lname in languages:
  9.         if lang == lc:
  10.             res.append(lname) # no sense chosing what you're using
  11.         else:
  12.             t = '<a href="/i18n/setlang/?language=%s">%s</a>' % (lc, lname)
  13.             res.append(t)
  14.  
  15.     return ' | '.join(res)

and use it in your context:

PLAIN TEXT
PYTHON:
  1. return render_to_response(..., { "lang_switch" : switch_language(request.LANGUAGE_CODE) })

and that's it.

The magic happens in the /i18n/setlang/ view, where Django sets the language in either a cookie or the current session (depending on what's available). It then returns to the current page (based on the Referer header).

November 10, 2006

Python :: flatten

Filed under: python — tb @ 10:52 am

Although it can seem like a CS excercise, everyone will sooner or later have to write a "flatten" function -- a function that takes a nested "list" and makes it one-dimensional:

PLAIN TEXT
PYTHON:
  1. >>> flatten([1,2, [3,4], 5])
  2. [1, 2, 3, 4, 5]

it should also handle Python datatypes in a reasonable manner:

PLAIN TEXT
PYTHON:
  1. >>> flatten(['hello', 'world'])
  2. ['hello', 'world']
  3. >>> flatten(i**2 for i in range(10))
  4. [0, 1, 4, 9, 16, 25, 36, 49, 64, 81]

The trick to the implementation is to realize that everything that is iterable should be flattened... except strings and unicode. To test if something is iterable, call iter on it and be prepared to catch a TypeError: iterator over non-sequence. To test if something is string or unicode use isinstance(item, basestring). That results in something like this:

PLAIN TEXT
PYTHON:
  1. def flatten(lst):
  2.     def _flatten(lst, res):
  3.         for item in lst:
  4.             if isinstance(item, basestring):
  5.                 res.append(item)
  6.             else:
  7.                 try:
  8.                     _flatten(iter(item), res)
  9.                 except TypeError: # iterator over nonsequence
  10.                     res.append(item)
  11.         return res
  12.     return _flatten(lst, [])

That generates the entire list in the res variable of the inner function as it's recursing through the datastructure. That could be a very long list, so perhaps a generator works better?:

PLAIN TEXT
PYTHON:
  1. def flatten(lst):
  2.     for item in lst:
  3.         if isinstance(item, basestring):
  4.             yield item
  5.         else:
  6.             try:
  7.                 flatten(iter(item))
  8.             except TypeError: # iterator over nonsequence
  9.                 yield item
  10.     return

That looks pretty good :-)

So what was the reason I had to write flatten this time?...

PLAIN TEXT
PYTHON:
  1. >>> from html import table, tr, td
  2. >>> td_num = html.mktag('td', width="20%", style="background:aqua")
  3. >>> table(tr(td_num(x), td(x**2)) for x in range(6))

Which gives

0 0
1 1
2 4
3 9
4 16
5 25

the __str__ for all the classes is only defined for the base class, which needed to flatten its contents...

November 8, 2006

Django :: making a date dropdown field

Filed under: django — tb @ 3:30 am

It seems like it is hard to input valid dates in a free-form text format for a significant portion of users. Personally, I don't like many of the popup calendar widgets since they usually rely on Javascript for rendering -- making your form useless to people that don't use Javascript in their browser. (To Javascript, or not to Javascript is a discussion for another time, but I'm quite sure Javascript shouldn't be used for pretty graphic effects.) Quite a number of the Javascript calendar widgets I've seen only allow you to skip backwards one month at a time, and that gets really tedious if you're as old as me and trying to find your birthday ;-)

An easy alternative is the traditional dropdown lists for months, days, and years. One way to implement that in Django is of course to have a separate SelectField for months, days, years and leave it to the template code to lay them out sensibly. I don't like that solution for several reasons: it's too much complexity at the wrong abstraction layer (I want to deal with a date-widget as one unit in my template code), formatting according to local customs becomes harder, and it will end up sprinkling conversion code all over your app if you're not careful. It turns out that it's not that hard to create a custom widget that does it all for you automatically...

Note: This code is minimalist in many ways. E.g. using localized month names, localized day/month/year ordering, using empty defaults, etc. is left as an excercise for the reader.

First lets define a convenience function for creating a dropdown list (we'll be responsible for the raw html in our widget, so it's probably not a good idea to use an existing Django Field -- I haven't tried that though, so I might be wrong):

PLAIN TEXT
PYTHON:
  1. def select(options, selected, name):
  2.     selected = str(selected)
  3.     options = [(str(v),str(v)) for v in options]
  4.     res = ['<select name="%s">' % name]
  5.     for k,v in options:
  6.         tmp = '<option '
  7.         if k == selected:
  8.             tmp += 'selected '
  9.         tmp += 'value="%s">%s</option>' % (k,v)
  10.         res.append(tmp)
  11.     res.append('</select>')
  12.     return '\n'.join(res)

Now for the DropDown (DD) DateField (it's about 35 LOC without all the explanatory comments):

PLAIN TEXT
PYTHON:
  1. class DDDateField(forms.FormField):
  2.     # you might want to add a year range to this...
  3.     def __init__(self, field_name='', is_required=False):
  4.         self.field_name = field_name
  5.         self.validator_list = [self.validator]
  6.         self.is_required = is_required
  7.  
  8.         # for convenience (we'll insert html select tags into the
  9.         # form to hold year/month/day data, and we'll name
  10.         # them as follows):
  11.         self.dayname = field_name + '_day'
  12.         self.monthname = field_name + '_month'
  13.         self.yearname = field_name + '_year'       
  14.  
  15.     # extract_data is called with a copy of the POST data.
  16.     # Remember that all the fields will be empty the first time
  17.     # around and that POST data is always text (and so should
  18.     # any default values that you set be).
  19.     # The return value from this method is later passed on to
  20.     # the render method (after the user has had an opportunity
  21.     # to mess with it).
  22.     # NOTE: if anything throws here, your widget won't show up.
  23.     def extract_data(self, post):
  24.         name = self.field_name
  25.         day = post.get(self.dayname, '1')
  26.         month = post.get(self.monthname, '1')
  27.         year = post.get(self.yearname, '1970')
  28.         value = post.get(name, '%s-%s-%s' % (year, month, day))
  29.         return dict(day=day, month=month, year=year, value=value)
  30.  
  31.     # now for the actual rendering. We're getting data from extract_data (above).
  32.     def render(self, data):
  33.         name = self.field_name
  34.         vals = (self.get_id(), name, forms.escape(data['value']))
  35.  
  36.         # put a hidden input tag here fun fun (we'll have to overwrite this value
  37.         # later..)
  38.         h = '<input type=hidden id="%s" name="%s" value="%s" />' % vals
  39.  
  40.         # using the convenience function from above
  41.         day = select(range(1,32), data['day'], self.dayname)
  42.  
  43.         # I haven't checked if it's possible to get localized month names from
  44.         # the calendar module, but if it is, they could go here:
  45.         month = select(range(1,13), data['month'], self.monthname)
  46.         year = select(range(1920, 2007), data['year'], self.yearname)
  47.         slash = '&nbsp;/&nbsp;'
  48.  
  49.         # use US ordering
  50.         widget = h + str(month) + slash + str(day) + slash + str(year)
  51.  
  52.         return widget
  53.  
  54.     # Unfortunately there is no way for the hidden field to get updated
  55.     # on a successful submission since render isn't called. The validator
  56.     # is called at this point though, and it has access to the entire set
  57.     # of form data. We're going to hijack it to tie up our lose ends
  58.     def validator(self, field_data, alldata):
  59.         # get the values from the subwidgets
  60.         y = alldata[self.yearname]
  61.         m = alldata[self.monthname]
  62.         d = alldata[self.dayname]
  63.  
  64.         # set the fieldname to the combined value
  65.         val = '%s-%s-%s' % (y,m,d)
  66.         alldata[self.field_name] = val
  67.  
  68.         # delete all the gunk we put in the form
  69.         del alldata[self.yearname], alldata[self.monthname], alldata[self.dayname]
  70.  
  71.         # there is no way for a user to enter an invalid date, so just
  72.         # return True.
  73.         return True
  74.    
  75.     # now all that's left for html2python is to extract data from above
  76.     # and return a Python object.
  77.     @staticmethod
  78.     def html2python(data):
  79.         # we're getting the data from the validator...
  80.         y, m, d = map(int, data.split('-'))
  81.         return datetime.date(y,m,d)

That's it, it can be used as any other field.

— Next Page »

Powered by WordPress