tkbe

June 25, 2008

django :: Debugging nirvana with werkzeug

Filed under: django — tb @ 11:27 am

I’ve previously written about my delight at the Django error pages. Well, I’ve found something even better: the Werkzeug Debugger! It’s developed by the Pocoo Team, which has also given us Pygments (the python code hi-liter that seems to be used everywhere lately), Jinja (the Django style template library used by Sphinx), and several others that I haven’t had the time to investigate yet.

Why is the Werkzeug Debugger so good? It is because it seems to integrate perfectly with Django, and it gives you a command prompt right there in your browser window, giving you the freedom to investigate the program state at that stackframe. Pure technogasm material! It also gives you the entire contents of the file, with the current function and the offending line highlit. And finally, you can view just the traceback (and supposedlyu automatically paste it to the lodgeit pastebin — I didn’t get that to work).

To install simply

easy_install werkzeug

then use the 12-line script found here to run your local django server (instead of python manage.py runserver).

March 25, 2007

Django :: admin search functionality

Filed under: django — tb @ 7:59 am

In-house staff has been getting spoiled lately by the simplicity of the search box in the Django admin interface. My favorite comment from last week was "the stupid thing doesn't even search the middle name field when I enter a name, I wish it could be more like the Django", in reference to an application for which we're paying a significant amount of licencing fees. (On a side-note I would suggest to everyone to re-brand the admin pages -- my users are now convinced Django is the stuff that I've put on their admin page...)

For a number of reasons, however, I'm starting to shift out parts of the admin interface for my own home-made pages. While they've served us well so far, we've gotten to the point now that the feature requests require more custom work than can be integrated into the admin interface. It's a good thing. It means users are thinking about how things can be done better, and it's also the way the Django admin interface is supposed to be used -- allowing everyone to focus on something besides basic admin functionality until later in a project.

It would seem important though, to not take a step backward and lose functionality like e.g. the simplicity of a single search box. I was to lazy to go look at the source, but a quick googling only turned up an entry from Petro Verkhogliad (http://petro.tanreisoftware.com/?p=22) that only deals with searching a single field (and a suggestion for an algorithm to search multiple fields that materializes way to much data to be practical). The second entry I found was from Steven Ametjan (http://www2.wolfsreign.com/archives/2007/01/22/writing-search-view-django/). His solution is unfortunately buggy if there is more than one search term.

A correct version should look something like this:

PYTHON:
  1. from django.db.models import Q
  2.  
  3. def search(terms=None):
  4.     if terms is None:
  5.         return Customers.objects.all()
  6.  
  7.     query = Customer.objects
  8.     for term in terms:
  9.         query = query.filter(
  10.               Q(fname__icontains=term)
  11.             | Q(lname__icontains=term)
  12.             | Q(email__icontains=term)
  13.             | Q(zipcode__icontains=term)
  14.             | Q(birthdate=term))
  15.     return query

That works ok, but searching the birthdate field requires using database syntax to enter dates (2007-03-25). People get very upset when they can't enter dates in their local format (I can't emphasize this too much). Where I'm sitting right now, dates are always entered as dd.mm.yyyy. We can fix this problem though, and at the same time make our searches faster by utilizing some domain knowledge. In this case we know that we don't need to search in non-date fields for terms that are dates (nor for non-date data in date fields). The only numeric column we're searching is the zipcode column, so we can limit terms that are matched against this column as well, and we end up with something like this:

PYTHON:
  1. def possible_zipcode(v):
  2.     try:
  3.         int(v)
  4.         return True
  5.     except:
  6.         return False
  7.  
  8. def local_date_format(v):
  9.     if len(v) == 10 and len(v.split('.')) == 3:
  10.         try:
  11.             day, month, year = map(int, v.split('.'))
  12.             datetime.date(year, month, day)
  13.             return True
  14.         except:
  15.             pass
  16.     return False
  17.  
  18.  
  19. def search(terms=None):
  20.     if terms is None:
  21.         return Customers.objects.all()
  22.  
  23.     query = Customer.objects
  24.     for term in terms:
  25.         if possible_zipcode(term):
  26.             query = query.filter(zipcode=term)
  27.         elif local_date_format(term):
  28.             day, month, year = map(int, term.split('.'))
  29.             query = query.filter(birthdate=datetime.date(year, month, day))
  30.         elif '@' in term:
  31.             query = query.filter(email__icontains=term)
  32.         else:
  33.             query = query.filter(
  34.                   Q(fname__icontains=term)
  35.                 | Q(lname__icontains=term))
  36.     return query

For my real code running against real data this approach turned out to be an order of magnitude faster than Django's generic algorithm... (In all fairness to Django I should probably mention that the only time Django's search box isn't almost instantaneous is when I'm running the admin interface on my personal machine against our production database over a vpn connection from home ;-)

January 17, 2007

django :: media, admin and otherwise

Filed under: django — tb @ 7:03 pm

Django's handling of media files such as images, css, etc. has confused me from the very beginning. I'm not going to claim that I fully understand the full ramifications yet, but at least I got something working... hopefully it'll be helpful for someone. I'm running Django from an Apache 2.0.59 server, and I will be serving media files from the same apache server. The Django documentation suggests that a separate webserver for serving media files will boost performance significantly... which is good, since it gives me something to worry about if I run out of things to do :-)

I've checked out my copy of Django to /home/djangosrc and created a symbolic link in Python's site-packages directory to /home/djangosrc/django.

My Django server lives in /home/django and I've got media files in /home/django/media, and template files in /home/django/templates.

I created a virtual host on Apache and set it up for Django:

XML:
  1. <VirtualHost 1.2.3.4:80>
  2.    ServerName www.mydomain.com
  3.    ServerAdmin webmaster@mydomain.com
  4.    <Location "/">
  5.       SetHandler python-program
  6.       PythonHandler django.core.handlers.modpython
  7.       SetEnv DJANGO_SETTINGS_MODULE datakortet.settings
  8.       PythonPath "['/home/django'] + sys.path"
  9.       PythonDebug on
  10.    </Location>

Then I created a symbolic link from the Apache DocumentRoot directory (htdocs) to the media files for the admin interface: ln -s /home/djangosrc/django/contrib/admin/media htdocs/admin_media. In the settings.py file I put: ADMIN_MEDIA_PREFIX = '/admin_media/' I'm pretty sure this tells Django that it should look for the admin interface media files in http://www.mydomain.com/admin_media/ (the first part of the url coming from the server Django is running on). Continuing the virtual host definition to reflect this I got

XML:
  1. <Location "/admin_media">
  2.       SetHandler none
  3.    </Location>
  4.    <LocationMatch "\.(jpg|gif|png)$">
  5.       SetHandler none
  6.    </LocationMatch>

That takes care of the admin interface.

For regular media files I added a symbolic link in Apache's htdocs directory to my media directory: ln -s /home/django/media htdocs/media, then over in settings.py I added MEDIA_ROOT = '/home/django/media' and MEDIA_URL = 'http://web.datakortet.no/media', and finally I could finish off the virtual host definition with:

XML:
  1. <Location "/media">
  2.       SetHandler none
  3.    </Location>
  4. </VirtualHost

So far it seems to be working well. The next project in my queue requires uploading of image files, so I haven't tested that yet, but hopefully I'm not too far off..

Next Page »

Powered by WordPress