Pre-compiled binaries for PyCrypto 2.6.1 (py27) on Win7

There is a “new” version of PyCrypto out, and I can’t find any simple way to install it on Windows.  This is hardly news, and usually the Michael Foord has ready made windows installers over at Voidspace. To be totally honest, there probably isn’t a great need for a 2.6.1 release for windows, since the reason for the release doesn’t seem to apply:

In versions prior to v2.6.1, Crypto.Random was insecure when using fork() in some cases. See the advisory for CVE-2013-1445 for more information. It is recommended that users upgrade to PyCrypto v2.6.1 or later.  (https://www.dlitz.net/software/pycrypto/)

My use case is that I’m developing on windows and releasing on linux, and I’d like the requirements.txt to stay as much in sync as possible.

Details:

  1. I’ve downloaded and used Visual C++ 2008 Express Edition
  2. I’ve used the 2.6.x branch of PyCrypto (*)
  3. The compiled versions do not include MPIR / GMP _fastmath
  4. The .whl file is created by converting the .exe installer.

You should probably not download crypto libs from some random guy with a website, so I’ve included my build log so you can roll your own by following my steps.  I’ve got gnuwin32 installed, so you’ll see the occasional *nix command.

To set up the correct msvc environment I started from the Visual Studio 2008 Command Prompt:

msvc2k8cli

and then I created a virtualenv to make sure I started from a clean slate:

c:\tmp>virtualenv --python=c:\Python27\python.exe dev
Running virtualenv with interpreter c:\Python27\python.exe
New python executable in dev\Scripts\python.exe
Installing setuptools, pip...done.

c:\tmp>cd dev
c:\tmp\dev>Scripts\activate
(dev) c:\tmp\dev>

The rest is in the build log.

For the brave and impatient there are pre-compiled versions:

(dev) C:\work\github\pycrypto\dist>md5sum *
aa791ce84cc2713f468fcc759154f47f *pycrypto-2.6.1-cp27-none-win32.whl
1a8cec46705cc83fcd77d24b6c9d079c *pycrypto-2.6.1.win32-py2.7.exe
Posted in Python | Leave a comment

Package dk

I spent a little time getting familiar with the life-cycle steps of publishing a Python package…

  • working with code on github (https://github.com/thebjorn/dk)  Hint:  use their windows application, it makes everything very streamlined.
  • figuring out setup.py so that the package is uploaded to PyPI and others can install it with `pip install dk` (https://pypi.python.org/pypi/dk/0.7.4).   Hint: if you don’t increase the version number, nothing good happens.
  • Creating a home page using GitHub Pages (http://thebjorn.github.io/dk/).  Hint: editing it after you’ve created it is the same process as creating it for the first time (don’t worry  you’re not going to clobber everything you did the first time).
  • Extracting docstrings using autodoc into Sphinxdoc documentation and integrating it with Read the Docs (http://dk.readthedocs.org/en/latest/). Hint: if using autodoc you’ll need to check the box for building in a virtualenv.
  • Setting up a trigger so the documentation is automatically re-built every time I push to github.

Everything was pretty straight forward, and I especially like the webhook to re-build the documentation (since developers + manual steps = fail).

The remaining steps are a bit more hazy…,  the remaining steps being the running of the test suite and reporting test failures + coverage.  Perhaps tox is the answer…?

Posted in Python | 1 Comment

django :: list_display can’t sort on attribute of foreign key field…

I’m not the only one surprised by the fact that you can’t use the double-underscore-foreignkey-attribute-accessor syntax in list_display:

class MyModelOptions(admin.ModelAdmin):
    list_display = ['fk_field__fk_attribute']  # ILLEGAL

There has been extensive discussions on the tracker (https://code.djangoproject.com/ticket/5863) and on the mailing list (http://groups.google.com/group/django-developers/browse_thread/thread/790484bfbe1b421f). It seems unlikely that this will be possible to do before hell freezes over (although someone commented that it works in Django 1.2 here: http://stackoverflow.com/questions/163823/can-list-display-in-a-django-modeladmin-display-attributes-of-foreignkey-field).

Luke Plant has suggested a solution using callables, which I personally find ugly, but YMMW:

def foreign_field(field_name):
    def accessor(obj):
        val = obj
        for part in field_name.split('__'):
            val = getattr(val, part)
        return val
    accessor.__name__ = field_name
    return accessor

ff = foreign_field

class MyAdmin(ModelAdmin):

    list_display = [ff('foreign_key__related_fieldname1'),
                    ff('foreign_key__related_fieldname2')]

The code from @lukeplant does all the work in the ModelAdmin, which keeps the Model class free of extra methods for the admin. This is generally a good idea, however I just had a use case where adding accessors made working with the model much easier.

The model in question looks like this:

class DailyEmployeeProjectHours(models.Model):
    employee = models.ForeignKey(Employee)
    empday = models.ForeignKey(EmployeeDay)
    project = models.ForeignKey(Project)
    seconds_worked = models.IntegerField(default=0)

The code was littered with “deph.employee.user.username”, “deph.empday.date”, and “deph.project.name”…

We needed something that could define an accessor/property, that would automagically be sortable in Django’s admin interface… which sounds like the job for a descriptor:

class FkeyLookup(object):
    def __init__(self, fkeydecl, short_description=None, admin_order_field=None):
        self.fk, fkattrs = fkeydecl.split('__', 1)
        self.fkattrs = fkattrs.split('__')

        self.short_description = short_description or self.fkattrs[-1]
        self.admin_order_field = admin_order_field or fkeydecl

    def __get__(self, obj, klass):
        if obj is None:
            return self  # hack required to make Django validate (if obj is None, then we're a class, and classes are callable <wink>)

        item = getattr(obj, self.fk)
        for attr in self.fkattrs:
            item = getattr(item, attr)
        return item

Usage:

class DailyEmployeeProjectHours(models.Model):
    employee = models.ForeignKey(Employee)
    username = FkeyLookup("employee__user__username")

    empday = models.ForeignKey(EmployeeDay)
    date = FkeyLookup("empday__date")

    project = models.ForeignKey(Project)
    name = FkeyLookup("project__name")

    seconds_worked = models.IntegerField(default=0)

and in admin.py::

class DailyEmployeeProjectHoursOptions(admin.ModelAdmin):
    list_display = "username date name hours".split()

    def hours(self, obj):
        return '%.2f' % round(obj.seconds_worked / 3600.0, 2)
    hours.admin_order_field = 'seconds_worked'

The admin list for EmployeeProjectHours now has columns named “Username”, “Date”, “Name”, and “Hours” — and all of them will be sortable! (if you’re on an ancient version of Django, you’ll need to apply r9212 from Django trunk, only the changes in contrib/admin/views/main.py are needed if you’re lazy).

In addition the DailyEmployeeProjectHours class now has a number of new properties…

deph = DailyEmployeeProjectHours.objects.get(...)
assert deph.username == deph.employee.user.username
assert deph.date == deph.empday.date
assert deph.name == deph.project.name

It’s cute and it uses descriptors… :-)

Posted in django | Tagged | 2 Comments

python :: Calculating the distance between two locations…

How far is it from point a to point b? There are complicated ways to answer this question, that e.g. takes into account whether you’re walking or driving etc. However, if you only need the approximate distance “as the crow flies”, some simple math is sufficient.

I’m assuming you’ve used Google’s geo-coding, or geonames.org’s postal code search, etc., and now have locations with lat and lng attributes.

The Earth isn’t a perfect sphere, among many non-spherical properties, it’s actually fatter around the equator than between the poles: The algorithm doesn’t take any of this into account, and instead uses a single value for the Earth’s radius. Assuming you’re not going very far (i.e. “halfway around the globe”, the result will probably not be too far off, YMMV of course).

Module haversine.py:

import math

def cosrad(n):
    "Return the cosine of ``n`` degrees in radians."
    return math.cos(math.radians(n))
    
def haversine((lat1, long1), (lat2, long2)):
    """Calculate the distance between two points on earth.
    """
    earth_radius = 6371  # km
    dLat = math.radians(lat2 - lat1)
    dLong = math.radians(long2 - long1)

    a = (math.sin(dLat / 2) ** 2 +
         cosrad(lat1) * cosrad(lat2) * math.sin(dLong / 2) ** 2)
    c = 2 * math.atan2(math.sqrt(a), math.sqrt(1 - a))
    d = earth_radius * c
    return d

def distance(a, b):
    "Return the distance between two points that have .lat and .lng members."
    return haversine(
        (float(a.lat), float(a.lng)),
        (float(b.lat), float(b.lng)))
Posted in Python | Leave a comment