Printing unicode to windows terminal from scratch (2.7)

Python 2.7 is dying, but still, here is a way to print a unicode string without requiring users to edit python stdlib files or changing code pages (both being both scary and inconvenient).

I can’t imagine this being anything but excruciatingly slow, but for small use-cases it should be OK.

(oh, and it is not thread-safe, nor inter-process safe if processes are sharing a terminal)

# -*- coding: utf-8 -*-
from __future__ import print_function

import codecs
import contextlib
import os
import sys
from encodings import aliases


class UniTerm(object):
    def __init__(self):
        self.state = self.get_state()
        aliases.aliases['65001'] = 'utf_8'
        self.chcp = '65001'
        sys.stdout = codecs.getwriter('utf8')(sys.stdout)

    def get_state(self):
        return {
            'cp': self.chcp,
            'aliases': aliases.aliases.get('65001'),
            'stdout': sys.stdout
        }

    def reset(self):
        self.chcp = self.state['cp']
        if self.state['aliases'] is None:
            del aliases.aliases['65001']
        else:
            aliases.aliases['65001'] = self.state['aliases']
        sys.stdout = self.state['stdout']

    @property
    def chcp(self):
        return os.popen('chcp').read().split()[-1]

    @chcp.setter
    def chcp(self, cp):
        os.system('chcp {}>NUL'.format(cp))


def uprint(*args, **kwargs):
    if sys.platform != 'win32':
        return print(*args, **kwargs)
    t = None
    try:
        t = UniTerm()
        return print(*args, **kwargs)
    finally:
        if t is not None:
            t.reset()


@contextlib.contextmanager
def uterm():
    t = UniTerm()
    try:
        yield
    finally:
        t.reset()


if __name__ == '__main__':
    uprint(u'\u2713')  # checkmark
    with uterm():
        print(u'\u26a1')  # lightening bolt

output:

c:\> python uniterm.py

Posted in Uncategorized | Leave a comment

Pre-compiled binaries for PyCrypto 2.6.1 (py27) on Win7

There is a “new” version of PyCrypto out, and I can’t find any simple way to install it on Windows.  This is hardly news, and usually the Michael Foord has ready made windows installers over at Voidspace. To be totally honest, there probably isn’t a great need for a 2.6.1 release for windows, since the reason for the release doesn’t seem to apply:

In versions prior to v2.6.1, Crypto.Random was insecure when using fork() in some cases. See the advisory for CVE-2013-1445 for more information. It is recommended that users upgrade to PyCrypto v2.6.1 or later.  (https://www.dlitz.net/software/pycrypto/)

My use case is that I’m developing on windows and releasing on linux, and I’d like the requirements.txt to stay as much in sync as possible.

Details:

  1. I’ve downloaded and used Visual C++ 2008 Express Edition
  2. I’ve used the 2.6.x branch of PyCrypto (*)
  3. The compiled versions do not include MPIR / GMP _fastmath
  4. The .whl file is created by converting the .exe installer.

You should probably not download crypto libs from some random guy with a website, so I’ve included my build log so you can roll your own by following my steps.  I’ve got gnuwin32 installed, so you’ll see the occasional *nix command.

To set up the correct msvc environment I started from the Visual Studio 2008 Command Prompt:

msvc2k8cli

and then I created a virtualenv to make sure I started from a clean slate:

c:\tmp>virtualenv --python=c:\Python27\python.exe dev
Running virtualenv with interpreter c:\Python27\python.exe
New python executable in dev\Scripts\python.exe
Installing setuptools, pip...done.

c:\tmp>cd dev
c:\tmp\dev>Scripts\activate
(dev) c:\tmp\dev>

The rest is in the build log.

For the brave and impatient there are pre-compiled versions:

(dev) C:\work\github\pycrypto\dist>md5sum *
aa791ce84cc2713f468fcc759154f47f *pycrypto-2.6.1-cp27-none-win32.whl
1a8cec46705cc83fcd77d24b6c9d079c *pycrypto-2.6.1.win32-py2.7.exe
Posted in Python | 2 Comments

Package dk

I spent a little time getting familiar with the life-cycle steps of publishing a Python package…

  • working with code on github (https://github.com/thebjorn/dk)  Hint:  use their windows application, it makes everything very streamlined.
  • figuring out setup.py so that the package is uploaded to PyPI and others can install it with `pip install dk` (https://pypi.python.org/pypi/dk/0.7.4).   Hint: if you don’t increase the version number, nothing good happens.
  • Creating a home page using GitHub Pages (http://thebjorn.github.io/dk/).  Hint: editing it after you’ve created it is the same process as creating it for the first time (don’t worry  you’re not going to clobber everything you did the first time).
  • Extracting docstrings using autodoc into Sphinxdoc documentation and integrating it with Read the Docs (http://dk.readthedocs.org/en/latest/). Hint: if using autodoc you’ll need to check the box for building in a virtualenv.
  • Setting up a trigger so the documentation is automatically re-built every time I push to github.

Everything was pretty straight forward, and I especially like the webhook to re-build the documentation (since developers + manual steps = fail).

The remaining steps are a bit more hazy…,  the remaining steps being the running of the test suite and reporting test failures + coverage.  Perhaps tox is the answer…?

Posted in Python | 1 Comment

django :: list_display can’t sort on attribute of foreign key field…

I’m not the only one surprised by the fact that you can’t use the double-underscore-foreignkey-attribute-accessor syntax in list_display:

[python]
class MyModelOptions(admin.ModelAdmin):
list_display = [‘fk_field__fk_attribute’] # ILLEGAL
[/python]

There has been extensive discussions on the tracker (https://code.djangoproject.com/ticket/5863) and on the mailing list (http://groups.google.com/group/django-developers/browse_thread/thread/790484bfbe1b421f). It seems unlikely that this will be possible to do before hell freezes over (although someone commented that it works in Django 1.2 here: http://stackoverflow.com/questions/163823/can-list-display-in-a-django-modeladmin-display-attributes-of-foreignkey-field).

Luke Plant has suggested a solution using callables, which I personally find ugly, but YMMW:

[python]
def foreign_field(field_name):
def accessor(obj):
val = obj
for part in field_name.split(‘__’):
val = getattr(val, part)
return val
accessor.__name__ = field_name
return accessor

ff = foreign_field

class MyAdmin(ModelAdmin):

list_display = [ff(‘foreign_key__related_fieldname1’),
ff(‘foreign_key__related_fieldname2’)]
[/python]

The code from @lukeplant does all the work in the ModelAdmin, which keeps the Model class free of extra methods for the admin. This is generally a good idea, however I just had a use case where adding accessors made working with the model much easier.

The model in question looks like this:

[python]
class DailyEmployeeProjectHours(models.Model):
employee = models.ForeignKey(Employee)
empday = models.ForeignKey(EmployeeDay)
project = models.ForeignKey(Project)
seconds_worked = models.IntegerField(default=0)
[/python]

The code was littered with “deph.employee.user.username”, “deph.empday.date”, and “deph.project.name”…

We needed something that could define an accessor/property, that would automagically be sortable in Django’s admin interface… which sounds like the job for a descriptor:

[python]
class FkeyLookup(object):
def __init__(self, fkeydecl, short_description=None, admin_order_field=None):
self.fk, fkattrs = fkeydecl.split(‘__’, 1)
self.fkattrs = fkattrs.split(‘__’)

self.short_description = short_description or self.fkattrs[-1]
self.admin_order_field = admin_order_field or fkeydecl

def __get__(self, obj, klass):
if obj is None:
return self # hack required to make Django validate (if obj is None, then we’re a class, and classes are callable <wink>)

item = getattr(obj, self.fk)
for attr in self.fkattrs:
item = getattr(item, attr)
return item
[/python]

Usage:

[python]
class DailyEmployeeProjectHours(models.Model):
employee = models.ForeignKey(Employee)
username = FkeyLookup("employee__user__username")

empday = models.ForeignKey(EmployeeDay)
date = FkeyLookup("empday__date")

project = models.ForeignKey(Project)
name = FkeyLookup("project__name")

seconds_worked = models.IntegerField(default=0)
[/python]

and in admin.py::

[python]
class DailyEmployeeProjectHoursOptions(admin.ModelAdmin):
list_display = "username date name hours".split()

def hours(self, obj):
return ‘%.2f’ % round(obj.seconds_worked / 3600.0, 2)
hours.admin_order_field = ‘seconds_worked’
[/python]

The admin list for EmployeeProjectHours now has columns named “Username”, “Date”, “Name”, and “Hours” — and all of them will be sortable! (if you’re on an ancient version of Django, you’ll need to apply r9212 from Django trunk, only the changes in contrib/admin/views/main.py are needed if you’re lazy).

In addition the DailyEmployeeProjectHours class now has a number of new properties…

[python]
deph = DailyEmployeeProjectHours.objects.get(…)
assert deph.username == deph.employee.user.username
assert deph.date == deph.empday.date
assert deph.name == deph.project.name
[/python]

It’s cute and it uses descriptors… 🙂

Posted in django | Tagged | 2 Comments