If you use logging.config.fileConfig (e.g. because you use paster serve something.ini to deploy your WSGI apps) you should know about this.

By default fileConfig disables all pre-existing loggers if they (or their parent loggers) are not explicitly mentioned in your .ini file.

This can result in unintuitive behaviour:

(if you don't see the embedded example, you can find it at https://gist.github.com/1642893).

If you have Python 2.6 or later (and you should), you can turn this off by passing disable_existing_loggers=False to fileConfig(). But what if it's not you calling fileConfig() but your framework (e.g. the above-mentioned paster serve)?

Now usually paster serve tries to configure logging before importing any of your application modules, so there should be no pre-existing loggers to disable. Sometimes, however, this doesn't work for one reason or another, and you end up with your production server suppressing warnings and errors that should not be suppressed. I haven't actually figured out yet who's responsible for those early imports in the application I'm working on (until today I assumed, incorrectly, that paster imports the module containing your WSGI app before it calls fileConfig).

If you're not sure if this bug can bite you or not, check that you don't have any disabled loggers by doing something like

import logging
assert not any(getattr(logger, 'disabled', False)
               for logger in logging.getLogger().manager.loggerDict.values())

while your application is running.

I've received more feedback about my last night's post on gnarly svn to git migration than I've expected. Thanks to that feedback (and, mostly, Raffaele Salmaso for doing almost all the work and emailing the result to me) eazysvn is now on GitHub.

The rest of this post will describe the conversion (and verification) in detail, because if I ever need to do this again, I do not want to start from scratch.

eazysvn lives in a Subversion repository. I want to bring it (kicking and screaming) into the 21st century and put it on Github.

git-svn is unsuitable for the conversion, because in revision 50 I moved / to /trunk and added the traditional /tags and /branches. With git-svn I either get one third of the history that ignores everything before the layout switch, or I get directories named 'trunk', 'tags' and 'branches'.

Then I thought maybe hg would be smarter about the conversion, and then I could use hg-fast-export to convert hg to git. I enabled the hgsubversion extension:

$ sudo apt-get install hgsubversion
$ echo '[extensions'] >> ~/.hgrc
$ echo 'hgsubversion =' >> ~/.hgrc

(blog posts like this are a good reason why hg ought to steal the 'git config --global foo.bar=baz' syntax from git).

Then I converted the svn repository to Mercurial:

$ hg clone svn+ssh://fridge/home/mg/svn/eazysvn eazysvn-hg

hg log -p confirmed that hg handled the conversion nicely looked right-ish at first glance, except the author information was nonsensical. To fix that:

$ rm -rf eazysvn-hg
$ echo 'mg = Marius Gedminas <marius@gedmin.as>' > AUTHORS
$ hg clone svn+ssh://fridge/home/mg/svn/eazysvn eazysvn-hg -A AUTHORS

Unfortunately, a closer look at hg log now shows that two thirds of the history is lost: hg ignored everything before the layout restructuring, despite printing the log messages of those revisions as it went about the conversion. *sigh*.

Dear lazyweb, surely svn layout reorganization can't be such a rare thing that no tools in existence support it? What should I try next?

P.S. I also tried Bazaar, to see what it would do:

$ bzr branch svn+ssh://fridge/home/mg/svn/eazysvn eazysvn-bzr
Repository with UUID 4fc293c4-4eed-0310-a01a-b4ad72f90fad at svn+ssh://fridge/home/mg/svn/eazysvn contains fewer revisions than cache. This either means that this repository contains an out of date mirror of another repository (harmless), or that the UUID is being used for two different Subversion repositories (potential repository corruption).
bzr: ERROR: exceptions.KeyError: 'missing revision paths for 78'

Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/bzrlib/commands.py", line 946, in exception_to_return_code
    return the_callable(*args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/bzrlib/commands.py", line 1150, in run_bzr
    ret = run(*run_argv)
  File "/usr/lib/python2.7/dist-packages/bzrlib/commands.py", line 699, in run_argv_aliases
    return self.run(**all_cmd_args)
  File "/usr/lib/python2.7/dist-packages/bzrlib/commands.py", line 721, in run
    return self._operation.run_simple(*args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/bzrlib/cleanup.py", line 135, in run_simple
    self.cleanups, self.func, *args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/bzrlib/cleanup.py", line 165, in _do_with_cleanups
    result = func(*args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/bzrlib/builtins.py", line 1263, in run
    from_location)
  File "/usr/lib/python2.7/dist-packages/bzrlib/bzrdir.py", line 919, in open_tree_or_branch
    return bzrdir._get_tree_branch()
  File "/usr/lib/python2.7/dist-packages/bzrlib/controldir.py", line 410, in _get_tree_branch
    branch = self.open_branch(name=name)
  File "/usr/lib/python2.7/dist-packages/bzrlib/plugins/svn/remote.py", line 420, in open_branch
    branch_path = self._determine_relpath(name)
  File "/usr/lib/python2.7/dist-packages/bzrlib/plugins/svn/remote.py", line 369, in _determine_relpath
    layout = repos.get_layout()
  File "/usr/lib/python2.7/dist-packages/bzrlib/plugins/svn/repository.py", line 701, in get_layout
    return self.get_layout_source()[0]
  File "/usr/lib/python2.7/dist-packages/bzrlib/plugins/svn/repository.py", line 720, in get_layout_source
    self._find_guessed_layout(self.get_config())
  File "/usr/lib/python2.7/dist-packages/bzrlib/plugins/svn/repository.py", line 743, in _find_guessed_layout
    revnum, self._hinted_branch_path)
  File "/usr/lib/python2.7/dist-packages/bzrlib/plugins/svn/layout/guess.py", line 143, in repository_guess_layout
    return logwalker_guess_layout(repository._log, revnum, branch_path=branch_path)
  File "/usr/lib/python2.7/dist-packages/bzrlib/plugins/svn/layout/guess.py", line 149, in logwalker_guess_layout
    logwalker.iter_changes(None, revnum, max(0, revnum-GUESS_SAMPLE_SIZE)), revnum, branch_path)
  File "/usr/lib/python2.7/dist-packages/bzrlib/plugins/svn/layout/guess.py", line 104, in guess_layout_from_history
    for (revpaths, revnum, revprops) in changed_paths:
  File "/usr/lib/python2.7/dist-packages/bzrlib/plugins/svn/logwalker.py", line 60, in iter_all_changes
    revpaths = get_revision_paths(revnum)
  File "/usr/lib/python2.7/dist-packages/bzrlib/plugins/svn/logwalker.py", line 295, in get_revision_paths
    return self.cache.get_revision_paths(revnum)
  File "/usr/lib/python2.7/dist-packages/bzrlib/plugins/svn/cache/tdbcache.py", line 187, in get_revision_paths
    raise KeyError("missing revision paths for %d" % revnum)
KeyError: 'missing revision paths for 78'

You can report this problem to Bazaar's developers by running
    apport-bug /var/crash/bzr.1000.2012-02-19T22:12.crash
if a bug-reporting window does not automatically appear.

Last weekend I attended the N9 Hackathon in Vienna. Nokia kindly sponsored all food and accommodation costs and, at the very end, surprised me with an entirely unexpected gift of a N9 phone.

Vienna: great transportation system, delicious food (either that, or I was always extremely hungry when I ate), huge portions, restaurants open until midnight. Shame I didn't have time to see the city itself.

The N9 is a gorgeous phone; much more so in real life than in pictures.

After some hassle upgrading the Qt SDK (the provided upgrade tool managed to somehow remove the Qt Creator IDE while purporting to upgrade it; I had to reinstall the entire SDK) and flashing my N950 with Beta 2 firmware (Qt SDK 1.1.3 produces apps incompatible with the old Beta 1 firmware of the N950) I started prototyping a time tracking application in QML, while learning QML and the N9 design guidelines at the same time.

Converting the pretty pictures into QML was harder than I expected, but at the end of the second day I had something that looked like a native N9 application.

Most useful reference pages were:

• The QML syntax introduction (which felt incomplete, but was almost adequate in the end).
• The list of Harmattan-specific QML components.
• The list of standard QML components.
• The UI building blocks pages mentioned above (pretty pictures! pretty colours! I like shiny things!).
• Harmattan Qt Components User Interface Guidelines: pixel and font sizes of the standard UI elements. (Ignore the "import UIConstants.js" red herrings; it's an internal thing apparently, and you can't use it directly from your 3rd-party apps. Unless you find and copy UIConstants.js into your project, after figuring out if the licence allows it, which seemed too much of a hassle for me to even start. So I hardcoded all the numbers directly, like a bad programmer who doesn't know about constants.)
• The TutorialApplication sample, finding the sources of which was unexpectedly difficult -- a straight git clone of the qt-components repository gives you something too recent to run with the older qt-components version on the N9. I ended up using apt-get source qt-components in Scratchbox to download the source tarball of version 1.0~git20110525-1+0m6. Look in qt-components/examples/meego/.

Finally, workflow. QML is parsed at run time (application startup time, unless you delay the loading), which means no recompilation ought to be required to make changes, which means short development feedback cycles ought to be possible. So I was not happy about having to wait several seconds after hitting Run in Qt Creator, while it built a .deb, copied it to the device over wifi or USB networking, installed and ran it there. Deploying to the Qt Simulator is quicker, but not as much as I think it ought to be. Plus, the Qt Simulator apparently cannot simulate the landscape mode of the N9, and it lies about the default font size of QML Text elements (if you do not specify a pixelSize, text elements will look all right in the simulator, but ridiculously tiny on the N9).

In the end I cobbled up a shell script that rsyncs my updated QML files to the device and runs a short Python script (over ssh) to launch them. You need rsync and PySide installed on the device for this, obviously, as well as having SSH set up for passwordless logins. As a bonus, I can now do QML development during lunch, directly on the device, with vim, enjoying proper syntax highlighting. :)

Oh, and my code is up on Github.

Andrew Olmsted built the first FBReader packages for Harmattan, after tweaking the build system a bit. The desktop version of FBReader already used Qt 4, and ran almost unmodified, but with some bugs (segfault on task switch) and ugly UI.

I started with the Ubuntu packages for FBReader, since they used a more sane build system for .debs (compared to upstream's funky shell script). Some tweaks were needed to make it build in Scratchbox: since GTK+ and Hildon libraries aren't available on Harmattan, I had to disable the building of -gtk and -maemo versions of libzlui. I also got to learn a new tool — quilt.

Fixing the segfault took a couple of days of debugging, studying the source code of both FBReader and Qt itself, and asking for help on IRC. Turns out FBReader was holding an active QPainter instance for too long, and its backing pixmap got destroyed (or, rather, converted from an OpenGL texture to a plain X11 pixmap) during a task switch, causing the crash. I'm probably describing this wrong BTW, but, in any case, adding QPainter::begin() and QPainter::end() calls in the paintEvent handler fixed the segfault.

Next, a small tweak in the .desktop file to make FBReader a single-instance application: change Exec=FBReader to Exec=single-instance /usr/bin/FBReader (I'm paraphrasing slightly).

Then, a more ambitious goal: making FBReader intercept volume keys and use them for scrolling. Google gave me a pointer to QmKeys, which was the wrong API to use here, but gave me a lead to qmkeyd2, which appears to be an open source daemon, which gave me a lead to sysuid, another open source daemon, which in turn gave me a lead to libresourceqt, and that was the right API at last.

Volume keys generate regular key events for XF86AudioRaiseVolume and XF86AudioLowerVolume, but they're also intercepted by qmkeyd2, which tells all subscribers (and sysuid is one) about them. Which subscriber gets to react is determined by the resource policy framework. So what I needed to do in FBReader was acquire the ScaleButtonResource when FBReader starts up or gets focus, and release it when FBReader quits or goes into background. That also required some IRC help until I discovered installEventFilter() and the ApplicationActivate/ApplicationDeactivate events. And QApplication::instance().

The various tools available in the developer firmware were invaluable: openssh, gdb, valgrind, strace, xev, xprop, lsof, netstat. Also, I would not have achieved my second goal without being able to look at the sources of Meego system components (qmkeyd, sysuid). Yay open source!

Here are my changes to the source code. You can find my modified Debian packaging files, as well as prebuilt binary packages (with full debug info, for gdb goodness), in my experimental harmattan apt repository. The UI is still ugly and non-native, but it doesn't matter much in fullscreen mode :) .

Note to self: when next building fbreader, make sure the 2 megabyte tags file doesn't end up in the .diff.gz. And speaking of crud in source packages, the vim package I built for Harmattan the other day contains the entire 50 meg .hg in the .orig.tar.gz. I need to figure out how to tell dh_make to omit it.