Seventh System Effect

It's somewhat official now, so I guess I have to announce it: Divmod is doing a massive refactoring of our application, starting with the database. Work began a week ago in my Quotient sandbox, and has been continuing around the clock since then.

Progress is now visible in a more public location:

the unified Divmod SVN repository at http://divmod.org/svn/Divmod/trunk/

I would have said "rewrite" rather than "refactor", but of course Everybody knows that's stupid. Plus, we are mostly migrating our old code base and cleaning it up along the way; the only component getting a complete rewrite is Atop - the rewrite of which is so fundamentally better that we came up with a better name: "Axiom".

To clarify the naming situation: the new Divmod repository has Axiom and a new version of Mantissa, Python package name 'xmantissa' to avoid package name conflicts during the transition period. eventually it will contain a package for Quotient, Sigma, and several other things. During the transition the Python package names will all start with 'x' but I will still refer to them by their project names, since the older projects will go away and the module names will eventually change.

Initially I was very concerned when we began the experiment that lead to this code revolution. It began, as bad ideas are wont to do, as a joke. I mentioned some client work that we are doing to JP, (who had already been rewriting some things) and my difficulty in choosing an appropriate persistence solution given some of the maintenance issues we'd been having with Atop.

Exasperated after an hour of discussion, JP said, "Why don't you just use SQLite". Now, I'd looked into it some time ago and (the ostensible punchline of the joke) it was garbage. However, that was SQLite version 1, not SQLite3, which has a different API and several critical features that made it considerably more appropriate to our tasks.

A few hours later I had a working prototype of maybe half of the functionality from our existing database. I was suitably impressed; SQLite was giving us all the benefits of SQL (ad-hoc queries and indexing, relational operations on data, a fast query engine) without any of the drawbacks (difficult to customize, unportable server, fragile and time-consuming deployment). I realized that we had got something radical on multiple levels.

In the past I've been very conservative about telling people when and whether to use Divmod's open-source released software, With this new system, I say: jump in. Use it. Only a week into implementation it might be a bit premature to launch a production system with it just yet, but indications are very positive that we will be able to do just that within the month.

The code is shorter, clearer to read, easier to maintain, and the database is Pickle Free℠.

(Okay, it's not really a service mark, but it should be. Pickle is the winner for causing problems for us.)

We are building from our experience with 5 previous persistence systems, 3 previous plug-in frameworks, and 4 previous authentication databases. A curious side-effect of all that experience - and the effect that the title is referring to - is that certain development methodology concepts become irrelevant. Most notably, "YAGNI", from XP, is no longer of any use: we know exactly how much extensibility we need. At every point in implementing this system we have known whether to fuse a component together because we'd built unnecessary additional complexity into previous systems, and where to use a plug-in architecture because we'd needed to inject ugly code into the middle of a monolithic routine.

As a result, where our architecture was heavily monolithic before, now it is almost entirely composed of plugins. It is so plugin-happy, in fact, that there is a database with Service plugins in it, which activate when the database is started from twistd; it contains its own configuration, including port-numbers, so nothing need live in a text configuration file. The web application system is built around this as well; so there is a plugin lookup for invoking raw nevow IResource implementors without sessions (for example, for XML APIs), then IResource implementors which do require a session, then IResource implementors which are specific to a particular user, and finally Fragment instances which plug into a generic hierarchical navigation system. At each level there is a distinct and clear place to put new plugin code, and large portions of it are self-similar. For example, you can install the hierarchical navigation both onto a public site and onto a user's private application pages, since the "web server" implementation is an IResource which can be installed either onto a toplevel database, or any user's personal database.

Oh, and did I mention - LivePage support built right in?

The net result of this is that you can build themeable, multi-user web applications with the code that's in SVN right now. The example isn't visually appealing, but the code is nice, and it's composed from a stack of plugins.

I'm very excited about the possibilities of what we'll be transforming our system and our application into within the next few weeks. I'd like to invite everyone who has been interested in Divmod's open source work in the past to have a look at the new repository, and consider coming to #divmod on irc.freenode.net to look for something to hack on. Considering the higher-level and easier-to-understand nature of the implementation of Axiom vs. Atop, I would also love it if we could find some people to help us document it right from the start.

So - anybody out there looking for an open source project's website to maintain?

Here we go...

Nothing interesting yet, but I figure you all should know about it.

http://divmod.com/users/glyph/blog/

Escape!

The last release was in may 2004.



Go get it before I change my mind.

I am from Typographic Circumstances (Review of Belkin MediaPilot)

The last keyboard I bought was a last-minute purchase, almost entirely at random. The keyboard that I really had my eyes on at the time was the Belkin
MediaPilot, which seems a rather transparent competitive answer to the DiNovo. Wandering through Microcenter this weekend, I saw a copy; and since I was still well within the "no questions asked" return period with my Eclipse, I purchased one to see how it would stack up.

See the "mood" header to see how it did.

Cons

The MediaPilot is a mediocre execution of a really great idea. On the surface this keyboard looks great: slimline form-factor; integrated pointing device; a few extended function keys without going insane; detachable "small" wireless keyboard while still providing full 104-key functionality when docked; dock to avoid batteries running out on you at an awkward moment.

Much as the Eclipse was a terrific keyboard that didn't look terribly good at first blush, this is a keyboard that looks terrific but somehow just doesn't measure up.

First of all, the key switches are absolutely the lowest quality that you can buy anywhere. Straight out of the box, this keyboard has keys that stick or randomly get double-typed. The keyboard actually shipped with the "0" key on the numeric keypad detached; I fixed it myself. The keyboard's plastic is so chintzy that it deformed before I opened the box; this means that when it's in "docked" mode, the keyboard has a tendency to rock diagonally, slightly.

While they haven't committed any unforgivable sins, there are still some serious problems with the layout of this keyboard. The right-most row of keys on the mini-board, for convenience, I suppose, is "Home, PgUp, PgDn, End, Right". This means that if I feel my way along the right edge of the keyboard to hit a key such as backslash or enter, I am always over one key too far. Because these keys are already provided on the mini-board, the dock-board provides "Insert/Sleep/ScrollLock/Delete/PrintScreen/PauseBreak" in the space where the 6-key set of Insert/Home/PgUp/Delete/End/PgDn is normally located,

Pros

It's not all bad, though. It is a slimline keyboard, it is relatively comfortable to type on; the wireless portion of the keyboard is extremely light and easy to toss around; the integrated pointing device also contains an integrated scroll wheel which is really handy. Pretty much all of the extended function keys work with Linux (although some only show up in 'showkey' and not in 'xev', so there are a few which would be impossible to bind to an X event out of the box).

Also, the "universal remote" functionality is very very cool if this is going to be hooked up to a computer hooked to a television set.

Conclusion

If I were going to buy a keyboard for a computer attached to a really nice plasma television, where I would frequently want to grab the keyboard and sit back on the couch to watch movies on it, but would sometimes want to sit closer-up to work on code, this keyboard would be ideal. In such a situation its sins would be forgivable and its features would be fantastic. However, in a normal office environment its warts are horrible and the fact that part of it is detachable does almost no good; in trying to experiment with different positions I could type with it in, I discovered that a major reason I like a wireless keyboard in an office is to move the keyboard aside to clear the desk real estate for some other activity, and the MediaPilot's wireless feature doesn't help with this at all.

I gave this keyboard a last chance by typing up this review on it, but my conclusion stands: as it is, though, I'll be returning this keyboard and sticking with the Saitek Eclipse.

Twisted.Child #1: Congratulations to Luc and Noa

... and I checked, I am not legally liable for this because it did not occur within the United States, even if it escalates Twisted to the literal status of a global cult:



His name is Ido Stepniewski, born 30 June 2005

Congratulations, and thank you for honoring the whole Twisted project by decorating your new child thusly.

(I just have to say it: doesn't this remind anyone of a certain comic?)

Exaggeration

So I mentioned typing at 140 words per minute in my keyboard post; this is apparently a pretty surprising speed. I am not the fastest typist I know; I do know there are people out there who can do 200+WPM on a good day, with 0 errors.

However, in my case, I'm talking about raw speed, errors and all, and I can't always get 140; if I'm typing at a comfortable pace it's a lot closer to 100. However, I figured I'd hop into gtypist to just provide some documentary evidence that 140(raw) is not such an insane number:



I still practice typing on a regular basis. Mainly I'm focused on reducing my error rate right now.

wacom tablet boot autodetection

This is not a tutorial for using wacom devices, just a fix for a particular problem. Maybe somebody else will run with it; the problem is now resolved to my satisfaction. It has only been tested on Ubuntu version 5.04.

I'm not liable for what you do with it. In fact, don't use it. It will harm you.

"Note that if I can get you to 'su and say' something just by asking, you have a very serious security problem on your system and you should look into it."
    - Paul Vixie, in the vixie-cron 3.0.1 installation notes

Any USB Wacom tablet owners out there running Linux? If you are, you're probably frustrated that plugging in any new devices will destabilize your tablet's event device location and make it impossible to reboot without editing your XF86Config-4/xorg.conf upon every boot.

Run this as part of your boot process, before X starts.

#!/usr/bin/python2.4

import os

def findtablet():
    data = file('/proc/bus/input/devices').read().split("\n\n")

    for d in data:
        if 'Wacom' in d:
            lines = d.split("\n")
            for line in lines:
                pdata = line.split('Handlers=')
                if len(pdata) == 2:
                    for pdev in pdata[1].split():
                        if pdev.startswith('event'):
                            return '/dev/input/'+pdev


device = findtablet()
os.symlink(device, '/dev/wacom')
print 'Tablet present at /dev/wacom =>', device

Then, rather than guessing at what event device the gizmo should be every time, just put something like this into your xorg.conf

Section "InputDevice"
        Identifier "stylus" # Wacom Graphics Tablet
        Driver "wacom"
        Option "Type" "stylus"
        Option "Device" "/dev/wacom"
EndSection

Area Flora

These were taken mainly to demonstrate that I have a halfway usable macro lens on my "cheap" camera (compared to Ying's, anyway).



but I think they look okay.

Won't You Be My Neighbor

I don't get out much.

Still, there are some strange things in my neighborhood. Simply on my way to the mall there are a surprising number of oddities. For example, there is a building next to the Museum of Science that I like to think of as the Dungeon of Science.



From the amount of ivy on that building, I have to guess that "Unauthorized Persons KEEP OUT" sign was placed there a good 6 decades ago; as they were constructing the Museum of Good Science, they had to have a place to trap all the Bad Science so that it wouldn't get out. There could be more mundane explanations, of course; it could just be where they keep the vampire.

If we continue along to the other side of the river, we come to this:



In the game Eternal Darkness, there is a point at which the villain must construct what he terms the "Pillar of Flesh", a grisly monument made of concrete and still-living people, designed to focus the evil energies required to bring his dark god forth to this world. You know, to raze it down to its component atoms and re-make it in its own image, etc. So, imagine my surprise when I discovered I live right next to a miniature; let's call it a "Pillar of Souls", since clearly whole bodies aren't trapped in there. You can see that most of the folks involved are smiling for the camera:



but there are a few who let their true feelings slip through:



Seriously, there is no inscription anywhere on this monument, nor any plaque in the area to explain what it is supposed to represent. Why would you make a sculpture of a group of people's heads smashed together in various states of dismay and/or amusement and then put that on a footpath near a mall? Why, I ask you? I note that it is pointed towards the sky - perhaps some kind of spectral battery for a weapon that shoots ghosts at the sun?

I am not an expert but I do not think that shooting ghosts at the sun would do anything, and if it did, it would not be a thing that is good for anyone.

I am weak (Review of Saitek "Eclipse" Keyboard)

This isn't an essay I've had in mind for any time, but the photo is kind of neat, and it happened today.

I have managed to avoid purchasing a keyboard for almost a year now. Apparently this is too long:



Today briefly before lunch, the "Alt" key on my Bluetooth DiNovo finally gave up the ghost. It's still under warranty, but I was in the middle of hacking with Emacs, and you can't really do that without a meta key. I really do destroy keyboards at an unbelieveable rate. As a backup, I purchased a somewhat sturdier and cheaper board; the Saitek Eclipse PC Keyboard.

While it doesn't meet any of my normal criterea for a good keyboard, i.e. it doesn't use mechanical keyswitches and it's not slim form-factor, something about this keyboard just exudes quality. This wasn't a dramatic or somehow amazing keyboard like the Tactile Pro, but it is a quietly comfortable and definitely high-quality keyboard. It's a bit paradoxical - I generally hate membrane keyboards with a lot of travel, but this one seems to be quite comfortable. The keys are a good size, the incline is comfortable, and I seem to be able to break 140wpm consistently in the hard drills at the end of gtypist. It has exactly the keys I need: no funky spatter of random "internet" function buttons that don't work with Linux anyway. All the keys are in exactly the right place; they didn't pull any weird tricks like expanding the Delete key, moving around the backslash or backquote, or sliding the Alt somewhere other than under the thumb. The nominal Control keys are nice and wide, when you need to use them, and the Caps Lock isn't castrated to make it useless as a Control.

But of course, these are all just mistakes they didn't make. It feels nice, and I don't know why. The only concrete difference I can really put my finger on (no pun intended) is the lack of "squeak". Perhaps there is a better word for this, but I have only tested this concept experientially - here's what I mean.

Try this with a normal $5 keyboard: push hard on one of the keys, then slide your finger back and forth. You will notice that it squeaks when you wiggle your finger back and forth, sending vibrations up your arm. I always take this as a sign of the quality of the construction of the keyboard, because the squeak is coming from bits of plastic rubbing against each other which are never really supposed to touch. On really awful keyboards this is bad enough that you can feel the squeak while typing; on others there is just the occasional hint of it.

While this particular feature isn't a deal-breaker for me (even the DiNovo has a tiny bit of squeak) it is definitely a good sign that the Eclipse doesn't have it. I'm enjoying typing on it immensely so far. Its first day it saw a prodigious amount of use, as I was preparing for a rather intense deadline :).

Twenty Megabytes

Today I'm playing with Flickr to see what the buzz is about (and of course, to get some ideas for Divmod's photo-sharing features). I often think of blog narratives that involve photos. I'm going to post a few samples of ideas that I've had for months, but just haven't found a moment to upload.

I maxed out my upload ratio on my first night: 20M of photos goes really quickly.

By the way, I uploaded all the photos with FlickrUploadr, an unofficial GTK+ tool to upload photos written, inevitably, in Python.