Twisted is the goddamn engine of your Internet.

Don't let anyone say otherwise.

Update: We're over the line now - 33 to 32!  Keep on voting though, make that bar bright green.

So, it seems that it is possible to get the last feature that I really, really wanted to use on my MacBook, NetworkManager, to work under Ubuntu Edgy without a custom kernel.  The steps involved are not difficult, but they are tricky, due to an interlocking matrix of bugs.  Thanks to some insomnia and procrastination I've decided to write it up.

Reader beware!  These steps are designed to work around a particular set of bugs on a particular revision of Ubuntu for a particular piece of hardware.  If you're reading this at some point in the future, chances are that the madwifi project has already produced a driver.  To find out, check to see if madwifi ticket 1001 has been resolved before you do any of this.

If you are running Ubuntu Edgy, with the default kernel, and you have a black rev.2 MacBook and you want to get NetworkManager working without screwing around with Feisty kernels, read on.

• First, you will have to build your own ndiswrapper.  You need at least version 1.29, which is quite a bit newer than the version packaged with edgy.  I chose 1.31 - not too old, not too new.  You need at least version 1.43.  (Earlier versions seemed to work, but 1.44 was the first version I installed which could suspend and resume reliably and did not very occasionally produce random crashes.) These instructions may very well work with older or newer revisions, but I am not going to build a big revision matrix; the whole point of this is a temporary workaround.  Install it with "make uninstall; make install" to make sure to remove ubuntu's packaged ndiswrapper driver first.  Keep in mind that if you upgrade your kernel via apt, you may need to repeat this step, so keep the sources around.
• Next, you will want to get the driver from d-link.  Go to this page, and get version 1.02.  Some discussion on the Madwifi page says to get version 1.01 instead because 1.02 crashes.  Ignore it.  As far as I can tell, it's just wrong.  I originally followed this advice, which is why I thought that it crashed ndiswrapper; I also had other problems with version 1.01 such as not being able to associate with various public access points.  See my later post about which driver to get.  Anyhow, install the driver into ndiswrapper by unzipping the downloaded archive and running "sudo ndiswrapper -i net5416.inf".
• Test to make sure the driver works.  If you are already running NetworkManager and nm-applet, simply doing 'modprobe ndiswrapper' ought to set it up nicely and you should get immediate visual feedback that it is working.
  
• Configure the module to un-load itself when you suspend or sleep and re-load itself when you resume.  This is important because otherwise ndiswrapper will not allow anything   This is accomplished by editing the file /etc/default/acpi-support and changing the 'MODULES' line to say 'MODULES="ndiswrapper"'.  While you're in there, you might want to also change the 'STOP_SERVICES' line to say 'STOP_SERVICES="mysql bluetooth "' instead of just mysql, since bluetooth is notoriously unreliable in the face of power management, and bluetooth connections, like wifi connections, will not survive a suspend/resume cycle anyway.
• • Make sure that ndiswrapper does not create a 'wlan0' alias for itself; you probably don't need to do anything, but if you're using a different version of the ndiswrapper script, it may create a file called "/etc/modprobe.d/ndiswrapper" with an alias for "wlan0" in it.  If you see this, remove it.  NetworkManager knows about the module name 'wlan0' and will constantly try to load it if it becomes unloaded for some reason.  This results in a particularly nasty race condition where the suspend machinery politely removes ndiswrapper in preparation for suspending and then NetworkManager loads it again, resulting in a hang from which it is impossible to recover without a hard reboot.  I managed to create this situation for myself through experimentation, so it probably won't happen to you, but just in case make sure that file doesn't contain any reference to 'wlan0'.
    
• Test suspending and resuming and make sure that the driver loads as expected.
  
• Configure the driver to load at boot.  This consists of editing the file /etc/modules and adding a line that says "ndiswrapper". DO NOT do this until you are sure the driver works with your machine; if it causes a crash, this might make your machine unbootable.
  

There you have it.  It's an unfortunately jury rigged situation, but now that I've gone through these steps, I can effectively (mostly) forget about ndiswrapper and compiling kernel modules and so on, and avoid the tedious and unpleasant typing "iwlist wlan0 scan | less" and instead just click on the access point that I want.

Last weekend, I attended PyCon 2007. I hadn't originally intended to go, but I could hardly miss my father giving the final keynote.

The keynote was riveting as expected. To be honest, I hadn't expected much of the conference beyond that. The first PyCon was an amazing experience, but as the topics drifted more towards "web frameworks" and away from more general programming, later ones were progressively less interesting. I missed last year's and nothing I read caused me to regret it much.

I'm happy to say that this expectation was completely wrong, and this conference was amazing. I thoroughly enjoyed it on many levels.

In previous years, a lot of my time was soaked up by justifications, answering questions like "why does Twisted make me do foo" or "why haven't you made Nevow do bar yet". This year, the discussions I participated in were all productive and engaging, including the two, multi-hour Twisted birds-of-a-feather sessions which I attended.

Many of the exciting developments that I enjoyed while I was there aren't ready for public consumption quite yet, so I can't say much here. I can give you some incredibly vague hints though! It was a very rewarding conference both from a business and community development perspective. (Those of you with privileged information, please do not add anything revealing to the comments. Seriously.)

One cool thing that I can shout from the rooftops already is that Guido, a group of concerned hackers, and I got to have a meeting of the minds, which Guido has already blogged about, addressing many upcoming concerns we all had about Python 3. That, and several other discussions with the responsible developers about the proposed transition plans for the 3.0 release have put my mind at ease. That's not to say that I agree with every decision that has been made - and I definitely need to participate in a few more mailing list discussions - but I feel much more comfortable that the whole thing is in good hands.

My major regret for this conference is that I was completely unprepared for the truckload of great stuff that happened. I thought I would take it easy for a few days and get back to work. If I had a better idea of what would be happening, I would have prepared at least a few lightning talks, a more structured BoF session, and better allocated my time to the many interesting folks who wanted to bend my ear so that I wasn't rushing from conversation to conversation.

I told a lot of people that I'd be doing a lot of things as soon as the conference was over. Unfortunately the first thing I actually did when the conference was over was develop the worst cold I've had in the last 5 years, and promptly stay home sick from work for a week, sleeping most of the time. To make matters worse, the conference and the illness coincided with the blackberry software on my phone crashing very badly and a not-quite-perfect beta-test of the Divmod migration process on my email account.

In other words, if I told you I'd get back to you at the conf, I probably haven't. I'm trying desperately to claw through my backlog right now, but it's going to take a while. Please be patient, and if you haven't heard from me in a week or so, feel free to send some repeat emails and nag me. If I said I wanted to get back to you, I really do.

I should warn you that I'm still not quite back at 100% HP/MP yet, and I have lots of actual work work to do as well, so you might want to wait a few days, but please, everybody, stay in touch.

The attendance list seems a bit anemic so far, so I thought I'd point it out:

Twisted Birds of a Feather session at PyCon 2007

I'll be there.

As I predicted, I purchased the black MacBook.

After every other laptop I've purchased, I experienced some form of buyer's remorse.  In this case, I didn't.  The overall experience has been positive.  I really don't have time to do a complete review (especially after spending all that time trying to remember how the heck bluetooth modems work - ugh!), but I'll offer the highlights, from an outline I was working on to do a fuller review:

• It's very fast.  It runs the unit tests even faster than my work machine, a beefy Opteron.
• More than any other Apple computer or appliance I've owned, its design is a work of art.  There are no special function keys (unless you count "eject"), no latches, no status indicator lights, clips or plug covers or anything.  The only moving parts are the keys and the disc drive.  It's also almost completely silent.  This can be slightly creepy: there's hard disk light, no sound from the disk, no network activity light, no external indicators to show that the machine is doing something.  I might have to get some blinkenlights panel applets just to make myself feel comfortable!
• • MagSafe is a really, really good idea.  It's not just about not knocking your laptop off of the table - it makes every other laptop plug you've ever used feel like utter crap.  The Vaio still holds a strong second in my book, because it allows for a fair amount of play on the plug before anything starts to bend or break, but... this is really in a class by itself.
    
• The keyboard is delicious.  For some reason I had the impression that macbook keyboards were bad, probably because of the silly placement of the 'ctrl' key if you don't re-map it.
• • Old school powerbook/linux and powerbook/emacs people, you'll know what I mean when I say this: capslock-as-ctrl doesn't stick.  In either OS.
    
• It works well with Ubuntu, with a few minor caveats:
• • The bad news:
    
  • • I had to use ndiswrapper for wifi, because apparently the rev2 macbooks have a totally different wireless card than the rev1.  Unfortunately this makes NetworkManager crash, and I have had to write a few shell scripts to access different wireless networks.
    • The 'fn' key is in software, which means I don't get pgup/pgdn/home/end or volume control keys in linux.  Also, the 'fn' key shows up in 'showkey' but not in 'xev' - it's not accessible to X for some reason. I don't know why this is, but it isn't really enough of an annoyance for me to bother with fixing yet.
    • Suspend has some issues around restoring the display properly when waking up.  I really don't care, so I'm unlikely to figure out how to fix this - I prefer the machine to hibernate when I close the lid anyway.
      
  • In general, however, lots of things worked out of the box or with minimal tweaking in linux:
    
  • sound

  • video (with accelerated 3D)
    

  • bluetooth

  • USB

  • firewire

  • wired ethernet
    

  • hibernate

  • CD/DVD burning

  • 3-button mouse emulation (using right-apple and enter keys as middle and right click)

The real litmus test for this laptop was that, immediately after purchasing it, I ended up having to drive all over Quincy for some reason, while also trying to catch up on several days of work.  The laptop performed like a charm; I connected to seven or eight different wifi networks, wrote several hundred lines of code, hibernated and resumed half a dozen times, sometimes just snapping it shut and tossing it in my bag without worrying about heat issues (the vents are along the top of this machine, where the bag is open, instead of along the side where they'll be instantly blocked).

This experience was actually quite pleasant, devoid of the usual "new laptop that doesn't quite have all the drivers installed" experiences I've come to expect.  Edgy can take some of the credit for this, of course, but the macbook itself is fast, light, comfortable to use, and has good battery life.