A collection of articles, ideas, and rambling from a guy who wrote some software that one time.
Monday, September 20, 2004
I guess this just goes to show that they have no respect for copyright law. Bunch of bloody anarchists, they are, those MPAA guys. Stealing the hard labor of innocent copyright creators of content, rapacity on the high seas, etc etc.
Thursday, September 16, 2004
I have started using Straw today in earnest, thanks to the ability to aggregate my LJ friends list. I particularly like Straw's UI. I don't think it's using Twisted - although it should be, since it's in python - but it behaves like it does. It is nice and responsive. News aggregators are cool. I'm even aggregating aggregators, most notably Planet Twisted, which if you aren't reading, you should be. It's a Planet thing.
Saturday, September 11, 2004
I started out here by writing a reply to href="http://garage.docsearls.com/node/view/453">r0ml's recent post on href="http://garage.docsearls.com/">Doc Searls' IT Garage. For some
reason, though, reading this article started me thinking about security. and so
its scope has expanded.
Based on my second- and third-hand knowledge, in previous office information
technology revolutions, security domains were well separated, or at least gave
a convincing illusion of being so. At first, things which ran on the mainframe
were the responsibility of the IT department, things which ran on the
workstations were the responsibility of the "rocket scientists", be they devs
or quants. Anything IT talked to that wasn't actually running on the mainframe
As IT subsumed the TCP/IP infrastructure, the professionals ran to IPX and
various transports for SMB: anything IT talked to on a Netware or NT share was
suspect. Now that IPX has gone the way of the dodo and CIFS is exclusively a
TCP/IP beast, there is a new problem: assuming the end-users are equally
sophisticated as your IT staff (unlikely) but less concerned with security
(very likely), if they attach a machine to "the network", meaning the
IT-maintained, TCP intranet, then there is no protection against the outside
world besides the firewall. There is no domain that IT can look at and
automatically say "ah-hah - data coming from there is suspicious, and sensitive
data should not go there".
With a new Outlook virus every two weeks, allowing users to download and run
things off the web puts corporate IT into an almost tragic position. An
industrial spy who writes a trojan that uses visual basic can act as if they
were actually a user, willy-nilly attaching any file on the user's hard drive
with the word "budget" in it to an email in outlook and mailing it to servers
in russia. When users inevitably fall prey to these problems, it becomes IT's
responsibility. Even - and perhaps especially - the machines sitting on the
users' own desks are part of the IT infrastructure.
They also must remain part of that infrastructure until there is some other
way to deal with assigning responsibility for security problems. If the CTO is
going to get the axe for poor security practices, you bet he's going to scream
bloody murder every time an IT staffer lets a user install a .scr file
This all creates a problem of in the mercurial worlds of legislation and
accounting, but I think the next IT revolution might put power back in the
hands of the users if these alternate universes come back into tune with
reality. The fact is, IT in the large is doing a pretty bad job with
security. Firewalls that block everything but port 80 and leave email in the
DMZ a good example of the sort of cargo-cult paranoia that drives modern
security design. Who is it out there that really believes that attackers can't
tunnel their outgoing information through HTTP or email?
What about incoming attacks? Outlook is always easy to pick on, but it is
hardly the only problem. Let me preface what I am about to say: I don't have
any connections in the white-hat or black-hat security communities. I do know
a few programmers though, and most of them work with the web. Every so often,
one of my colleagues will hit a website, look at a peculiar URL, say
"Hmm... that's funny" and try passing some obviously invalid data as a
parameter. On several occasions, this has resulted in me getting an IM or an
IRC connection saying something like "Hey, look at this:
http://...?PROGRAM=/bin/cat%00/etc/passwd". Or maybe,
These simple attacks, performed through the public web-sites made available by
the companies themselves, result in password files being exposed, or even
customer information being inadvertently granted to outsiders. In every case,
the problems have been promptly reported to the proper authorities within the
companies with the problem (and in at least one case, where the company was
unresponsive, to the police).
Aside to the viewers at home: before you ask, no, I will
not tell you who discovered the problems, or where the problems were
discovered. Most especially, I will not "teach you to hack", and if
you asked, you are href="http://www.catb.org/~esr/faqs/hacker-howto.html#I_want_to_crack_and_Im_an_idiot">sub-human
scum. I thought long and hard about posting this part of the
entry, and if any discussion about this breaks out, I will immediately
remove public access to it.
I've never heard two reports from the same person, or at the same company;
these are not hardened criminals breaking into sites, or one company whose
security is abysmally bad. Security on the internet really is so bad that a
casual observer with no security training and only a smattering of knowledge
about the potential configuration of a website is often able to accidentally
break into it. It's a quiet epidemic in the technology industry, one that goes
to the roots of the unexpected success of the internet and the hyper-speed at
which programmers have been forced to produce code and be the first to market.
This epidemic will become a harsh reality for consumers soon, as computational
trust issues have now taken on clear
and present political consequences.
Back to the issue at hand, though. In "Do It Yourself" IT, who is
"Yourself"? The implied "Me" shifts back and forth between IT companies and
vendors, but the "You", the "real people" who need to do their work with
computers, are hamstrung by mistakes "We" have made. It seems to me that the
most serious among these mistakes, the really limiting ones, are related to
security. The limiting factor isn't one particular aspect, but both problems
and solutions, both perceptions and misconceptions about security and real
Business technologists need to get serious about security, and start
considering attacks against their software in a real way. That means getting
security where it counts: in the applications and in the operating system. IT
management needs to take drastic action and hold vendors responsible for even
potential security problems. There is a tendency to whitewash these things or
to put them on the back burner, since when security is not an emergency, it's
not a visible problem at all.
Until that climate changes, the user's computer will be a prisoner of IT's
fear that it will cause security problems. I don't have any illusions that
suddenly everyone will start getting better at security auditing, but the
fundamental technologies underlying our infrastructure need to be cleaned up
significantly. I'll call out a few by name by way of example - Perl, PHP, and
ASP. Every compromised site I've ever seen was using one of those three
technologies, and it was a problem at that level or a very bad, but very common
idiom that made the sorts of mistakes I've seen easy to make for
I know I plug high level languages a lot, but I don't want to end on a glum
note of "and that's how things are". You can improve your code's foundation
today, if you just pay attention to security. If you're starting a
new project, Lisp, Smalltalk and Python may not be perfect, but applications
written with them (and with an eye to security) can set you free, to be You, or
Me, and let you define who Yourself is.
In my previous entry, Mike Dartt asked me "how I did it". I tried to
answer in a reply, but it was so long LJ wouldn't let me post it. So,
here's another top-level article for you all, describing how I've gotten
organized and what I've been doing.
I've read lots of books on time management. None of them really helped.
I had to reduce this to the bare minimum, because I certainly couldn't have
paid attention to anything longer than this blog entry when I was trying to
get organized. If I could have, I probably wouldn't have needed it. So
here's my schedule:
- 8:30: Get up
- grab a Red Bull from the fridge and drink it
- Check email before the call
- 9:30: time for the group call, discuss what I'm going to do today.
The morning is a little nebulous, as I've reserved this time for meetings with co-workers and planning discussion. I also update the issue tracker to reflect what I'm really doing, writing any new to-do items down and assigning them an appropriate priority.
- 12:00: break for lunch even if I am in the middle of something.
- 1:00: "close my door". My office doesn't actually have a door, so it's a bit of an act of will to do this properly.
- 6:00: "leave the office".
Again, I often can't literally leave the room where I work, because that is where pretty much all of my stuff is, but it is time to mentally shut down for the day. I start writing a description of what I've done, since this usually takes me about an hour.
Since I'm responsible for "managing the team", I thought long and hard about
what had been effective for me in the past, and I took a leap of faith. I
didn't know that this sort of simplistic "management" technique would work on
anyone else, especially because I have been historically unsuccessful with such
plans, but it was my responsibility and I had to try something. So,
the three axioms and two corollaries I used to put together this schedule for
myself and the strategy for the team are1:
- If you want to avoid getting sidetracked, know what you are doing now and what you are doing next. Remind yourself of your to-do list at every opportunity.
Therefore: Keep a list, or many lists. Make sure they're
updated, and make sure they're in order. Having an issue tracker that
by default sorts by priority and shows you the top 10 things you have
to do in order is handy. Right now we're using href="http://www.bestpractical.com/rt/">Request Tracker, which has
a lot of problems, but is worthwhile because of that one feature that
many other trackers seem to lack. Our tracker wasn't exactly getting
ignored before, but everyone regarded the goals as extremely abstract
and high-level and they would get updated only rarely. I didn't trust
it to have the right things for me to do, and I assumed I had to put
together my own list of priorities daily. This would shift around and
sometimes not reflect the state of the tracker at all.
Now, I am
totally annoying all the time about the tracker, and I make sure to use
it for any random ideas that I might have. Repeating over and over
that it's important for everyone else to keep it in good shape has
forced me to do so as well.
If you want to actually get tasks done, you have to have enough energy to accomplish a specific task, and
- You must have a sense of urgency about actually working now.
As a fellow Hampshire-ite of the Dark Times ( href="http://www.livejournal.com/users/z3p/355249.html">kids these
days don't seem to have the same issues we did) Mike and href="http://www.livejournal.com/users/ordinalten">others will
certainly appreciate the soul-sapping power of work that is never really
due and can never really be complete. (If you can't fail, you can't
really succeed either. That's some taoist shit right there.)
Therefore: make sure the work is something you can walk away from. If you don't have a place you can leave, make it a time you can leave. Make sure that you show up and are focused 100% when you are working.
Oddly enough, the way that this started with myself and with my team, because we are all highly motivated, was bashing people about the head to stop working when the end of the work day rolled around. It was difficult to tell people to do, and it was difficult to do myself. It's especially hard when you're just starting with this, still tired all the time and you feel behind, and it's 6PM and you haven't done jack all day, to tell yourself "time to write an update that says I didn't do jack all day" and stop working.
The paradoxical effect of stopping working is that you'll get more work done. When you don't linger all night, only half-paying-attention to what you're doing, you come to work the next day ready to get something done. Moreover, you are aware that if you don't do it during the allotted hours, it's going to have to wait for the next day: personally, this has been a great motivator to get working now and not putter about reading comics and suchlike.
The fourth point, which is entirely personal, is "keep my body chemistry from interfering with my day". I have been attempting to force myself to eat snacks on a regular basis, drink caffeine at regular intervals, so that my blood sugar doesn't drop precipitously and my ADD doesn't start acting up at around 2pm (right as I should be getting into the thick of work) as it generally does. I am of course also trying to eat right and exercise but the main point I am trying to get to is just that my body's state of activity remains relatively consistent throughout the day, so that I'm not struggling with having huge bursts of energy when I'm supposed to be asleep and then exhaustion when I'm trying to focus on work. That's far more important to me, on a practical level, than the ability to run several miles at a good speed or do 100 pushups in a row.
To be honest, most everybody on the Divmod team is better at
self-management than I am, so this has made a bigger difference for me than
for everyone else. I doubt that most of them are desperately trying to
manage their glucose intake just so they don't collapse. I think it's
helping all around though, especially the carefully maintained group to-do
list. Now when I make a request of someone, I'm sure about what I'm asking
them to stop work on, so I can avoid interrupting important things.
Some of this is probably specific to working in a self-scheduled group environment, responsible for your own tasks, on creative projects. I hope it helps you with something. I'm just about out of time on my email/blog time budget, and this is only tenuously on-topic for work, so I'll have to stop here :).
1. I sure am loving these itemized lists today.
Thursday, September 09, 2004
It has been a rare occurrence in my life to be working regular hours, sleeping at the appropriate time, eating some number of square meals in a day that is greater than one and getting to sleep at a reasonable hour. Normally, working happens in fits and starts for me, and I struggle painfully, waiting for inspiration to strike. As I've been hammering myself into a routine and trying hard to eliminate distractions, though, I find that my day has a certain natural rhythm.
Over the past few days, I have noticed a serious improvement emerging in my habits. When 1pm rolls around, I am already thinking about what I'm going to be coding. If I have a design problem, I start mulling it over automatically as lunch approaches. I have also started a more regular schedule for consuming caffiene, which seems to be helping me cope with A.D.D. Today, I found a sushi place with really cheap lunch specials, and of course the focus-friendly properties of raw fish are nice to have at reasonable rates.
What does this have to do for the company? Well, I feel like I'm making some serious progress on our most serious impediment to open signup right now, which is really exciting. This is an invisible barrier which has haunted me almost since my ill-fated experience with E.K. (my first software startup project, when I was 14). Never got to see an actual box to put that software in. Never got Divunal to an open beta, into the hands of actual players. Never got any game at Ninjaneering off the ground to the point where real customers were playing it. Finally, with real customers actually testing the software, I'm getting ready to cross a finish line both personally and for Divmod.
I suppose the eating and sleeping stuff sounds pretty mundane in comparison to the eve of such an achievement, but for me, they're the same thing. I don't know how to put it, if you haven't had a few hundred sleepless nights and groggy afternoons yourself. It's a welcome and long-overdue change for me, though, and I imagine there are a few readers out there hoping for the same sort of transformation; I hope it sticks.
I'm going to have to stop with this blog entry now, because it's late, and I'm tired. I'm going to sleep.
Friday, September 03, 2004
When I was very young, my mother was concerned that I never laughed or smiled, and having forgotten to pre-load my positronic net with the "humor" module, she realized she would have to do some work from scratch. I am told that the original transcript went something like this.
Mom: Do you know how humor works?Now, my mother actually kept balls of yarn in various places around the house, and I had seen the moon, so this didn't strike me as very funny. I thought about how big balls of yarn were, how surprisingly long they were when unrolled, and how slowly they got smaller. Then I attempted to mentally estimate the distance to the moon, in terms of how quickly the balls unrolled, how quickly they got smaller, pictures in books of the relationship between the moon and the earth, and how far away other things I had seen were. I don't remember the rest of the conversation, but I distinctly remember the mental image that I built during this process, as it has stayed with me during the years. It looked like this:
Mom: I am going to tell you a joke, then. It is one of the first jokes that my brother used to tell.
Mom: How many balls of string does it take to get to the moon?
Me: Three.My mother thought this was hilarious, so my initial understanding of humor was that I should run up to everyone I met and say: "howmanyballsofstringdoesittaketogettothemoon?doyougiveupyet?THREE!HAHAHAHAHAHAHAHAHAHA". Reading doc/fun/Twisted.Quotes in the Twisted distribution can show you how little it's progressed since then.
When I estimate programming tasks, I still have a similar sensation to when I was 2 years old and building that little picture in my head. Then, I grossly underestimated because I didn't have a mapping between astronomical distances and inches, because I didn't know what units distance was measured in among the stars. Now, I grossly understimate because I don't know what unit you can measure programming effort in. It's not "hours" because I can't reason about that - one does not do a uniform amount of work within one hour on a program, especially since several hours are spent thinking. I know various ways to measure finished programs, and I know of various ways to measure programs by specifying them to death - however, neither of these gives me the accurate estimate when I want it, which is to say, before work has begun and a great deal of resources have been invested. It is harder, and takes longer, in my experience, to accurately estimate (in hours) how long a program will take than to just write it in the first place; and even if you do go through that process, you can't estimate how long the esimation will take (and the estimation process cheats, by stealing work from the programming process so that it is shorter.)
All this thinking doesn't do anything to make the need for good estimates go away though. So how do you tell how big, or how hard a program is, without first writing the program several times and getting lots of different people to do it? When you know how hard it is, what units do you express it in?