A clear undercurrent pervades all discussion of potential contributions to the open source projects I have been involved with. Everyone involved, contributors and maintainers both, assumes that open source projects should be glad of all the contributions they receive, that contributors are in short supply, and that every effort should be made to accommodate those willing to donate their time.
To some extent, this is true, to be sure, but I think the case is frequently overstated. One should be friendly to contributors, and honor their contributions, but a contribution to a running project is not a one-sided transaction. In fact, open source projects offer a precious commodity: maintenance in perpetuity, at least for the project's lifetime. In larger organizations, this sort of maintenance is quite often the subject of multi-million dollar "support" contracts, and is in fact the bedrock upon which the towering edifice of the software-support industry is built.
It has been gradually dawning on me, mostly since the institution of UQDS on Twisted, that this is not the only benefit a contributor receives from a good project. The careful feedback that an experienced peer reviewer can give to a contributor is valuable professional development tool. This is true even if the reviewer isn't particularly more experienced as a programmer or as a reviewer: simply getting a different set of eyes, and a fresh perspective on your code can show you problems with it that would never have occurred to you on your own.
Perhaps more importantly, it is actually prestigious to contribute to certain projects, and not in the cutesy-happy way that terms like "gift culture" imply. "Rank" in a merit-based hierarchy such as the Apache project can clearly contribute to one's employment prospects, as much as any other professional qualification.
I often wrestle with the problem of "recruiting" for projects which cannot pay their contributors, and has apparently very little to offer. As these notions occurred to me, I noticed a similarity in these sorts of benefits to another sort of contribution to a community: academia.
Recruiting for an open-source project is sometimes laughed off by potential contributors. "Yeah, right, like I'd spend my time writing code for free." Personally, I've always found the best of any field like what they do enough to do it for free, but still... this objection seems reasonable. It is ridiculous to give your time away when you could be getting paid for it, right?
Let's examine the average college student at a prestigious technical university - let's say CalTech or MIT - for a moment. They (or their parents) pay substantial sums of money for the privilege of attending the school. They are forced to do rote work - often in excess of 60 hours per week. At the end of four years of this gauntlet, they are presented with a piece of paper. As the son of one MIT alum and the fiancé of another, I can vouch for the fact that this is no cake-walk.
Of course, presented like this it doesn't sound so glamorous. As a dropout myself, I have an incentive to make it sound bad; but in our high-tech society the value of education is well-understood. I don't think I need to explain to any likely reader of this blog why that piece of paper might be worth something to a few of those students one day. Although many students might receive some form of scholarship or free tuition, nobody in their right mind would suggest that these schools should compensate their students, or frantically beg students to enhance their academic community by giving them whatever research they feel they have time for...
So, let's look at the Twisted project from this perspective. Community dedicated to technical excellence? Check. Continuous, harsh abuse of its participants self-esteem? Check. Years of incredibly tedious drudge-work to complete? Check.
We don't have a piece of paper, though.
This is actually very important, and I think can explain a lot of difficulty in recruiting. Often, contributing to an open source project such as Twisted is described as a happy, fun exercise, in the hopes that it will attract contributors. It can be fun, of course, but on the whole that is not the real motivator, and so this is generally recognized as false by potential participants and they move on. As we have moved further towards honesty about that fact - for example, instituting an unapologetic review process that rejects a substantial number of contributions - we have paradoxically gotten more activity, not less. Contributors actively engaged in the project have noticed an improvement in their own skills and are excited about that fact.
Still, it's an uncommon skill to be able to recognize that you're improving and be happy with that alone. I think that some public form of recognition could go a long way towards getting more people involved. I've been pondering different ways that we might honor contributors, from a branded USB drive, to some kind of offical certificate proclaiming a "rank", all the way to something like a "class ring" for contributors above a certain level.
That's probably not all it takes to get people excited about contribution, though. Institutions like MIT have centuries of cultural context associated with them. You can see this in movies: when a hacker character is being introduced in movie which prominently features the tubes, the whispers in the background often talk about the hacker in question's academic history at a selective university. I have been trying to imagine what the world would be like if, in the dramatic scene where Extreme Hacking Prodigy #2 is being introduced, the muted voices in the foreground of the scene were instead exchanging knowing mutters about his near-impossible record of over one hundred patches accepted to Twisted - in less than a year!
More importantly than what that would would be like - how do we get there from here?
9 comments:
...they should contain microcontrollers running public key software. : )
Twisted Feelies!
I feel like I should have some thoughts on this, dammit, but I've gone looking for them and they aren't there. It seems like the kind of thing I need to have a rambling conversation with someone about before I can write about it. Bugger.
Isn't this why there are commit logs? :-)
But more seriously, a USB drive or class ring isn't the right idea. The piece of paper isn't the right idea either. The piece of paper doesn't have any value except as decoration .. you don't even bring it to job interviews. Unless you're a lawyer or doctor, nobody who's likely to give you money will ever see the piece of paper.
The thing that has value is the fact that someone can call the university and ask whether you were there. It's verifiable. But that by itself wouldn't be valuable if the people in charge of giving you money didn't hold a university education in high esteem.
Things are moving in the right direction. Lots of employers do take open source contributions seriously. But maybe they'd take them more seriously if projects provided some way to call and verify that you worked on them and achieved some level of success. This also means creating a standard for identifying that level of success. You graduate by doing some fixed amount of work. It might not hurt to establish that level of work as some kind of Open Source Industry standard for SLOC patched or something. What's the Twisted equivalent of 120 units?
Take as step one the act of creating that standard and making sure it's verifiable by outside parties. Step two would be getting other projects to adopt it, and step three would be getting employers to acknowledge that standard as something useful to have on a resume. They already know why Open Source labor contributes to your value as an employee, but they have no way to quantify it. Is it like having a Master's degree? Or is it like having a work-related hobby? A verifiable standard would help them answer that question.
This is a very interesting idea.
One minor quibble: of course, the "piece of paper" has only symbolic value. Before the advent of the telephone network you did actually show it to potential employers. The fact that you can now call and ask is more of a defense against forgery than it is a different symbol. I assumed that the "piece of paper" I was proposing would be virtual. The various other decorations I proposed were not the actual reward for status themselves, just symbols of a status achieved...
Anyway, I personally really dislike lines of code as a metric. It incentivizes code bloat. "tickets resolved" probably works better, especially because that involves an explicit verification process (somebody else has to identify that your work actually fixes the ticket), if not pre-commit code review (which is, in my humble but entirely correct opinion, better). I don't think that the actual rewards have to be terribly fine-grained though. For example, nobody really looks past the first digit of your GPA, and Apache only has 4 or so "levels". I need to do some math to figure this out here, but you should get one status for your first "year's worth" of tickets (however many that is), and another for the next 3 or 4 years.
I do think that this ranking should be per-project. Much as you don't get credit for doing a few courses at 10 different colleges, part of the value of honoring contributors is rewarding those who follow through and complete major features and enhancements more.
Right, that's why I said "120 units" (a whole degree) rather than one unit.
I like the "tickets resolved" metric very much. Not all tickets are equally easy to resolve, but that's okay because not all university courses are equally easy for the credits you earn. It also conveys a little bit of credibility to the open source projects involved, much in the way ISO certification might convey some credibility: You have to be at least organized enough to need a bug tracker. Granted, this isn't a very high bar, and there should be others. I can't give my stepson 4 university credits for showing him how CSS works on his laptop, nor should my pet project convey any industry clout; standards have to apply to the project as well as the worker.
As an alternate (and probably easier) idea: Promoting your open source project to universities, and let *them* accredit you as some kind of self-study course, so that students can earn industry clout in the way they're most familiar with: as units toward a degree.
The biggest thing you talk about here is how the peer review process enforces learning, and how people gain experience from having others review their code. But right now all of that history is kept in Trac conversations and emails, and you can't really make any useful metrics out of that type of feedback. It seems like if reviewers could thumbs-up / thumbs-down on peoples' submissions, Digg-style, in the same place that they comment on submissions, you might start to get some interesting trends out of it.
The alternative is to do what a lot of small companies seem to do for recognizing talent internally: put metrics on the back burner, and just make a policy of handing out discretionary awards for kicking ass. Any rewards system that is entirely automated is easier to manipulate and "cheat" than a rewards system that ultimately trusts the judgment of a few highly-regarded decision-makers.
Now, if you're just asking what KIND of gifts to hand out, what about getting a Twisted-friendly company to sponsor a few tickets to PyCon for the best contributors? :)
I think the metric we've proposed deals with this.
On Twisted, at least, each ticket resolved is beholden to a subjective judgement on the part of an empowered reviewer. That defeats the gameability of a purely-automated metric.
However, it also defeats the inherent vulnerability of "discretionary awards for kicking ass" - if you couple the judgment of the work with the decision to grant an award, all kinds of other political crap gets involved. You end up mis-estimating the work of people who have done low-visibility but highly critical crap-work. So, lots of smaller judgments gives you some level of objectivity.
Glyph,
Interesting topic. For twisted, it might be nice to have a trac page that list listed users names, and their contributions to the project.
Complete with links to the tickets that highlight their contributions.
ex:
bugs reported:
name: ticket number ticket link
patch contributed:
name: ticket number ticket link
ticket reviewed:
........
branch created and merged ???:
.........
This would show user contributions to the project on the project page. This could be similar to a transcript from a school, complete with links to the contributor's code.
food for thought,
Mike
Post a Comment