I've been mulling over these ideas for quite a while, and I think I may still have more thinking to do, but recent events have gotten me thinking again about the increasing urgency of the need for a professional code of conduct for computer programmers. Mark Russinovich reported on Sony BMG's criminal contempt for the integrity of their customer's computers, and some days later CNET reported on Sony BMG's halfhearted, temporary retraction of their crime. A day later, CNet's front page has news of Apple trying to institutionalize, as well as patent, a similar technique. While the debate over DRM continues to rage, there are larger issues and principles at stake here that it doesn't seem like anyone is talking about: when you run a program on your computer, who is really in charge?
I posit that, in no uncertain terms, it is a strong ethical obligation on the part of the programmer to make sure that programs do, always, and only, what the user asks them to. "The user" may in some cases be an ambiguous term, such as on a web-based system where customers interact with a system owned by someone else, and in these cases the programmer should strive to balance those concerns as exactly as possible: the administrator of the system should have no unnecessary access to the user's personal information, and the user should have no unnecessary control over the system's operation. All interactions with the system should faithfully represent both the intent and authority of the operator.
Participants in the DRM debate implicitly hold the view that the ownership of your operating system, your personal information, and your media is a complex, joint relationship between you, your operating system vendor, the authors of the applications you run, and the owners of any media that pass through that application. Prevailing wisdom is that the way any given software behaves should be jointly determined by all these parties, factoring in all their interests, and that the argument is simply a matter of degree: who should be given how much control, and by what mechanism.
I don't like to think of myself as an extremist, but on this issue, I can find no other position to take. When I hear lawmakers, commercial software developers, and even other open source programmers, asking questions like, "how much control should we afford to content producers in media playback programs?", I cannot help but think of Charles Babbage.
On two occasions I have been asked [by members of Parliament!], 'Pray, Mr. Babbage, if you put into the machine wrong figures, will the right answers come out?' I am not able rightly to apprehend the kind of confusion of ideas that could provoke such a question.The "you don't own your computer" paradigm is not merely wrong. It is violently, disastrously wrong, and the consequences of this error are likely to be felt for generations to come, unless steps are taken to prevent it.
Computer programmers need a socially, and legally recognized code of professional ethics, to which we can be held accountable. There have been some efforts in this direction, the most widely-known one being the Software Engineering Code of Ethics and Professional Practice. As long as I'm being extreme: this code of conduct is completely inadequate. It's sophomoric. It's confused about its own purpose. It sounds like it was written by a committee more interested in promoting "software engineering" techniques, as defined by the ACM, than in ethics. I'll write a bit about exactly what's wrong with it after I describe some similarities in existing professional codes of conduct which themselves have legal ramifications.
Although there are many different codes of ethics for medical doctors, a principle which echoes through them all is one which was formulated in ancient history, originally by Hippocrates but distilled into a catch-phrase by Galen: "First, do no harm."
The idea is that, if you are going to be someone's doctor, you have to help them, or at least, you shouldn't ever harm them. Doctors generally regard this as a sacred responsibility. This basic tenet of the doctor-patient relationship typically overrides all other considerations: the doctor's payment, the good or harm that the patient has done or may do, and the advancement of medical science all take a back seat to the welfare of the patient.
In this modern day and age, where doctors often perform general anesthesia on their patients to prepare them for surgery, this understanding is critical to the credibility of the medical profession as it stands. Who would knowingly submit themselves to a doctor, knowing that they might give you a secondary, curable disease, just to ensure they got paid?
Lawyers have a similar, although slightly more nuanced, principle. Anybody who has watched a few episodes of Law and Order knows about it. A slightly more authoritative source than NBC, though, is the American Bar Association, who in their Model Code of Professional Responsibility (the basis for the professional responsibility codes of most states' Bar associations in the United States) declare:
The professional judgement of a lawyer should be exercised, within the bounds of the law, solely for the benefit of his client and free of compromising influences and loyalties. Neither his personal interests, nor the interests of other clients, nor the desires of third persons should be permitted to dilute his loyalty to his client.For criminal defense lawyers, these "compromising influences and loyalties" may include a basic committment to the public good. A lawyer who represents a serial murderer who privately admits to having committed heinous crimes must, to the best of their ability, represent the sociopath's interests and try to get them exonerated, or, failing that, the lightest sentence possible. Low as we as a society might consider a lawyer who defends rapists and murderers, we would think even more poorly of one who gave intentionally bad advice to people who he personally didn't like, or sold out his client's interests to the highest bidder.
(emphasis mine)
A doctor's responsibility is somewhat the same. If a doctor is treating a deeply evil person, they are still obligated by the aforementioned sacred patient/doctor pact to honestly treat that person, not use their position as a doctor to proclaim a death sentence, or cripple them. They are obligated to treat that person equitably, even if that person's evil extends to not paying their medical bills.
This pattern isn't confined to professional trades. Catholic priests have the concept of the "seal of confession". If you confess your sins to a catholic priest, they are not to reveal those sins under any circumstances, regardless of the possible harm to others. A priest certainly shouldn't threaten their flock with knowledge of their confessed sins to increase contributions to the donation plate, even if one of them has confessed a murder.
In each case, society calls upon a specialist for navigating a system too complex for laymen to understand: the body, the law, and the soul. In each case, both society at large and individuals privately put their trust completely into someone allegedly capable of navigating that system. Finally, in each case, the trust of that relationship is considered paramount, above the practitioner's idea of the public good, above the practitioner's (and other's) financial considerations.
There is a good reason for these restrictions. Society has systems in place to make these judgements. Criminal defense lawyers are not allowed to judge their clients because that's the judge's job. Doctors aren't allowed to pass sentences on their clients because that's the legal system's job. Catholic priests don't judge their confessors because that's God's job. More importantly, each of these functions may only be performed with the trust of the "client" - and it is important for the client to know that their trust will not be abused, even for an otherwise laudable goal, such as social welfare, because notions of social welfare differ.
I believe that computer programmers are a fourth such function.
Global telecommunications and digital recording are new enough that I think this is likely to be considered a radical idea. However, think of the importance of computer systems in our society today. Critical functions such as banking, mass transit, law enforcement, and commerce would not be able to take place on the scale they do today without the help of computer systems. Apropos of my prior descriptions, every lawyer and doctor's office has a computer, and they rely on the information provided by their computer systems to do their jobs.
More importantly, computers increasingly handle a central role in our individual lives. Many of us pay our bills on a computer, do our taxes on a computer, do our school work or our jobs on computers. Sometimes all of these things even happen on one computer. Today, in 2005, most of those tasks can be accomplished without a computer (with the exception, for those of us with technical professions, of our jobs) but as the public systems we need to interact with are increasingly computerized, it may not reasonable to expect that it will be possible to lead an average modern life in 100 years without the aid of a personal computing device of some kind.
If that sounds like an extreme time frame, consider the relative importance of the automobile, or the telephone, in today's society versus 1905. It's not simply a matter of convenience. Today it is considered a basic right today for accused criminals to make a phone call. Where was that right when there were no telephones?
Another way to think of this relationship with technology is not that we do a lot of things with computers, but that our computers do a lot of things on our behalf. They buy things. They play movies. They make legal claims about our incomes to the federal government. Most protocol specifications refer to a program which acts on your behalf (such as a web browser) as a user agent to reflect this responsibility. You are not buying a book on Amazon with your computer; you click on some links, you enter some information, and you trust that your computer has taken this information and performed a purchase on your behalf. Your computer could do this without your help, if someone has installed a malicious program on it. It could also pretend to have made a purchase, but actually do nothing at all.
Here is where we approach the intersection between programming and ethical obligation. Every time a user sits down to perform a task with a computer, they are, indirectly, trusting the programmers who wrote the code they will be using to accomplish that task. Users give not only the responsibility of performing a specific task, they trust those programs (and thereby their programmers) with intensely personal information: usernames, passwords, social security numbers, credit card numbers - the list goes on and on.
There may be a technological solution to this problem, a way to limit the amount of information that each proram needs, and provide users with more control over what different programs can say to each other on their own computer. Some very smart people are working on this, and you can read about some of that work on Ka-Ping Yee's "Usable Security" blog. Still, one of the experts there contemplates that perhaps, given the abysmal state of software today, perhaps the general public shouldn't even use the internet.
DRM is definitely a problem, but the real problem is that it's the top of a very long, very slippery slope. Its advocates point at the top of that slope and say "See, it's not so bad!" - but where will it end? While I am annoyed, I'm not really that concerned with the use of this kind of technology to prevent copyright violations. It's when we start using it to prevent other sorts of crimes that the real fear sets in.
Today, it's considered almost (but not quite) acceptable that Sony installs the digital equivalent of a car-bomb on my computer to prevent me from copying music. As I said at the beginning of this article - they don't think that the practice is inherently wrong, simply that there are some flaws in its implmentation. Where will this stop? Assuming they can perfect the technology, and given that my computer has all the information necessary to do it, will future versions of Sony's music player simply install themselves and lie in wait, monitoring every download, and automatically billing you for anything that looks unauthorized, not telling me about it until I get my credit card statement?
Whether unauthorized copying should be a crime or not, preventing it by these means is blatantly wrong. Let me be blunt here. It is simply using a technique to wring more money out of users because the technique is there. Much like the doctor who cuts off your nose and won't reattach it until he gets paid for his other (completely legitimate) services, this is an abuse of trust of the worst order. It doesn't matter how much money you actually owe the doctor, or Sony: in any case, they don't have the right to do violence to you or to your computer because of it.
What of "terrorism"? Will mandatory anti-terrorism software, provided to Microsoft by the federal government, monitor and report my computerized activities to the Department of Homeland Security for review? From here, I'll let you fill in the rest of the paranoid ravings. I don't see this particular outcome happening soon, but the concern is real. There is no system in place to prevent such an occurance, no legal or ethical restriction encumbent upon software developers which would prevent it.
This social dilemma is the reason I termed the IEEE/ACM ethics code "sophomoric". With the directionless enthusiasm of a college freshman majoring in philosophy, it commands "software engineers" to "Moderate the interests of [themselves], the employer, the client and the users with the public good.", to "Disclose to appropriate persons or authorities any actual or potential danger to the user, the public, or the environment", to "Obey all laws governing their work, unless, in exceptional circumstances, such compliance is inconsistent with the public interest." These are all things that a good person should do, surely, but they are almost vague enough to be completely meaningless. These tenets also have effectively nothing to do with software in specific, let alone software engineering. They are in fact opposed to certain things that software should do, if it's written properly. If the government needs to get information about me, they need a warrant, and that's for good reason. I don't want them taking it off my computer without even asking a judge first, simply because a helpful software engineer thought it might be a "potential danger to the public".
Software developers should start considering that accurately reflecting the user's desires is not just a good design principle, it is a sacred duty. Much as it is not the criminal defense lawyer's place to judge their client regardless of how guilty they are, it is not the doctor's place to force experimental treatment upon a patient regardless of how badly the research is needed, and it is not the priest's place to pass worldly judgement on their flock, it is not the programmer's place to try and decide whether the user is using the software in a "good" way or not.
I fear that we will proceed down this slippery slope for many years yet. I imagine that a highly public event will happen at some point, a hundred times worse than this minor scandal with Sony BMG, and users the world over will angrily demand change. Even then, there will need to be a movement from within the industry to provide some direction for that change, and some sense of responsibility for the future of software.
I hope that some of these ideas can provide direction for those people, when the world is ready, but personally I already write my code this way.
I've written about this a couple of years ago, and I think there's more to the issue, but I feel like this key point of accurately relaying the user's intent is the first step to anything more interesting. I don't really know if a large group of people even agree on that yet.
So, like I said, this post isn't about Divmod - exactly - but when we say "your data is your data"... we mean it.
Posts
45 comments:
Wonderful essay! What you have expressed here is critically important.
We had a glimmering of this working at Origin, where a respected leader advocated 'fascist' management policies for our network services, the virtual worlds. In a world where virtual and physical entities have increasing interplay and influence on each other, the policies governing the crafting and operations of the virtual will also increasingly impact the real, for good or ill.
Examples: Cell phones reporting user locations, unauthorized RFID scanning, traffic cameras IDing license plates, zombie PCs, Magic Lantern, Carnivore, autodeleting Tivos, and that's just what we know about.
Please develop this idea further. We're going to need it.
traffic cameras IDing license plates
In what sense do you feel that this falls within the scope of the essay? Should it? The connection isn't clear to me: I as a driver am neither owner nor user of the traffic cameras (at least, not if you mean ones like the ones in Australia, which are automatically taking photographs for the purpose of charging people with traffic offenses). The author of their software does not therefore seem obliged to serve my interest in not being tracked and fined, they seem obliged to serve their user's interest in harming me.
There's probably an open question in this particular essay about the extent to which inflicting harm is a sacred duty by programmers, because an analogy is drawn to two professions with opposed ethical requirements on that front.
For doctors, the situation is clearer, because it's difficult to directly harm a third party by giving medical treatment (even though indirect harm, such as that patient using their healthy body to kill someone may result). There are some grey areas, such as allowing a patient with HIV to actively deceive their sexual partners (who may also be that same doctor's patients) about their status since that is often the only way to get that person to continue to seek their advice, and these are actively debated. But essentially, doctors do not harm patients or third parties.
For lawyers, it is required to inflict harm, if it is harm inflictable through legal processes and is likely to benefit their client. Their role is to harm their client's opponents as directly as they professionally can in fact: to destroy their reputations, to confuse them or induce panic attacks in cross-examination, to have them convicted and their livelihood destroyed. Lawyers do not harm clients, but they are compelled to harm third parties as a matter of professional ethics.
So, for computer programmers, where the user of the program and the owner of the equipment want to inflict harm (for example, fining people for traffic offenses) and where they are not intending to use data and equipment that belongs to others to inflict that harm, this essay tends to suggest that the programmer should aid them. But that's me inferring from Glyph's analogies, I'd certainly be interesting in hearing whether he thinks the analogy to lawyers holds that far, or if not, what the difference is, and what the ethics are of writing a program intended to harm people, but not by using or abusing their private data or equipment.
There is also a definition question still open about who exactly owns data like "car with plate ABC 123 has just exceeded the speed limit". Australian law compels me to display these plates (or would if I owned a car), which are linked to my personal information and which allow me to be much more easily identified and punished as a traffic offender than a mere description of my vehicle would. My passport allows the Australian government's border control to be able to tell at any time whether I was inside or outside our borders. (They consider their tracking so good -- because we have no land borders -- that it's usually presenting in court as essentially flawless information.) This is all part of a legal compulsion to expose my personal information when doing certain things (travelling, driving) in ways that open me to harm. To what extent are authorities obliged to limit their use of that information? Is automated use of it subject to different ethical boundaries?
You are right Mary, "first, do no harm" doesn't really allow room for the implementation of justice, which usually (and sadly) requires a certain degree of violence (forceful seizure of property, imprisonment, enforced labour etc)
I'm not entirely sure of the relevance, but doctors also inflict harm (in its broadest sense) in the interest of their patient. Examples include amputation and abortion. In fact I'm told that surgery would technically fall under Australian assault laws, if it weren't for the fact that its specifically excluded (IANAL, of course)
I hope I was actually summarising your view in the first paragraph, btw :)
It's not a complete summary: I was using punishment (which is usually at least desired to be an "ethical harm" or "harm in cause of justice") as an example of harm because it's one of the more acute ones philosophically, but I'm also interested in the ethics of inflicting via software harm beyond that.
Lawyers especially don't have a code of ethics that says "do (legal) harm in the name of justice", it says "do (legal) harm to help your client" and the overall effect is meant to be justice. (As with doctors, lots of debate at the edges: eg is breaking a PTSD suffering ten year old in cross-examination one of these things or is it beyond the pale?) Glyph's essay suggests a similar idea to me: that getting developers to follow a narrow and not necessarily just local code of ethics will result in justice overall.
Holy schnikies. Thank you, glyph. That was so well put together. This needs to get passed around until everyone's read a copy of it.
we need not only the ethical framework but actual details of what constitutes ethical behaviour in given circumstances, for a software developer
Thank you for clarifying my admittedly muddy thinking :)
The first draft of this essay specifically mentioned MMPs, in the section on the 100-years-hence role of a computer.
Since you brought it up, I'll paste that paragraph back in: There are already examples today of interactions which are impossible unmediated by technology. You can't engage in an online game via postal mail. While many players say that the important aspects of the game are really the social aspects, those social aspects are impossible without the technological scaffolding around them.
The "respected leader" you mention is a pretty good example of why the issue is not as straightforward as "be a good person and you'll be a good programmer". I think that his *personal* integrity was great, and he very honest and forthright, but certainly his operational policies fell outside the bounds of this framework I'm setting up.
I do plan on doing some more writing on this topic. But don't expect to see it on a regular basis :).
I did actually read your earlier essay, but you're right, it is complex: I usually re-read it every time I want to discuss it with someone.
I agree with what you're trying to do, in a way I'm being unfair by jumping to later parts of the discussion, primarily because I thought was being a bit sneaky when (perhaps?) claiming particular greyer acts at an early phase of the discussion.
Mary, the reason I used "Primum non Nocere" as the title was because much as the doctor's obligation is to do no harm to their patient, the programmer has a responsibility to do no harm to the user, similarly to the lawyer's repsonsibility to do no harm to their client. As you've said, even the doctor may be indirectly enabling severe harm to others that will be done by their patient when they recover.
Outside those bounds, the lines are a lot fuzzier. Since you asked this question directly: I think that the analogy to lawyers falls apart once you start talking about directed harm; the better analogy might be towards the general category of engineers vs. the specific category of handgun engineers.
Programmers who write desktop software are like industrial designers for home appliances. They don't really think at all about whether the appliance could be used as a weapon, they just try to make it perform its task well. The designer of an electric carving knife simply doesn't (and shouldn't) spend any mental effort on its utility as a weapon; it should just be a good carving knife, and not be unnecessarily unsafe. If the user is going to use it on human flesh... well, that's really the user's responsibility.
Programmers who write software designed to target ICBMs or otherwise kill people are in grayer area, more like firearms, but the general rule is the same: build to the function, not to the purpose. Guns are to be used to inflict harm, and the designer of a firearm has the responsibility of making sure that the harm is directed exactly and only where the user of the firearm wants. An arms manufacturing company can be held liable if there is excessive and undisclosed collateral damage for firing one of their guns, but it's hardly their fault if the gun can be used to kill someone, if that's what it's for. Perhaps local laws prohibit the manufacturing or sale of firearms, or require certain safety measures to be included, but it's not the gun manufacturer's job to make a moral judgement of the use of the weapon. Ultimately it is the user of the firearm who is responsible for its use.
This is important, because in order for the user of a firearm (or a carving knife, for that matter) to be held liable for their own actions, their actions have to be the result of their own decisions, not of the manufacturer's heuristics about whether the weapon is pointed at someone evil or not. Programs should be the same. Perform the function, and leave the moral reasoning to the user.
As far as the things Jason mentioned which don't fall directly under the scope of this discussion - traffic cameras and RFID scanning - they are still examples of interaction between the real and the virtual, and badly in need of their own ethical constraints.
At a bare minimum (which is really where this idea is positioning itself: a bare minimum for ethical coding) a system designed to capture license plates should reflect the needs of the user, which in this case, is one-half the police, one-half the same abstract "the people" that is referred to by legal prosecutors. As I said, a system should accurately reflect the intent and authority of the operator. In the case of a traffic-monitoring system, the authority of the user will never give them permission to alter existing records, because that would constitute evidence tampering.
There is also the unrelated question of to what extent the evidence from traffic cameras should be admissible in court. My own opinion? The police should need some kind of warrant to obtain it. However, I consider that a legal issue and not a technical one, so I'm not an expert.
>.> programmers don't need licenses. people don't usually die from bad programming.
I never said that they did. Are you extrapolating from something?
Thanks :).
I've just read the mefi commentary though, and it definitely doesn't sound like people are getting it. I think I'll have to refine the ideas here a bit more, post some clarifications, and maybe outline some kind of an actual plan, so that people can have an idea of what I'm proposing. (Hint: it isn't "professionalize programming", or "arrest the F4I programmers")
Indeed. I think people are getting distracted by a love of dialectic (at best). The spirit of this is captured clearly in your direct reference to the Hippocratic Oath. What's more, it seems fairly clear to me, as a reader, that this essay is intended to encourage discussion (and thus ideological exploration and fine-tuning) along these lines.
Interesting, but various other less code-centric buisness practics may have just as much impact. Not sure how the line should really be drawn.
Copyrights and patents were historicallly to force disclosure of inventions. Why not just require all source code to be published for a copyrigh to be valid? A less invasive requirment, for a less invasive profession.
That would definitely be a good start. I'm thinking more about the basis for such laws at the moment though, rather than a specific remedy.
Basis is trivial: We don't feel like giving you a monopoly via copyright without full disclosure of the source code.
You don't really want to say what people are and are not allowed to develop. One could even say that the hypocratic oath itself has been outdated by plastic surgery, x percent chance of looking better, y percent chance of looking worse, z percent chance of death. How big does z need to be to violate the oath when the patiaent was never in any risk? I'm not much of a fan of plastic surgery, but I'd definitely consider risky gene therapy or implants to make me smarter.
In general, most "modern problems" seem to trace back to a mix of
1) stupidity & ignorance - can slowly be fixed by learning about how humans make decissions (for example, read up on group think and deliberative democrasy).
2) monopoly - partially fixable here by replacing the individual tax with a *progressive* income tax on corperations, i.e. big corperations pay a higher precentage of their profits.
3) secrecy - start by not recognizing IP without complete discosure, i.e. no copyrights on music without pushished guitar tablature, no copyright on Titanic with publishing the code to draw the waves, no copyright on MS Windows without full compilable sourcecode, etc.
You don't have to expect people to be good or intelligent all the time, but you should expect them to disclose their actions, and not prevent others from taking competing actions. You don't need to trust your programmer, but you do need to trust that he can't prevent other programmers from understanding what he has done.
I still think you're making assumptions here :).
This is essentially the same position that my father takes, and I've certainly discussed this topic with him at length. Being a second-generation programmer is pretty handy sometimes.
When you say "basis is trivial" though - it may be from a philosophical perspective, but not from a PR perspective. We need to get the public thinking about this stuff. The basis isn't trivial if you think that copyright is a natural right and should be protected like property; I'm pretty sure that the general public hasn't been widely exposed to the idea that it's an artificial monopoly "to promote science and the useful arts", especially considering that there are massive advertising campaigns by media icons specifically contravening that view.
Oh, also - I don't think what you're saying necessarily conflicts with what I'm saying.
Forced disclosure of source code, although it may be a good idea for other reasons, is one way to put public pressure on software shops to not do anything nasty. What I'm putting forward here is a more formal idea of what "anything nasty" is.
One more thing: "do no harm" still applies to the plastic surgeon. There are risks involved in any medical treatment, sure, but the point is the surgeon isn't trying to hurt you, or doing anything where they know the risk/reward ratio isn't what you understand or want.
The hippocratic oath proper has certainly been outdated by a variety of things. It specifically forbids physicians from surgery of any kind, for example :). At the time, that was a barber's job.
No, "do no harm" just has an interpretation which applies to the plastic surgeon. However, he is most definitely "trying to hurt you" from some reasonable perspectives. But you've told him that you don't see it that way. This execption does not save your oath, however, as societies views of hurt often take precedence over yours. Legalized euthanasia is clearly a good thing too, but does not sit well with your oath.
What the doctor can't do is lie to you about what he will do. You could be going into a heart op. under a national health program with mandatory euthanasia for non-paying people in some limited situations. All of that seems reasonable to me, but you should be told about the small risk.
Sadly, you are correct that PR is non-trivial. But it pretty clearly the right tact to take, at least to me. I just don't like curing the symptoms while leaving the root cause ignored. Just let the symptoms fester, make everyone see it as a problem with copyright itself.
Anywho, you do eventually need a notion of "harmful" for software, in so far as it is possible, no real point in delaying figuring that out. I'm really just speeking about where activisim should focus. Forced disclosure of source code is probably achievable, eventually.
I don't get what you're trying to say. "society's views of hurt often take precedence"? Society understands the role of a plastic surgeon, and doesn't consider it harm, as the other threads here about law and surgery say. Euthenasia for non-paying people seems reasonable to you? we kill people because they're old and poor? what???
Speaking of automated traffic monitoring:
"Britain is to become the first country in the world where the movements of all vehicles on the roads are recorded. A new national surveillance system will hold the records for at least two years.
Using a network of cameras that can automatically read every passing number plate, the plan is to build a huge database of vehicle movements so that the police and security services can analyse any journey a driver has made over several years.
The network will incorporate thousands of existing CCTV cameras which are being converted to read number plates automatically night and day to provide 24/7 coverage of all motorways and main roads, as well as towns, cities, ports and petrol-station forecourts.
By next March a central database installed alongside the Police National Computer in Hendon, north London, will store the details of 35 million number-plate "reads" per day. [...] Already there are plans to extend the database by increasing the storage period to five years and by linking thousands of additional cameras so that details of up to 100 million number plates can be fed each day into the central databank."
http://www.livejournal.com/users/jwz/581641.html
I ABSOLUTELY agree with this excellently written piece. I think such a code is needed ASAP.
Trouble is, the United States is hellbent on using all our technology to massacre the world's poor and abrogate our rights to speak out against it.
I've thought of a technician labor union that would get involved in political causes and organize mass strikes. That would them in the wallet place...
You found the Software Engineering Code of Ethics, and mentioned the ACM, but you apparently did not find the ACM Code of Ethics. As a member of the ACM I am bound to uphold it (afaik the same does not apply to the SE Code of Ethics you reference), and its just as binding and similar in shape to the Code of Ethics for any of the other Engineering Professional Societies.
The biggest concern is not that it exists, its that it isn't well enough known, and that right now there aren't enough companies that know the difference between Codemonkey and Professional Software Engineer, nor that organizations like the ACM exist to help determine that distinction. That is a market failure (too many jobs for too few true Professionals) not an ethical one. In the same vein, a good University is going to have an Engineering Ethics class of some sort (we had a focused Computer Engineering Ethics Course), and again, right now companies aren't exactly choosy right now in whether or not to look specifically for an Ethics course in a Software Engineer's background.
Hope that was informative.
Get your head out of your ass. Maybe you only aim to work on the newest NVidia driver for linux, but some programmers are working on navigation software, firmware for medical hardware, and other sorts of mission critical apps. As the world becomes more computerized, the possibility -- and liklihood -- of people dying from a poorly written app is more apparent.
http://en.wikipedia.org/wiki/MIM-104_Patriot#Failure_at_Dharan - 28 deaths
and more generally
http://en.wikipedia.org/wiki/Computer_bug
Not usual occurrances, but deaths can be caused by programming.
AFAIK, one does not need ACM certification to be a computer professional in the same was as one needs a license to become a medical professional. In what way is the ACM Code of Ethics binding?
While I agree with your basic premise, I find it somewhat idealistic. In most cases, programmers are paid employees of a company where other people control what the software ultimately does. I can't imagine that the decision to add DRM to Sony's music CDs was made by a programmer, or even someone familiar with software development. It was almost certainly made by an executive who simply said "Get this done". Software companies constantly release software that is not ready to be published, with no intention of fixing the problems, not because the programmers are incompetent or unethical, but because the person who controls the ship date cares more about making that date than shipping working software.
A code of ethics for programmers is a great idea, but it doesn't address the consequences. As it stands, if I don't like what my company produces, the only choice I really have is to leave the company, and figure out some other way to pay for my mortgage. How many of us are in a position to do that the moment an assignment comes in that we disagree with?
That said, I think that there is great need for change. As a user, I am constantly disappointed by the quality of software, which has gone down in direct proportion to the price of computer hardware and software. I am tired of buying products that technically work as advertised, but are full of little bugs and silly restrictions. Executives need to learn that software isn't done just because it's time to ship it to make the quarterly revenues.
I said in the last message that what needs to happen to make it more binding on the industry as a whole is wholly reliant on market forces: if companies expect to see their employees bound by a professional code of ethics, they need to start placing ACM membership as a requirement to their position. That's how licenses come about, they are wanted by either 1) enough employers, or 2) enough legislators.
There are businesses that require IEEE membership for their Electrical Engineers. There are businesses (mostly for Civil Engineers) that require the Professional Engineering certification exam.
Just because businesses, right now, in Computer/Software Engineering are not requiring such things does not mean that there is no Code of Ethics nor that there are no ethical Professionals. All it means is that right now to Businesses, and to our Legislators, there isn't enough reason to demand that their Computer/Software Engineers be a member of their Professional society nor be bound to the Code of Ethics of said society.
I think that is the difference that tells an etical person from an unethical one, and thats why we need ethics on our professional. Sometimes I tell my cooworkers we are worse than whores, because the good ladies at least deliver to the user what he wanted at first place.
We don't do that, we are inescrupulous mercenaries, we excuse ourselves by shifting the blame to the ones making financial decisions, but we are as responsible for our failures as anyone else.
When a doctor is asked to cut of his patient just because he is a rober does he do so? If he does, is it well looked upon by the medical society or his peers? I don't think so. Because they abide to an ethical standard.
We don't abide(as a group) to any ethical standard, and each professional goes for what he thinks is right or wrong. To add injury to damage some ill conceived ideas have entered and installed in our day to day thinking like plague. For example "the the good is the excelence enemies" or "nobody dies due to computer programs" and such. Some professional recite them like mantras, throwing even more darkness on our profession profile.
Of course we lack legal support to refuse to do certain jobs or things... Yet it is the responsibility of each one what each one does. And by accepting to do anything you are agreeing to what is being done.
Very well put, and quite true.
FWIW: I learned of your blog through www.userfriendly.org so your voice has been heard by a great number of people in a position who are in the industry.
One problem I see arising from this is when one particular piece of software (or other product for that matter) has several users with different goals.
In the following I'm giving examples and trying to be objective so please don't extrapolate from it to try and guess my meaning - cause we all know (hopefully) that extrapolation can lead to extremely bad conclusions ;-)
Is the user of Sony's copy protection software Sony or the person who listens to the music on her/his computer?
Is the user of a car tracking module (can't remember the exact name?) the owner of the car? The driver of the car? The car thief who drives the car after stealing it? The police trying to track the car? The non-licensed entity using the signals emitted from the tracking device to gain information about the car's position?
I don't see any easy and simple code to adhere to because the concept of "good engineering practise" is either too vague or far too detailed to be applied in practise. If it's too vague you'll soon find yourself in an uncovered grey area. If it's too detailed you'll soon find yourself outside its defined "world".
Anyways, just my 2 cents and while they're Euro cents I don't really believe they're worth more than anyone else's cents. The opposite is more likely the case ;-)
and guess my meaning
Erm, make that "and guess my opinion"
http://en.wikipedia.org/wiki/Therac-25
People burned, and in at least five cases to death, by bad software engineering.
Common? No. But possible. Even writing stupid business software gives you the chance to negatively impact untold numbers of people's lives if you screw up (e.g. mis-reporting the financial performance of a company, leading to loss in investor confidence, leading to poorer results, leading to layoffs...), let alone the more critical things enumerated above.
I lecture a module in Information Systems Practice.
It's an excellent post for opening up a discussion in one of my classes. I hope you don't mind me emailing this post to my students.
Not at all! In fact, I'd be honored.
I think you have to split this into two halves.
1. I will never program a computer to disobey its owner.
2. I will never program a computer to harm its user.
Private property solves the complications. A computer must obey its owner, no question about that, but must only avoid harming its users. You aren't obliged to help them. This gets around the questions like: what if I want not to let the users of my online service upload porn. You're allowed to write code to refuse the upload, but you're forbidden to write code that reports them to the police.
I am a graduate student in Information Studies at UT-Austin. Recently, the field has started to shift from strictly books (i.e. Librarianship) to Digital Media. I have been, as a young student, very vocal about the need to start implementing a code of Ethics in ALL disciplines. The simple fact is, we have been training people to just do jobs but have not, at all, made any attempt to make them think on a philosophical, moral, or ethic level. What comes from this is a an army of workers with no regard for the impact of what they do. To them, the ends justify the means. I am SO relieved to see a person willing to say "we need a return to ethics".
Anybody who takes the time to read and understand the Debian software distribution system documents ("Social Contract", "Policy and Procedures") would find that we have already been considering it our sacred duty for at least a decade and arguably much longer. It is more a question of education and popularization but there are already many thousands of people signed up in the cryptographically secure, grassroots Debian Web of Trust that is possible via the public cryptography we use in our procedures.
http://debian.org/
1. Other than an ethic to “obey the law”, there is no need for computer professionals to hold any other ethical values associated with their knowledge and skills in computer and Information Technology.
If anyone could help shed some light on this question, i would really appreciate it :). I really enjoyed the essay as I am having to do a unit on Ethics in IT and Multimedia and it has certainly opened up a lot of questions.
The ACM code of ethics is equally problematic.
First, it's not a code of professional ethics. It's just a code of ethics, with a few offhand mentions of professionalism. 95% of it is completely generic - and if I might add, somewhat banal. For example: "Strive to achieve the highest quality, effectiveness and dignity in both the process and products of professional work.", "Honor contracts, agreements, and assigned responsibilities.", "Articulate social responsibilities of members of an organizational unit and encourage full acceptance of those responsibilities.", "Be honest and trustworthy.", and most tellingly, "Contribute to society and human well-being.".
Of course everyone wants to contribute to "human well-being", but everyone has a wildly different idea of what that is. The sort of professional ethics that I'm looking for are a more specific and narrowly-defined set of rules for how programmers (and software) should interact with society: i.e. that software designers should not consider themselves to have the freedom to contribute to society and human well-being in any way that they see fit. Doctors, for example, are obliged by their professional ethics to allow patients to refuse treatment even if the doctor knows that the patient will die without it.
The hard part of dealing with codes of ethics is understanding and dealing with the inherent conflicts they represent. Sometimes being a good citizen demands being a bad employee, or being a good employee means being a bad parent. Codes which try to bundle in "be a good overall person" with specifics of professional conduct are attempting to dodge these inherent conflicts and end up, in my opinion, providing very little value.
Greetings. I found your essay while googling for software developer ethics and conduct, and I thank you for writing it. Although it's been a while since you wrote it, not much has changed, sadly.
One aspect of a code of conduct I would more directly address is the ability of individuals within the profession to be able to say they have a call to standards higher than their employer's wishes when they are asked to do something unethical. In the same sense that doctors, lawyers, and many other professions can appeal to their responsibility to their licensing organizations if they are asked to do something unethical, programmers need the same safety net. Right now, if you or I try to respond to our employer's unethical request with reasons to refuse, they are completely able to terminate our employment and replace us with someone who will implement their wishes with no consequences, real or threatened, at all.
When broken or malicious software gets shipped, it might be the incompetence of programmers, but how often is it the demands of the employer to cut corners, ignore possible consequences, and meet the deadline that result in bad software? Programmers have little to stand on when the ethical requirement to do the right thing comes up against the employers desire to wring profits out of small margins arises.
This situation will continue until we, like a lawyer asked to violate client the attorney/client privilege, can firmly stand up for what's right.
Post a Comment