A few years ago I was talking to Ying about my aspirations to one day develop my own game (Iä! Divunal! May its slumber soon be ended!). I was telling her about its design — naïve student of interactive fiction that I was, I had decided that there would be two salient features: permanent death, and "no numbers anywhere". Everything would be relayed to the player by way of descriptive phrases, because that was, like, more real, man.
Never one to pull any punches on my account, Ying told me she didn't think it would be any fun. I asked her why she thought this game — which, if I could pull it off, would be artful, like a reading a novel that was written for you every day — would not be fun, whereas a grindy stat-monster like DragonRealms was fun. She said:
Now, I am obsessed with the power of numbers that go up. It's not just a trick for game designers. It's a basic part of the human condition. A power so great it can be used only for good, or evil. A tool for positive social change. A force which, every day, keeps little babies from dying.
You think that last bit was hyperbole, right? Wrong. Doctor Virginia Apgar developed the Apgar score, which is a way of rating how healthy a newborn baby is. The development of the score itself, not any particular technique for improving the score, was responsible for drastically reducing infant mortality. (I believe one of the largest drops in recorded history, but I can't find an online citation for that.)
Lest you think this had something to do with the culture around video games, she did this in 1952.
Many people have observed that there is also a dark side to this phenomenon. In 1987, Alfie Kohn famously wrote an article that is now distributed with every copy of emacs which notes that giving people incentives gets them to focus on the incentive rather than the task; and enjoy the incentive more than the task. In 2000, Joel Spolsky wrote "Incentive Pay Considered Harmful", which details the numerous problems with HR reviews and employee incentive pay. Just this month, he expounded again in "How Hard Could It Be?", noting that when you pay people to optimize something, they will optimize it, whether that helps you or not.
I don't believe there's a contradiction here. What these studies are observing is that, if you crudely design and crudely present an incentive, it will have crude effects. There's an art and a science to designing incentives, and the people who write employee incentive plans (and incentive impact studies) are not really using interested in using incentives in the way that makes them effective: to make activities more fun. The only people who really get this right are game designers. Unfortunately, the insight that game designers have is rarely shared with other disciplines. Game studios make even software startups look tame by comparison, leaving their employees little time for professional development, or, you know, sleeping. When their ideas are shared, they are frequently, almost implicitly dismissed — after all, it's "just a game". The "serious" folks want measurable objectives, clearer research, not "fun".
But tell a "serious" cognitive scientist or a "businesslike" incentive plan designer to produce a scheme which will cause the user's brain to release large amounts of Dopamine on demand, and they're not likely to deliver anything useful.
I certainly don't have as much experience with this as I'd like, but I've made plenty of observations. My personal hypothesis is that the key factor here is subtlety. The apgar score is an arbitrary number. It means nothing beyond what it means. Your progress, or health in a game has no meaning beyond the game. It's an obvious yardstick by which you can judge your progress, but it doesn't really matter.
Stack Overflow is, I think, a great example of this type of subtlety. Your reputation is arbitrary, and there are lots of arbitrary little landmarks you can achieve. "Badges", originally from City of Heroes, and also known as "achievements" on Steam, are a great way to motivate users to stick around just a while longer. "Oh, I'm done for the day, but I'm only 30 votes from civic duty. Let me vote on a few more things." Because the motivation is there, you stick around; but because it's subtle, it's not worth aggressively gaming the system (and thereby wrecking it).
You can see the flip side of that pretty quickly on similar sites that try to motivate participation based on money. I can't tell how good Experts Exchange is, because it throws up roadblocks, to protect their precious "content" and make sure they can make money on it. Those who I know who have tried it assure me that it is full of spam and fraud, largely because the incentive structure is all based around money. The "score" doesn't represent progress or mark status, it is progress and it is status. The point of Experts Exchange is to get money, and the questions are just there to provide you with a mechanism to do so. The point of Stack Overflow is to provide good answers to questions, and the reputation score is just there to get a rough idea of who does that best.
I think that this principle can be applied in lots of places in everyday life. To give you a hint of where I hope to apply it at Divmod, in Blendix: consider the feeling of getting a point in a game, and the feeling of checking off an item on a to-do list. Compare beating a level to seeing a page full of "done" items. Just imagine that page full of ticked-off checkmarks. Makes you want to write a to-do list, doesn't it?
Never one to pull any punches on my account, Ying told me she didn't think it would be any fun. I asked her why she thought this game — which, if I could pull it off, would be artful, like a reading a novel that was written for you every day — would not be fun, whereas a grindy stat-monster like DragonRealms was fun. She said:
People like numbers that go up.This is a phrase which I have now both said and heard in countless conversations with professional game designers. At the time it struck me as insightful, but I didn't realize how insightful it was for many years.
Now, I am obsessed with the power of numbers that go up. It's not just a trick for game designers. It's a basic part of the human condition. A power so great it can be used only for good, or evil. A tool for positive social change. A force which, every day, keeps little babies from dying.
You think that last bit was hyperbole, right? Wrong. Doctor Virginia Apgar developed the Apgar score, which is a way of rating how healthy a newborn baby is. The development of the score itself, not any particular technique for improving the score, was responsible for drastically reducing infant mortality. (I believe one of the largest drops in recorded history, but I can't find an online citation for that.)
Lest you think this had something to do with the culture around video games, she did this in 1952.
Many people have observed that there is also a dark side to this phenomenon. In 1987, Alfie Kohn famously wrote an article that is now distributed with every copy of emacs which notes that giving people incentives gets them to focus on the incentive rather than the task; and enjoy the incentive more than the task. In 2000, Joel Spolsky wrote "Incentive Pay Considered Harmful", which details the numerous problems with HR reviews and employee incentive pay. Just this month, he expounded again in "How Hard Could It Be?", noting that when you pay people to optimize something, they will optimize it, whether that helps you or not.
I don't believe there's a contradiction here. What these studies are observing is that, if you crudely design and crudely present an incentive, it will have crude effects. There's an art and a science to designing incentives, and the people who write employee incentive plans (and incentive impact studies) are not really using interested in using incentives in the way that makes them effective: to make activities more fun. The only people who really get this right are game designers. Unfortunately, the insight that game designers have is rarely shared with other disciplines. Game studios make even software startups look tame by comparison, leaving their employees little time for professional development, or, you know, sleeping. When their ideas are shared, they are frequently, almost implicitly dismissed — after all, it's "just a game". The "serious" folks want measurable objectives, clearer research, not "fun".
But tell a "serious" cognitive scientist or a "businesslike" incentive plan designer to produce a scheme which will cause the user's brain to release large amounts of Dopamine on demand, and they're not likely to deliver anything useful.
I certainly don't have as much experience with this as I'd like, but I've made plenty of observations. My personal hypothesis is that the key factor here is subtlety. The apgar score is an arbitrary number. It means nothing beyond what it means. Your progress, or health in a game has no meaning beyond the game. It's an obvious yardstick by which you can judge your progress, but it doesn't really matter.
Stack Overflow is, I think, a great example of this type of subtlety. Your reputation is arbitrary, and there are lots of arbitrary little landmarks you can achieve. "Badges", originally from City of Heroes, and also known as "achievements" on Steam, are a great way to motivate users to stick around just a while longer. "Oh, I'm done for the day, but I'm only 30 votes from civic duty. Let me vote on a few more things." Because the motivation is there, you stick around; but because it's subtle, it's not worth aggressively gaming the system (and thereby wrecking it).
You can see the flip side of that pretty quickly on similar sites that try to motivate participation based on money. I can't tell how good Experts Exchange is, because it throws up roadblocks, to protect their precious "content" and make sure they can make money on it. Those who I know who have tried it assure me that it is full of spam and fraud, largely because the incentive structure is all based around money. The "score" doesn't represent progress or mark status, it is progress and it is status. The point of Experts Exchange is to get money, and the questions are just there to provide you with a mechanism to do so. The point of Stack Overflow is to provide good answers to questions, and the reputation score is just there to get a rough idea of who does that best.
I think that this principle can be applied in lots of places in everyday life. To give you a hint of where I hope to apply it at Divmod, in Blendix: consider the feeling of getting a point in a game, and the feeling of checking off an item on a to-do list. Compare beating a level to seeing a page full of "done" items. Just imagine that page full of ticked-off checkmarks. Makes you want to write a to-do list, doesn't it?
Posts
6 comments:
> Just imagine that page full of ticked-off checkmarks. Makes you want to write a to-do list, doesn't it?
Absolutely not. Maybe it's just me, but I take no satisfaction in completion.
I take satisfaction in exploration, in acquiring insight, and in unfolding a plan. In real life, this part is at most 50% of the work. The major part of any work is productization: going from "I did the hard part" to "I delivered a solution".
A fully ticked off todo list means a lot of tedious and not-so-interesting stuff was done. That is essential to delivering value, but it's not fun.
There's starting to be a bunch of papers on the best way to design incentives for Mechanical Turk workers to complete tasks in a way useful to experimenters. (Computational linguists are perpetually starved for human linguistic intuitions to either feed into or evaluate computational results. Those with access to US bank accounts are starting to use Mechanical Turk for below-minimum-age annotation.)
It's more about making cheating time-consuming than making annotating fun though.
You can also show people the rate at which the number is going up. They like that.
The second derivative (that IS what the "rate of the rate" is called, right?) is even better if you can convey it's meaning.
> You can also show people the rate at which the number is going up. They like that.
That's actually what I did like about DragonRealms - it showed you the rate at which you were gaining points as well.
Reading your post reminded me of an idea I meant to post to Project 10^100. Sadly having now submitted to it, I can't link to the submission. I'll put part of it in here:
"Education is most interesting when it involves *doing* things, actively creating, and when learning is in response to personal motivations. I propose building online communities centered around specific domains (creative writing, biology, carpentry, etc). In the context of a community students would "level-up", increasing rank as they gain mastery of the subject. The leveling up is specifically targetted at an almost irrational desire to progress. That mastery would be shown through specific tasks, and the successful completion of those tasks would give points (XP). The tasks could be rote (solving a math problem) or more personal (identify a tree in your yard, upload a picture of it and its name). Your solutions would be judged by your peers (typically people with a higher rank than you), and if you make mistakes your peers would both identify the problems and help you through the task. There would be no effort to automate the judgement, and so completion of a task would always be accompanied with acknowledgement from a real person.
"Participation in judging other student's tasks, and assisting them when necessary, would itself be a requirement for movement further up in ranks. Also students would be able to create new tasks for their peers. Children of different ages would be encouraged to teach each other, as well as adult mentors. Most aspects of the site could be transparent -- tasks completed, help provided, discussion of tasks, etc."
This idea came to me when I was thinking of how motivating leveling up is, even when it's stupid. I think this actually was best clarified to me by some essay I saw that Lore Sjöberg wrote, though I can't find it now since he's been doing game reviews. Anyway, the idea was intended to direct that need for leveling up toward real education, as opposed to the typical crap that is computer education and educational game scoring.
Ian:
Yes. Exactly. I have a deep and abiding interest in education as it intersects with this idea.
I think that if you can structure the rewards properly to motivate people, but also be transparent about them (so that people know what the numbers they're seeing really mean), you can motivate participation in lots of different fields where their interest would otherwise have fizzled out.
Post a Comment