I am occasioned to write this essay by a press release from the Entertainment Software Association dated August 26th, 2003. This press release carries the headline:
"GAME PLAYERS ARE A MORE DIVERSE GENDER, AGE, AND SOCIO-ECONOMIC GROUP THAN EVER, ACCORDING TO NEW POLL"
It goes on to explain that the new poll was carried out by Peter D. Hart Research Associates, Inc, and includes a number of startling discoveries, such as:
"...a full 17% of game players are over age 50..."
"Women age 18 and older now make up a larger percentage of the gaming population - 26% -- than boys ages 6 to 17, who represent 21% of gamers."
"The average age of game players is now 29 years old."
We’ve been seeing studies like this for the last ten years or so. Their common message is always the same: videogames are played by mature adults, upstanding citizens, responsible people. It won’t be long before we see a commercial with a chap in black robes announcing, "I’m not really a Supreme Court judge, but I play one on television, and I love to play Death Orgy 2003 with all my other serious, responsible, respectable friends."
What’s really there
Let us look more closely at the professionally obtained poll. I sent an email to the Entertainment Software Association requesting the questions that were asked; I received a very polite letter explaining that they could not release that information. They published the results of the poll -- according to their interpretation -- but they did not publish their methods or their sources.
This violates one of the fundamental rules of science. To be reliable, any measurement or experiment must be repeatable. In other words, the rules of science require the experimenter to publish his methods as well as his results, so that any other scientist can repeat the experiment. If you don’t publish the method, it ain’t science. And the ESA poll refused to divulge its method. It is impossible for anybody else to repeat the experiment to determine if the results are reliable.
This leads us to a surprising conclusion: the ESA results are unscientific. We can’t place any confidence in them because we have no idea what they really mean. It doesn’t matter how representative the sample is or how large it is or even if the researchers wore white lab coats. If you don’t publish your methods, your work simply doesn’t count as science. That doesn’t make their poll wrong or nasty or incorrect or evil -- it means only that we can’t give the same credence that we would give a truly scientific study.
Here’s an example of how the unscientific nature of the poll ruins the value of the results. The press release claims that "a full 17% of game players are over age 50". That may seem clear enough, but how exactly do they define the term "game player"? If somebody has ever played a game at any time in their life, does that make them a "game player"? For all we know, that’s how they defined the term -- in which case their data is certainly believable, but it doesn’t tell us much.
Perhaps they weren’t so devious in their definition. Perhaps they define a "game player" as "anybody who has played any electronic game in the last year". That’s a more reasonable definition -- but it still includes anybody who has played a quick game of solitaire.
Of course, some could argue that such a definition is perfectly appropriate, and I wouldn’t want to argue against such a definition. But we’d still want to be honest about our results, and make it clear what they mean. If we want to say that 17% of all the people who played a quick game of solitaire in the last year were over age 50, that’s fine with me. But it would be deceptive to lump such people into the same group with the die-hard videogamers as if there were no difference between the occasional game of solitaire and the daily dose of death and destruction.
And the most important problem is that we simply don’t know what was going on with the study. All my speculation about the study is just that: utter speculation. The truth is, they could be counting vegetative mental patients as game players and we’d have no way of knowing. We simply don’t know what they mean by "game player". So we’re left with an utterly useless study -- at least, in terms of scientific truth. As a marketing device, it was probably quite effective.
My own poll
What really bothered me, though, is that the results clash with my own experience. Despite the breathless claims of the surveys, I don’t know ANY grandmothers who play Quake Deathmatch three hours a day. In fact, I don’t know many people (outside of my professional colleagues) who play games. So I decided to carry out my own little survey.
I didn’t carry out a typical survey with a carefully selected, representative group of respondents; I simply asked some of my friends and acquaintances. My selection rule was simple: if I had a few moments of spare time with somebody, and felt a close enough acquaintance to ask personal questions, then they were included in the sample. This would not yield anything like a representative sample of Americans; it would yield only a representative sample of my friends and acquaintances.
The average age of my respondents was 46 -- that’s nowhere near representative! My sample included nobody under 10, just two teenagers, and only three people in their 20s. The other 20 respondents were mostly in their 40s and 50s -- which is what you’d expect for friends and acquaintances of a 53 year old. But it certainly isn’t representative.
My sample is also unrepresentative in that most of my respondents are educated, middle to upper middle class, and have computers. This biases my data in favor of the "everybody plays games" hypothesis.
So here’s the crucial item: how many "game players" are there in the group? That depends on what the definition of "game player" is. Here are the counts and percentages of game players depending on a variety of possible definitions:
Anybody who has played a game in the last ten years: 24, or 96%
Anybody who has played a game in the last year: 18, or 72%
Anybody who has played a "heavy" game: 9, or 36%
Anybody who has purchased a game in the last year: 5, or 20%
The point of this is not to assert that 20% or 36% or 72% of all adults are game players; the point is that the very notion of game player can be twisted around any which way by rigging the definition or the question asked. I didn’t ask this question, (and perhaps I should have), but if I defined "game player" to be someone who plays heavy games at least one hour a week, I’m sure that I could have gotten even lower figures. It’s all in the way you load the dice.
But wait! There’s more! I wanted to put some sort of solid comparison on my results, so I asked the questions about paying money for movies and books. These results are best represented by the median values of the elapsed time since last purchase:
Games: never
Movies: 7 days
Books: 10 days
Thus, people may play games, but when it comes to actually purchasing them, games don’t even show up on the scale with movies and books; they’re a complete non-entity.
A different sample
As an additional check on my results, I then distributed the same questions to my mailing list for interactive storytelling; it was forwarded to another mailing list of people interested in interactive entertainment. I got 17 responses. Here are the same results for the different definitions of "game player”:
Anybody who has played a game in the last ten years: 17, or 100%
Anybody who has played a game in the last year: 16, or 94%
Anybody who has played a "heavy" game: 14, or 82%
Anybody who has purchased a game in the last year: 13, or 76%
Wow! This makes it look as if almost everybody is a game player -- which is a major point to consider. By selecting a group of people who are interested in interactive entertainment, I came up with a completely different conclusion. This could be treated as an object lesson in the hazards of poor sampling. But there’s another lesson here, which I’ll get to later.
Here’s the data on median elapsed times since last purchase:
Games: 30 days
Movies: 7 days
Books: 21 days
Interestingly, even though this group was clearly heavy on game players, games still came in third place in recency of purchase. One could argue, of course, that games are more expensive than movies and so represent a larger, rarer investment. That’s a complicating factor that deserves consideration. But the data certainly suggest that games play a smaller role in the entertainment budgets of most people. Particularly interesting is the fact that books outperform games, something that most die-hard gamers will find impossible to believe -- which only goes to show how out-of-touch most die-hard gamers are.
It is tempting to dismiss these results as utterly unreliable because they were obtained through such an amateurish poll. But before we dismiss these results out of hand, consider: this poll, while obviously flawed, is scientifically valid. Anybody who doesn’t like my results is welcome to repeat the experiment and present their results. The difference between my poll and the ESA’s poll is that the ESA results deserve no credence whatsoever, and mine deserve a tiny shred of credence. What we’d really like to see is a well-defined poll of a properly representative group of respondents and transparent methods. The ESA poll has half of this, and my poll has the other half, but neither poll has both.
Why?
There’s another consideration here: why are such polls so common in the games biz? It’s not just a marketing slant; everybody in the games industry passionately believes this nonsense. Question this dogma and you’ll be regaled with anecdotes of unlikely friends and relatives who are avid game players:
"My 83 year old grandmother uses a walker, but she still has to have her daily game of If It Moves, Shoot It!"
"Oh yeah? Did you know that one of the world’s top ten players of Death To Everybody is a Carmelite nun?"
"There’s a Nobel Laureate nuclear physicist who owns 63 X-Box games."
And so on. There’s much more going on here than disagreements about demographics; there’s an underlying and not-so-hidden agenda. Games people are smarting about the tawdry, unhealthy image that their industry suffers. The Columbine High School shootings really stoked the critics. Videogames made ’em do it! Sure, that’s simplistic and likely wrong, but it struck a chord with the public. Many people regard games as unhealthy, addictive, morally tawdry, and generally distasteful. One of the respondents to my poll answered the question, "Have you ever played a ’heavy’ computer game?" with "Certainly NOT!" She didn’t want to be associated with that trash. People are willing to acquiesce to kids playing games, the same way that they acquiesce to body piercing or purple hair. They figure it’s a phase, and in fact most kids grow out of it.
That’s the reality that the games people are trying to hide from. In their search for respectability, they will seize upon any anecdote, doctor any statistic, and minimize any truth. It’s understandable; how would you feel if your job was regarded as one notch above pornographer? But in refusing to confront the reality, games people only prolong their agony. You don’t solve problems by denying them.
What to do?
What’s especially sad about this is that the solution to these problems is obvious: start making respectable games. There is absolutely no technical or design reason why games have to be so tawdry. Sure, the primary market of adolescent males much prefers the violent rebellious stuff, and the industry can continue to make billions supplying this market. But the games industry needs to learn a lesson that the movie industry and the book industry learned long ago: you gotta push the noble fringe. Every year there are dozens of high-minded movies that get funding. Most lose money, but a few break even. Hollywood is willing to fund these efforts for three reasons:
1. They support the image of cinema as an art form worthy of respect. Sure, Hollywood makes tawdry movies; moviemakers are certainly not above showing some skin to boost sales. But while there are plenty of individually sleazy movies, Hollywood’s output taken as a whole is a healthily heterogeneous collection. For every Betty Bigtits there’s a Mickey Mouse; Koyaanisqatsi played just down the road from Debbie Does Dallas.
2. They provide a steady stream of new ideas and new talent. Think of them as the minor leagues where oddball ideas and people get their chance at bat. The games biz has nothing like this; we have hobbyists who slap together amateur efforts, and we have big-time people who develop the mainstream products. There’s nothing in between, which is one reason why the games industry is so creatively stagnant.
3. They occasionally hit a home run. Movies like Pulp Fiction and Blair Witch Project are perfect examples of minor-league movies that hit the bigtime. These movies are fantastically profitable; their return on investment greatly exceeds any of the mainstream movies. It’s rather like venture capitalism in the high-tech biz -- you lose money on most of your investments, but the few that make money more than make up for the losers. Again, the games biz has nothing like this.
The book publishing industry does exactly the same thing; indeed, they are even more venturesome than Hollywood. How do you think that unknown writers like Tom Clancy or J.K. Rowlings get started? Some editor somewhere decides to take a chance. Book publishers take a lot more chances than games publishers, and they are occasionally rewarded for their efforts. In the end, they make money too.
Will it happen?
Probably not. The most likely response to this essay will be a collection of denunciations and diatribes. Most of the responses will boil down to the following:
"Chris, you have some good ideas but your conclusions are extremist. You shouldn’t be so narrow-mindedly driven by facts and principles; you should balance your conclusions by averaging them with mainstream opinion."
"The ESA poll was better than your poll, so we can dismiss your conclusions."
"Your own data prove that games are popular, if you analyze it like so..."
"Your poll wasn’t scientific, so we can dismiss your conclusions."
"All my friends play games, so there!"
"You’re just a big poop-head!"
These responses are most unlikely:
"While your analysis has many flaws, the overall gist seems worthy."
"You’re right: the games industry really has to undergo substantial change."
"We should undertake a really honest study of consumer attitudes towards games."
So why do I bother?
Because 50 years from now, historians will have a crucial counterpoint to understand why the industry had its head in the ground.
Postscript: June 5th, 2004
I recently received an email from Dr. Kim Thompson at Harvard directing me to her own studies on videogames:
http://www.kidsrisk.harvard.edu/faqs4.htm
http://www.kidsrisk.harvard.edu/faqs5.htm.
Now THIS is the way studies should be done! Everything is laid out cleanly for anybody to see and evaluate. The authors define EXACTLY what they mean by each of their terms. For example, here is their definition of violence:
"We defined violence as intentional acts in which the aggressor causes or attempts to cause physical injury or death to another character. We did not include actions that led to unintentional physical harm, the effects of natural disasters, or the presence of dangerous obstacles not attributed to another character. We also did not include intentional acts of physical force that represent normal play in a sports game (e.g., tacking in football), but we counted all punches and kicks in boxing and wrestling games are violence because the intention in these sports is to cause injury. We noted whether each game contained violence and whether the game rewarded or required the player to injure or kill characters. To quantify the amount of violence in each game, we coded whether each second of recorded game play contained acts of violence and then calculated the percentage of violent game play. "
And here’s the quote that really grabbed me:
"We observed a total of 11,499 character deaths in the 81 games, occurring at an average rate of 122 deaths per hour of game play time. This total included 5,689 human deaths, occurring at an average rate of 61 human deaths per hour of game play time."
To put this in perspective, a boy who plays videogames for an average of ten hours a week will over the course of his junior high school and high school years experience some 36,000 deaths. It would take only about 200 such kids to generate the same body count (virtual, of course) as the Holocaust. How’s that for a sensationalized representation? Here’s an even more sensationalized way to say it: if there are at any given moment 10 million kids around the world playing such games, then they are generating approximately 1 billion virtual deaths every hour. Isn’t that sweet?