2013/05/16

What do tests test for?

What is the point of a test? To show what the tester can do? To create a barrier? To ensure competence in the tested group? In 'Why I Let My Students Cheat On Their Game Theory Exam' the author shows us a novel way to test students and the surprising results.


An entire field of study, Game Theory, is devoted to mathematically describing the games that nature plays. Games can determine why ant colonies do what they do, how viruses evolve to exploit hosts, or how human societies organize and function.

Tests are really just measures of how the Education Game is proceeding. Professors test to measure their success at teaching, and students take tests in order to get a good grade. Might these goals be maximized simultaneously? What if I let the students write their own rules for the test-taking game? Allow them to do everything we would normally call cheating?

A week before the test, I told my class that the Game Theory exam would be insanely hard—far harder than any that had established my rep as a hard prof. But as recompense, for this one time only, students could cheat. They could bring and use anything or anyone they liked, including animal behavior experts. … They could surf the Web. They could talk to each other or call friends who’d taken the course before. They could offer me bribes. (I wouldn’t take them, but neither would I report it to the dean.) Only violations of state or federal criminal law such as kidnapping my dog, blackmail, or threats of violence were out of bounds.

Gasps filled the room. The students sputtered. They fretted. This must be a joke. I couldn’t possibly mean it. What, they asked, is the catch? “None,” I replied. “You are UCLA students. The brightest of the bright. Let’s see what you can accomplish when you have no restrictions and the only thing that matters is getting the best answer possible.”

Once the shock wore off, they got sophisticated. In discussion section, they speculated, organized, and plotted. What would be the test’s payoff matrix? Would cooperation be rewarded or counter-productive? Would a large group work better, or smaller subgroups with specified tasks? What about “scroungers” who didn’t study but were planning to parasitize everyone else’s hard work? How much reciprocity would be demanded in order to share benefits? Was the test going to play out like a dog-eat-dog Hunger Games? In short, the students spent the entire week living Game Theory. It transformed a class where many did not even speak to each other into a coherent whole focused on a single task—beating their crazy professor’s nefarious scheme.

On the day of the hour-long test they faced a single question: “If evolution through natural selection is a game, what are the players, teams, rules, objectives, and outcomes?” One student immediately ran to the chalkboard, and she began to organize the outputs for each question section. The class divided tasks. They debated. They worked on hypotheses. Weak ones were rejected, promising ones were developed. Supportive evidence was added. A schedule was established for writing the consensus answers.

As the test progressed, the majority (whom I shall call the “Mob”) decided to share one set of answers. Individuals within the Mob took turns writing paragraphs, and they all signed an author sheet to share the common grade. Three out of the 27 students opted out (I’ll call them the “Lone Wolves”). Although the Wolves listened and contributed to discussions, they preferred their individual variants over the Mob’s joint answer.

In the end, the students learned what social insects like ants and termites have known for hundreds of millions of years. To win at some games, cooperation is better than competition. Unity that arises through a diversity of opinion is stronger than any solitary competitor.

The shared grade for the Mob was 20 percent higher than the averages on my previous, more normal, midterms. Among the Lone Wolves, one scored higher than the Mob, one about the same, and one scored lower.

Is the take-home message, then, that cheating is good? Well … no. Although by conventional test-taking rules, the students were cheating, they actually weren’t in this case. Instead, they were changing their goal in the Education Game from “Get a higher grade than my classmates” to “Get to the best answer.” This also required them to make new rules for test-taking. Obviously, when you make the rules there is no reason to cheat. Furthermore, being the rule-makers let students behave in a way that makes us a quintessentially unique species. We recognize when we are in a game, and more so than just playing along, we always try to bend the rules to our advantage.

Morally, of course, games can be tricky. Theory predicts that outcomes are often not to the betterment of the group or society. Nevertheless, this case had an interesting result. When the students got carte blanche to set the rules, altruism and cooperation won the day. How unlike a “normal” test where all students are solitary competitors, and teachers guard against any cheating! What my class showed was a very “human” trait: the ability to align what is “good for me” with what is “good for all” within the evolutionary games of our choosing.

In the end, the students achieved their goal: They earned an excellent grade. I also achieved my goal: I got them to spend a week thinking like behavioral ecologists.

The best tests will not only find out what students know but also stimulate thinking in novel ways. This is much more than regurgitating memorized facts. The test itself becomes a learning experience—where the very act of taking it leads to a deeper understanding of the subject.