Exam security

Canterbury takes its invigilated exams pretty seriously.

Students in our big 100-level exams have to leave all of their materials at the front of the room save a pencil, pen, ruler or non-programmable calculator (as needed for the exam). Hats have to be left at the front of the room. We arrange students through the room in a checkerboard; we know where each student is seated so we can check scripts later on if we suspect somebody's copied from a neighbour. Two invigilators stalk the room throughout. We leave a blank row after every second row of students so the invigilators can get up to any student who's asking a question, or who is peeking a bit too often up a shirt sleeve.

What are the key features of this set-up?

• Students can't tell where they'll be seated before entering the room. They'll likely be seated near somebody with an alphabetically close last name, but they can't tell where in the room they'll be.
• Students can't easily smuggle answers in; the only materials they're supposed to have in front of them are the ones they have to have, though we'll usually allow a water or drink bottle.
• Students don't have access to the blank exam booklets, which vary colour from year to year and, occasionally, by whatever the lecturer happens to have had in his older stock. It's not always easy to predict what the colour of the exam booklet cover will be, so it's harder to smuggle in one that has notes inside.
• Invigilators can approach any student from an oblique angle; as they can be behind a student, it's hard for a student to tell when he's being watched.

Turns out that those measures would thwart most of the exam hacks that students taking this government cybersecurity course tried when they were instructed to try to cheat on the exam. The quiz asked them to write the first hundred digits of pi. Some of the hacks were very nice, but students didn't have to try all that hard given the reasonably lax exam setting.

 

Here's the working paper: Embracing the Kobayashi Maru. What did the students try?

• Variants of encoding the answer onto materials they were allowed to bring into the exam with them: textbooks; the back of name tags that always sit on desks; notebooks; course schedules; post-it notes; food; coffee cup sleeves; random materials that might be in a pencil case; hidden in a watch bezel; bringing in a pre-completed answer sheet hidden among other papers.
• Variants of hacking the exam room: writing on ceiling tiles; writing the answer on sheets of paper in the pile that the invigilators were likely to distribute on request for blank papers; hiding the answer within the computers on the desks;
• Hacking the grader's or invigilator's laziness: Memorize the first ten digits, then provide random numbers.

Most of these won't work given our processes. To what sorts of attacks could we still be vulnerable? Security through obscurity is a pretty poor solution and I'm sure our students are more creative than I am anyway. So here are a few potential options:

• Notes written on or around drink bottles or coffee cup sleeves;
• Notes hidden on one's person and accessed in the privacy of a toilet stall, especially if you've drawn the lucky straw and both invigilators are of not-your-gender;
• Small notes written on erasers or taped inside the sleeve of a programmable calculator;
• Small notes hidden inside pencil cases if the invigilator isn't strict enough on the "pencils and pens only" rule;
• A note hidden under a pony-tail or in dreadlocks.

Options are near limitless for variants on open book or open notes exams; fair systems would let students either bring in anything (completely open book) or restrict students to bringing in only one or two sheets of paper of specified size. Otherwise you're just encouraging students to encode notes on the cover or in the text of the book.

Perhaps I should include a "Please cheat on this question" question in the midterms next semester. Academic misconduct and a visit to the proctor for cheating on other questions, but only a zero on that question if caught cheating on it with bonus points for demonstrating the hack afterwards; then we'd have a better list of things for which to watch out for subsequent cohorts. But as I'm not even sure it's a good idea, it's pretty unlikely that I'd be able to convince the various "Powers That Veto" that it's all that hot.

No exam system will ever be hack-free. It's important to set a strong anti-cheating norm, demonstrating that students could reasonably expect to be caught if they try so they don't expect to be disadvantaged if they don't cheat. Our processes probably do more to encourage honesty by showing that we care about honesty and setting the right expectations than by thwarting the most determined of exam hackers.