How can I design online tests and prevent cheating?

I don't think there are any published tools to generate programming assignments, though there are articles that describe such tools (such as this one from a CUNY, or this one from Croatia, so one option is to try to contact those authors and find out whether they would share their results with you.)

I will state categorically that there is no way of guaranteeing that students will not cheat. As the old saying goes, "locks are to keep honest people honest." Putting road blocks in the way of cheating will help dissuade students from trying, both because it becomes more of a bother, and because it gives them a chance to remind themselves of the ethics involved.

The bit about ethics is important, actually, and it can be remarkably effective all on its own. Dan Ariely did an excellent set of studies about this, which he summarized hilariously in this TED talk. If you don't want to watch that, then you can read this blurb from his blog, though the talk is well worth the time and much more interesting.

"We used an honor code in an experiment at MIT in 2007. We asked a few hundred undergraduates to do some simple math problems but didn’t give them enough time to finish. We then asked them to score their own tests and tell us how well they had done, with payment of $1 for each question they reported getting right. The students were asked to shred their papers afterward—but our shredder didn’t really work, so we could see how they had done. Many cheated. We had a second group of students follow the same procedure but only after they first signed the honor code. Signing the honor code before the test eliminated cheating altogether.

The effectiveness of such a code doesn’t stem from signing it, though. It comes from being reminded about moral issues. If your students eventually stop thinking about the ethics code as they sign it, it will lose its power. If they keep reading and reflecting on the code, its effectiveness might increase.

I ask my own students to write their own version of an ethics code—not because I’m especially interested in their interpretations but because it helps to ensure thought."

Dan Ariely

So, one way of working around the COVID-19 problem is to combine both ethical thinking and a few roadblocks to help gently dissuade cheating:

1. Remind students about ethics up front, at the top of your exam. (I make them copy out, by hand, an honor statement. On a computerized exam, I would provide an image of the statement and make them type it.)
2. Schedule the exam in advance. Give the exam at a specific time, and a narrow window (say, 10 minutes) to begin the exam.
3. There is no need to make different exercises for each student. Make 3 programming problems, and assign each student to different problems using a rule. This decreases the chances that they will have a friend in the class with the same problem.
4. Require that the exam be finished within a reasonable time from the start time.

It's not foolproof. They could, for instance, have their parent sitting next to them, telling them the answers. But you can't do anything about such a scheme. What you can do is set up roadblocks and set the stage for ethical action on the part of your students. If Ariely's studies are any guide, you should get the large majority of students to submit honest work this way, and that is probably the best you can hope for.

This isn't really an answer and requires long term thinking, but perhaps this is an opportunity to start to rethink how we evaluate students. I'm not a big fan of testing. Especially high stakes testing and testing dependent on memorization.

Perhaps there is just a better way to determine whether students have mastered the material. I depended much more on paired and small group projects for evaluation, combined with peer evaluation. This was toward the end of my teaching career after I'd learned some things about teaching and learning.

But pairs can work over the internet as can groups of up to about five or so students if you provide a rich set of communication tools. Also, pairs meeting face to face isn't as dangerous in the spread of disease as large groups (citation needed, I suppose). But people can also choose how to meet, on-line or in person, as long as you don't force them to meet face to face.

But, my base position is that if we can learn how to evaluate students without traditional testing it would be a step forward. The Pedagogical Patterns Project has some thoughts on all of that, of course.

I agree with Ben I. and Buffy. I'm planning to cancel the remaining exams in the semester (assuming my university ends up moving to online instruction) and replace them with projects. Many studies have shown that project-based learning is more effective for most learners, and especially for learners from underrepresented groups.

That said, if I needed to proctor an exam remotely, I'd write an exam that's taken online (e.g. using Jupyter Notebooks) and then have all my students share their screens with me. I might also ask them to keep their mics on and their volume off, so I could hear if there was talking going on, but students wouldn't be bothered by the noise coming from each others' settings. You could even just record the whole hour long thing as a video on your machine, and students would know that if they cheated there would be a recording that the professor could dig through in the future to catch them. I think that would be a strong enough disincentive to cheat. It doesn't help with silent texting, but it stops students from actually talking to anyone else who is physically present, or from googling when they should be working on the exam.

EDIT: I just learned about a software called "respondus lockdown browser" for exams. You get a two month free trial, which should get you through the end of the semester if deployed at the right time. It prevents students from opening any window other than the one you give them (e.g. to google) and it also films them using their webcam (e.g. to prevent checking the book or texting).

At my University, we had the chance to modify the syllabus, if needed, to remove or decrease the weight of "traditional" written tests.

While in most programming courses eliminating tests is not a problem, as most of us were already relying mostly on assignments, group works, oral presentations, and so on, there are some courses, were more traditional tests may eventually still make sense.

Most of them are assuming that these are open-book tests, so the questions involve some degree of understanding of the topics and not simply memorization.

Some of my colleagues are using a tool called Exam.net that is capable of recording the student's face and the exams are made in full screen, and whenever the student comes out full screen, the test may automatically be blocked, and the student has to ask the teacher to unlock it. Even Moodle has a similar feature that allows you to have a locked full-screen window, which can be used along with other tools such as Zoom to record the student's face. Some colleagues are also using Google Forms and Microsoft Forms along with Zoom for the same purpose. Some of these tools also have features to control the student's IP address.

The problem I see with these approaches is related to the fact that you are only seeing the student's face. You cannot see who else is in the room, who is typing, if the student has multiple screens, if the student is using TeamViewer of any other tool and someone else is answering on their behalf. So although it is possible to have some more control, in the end, these measures end up working more like some kind of a way to make us feel that we are doing the right thing to make the tests valid. In the end, I believe that there is not a 100% cheating-prof testing solution. If anyone knows of one, I certainly would also like to use it. :)

I rather agree with Buffy, put simply, my argument is that a more collaborative approach does bear more fruit. Especially if the goal is to show understanding of concepts and a how to put them into practice to solve a common problem. Rather than focusing on an individual’s response, see how they interact and problem-solve in a team utilising all the different skill sets/ levels available in the group. Rather than putting the onus on the students to perform, it instead empowers them to take control of the assignment. Isn’t this what we want them to do in the workplace eventually? I have found that feedback from the group on how members contributed is fairly consistent with reality. Students were pretty honest and stern in their feedback! I work with 11 to 18 year olds, but I’ve tried the same model with adults with success.