Test Security in the Age of AI: Why the Whole Community Needs to Be in the Room

With Jim Wollack, Professor at the University of Wisconsin-Madison and President of the Council on Test Security

Published: 10/17/2026
Test Security in the Age of AI: Why the Whole Community Needs to Be in the Room

Jim Wollack has spent thirty years developing methods to detect and prevent cheating on tests. As a psychometrician, professor at the University of Wisconsin-Madison, and president of the Council on Test Security, he sits at the intersection of research, practice and policy. He joined the Test Community Network podcast ahead of the Conference on Test Security (COTS) 2026, the first time the event will be held outside the United States, to talk about what is changing in test security, why process data is becoming a game-changer, and whether the sector is keeping pace with AI.

Watch the episode in full

Why does test security matter now more than ever?

Jim's argument is straightforward: testing is important because it gives us information about what people know and can do that simply is not available through other channels. Scholarships, admissions, licensure decisions: all of these depend on the integrity of test scores. But the other side of that equation is less comfortable to acknowledge.

"People really do cheat and it happens a lot more than we care to admit, and it's become really entrepreneurial. It is a big business, heavily aided and supported by technology."
— Jim Wollack, Professor, University of Wisconsin-Madison

The strategies the industry has traditionally relied upon to combat cheating are no longer as effective as they once were. Technology has changed the game for both sides: those trying to protect tests and those trying to exploit them. That reality is what makes conferences like COTS so essential. They bring the entire test security ecosystem into one room.

What makes COTS different from other conferences?

COTS began in 2012 under a rather more specific name: the Conference on the Statistical Detection of Potential Test Fraud. In those early years, it was focused on getting researchers interested in developing new statistical methodologies to detect cheating, at a time when very little was available and what did exist was held close to the chest by testing companies.

The conference has since broadened considerably, but what distinguishes it from other events is its insistence on bringing together every part of the community: psychometricians, test developers, administrators, policy makers, lawyers, investigators and IT specialists.

"We need the psychometricians talking with the test developers and the test administrators and the policy makers and the lawyers and investigators and IT folks. We all need to be there."
— Jim Wollack, Professor, University of Wisconsin-Madison

Other research conferences touch on test security, but Jim argues that for the field to move forward, everyone who is part of the ecosystem needs to be learning from one another, not just the researchers in isolation.

What are the trending topics in test security?

Unsurprisingly, AI dominates the programme. Jim notes that AI has been a central theme for the past couple of years and that this is not going to change, partly because the technology itself is evolving so rapidly that the community is constantly trying to keep up.

What is notable, however, is the diversity of the AI conversation at COTS. It is not simply about the threat AI poses to test integrity. Sessions explore how testing programmes can use AI to thwart or prevent cheating, as well as how examinees are using AI, so that systems can be made more robust against it.

Beyond AI, two other areas are gaining significant ground: proctoring and process data.

"We know that proctoring is changing a lot. Some of this is obviously related to AI and how AI is being used within proctoring, but there have just been a lot of innovations to proctoring and to remote proctoring models over the last few years."
— Jim Wollack, Professor, University of Wisconsin-Madison

Why is process data becoming so important?

Tests are now overwhelmingly administered on computer, and companies have become adept at capturing far more than just the candidate's answers. Process data, the record of how a candidate arrived at their responses, is emerging as a rich and largely untapped resource.

Until recently, process data in the psychometric literature was treated almost synonymously with response time. Response time is useful: it tells us whether candidates are genuinely engaging with test content. But the field is moving well beyond that single measure.

"We're seeing programmes collect information like eye gaze, where on the screen and for how long are test takers looking. We're capturing clickstream data now. Every mouse move, every mouse click is information that we can now get at."
— Jim Wollack, Professor, University of Wisconsin-Madison

Jim frames process data not just as a security tool but as a source of validity evidence. One of the pillars of validity is collecting information on the response process, and these new data streams allow testing programmes to do exactly that. From a security perspective, many cheating behaviours produce answers through an atypical response process, and process data gives programmes the tools to identify misconduct that might otherwise go undetected.

How is machine learning changing cheating detection?

The volume and variety of data now available, from process data to response accuracy to biometric information, creates an opportunity for machine learning that was not possible a decade ago. Jim describes the growing use of machine learning models to consider multiple variables simultaneously, hunting for anomalies in a data-driven, empirical way.

The strength of machine learning lies in pattern recognition: clustering groups of examinees who share similar patterns across a range of background variables, and flagging groups that look very different from everyone else on key indicators.

"Machine learning has been very useful at recognising patterns that aren't readily apparent to people, to researchers."
— Jim Wollack, Professor, University of Wisconsin-Madison

Jim is careful to note that flagging an anomaly is not the same as proving cheating. An investigation would still need to follow. But the ability to surface groups that warrant closer attention, groups that traditional methods would miss, represents a significant advance for the field.

Do you need proof of cheating to act on it?

This is one of the more nuanced areas in test security, and Jim's perspective reflects an evolving consensus, at least in the United States. He argues that a testing programme's primary job is to certify scores it is confident in. If the psychometric staff see enough anomalies that they cannot certify a score in good faith, the score should be cancelled, even without a smoking gun.

"The job of the testing programme ultimately is to certify scores when they are confident in them. If you're seeing enough that you are not confident in that score, you should be able to cancel that score."
— Jim Wollack, Professor, University of Wisconsin-Madison

He notes that in the US, courts have been very deferent to testing companies on this point, upholding score cancellations made in good faith on the basis of statistical anomalies, without requiring proof of the specific method of cheating. Most testing programmes avoid the word "cheating" entirely in their materials and communications, framing it instead as a score anomaly that prevents certification.

How this plays out in other jurisdictions, including the UK, where Tim observes a somewhat different culture around evidence and litigation, is one of the questions Jim is most eager to explore at COTS in Cambridge.

Is the sector solving for today's AI or tomorrow's?

Tim raised a concern that many in the assessment community are proposing solutions based on what AI could do six months ago, without fully understanding its current capabilities, let alone where it will be in another six months. Jim acknowledges the tension but frames it pragmatically.

"Although what you say is no doubt true that there will be things that we can do today that we won't be able to do six months from now, if we can do it today, we should be doing it today. And as we learn where we're going, we'll cross that bridge when we get there."
— Jim Wollack, Professor, University of Wisconsin-Madison

He points to a longer-running dynamic: the cat-and-mouse game between testing programmes and those who exploit their vulnerabilities has always been part of the field. Fraud rings explore weaknesses, testing programmes respond, and the cycle continues. AI accelerates that cycle, but the fundamental pattern is not new.

Tim illustrated the point with a live demonstration he had built for the EATP conference: a simulated patient triage test where he handed the entire assessment over to an AI agent using computer use. The AI navigated the test, answered the questions, and even drew a stick man illustration to indicate where triage should be applied. The data trail would give the game away, but the capability itself is a reminder that the goalposts are moving faster than many realise.

What can people expect from COTS 2026?

The in-person conference in Cambridge is fully booked, with a waiting list, a reflection of the growing urgency around test security. For those who cannot attend in person, COTS offers a virtual conference a couple of weeks later, with non-overlapping content, live presenters, two tracks and sessions running three to four hours a day over three days.

Jim is particularly excited that this is the first COTS held outside the United States, bringing a new set of presenters from the UK and Europe and, he expects, some important nuanced differences in how test security challenges are discussed and addressed.

"I'm just excited for new perspectives that are going to be there and to see the impact of it in the session content, in the conversations that we have at lunch."
— Jim Wollack, Professor, University of Wisconsin-Madison

Conference on Test Security (COTS) 2026: conferenceontestsecurity.org

Connect with Jim Wollack: LinkedIn

Connect with Tim Burnett: LinkedIn