Training Doctors for an AI-Saturated World

Training Doctors for an AI-Saturated World: A Conversation with Ebrahim Talaee

What does it mean to assess a doctor's judgement when that doctor already works inside an environment thick with AI — where patients have been triaged, histories summarised and examinations written up by an algorithm before the clinician even reaches the bedside? Ahead of the IAEA's 51st Annual Conference in Toronto, Tim Burnett sat down with fellow ambassador Ebrahim Talaee to find out.

Ebrahim Talaee is Associate Professor in the Department of Biomedical Informatics and Assistant Dean for Curricular Assessment and Evaluation at the University of Arizona College of Medicine – Phoenix. His background is in education, curriculum, assessment and evaluation, all developed at the University of Oxford's Department of Education, and his focus throughout has been on improving learning by improving the assessment and curriculum that surround it. Six months into his post, he is already running two strands of work that speak directly to this year's conference theme of trust, transparency and technology — one on AI-assisted item development, the other on performance-based assessment in the clinical years.

On the first strand, medical multiple choice questions are far more elaborate than the format most people picture — typically a detailed clinical scenario followed by leading questions and four or five options. AI has already proved useful for cloning and varying these items: shifting the setting from emergency to inpatient, altering vitals, or changing a patient's age, gender or cultural background. But Talaee is clear about where the expertise sits.

"Content experts are content experts. We don't expect them to be prompt engineers."
— Ebrahim Talaee, Associate Professor & Assistant Dean, University of Arizona College of Medicine – Phoenix

Rather than asking faculty to master prompting, his team has built a small set of proven, template-based prompts that specify exactly what should and shouldn't go in. The reason matters: in medicine, a correct answer alone isn't enough.

"What we are interested in is not the answer that the students give to a question, but the process of getting to the answer — what we call reasoning, or the professional judgement of a doctor."
— Ebrahim Talaee, Associate Professor & Assistant Dean, University of Arizona College of Medicine – Phoenix

That focus on reasoning carries into the second strand of his work: the clerkship years, when students are assessed largely through workplace-based assessment. These judgements are inherently subjective — different preceptors, observing at different times, in different conditions. Talaee began with a job analysis of how doctors actually spend their day, and what he found reframes the whole question of what we are assessing.

"Doctors live in an AI-saturated environment. It is not only my individual reasoning, but a combination of my reasoning plus the algorithmic products that have been put forward to me."
— Ebrahim Talaee, Associate Professor & Assistant Dean, University of Arizona College of Medicine – Phoenix

Scholars call this distributed cognition, and Talaee argues it demands a specific competency in return — what he describes as epistemic judgement, the ability to interrogate rather than simply accept what the machine offers.

"Doctors should be able to check the credibility of the knowledge that they get, and not rely only on the algorithmic outputs they receive from AI."
— Ebrahim Talaee, Associate Professor & Assistant Dean, University of Arizona College of Medicine – Phoenix

He illustrated the risk with a striking example. Deface a stop sign with a little dirt and a human still reads it as a stop sign; a self-driving car may read it as no sign at all and drive straight through. In a high-risk clinical setting, that capacity to spot when something is wrong — and to override the output — is exactly the competency medical education must protect. Tim framed it as moving beyond "human in the loop" to expert in the loop: a human with the authority, knowledge and skill to actually exercise judgement, not just a passenger in the cockpit.

In Toronto, Talaee is presenting a paper on making clerkship grading more valid — including the thorny question of what different grading schemes, from simple pass/fail to four-tier honours, actually signal to students. He's most looking forward to connecting with the MedEd assessment community and comparing notes on the challenges they all share. With a few unhurried days in one of his former home cities — and Niagara Falls within reach — it promises to be a rich conversation.

Learn more about the conference: IAEA 51st Annual Conference — Trust, Transparency and Technology in Educational Assessment (EQAO, Toronto, September 2026)

Connect with Ebrahim Talaee on LinkedIn: linkedin.com/in/ebrahimtalaee