Automated item generation uses AI to draft, transform, or help produce assessment items, but the real question is whether it can do so without weakening validity, security, or defensibility. The strongest sources point to value in speeding up item drafting and easing subject-matter bottlenecks, provided human review remains central. The open question is not whether AI can produce item-like text, but how far that output can be trusted across subjects, levels, and assessment formats. In practice, item generation is a governance and validity problem first, and a productivity tool second.

Automated item generation uses AI to draft, transform, or help produce assessment items, usually with human review and governance controls around quality, construct alignment, and acceptance. The central assessment question is not speed on its own, but whether item development can scale without weakening validity, security, or defensibility.

Item development is expensive and often limited by subject-matter expert time. If AI can help produce usable first drafts, assessment organisations may expand content pipelines and reduce bottlenecks. But item quality is not just a throughput problem: weak prompts, weak source control, or weak review can create items that miss the construct, leak unintended cues, or fail defensibility tests.

- **Drafting vs transformation vs generation**: AI may help write new items, rework existing ones, or generate variants. - **Construct alignment**: the item still needs to measure the intended knowledge, skill, or ability. - **Human review**: AI output should not be treated as operationally ready without checking wording, content, and fit. - **Audit trail**: defensibility depends on being able to show what was produced, reviewed, and approved. - **Exposure risk**: scale can create security issues if item patterns become predictable or overused.

The source set points in a consistent direction: AI is being positioned as a way to speed up item drafting and ease content-production pressure, especially where subject-matter expertise is scarce. Stronger guidance and implementation material also converge on the idea that AI can support item production, but it does not remove the need for human authority over construct fit, wording, and acceptance. There is also broad convergence that structured workflows matter more than one-shot prompting. Approved source material, misconception-based distractor design, and clear review points are treated as the sensible control pattern. The deeper assessment issue is governance: can the organisation evidence a trustworthy item-development chain rather than just generate text quickly?

The main open question is not whether AI can produce item-like text, but how far that output can be trusted across different subjects, levels, and assessment formats. Vendor and case-study material tends to frame the opportunity in terms of efficiency and scale, while the independent evidence base on item quality, bias, and operational performance is still thin. Another unresolved issue is where the line should sit between assistance and authorship. The sources suggest that more mature organisations will define clear acceptance thresholds, but there is not yet enough evidence to treat any single operating model as settled practice.

- Items may drift away from the intended construct. - Weak review controls may let flawed items into live use. - Predictable generation patterns may increase exposure or reuse risk. - Vendor efficiency claims may outrun evidence of quality. - Poor governance may create audit, appeal, or reputational problems.

Treat automated item generation as a validity and governance problem first. A sensible decision framework is: 1. Define what the learner must demonstrate unaided. 2. Decide where AI can support drafting without changing the construct. 3. Use approved source material and item specifications. 4. Require human review before any operational use. 5. Check construct alignment, wording, bias, and cueing explicitly. 6. Keep an audit trail showing what was generated, reviewed, changed, and approved. 7. Reassess whether the workflow is still defensible once items are reused at scale. Useful questions for assessment teams include: - What evidence supports the quality and validity of AI-generated items in this subject area? - Which review controls are mandatory before operational use? - How is construct alignment checked when AI produces first drafts? - What audit trail shows who approved what, and why? - How are speed claims balanced against independent evidence on item quality?

A practical comparison usually sits on a spectrum rather than a binary choice: | Option | What it looks like | Main benefit | Main risk | |---|---|---|---| | Prohibit AI generation | All items written by humans only | Maximum control and simplicity | Slower development and higher SME burden | | Permit AI for drafting only | AI creates first drafts, humans approve everything | Faster production without full automation | Weak review can still let poor items through | | Integrate AI into a governed workflow | AI used with specifications, source controls, and audit trails | Best chance of scaling with defensibility | More governance overhead and process discipline needed |

An awarding organisation under pressure to increase question-bank coverage uses AI to draft multiple variants from an approved syllabus and a bank of misconception patterns. Subject experts then reject items that drift from the construct and rewrite anything with unclear wording or unintended hints. The useful outcome is not that AI “writes the exam”, but that the team can expand throughput while keeping human judgement at the point of acceptance.

- Practical guidance on workflows, controls, and review points. - Wider governance framing for AI adoption in awarding. - Case-study material showing how suppliers and partners are positioning AI-assisted item development. - Market-facing article on AI becoming part of modern assessment infrastructure.

Supplier messaging generally frames automated item generation as a route to faster production and better scale. That is useful as a market signal, but it does not by itself validate item quality or defensibility. The current market story is stronger on efficiency than on independently demonstrated assessment performance.

### Can AI generate exam questions safely? Potentially, but only if item quality, construct fit, and approval controls are strong enough to support defensibility. ### Does automated item generation remove the need for subject experts? No. The source set suggests subject experts remain necessary for review, acceptance, and judgement about construct alignment. ### What is the biggest risk with AI-generated test items? Weak governance: items that look usable may still miss the construct, leak cues, or fail review and audit expectations. ### How do I explain AI-generated items to a regulator or verifier? Focus on the specification, the review steps, the approval thresholds, and the audit trail that shows human control over what entered operational use.

Tim Burnett (Admin)

Test Community Network. "Automated item generation." TCN AI & Assessment Wiki. Last reviewed 2026-04-22. https://www.testcommunity.network/wiki/automated-item-generation.html

- OpenEyes article on AI becoming infrastructure in modern assessment and GenQue item generation. - Surpass/Inteleos case study on AI-assisted item development. - TCN guidance on AI for test item generation. - TCN guidance on AI adoption for awarding organisations.