The role of AI in software quality assurance

AI changes quality assurance because the same input can produce different outputs and a plausible answer can still be wrong. Teams need deterministic software tests, model evaluations and clear human accountability within one release process.

Why traditional QA is only part of the answer

Traditional checks still matter for authentication, permissions, integrations, data contracts and error handling. Model behaviour adds another layer: relevance, grounding, safety and consistency often require scored evaluation against representative examples rather than a single pass-or-fail assertion.

The evaluation set should reflect the production workflow, including ordinary requests, ambiguous inputs, adversarial prompts and cases that must be refused or escalated. If reviewers cannot agree what a good answer looks like, the acceptance criteria are not ready.

Measure quality against the decision

The right quality bar depends on what the system is allowed to do. A low-risk drafting assistant may prioritise usefulness and editing time. A system that changes a customer record needs stricter accuracy, permission and audit requirements. A consequential decision may require a person to approve every action.

Measure at least task success, unsupported claims, refusal behaviour, latency, cost and escalation quality. Define the threshold, the owner and the response when performance moves outside the agreed range.

Testing extends beyond the model

An AI service is a chain of components: prompts, retrieval, models, tools, business rules and downstream systems. Test each component and the end-to-end workflow. A strong release process combines:

• offline evaluations against versioned test sets;
• security and adversarial testing for prompts, data and tool use;
• integration tests for permissions, failures and rollback;
• human review for subjective or high-risk criteria; and
• production monitoring for quality, latency, cost and drift.

Challenge the system before users do

Adversarial testing should challenge assumptions about input quality, user intent, permissions and downstream effects. The goal is to discover failure modes before release and decide which can be prevented, contained or escalated.

Production feedback does not become automatic learning. Teams should review failures, update the evaluation set and approve prompt, model or workflow changes through the same governed release path.

Trust comes from visible control

Trust is an operating outcome. People need to know what the system did, what evidence it used, what it could change and who approved the result. These controls also give engineering teams the information required to investigate incidents and improve performance.

A practical assurance model

Define the decision, risk tier and accountable owner before selecting the evaluation method. Keep the evidence from every release, make exceptions visible and give users a clear route to human review. We help organisations build those controls into delivery rather than adding them after a model reaches production.