Self-assessment

Six questions before you rely on an AI result

Think of one thing you are about to let AI do or decide, and answer eighteen questions about that one thing.

For an owner, director, finance lead or professional about to act on something an AI tool has produced. 18 questions about 6 minutes

Izam Ryan is a chartered accountant who builds with AI. He has spent years checking other people's figures on behalf of whoever was paying for the answer. More recently he has been directing AI models to design and ship working software. Both come down to the same habit: before relying on a result, ask how it was arrived at and who would answer for it.

CANDOR is the list he runs through before trusting an AI result. Consequence: should AI touch this at all? Accountability: who is answerable, and who can switch it off? Non-discrimination: is it fair to the people it affects? Data: can the inputs and the model be trusted? Operational integrity: is the system sound and behaving? Reperformance: can what it did be shown and repeated? (Re-perform: a second person can repeat the work and get the same answer.) Candor is named for it.

Pick one use. A price change, a letter to a customer, a shortlist of candidates, a set of figures for the board, a contract review. Answer every question about that one use, not about AI in general. The read-out does not say whether to go ahead. It shows which of the six questions could not yet be answered, which is usually the one worth asking.

The answers stay in this browser. Nothing is recorded anywhere unless the Send control at the foot of the read-out is pressed.

Section 1 of 6

Consequence: should AI touch this at all?

These tools are wrong some of the time by their nature, and that does not get fixed later. So the first question is not how good the tool is but what a wrong answer would cost and whether it could be taken back. This lens is inspired by the work of Alexandra Najdanovic at Aieutics (aieutics.com) on error tolerance and reversibility.

01 If this result were wrong and acted on, do you know what it would cost, in money, in a relationship or in a legal position?
02 If a wrong result were acted on, could it be undone, and how soon would it have to be caught?

A draft can be torn up. A letter that has been sent, a price that has been given, a person who has been turned down, cannot.

03 Has someone decided how often this may be wrong before it stops being worth doing this way?

One wrong answer in a hundred is fine for a first draft and ruinous for a payment run.

Section 2 of 6

Accountability: who is answerable, and who can switch it off?

A tool cannot be blamed and cannot be sacked. Somebody has to own the result as if they had produced it themselves, and that somebody has to be able to stop it.

04 Is one named person answerable for what this produces, whether or not they pressed the button?
05 Could that person stop it today, and would the work carry on without it?

If stopping it would stop the work, the decision to keep it running is being taken by default.

06 Is it written down at which point a person has to look at the result before anything happens on the strength of it?

Section 3 of 6

Non-discrimination: is it fair to the people it affects?

Anything that ranks, filters, prices or scores people can favour some and disadvantage others without anyone having decided that it should. The tool will not say so. Somebody has to look.

07 Do you know whether this treats some people differently from others, and which people?

It applies to customers, applicants and staff alike, and to anything that decides who gets a price, an offer, a place or a reply.

08 Has anyone compared what it produces for different groups of people, rather than assumed it is even-handed?
09 If someone on the receiving end thought the result was unfair, is there a way for them to say so and have a person look again?

Section 4 of 6

Data: can the inputs and the model be trusted?

A result is only as good as what went in and what did the work on it. That means knowing what the tool was given, whether it was allowed to have it, and whose model is behind it.

10 Do you know what information this was given, and whether it was current, complete and the right version?

A tool working from last year's price list gives a confident answer about last year.

11 Has someone checked that what goes into it may go there at all: confidential, personal or a customer's information?
12 Do you know whose model this is, what it was built for, and what the supplier does with what you put in?

The answer is in the supplier's terms, which change, and which someone has to have read.

Section 5 of 6

Operational integrity: is the system sound and behaving?

A tool that gave good answers in the demonstration is not the same as a tool that gives good answers on a Tuesday afternoon after its supplier has updated it. Soundness is shown by testing, watching and having a way out.

13 Was it tried on cases where the right answer was already known before it was used on live ones?
14 Would anyone notice if it started giving different answers next month from the ones it gives now?

Suppliers change models without asking. The same instruction can give a different result after an update.

15 If it went down or went wrong part way through, is there a way of finishing the work without it?

Section 6 of 6

Reperformance: can what it did be shown and repeated?

This is the one Izam cares about most. A result that cannot be traced back to its sources and arrived at again by someone else is a claim, not evidence. It is also the difference between a tool that saves time and a tool that has to be re-done by hand before anyone will act on it.

16 For each result that was acted on, is there a record of what the tool was asked, what it was given and what it produced?
17 Does the result point back to the sources it drew on, so that someone else could check them?

A number without a page reference is an assertion. A number with one is a fact that can be disputed.

18 Could a second person, starting from the same inputs, arrive at the same result or explain why they did not?

0 of 18 answered