Ask any assistant a compliance question and you get a number. The number is the least informative part of the answer. What tells you whether to rely on it is what comes with it: a working, a citation, a date, and something about who stands behind it. This piece is about reading those four things, because each proves something specific and none of them proves the thing people assume.
The figure
A figure on its own proves nothing. It might be computed, recalled, copied from a forum post or made up; the digits look the same. The first thing to check is whether the figure changes when the inputs change in the way the rule says it should. Ask the same question with a slightly different wage or date. A deterministic tool gives a different figure by the right amount every time; a model gives a figure that is usually right and occasionally drifts, and you cannot tell which from the digits.
The working
A working is the sequence of steps from inputs to figure: this amount, times this rate, less this threshold, rounded this way. It proves that the figure is the result of a calculation rather than a recollection, and it lets you check the arithmetic yourself. What it does not prove is that the rate and threshold in it are correct. A working can be perfectly consistent and start from the wrong number. When you read a working, look at the inputs it takes for granted; those are where the errors live.
The citation
A citation names the source: an Act and section, a ruling, a published table, a Commission determination. It proves that somebody could look the figure up, if the citation is real. It does not prove the figure came from there, and it does not prove the passage says what the answer says it says. The test is to open the source at the pinpoint given and read the paragraph. A good answer gives you the paragraph, not the document, because a section number in a 900-page Act is a hunt, and a hunt is how a wrong citation survives. Be especially careful with a citation that reads well but is vague at the end: "see the ATO website" is not a citation, it is a gesture.
The date
Rates, thresholds and awards change on a schedule and the question is always about a particular time: this financial year, this quarter, the period the employee worked. An "as-at" date says which version of the rule was used. It proves the answer is anchored to a point in time and it lets you reject an answer anchored to the wrong one. It does not prove the version at that date was the one in force; that depends on whether the source behind it was itself current, which brings us to the last item.
Who stands behind it
Somewhere behind every number is a person who typed a rate into a table, or a model that recalled one. Ask which. A tool that keeps its rates and thresholds in a table a named person has signed off against the published source, with the sign-off date recorded, has an answer to "how do you know". A tool whose figures come out of a model at answer time does not; it has a probability. Neither the working nor the citation nor the date can substitute for this. They tell you the answer is well formed. Only this tells you it is well founded.
Reading the whole
Put together, the four questions to ask of any compliance answer, in the order that saves time:
- Is there a working, and do its inputs look like the ones the rule actually uses?
- Does the citation point to a paragraph, and does the paragraph say what the answer says?
- Is the as-at date the period the question is about?
- Who signed off the inputs, and when?
An answer that fails the first is a guess. An answer that fails the fourth might be right, and you carry the risk that it is not.
What this looks like at TobyAI
Each answer TobyAI states carries all four: the figure, the working line by line, the citation to the paragraph in the primary-source text we hold, the as-at date the figures were selected for, and the name and date of the Compliance Lead sign-off on the table those figures came from. AI writes the explanation around that; it does not produce any of the four. You can see the shape on the public worked answers, which are generated from the same tables the product uses.
Next
Where the data goes: a walk through what leaves your accounting file when you ask an AI assistant a question, and the residency questions worth putting to a vendor before you do.