Four things called AI

1. Machine learning classifiers. The oldest and quietest. A model trained on millions of past transactions guesses that "WOOLWORTHS 1234 SYDNEY" is groceries and "TELSTRA" is telephone. It has been in your bank feed for a decade. It learns from what you and every other user accepted, and it is right most of the time because most transactions look like previous transactions. It does not know what a transaction is. It knows what it resembles.

2. Document extraction. Point a camera at a receipt and the software fills in the supplier, date, amount and GST. This is optical character recognition (reading the characters) plus a model that guesses which number is the total. Very good on a clean printed invoice, poor on a crumpled handwritten one, and it will confidently read a 3 as an 8 in the wrong light.

3. Large language models. The thing everybody now means by AI: ChatGPT, Copilot, Gemini, Claude, and the chat assistants that appeared in your software in the last two years. A large language model (LLM) has read most of the public internet and learned to predict the next word. That single skill turns out to produce fluent prose, plausible summaries, working code and confident answers to almost any question, including questions about Australian tax law that it has no way of checking.

4. Agents. The newest. An agent is an LLM given tools and permission to act: read your ledger, draft the journal, send the email, file the form. The word "agentic" in a product brochure means the software is being allowed to do things, not only say things. That is a different risk category, and we come back to it below.

Most products blend two or three. The chat assistant that "explains your P&L" is an LLM reading numbers a classifier categorised. Knowing which layer you are talking to tells you how much to trust the answer.

How a language model actually answers

It is worth being precise here, because the intuition most people carry is wrong. An LLM does not look anything up. When you ask "what is the super guarantee rate", it does not open a table. It produces the sequence of words that, given everything it has read, most probably follows your question. If the internet mostly said one figure when the model was trained, it will say that figure, fluently, with a citation-shaped sentence after it. If the rate has changed since, or the training data was mixed, it will still answer fluently. The fluency is constant. The correctness is not.

This is what the word "hallucination" means: a confident, well-formed answer with no fact behind it. It is not a bug that will be patched next quarter. It is what next-word prediction does when the next word is not in the data. Vendors reduce it by handing the model documents to read at answer time (retrieval-augmented generation, or RAG) and by bolting on calculators, and both help. Neither turns the model into something that knows.

For drafting an email, that is fine; you read it before sending. For a tax figure, it is the whole problem. A number that is right 95 per cent of the time and wrong the other 5 per cent, with no visible difference between the two, is not a number a bookkeeper can sign.

Where the data goes

Every one of the four sends data somewhere. The classifier in your bank feed sends transaction descriptions to your accounting vendor's servers, which for the large vendors are usually in Australia or the United States depending on the product, and their privacy policies say which. Document extraction sends the image of the receipt. An LLM assistant sends whatever you type or paste, plus whatever context the software attaches: often the client's name, the figures on screen, sometimes the whole document.

Three questions to ask of any AI feature, and the answers are almost always in the privacy policy if you read the section on sub-processors:

• Which country is the processing in? Australian privacy law lets you send personal information overseas, but you carry the obligation to take reasonable steps about what happens to it there.
• Is the data used to train or improve the model? Consumer tiers of the big assistants often say yes by default; business tiers usually say no. The difference is the tier you pay for, not the product name.
• Who is the model provider? Your accounting vendor rarely builds the model. It rents one from OpenAI, Google, Anthropic, Microsoft or Amazon, and that provider's terms sit underneath.

None of this means do not use it. It means know which layer is doing the work and where.

What each is good for

A fair scorecard, from someone who builds this for a living:

• Classifiers are excellent at volume and boring at judgement. Let them categorise; review the exceptions; never let one decide whether something is a fringe benefit.
• Extraction saves hours and makes small errors at scale. Reconcile totals, not line items.
• LLMs are superb at language: summarising a 40-page ruling into a paragraph you then check, drafting the client email, explaining a concept, turning a messy spreadsheet into a clean one. They are poor at arithmetic, dates and anything where being 95 per cent right is the same as wrong.
• Agents are worth having when the action is reversible and reviewable (drafting, sorting, flagging) and dangerous when it is not (paying, lodging, posting to a ledger without a human in front of it).

The pattern across all four: use the model for the part where a wrong answer is cheap to spot and cheap to undo, and keep a deterministic process (a table, a calculation, a human) for the part where it is not.

Our view, for what it is worth

At TobyAI we hold to a hard line on the third and fourth categories. A language model may explain a figure, draft the working in plain English and route your question to the right tool. It may not originate the figure. Rates, thresholds and dates come from a versioned table that a named person has signed off against the published source, and the working is done in ordinary code that gives the same answer every time. The model then talks about that answer. We think this is the only honest way to put an LLM next to a compliance number, and later pieces in this series will show the measurements that led us there. Disagree with us in the meantime; the pieces are here so you can.

Next week

The glossary: LLM, token, prompt, context window, RAG, fine-tuning, agent, MCP, hallucination, guardrail, inference, residency. One paragraph each, in the order a practitioner meets them.

Sources

1. Office of the Australian Information Commissioner, Guidance on privacy and the use of commercially available AI products (October 2024).
2. Office of the Australian Information Commissioner, Australian Privacy Principle 8, cross-border disclosure.
3. Lewis et al., Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks (2020).
4. Xero, Privacy notice, section on where data is processed.
5. OpenAI, How your data is used to improve model performance.