Every vendor demo now comes with a vocabulary. Most of it is ordinary engineering dressed as magic, and once you know the twelve words below you can read a product brochure the way you read a bank statement: quickly, and for what is missing. One paragraph each, in the order a business owner, accountant or bookkeeper meets them.

LLM

Large language model. A program trained on a very large amount of text to predict the next word. That single skill produces fluent writing, summaries, code and answers. It does not look things up and it does not know when it is wrong; see hallucination. ChatGPT, Gemini, Claude and Copilot are LLMs with a chat window in front.

Token

The unit an LLM reads and writes in. Roughly three quarters of a word in English; "superannuation" is several tokens, "tax" is one. Vendors price by the token, which is why a long document costs more to ask about than a short one, and why a "context window" (next entry) is quoted in tokens.

Prompt

Everything sent to the model in one go: your question, plus the instructions the vendor wrote and hid, plus any document or ledger extract the software attached. When a product says "we never send your data to the AI", the prompt is the thing to ask about, because that is where the data would be.

Context window

How much the model can hold in front of it at once, in tokens. Anything outside the window does not exist to the model. A long client file can exceed it, in which case the software either trims it, summarises it or silently drops the end. Ask which.

RAG

Retrieval-augmented generation. Before the model answers, the software searches a set of documents, pastes the best matches into the prompt, and asks the model to answer from them. This is how most "ask your documents" features work. It improves accuracy because the model is reading rather than recalling, but the model still writes the answer, and it can misread what it was given or fill a gap from memory. A citation produced by RAG tells you which document was retrieved, not that the answer is in it.

Fine-tuning

Training an existing model a little further on your own examples so it adopts a style or a task. Useful for tone and format. It does not install facts reliably, and it does not update a rate that changed after the examples were written. If a vendor says the model is "fine-tuned on Australian tax", ask whether it is tuned to sound right or to be right; only the first is what fine-tuning does well.

Agent

An LLM given tools and permission to act: read a ledger, draft a journal, send an email, call another system. "Agentic" in a brochure means the software does things rather than only says things. The right questions are which actions it may take, which of them are reversible, and who reviews the irreversible ones before they happen.

MCP

Model Context Protocol. An open standard for how an AI assistant connects to a tool or a data source, the way a printer driver lets any program print. A product with an MCP server can be plugged into the assistant you already use rather than making you use its own chat window. It says nothing about the quality of what comes back; it is a plug, not a promise.

Hallucination

A confident, well-formed answer with nothing behind it. It is not a defect that will be patched; it is what next-word prediction does when the next word is not in the data. Vendors reduce it with RAG and calculators and it still occurs. For an email draft it costs a reread. For a threshold or a due date it costs the same as being wrong, because there is no visible difference between the right answer and the fluent one.

Guardrail

A check placed before or after the model: block a topic, strip a tax file number from the prompt, refuse an answer that lacks a source, run a calculation in code instead of in the model. Guardrails are ordinary software, and they are the part of an AI product you can actually audit. Ask to see the list.

Inference

The act of running the model to get an answer, as distinct from training it. When a vendor talks about "inference costs" or "where inference happens", they mean the servers your prompt is sent to. That is the location that matters for the next entry.

Residency

Which country your data is processed and stored in. The model provider, the vendor's own servers and any logging can each sit somewhere different. Australian privacy law permits sending personal information overseas but leaves your business or your practice carrying the obligation for what happens to it there. The residency question to put to a vendor is not "is the data in Australia" but "name every place a prompt or its answer is processed or kept, and for how long".

How we use these words

At TobyAI the LLM explains and routes; it does not originate a figure. Rates, thresholds and dates come from a versioned table signed off by a named person against the published source, the working is done in ordinary code, and retrieval runs over primary-source text we hold ourselves, in Sydney. The glossary above is the vocabulary we use in our own documentation, so if a later piece uses one of these words it means exactly what is written here.

Next

Reading a compliance answer: what a figure, a working, a citation and an as-at date each prove, and what none of them prove on its own.

Sources

  1. Office of the Australian Information Commissioner, Australian Privacy Principle 8, cross-border disclosure.
  2. Office of the Australian Information Commissioner, Guidance on privacy and the use of commercially available AI products (October 2024).
  3. Lewis et al., Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks (2020).
  4. Model Context Protocol specification, modelcontextprotocol.io.