Twelve questions to ask before putting an AI assistant in a business tool

Every business tool is acquiring an assistant. Most of the marketing describes what it can answer, which is the least interesting property. The interesting question is what happens when it is wrong, or when someone tries to make it be wrong on purpose.

We built one into Matrix, so this list is partly a description of the decisions we had to make. It is written to be used against any vendor, including us — each question is followed by what a good answer sounds like, and then by ours.

Identity and access

1. Does the assistant have its own credential?

If it authenticates as a service account, it can see everything that account can see, and the only thing standing between a user and someone else’s data is the model choosing not to show it. That is not an access control.

A good answer: it runs inside the user’s existing session and has no credential of its own.

2. Where is the user’s permission checked?

“The prompt tells it to respect permissions” is not a check. Ask which code refuses the request, and whether it runs before or after the database is touched.

A good answer: before any work happens, and again in the underlying service that the assistant calls — so the check does not depend on the assistant layer being correct.

3. Can the model assert who the user is?

If any tool takes a user id, an organisation id or a role as an argument, the model can set it — and a model can be talked into setting it to something else. The identity has to come from the session, out of band.

A good answer: tools are structurally forbidden from declaring identity parameters, and the check is at startup rather than per call. In Matrix, a tool that declared one fails to register and the application does not start.

4. How is multi-tenancy enforced on the assistant’s path?

Ask specifically whether the tenant filter can be absent. A system that defaults to an empty tenant when none is supplied will, one day, return everybody’s data.

A good answer: the data layer errors when no tenant is present rather than defaulting.

Scope of action

5. What is the complete list of things it can do?

There should be a list, it should be finite, and someone should be able to show it to you. “It can query your data” is not a list.

6. Can it write, generate or execute SQL?

Text-to-SQL against a live business database is the highest-risk pattern in this category. It is powerful, and it means the blast radius of a bad generation is your whole schema.

A good answer: no. Fixed tools registered at startup, no path that accepts a query string from a request.

7. Can it change or delete anything, and what happens first?

Read-only assistants are a much smaller decision. If it can write, ask what stands between a misunderstanding and a deleted record.

A good answer: destructive actions are proposed rather than performed, and the confirmation is cryptographically bound to the specific action, its arguments, the user, the tenant and an expiry — so a confirmation cannot be replayed against a different action.

8. How much of what it answers involves a model at all?

This is a quality question as much as a security one. “How many open bugs are there” has an exact answer that a database can compute. Sending it through a model introduces the possibility of a wrong number stated confidently.

A good answer: lookups and aggregations are answered by queries, and the model is reserved for questions that genuinely need judgement. In Matrix the router is deliberately biased towards giving up and taking the slower, more careful path, because a confidently wrong fast answer is worse than a correct slow one.

Evidence and exhaust

9. What is recorded when it acts?

A good answer: every non-read action, including the ones that failed — because a refused attempt is exactly what you want to see when investigating something.

10. What ends up in logs and metrics?

Assistants are unusually good at leaking. Question text goes into a trace, arguments go into a metric label, and now your observability platform holds employee names and client details.

A good answer: question text, tool arguments, answers and identifiers are deliberately excluded from telemetry, and the vendor can say which.

11. Where does the data go, and who else sees it?

If a third-party model provider is involved, that is a data-processing question with a contractual answer, not a technical one. Ask which provider, what is sent, and whether it is optional.

Our answer: the questions teams ask most — my projects, my tasks, what is overdue, how many, what percentage — are answered directly from your own database in a single query, so they never leave your workspace at all. Open-ended questions such as compare and summarise use Anthropic’s Claude, still inside the asker’s permissions and still recorded in the audit log. See how Gini Assistant works.

12. Who can see what was asked?

Oversight is usually an afterthought, and it should not be. If a team is going to ask an assistant about live business data, someone accountable should be able to review how it is being used — and that review should itself be permissioned, not a log file on a server.

Our answer: Chat History is a report inside Matrix with its own named permission, so a company grants visibility of assistant use exactly as it grants any other report.

The underlying principle

Every question above is a variation on one test: does any security property depend on the model behaving? If the answer is yes anywhere, that is where the system will fail — not because models are especially untrustworthy, but because a component whose behaviour is probabilistic is a poor place to put a rule that has to hold every time.

The corollary is reassuring: an assistant built so that nothing depends on the model behaving is a small, boring addition to a system you already trust. It is a new way to ask, not a new application, and it should be evaluated as such.

Related reading: the security controls Matrix actually implements, and designing a role and permission model.