The 16 Questions Every Council AI Deployment Now Faces

In our last brief on the NSW Audit Office report, we said the real rules for council AI would be written at sector level — by frameworks, not a grand AI Act. The NSW AI Assessment Framework is exactly that rule, already in force. Issued by the Office for AI under NSW Circular DCS-2024-04, the AIAF is a mandatory risk self-assessment that every NSW Government AI use case must pass, from concept through deployment. Its logic is spreading: a parallel design now sits in the federal government's architecture guidance, and other states are watching. If you run AI in the public sector, this is the shape of the gate you go through.

The encouraging news is that it has been made usable. The previous version took 40-plus hours and leaned on subjective self-scoring. The redesign is a 15-minute, 16-question intake that automatically calculates a risk level — low, medium, high or critical — and tells you exactly which assurance activities are mandatory for that level. The deeper toolkit only activates for the cases that warrant it. As the Office for AI puts it, all the old knowledge is retained; it is just surfaced at the appropriate time.

What it actually measures

The 16 questions probe the things that make an AI system risky: how autonomous it is, what kind of data it touches, whether it influences decisions about people, and who is affected if it gets something wrong. Underneath sits the NSW AI Ethics Principles — community benefit, fairness, privacy, security, transparency, and accountability. Those six words are the spine of the whole framework. Every assurance activity it asks for is, in the end, a request to evidence one of them.

And it is not a one-time form. The AIAF expects re-assessment whenever the features, the datasets, the purpose or the decision context change. That matters for councils, because an AI tool that starts out answering opening-hours questions and later begins taking payments or triaging after-hours emergencies has moved up the risk scale — and the framework expects you to notice.

A phone assistant will rate medium-to-high. That's correct.

Here is the part councils sometimes hope is not true: a customer-facing voice assistant that handles personal information and payments, and that interacts with residents who may be in distress, is not a low-risk use case. Run it honestly through the 16 questions and it lands medium-to-high. That is not a mark against the technology — it is the framework working as intended. Anything touching citizens' data and money should attract scrutiny.

The mistake is treating the risk rating as a verdict rather than a checklist. A medium or high rating does not mean "don't deploy." It means "here are the controls you must be able to evidence." The decisive question is no longer is this risky? — of course it is — but can we show the controls that make it safe? And most of those controls are not policies you write after the fact. They are properties of how the system was built.

Why the answer is mostly architecture

Walk the six principles and you find that each one resolves, for a deployed AI tool, into an engineering question with a yes-or-no answer:

Privacy — does the data stay in Australia, and is it walled off from everyone it shouldn't reach? Security — in a system serving many councils, can one council's data ever surface to another? That is not rhetorical: multi-tenant isolation is precisely where shared AI platforms fail, and it is the single control a security assessor will press hardest. Either every request is bound to the organisation that owns it, with the boundary enforced and tested, or it isn't. Transparency — is every interaction captured, so a resident or an auditor can see what the system did? Accountability — is there a durable, tamper-evident record of who was told what, and when? Fairness and community benefit — when the system reaches the edge of its competence, does it hand off to a human rather than press on?

You cannot bolt these on for an audit. A platform that wasn't built with tenant isolation can't credibly claim it under questioning; a tool that doesn't log can't produce a transcript that was never recorded. This is why the framework, for all that it reads like paperwork, is really a procurement test. It asks you to buy AI that was engineered for assurance from the first line of code.

Where Cassie sits in this

We build for these questions deliberately, because we field them from every council we work with. Cassie is hosted in Australia. Every request is bound to a single organisation, so one council's calls, rosters and records are isolated from another's — a boundary we treat as core engineering, not a setting, and one we have hardened and tested precisely because it is where multi-tenant systems are most likely to leak. Every call is transcribed and logged, giving residents and auditors a full record of what was said and done. Acknowledgements, escalations and outcomes are captured as a durable audit trail. And when a call exceeds Cassie's competence or sensitivity threshold, it transfers to your team — with the transcript — rather than improvising.

We will be straight with you, in the same spirit the framework asks of agencies: a phone assistant is not a low-risk use case, and we don't pretend otherwise. What we offer is the evidence that clears the assurance the rating demands — the residency, the isolation, the transcripts, the audit trail, the human-in-the-loop — laid out in the language the AIAF uses. That is the difference between a tool that complicates your assessment and one that helps you complete it.

Help us complete it with you

If your council is staring at an AI Assessment Framework for a phone or customer-service tool, you don't have to start from a blank workbook. We've done this assessment for Cassie, and we can sit alongside your project sponsor, technical lead and data governance lead to map our controls to the framework's questions — principle by principle, with the evidence attached. Bring your risk register; we'll bring the answers. The assessment is yours to own and sign off, but completing it shouldn't be a research project. Let us help you get it done.