Private AI is an operating model, not only a hosting location

Start with the control boundary

Private AI is often discussed as a choice between an on-premises server and a cloud service. Location matters, but it does not determine privacy, security or organisational control by itself. A locally hosted application can send prompts to an external model, expose knowledge through weak permissions or depend on unmanaged software updates. A cloud-hosted service can still operate within a carefully designed and governed boundary.

For this article, private AI means an operating approach in which an organisation can identify and govern who can use the service, what information it can access, where that information travels, which models and suppliers are involved, what actions the system can take and who runs and changes it. It is a design objective, not a certification or a universal guarantee.

Map the complete data path

Draw the path from source to final action before deciding that a deployment is private. Include:

• source systems, documents, email, databases and user input;
• ingestion, extraction, transformation and temporary working files;
• search indexes, embeddings, caches and derived metadata;
• prompts, retrieved context and model endpoints;
• generated outputs, approvals, exports and downstream actions;
• application, model, security and workflow logs;
• analytics, diagnostics, support access and error reporting; and
• backups, replicas, archives and disaster-recovery copies.

For every step, record the organisation or supplier that can receive the information, the purpose, location, retention, access controls, encryption, deletion method and evidence available for review. The boundary is incomplete when a team can name the hosting region but cannot explain where prompts, indexes, logs or support data go.

The New Zealand National Cyber Security Centre's AI data security guidance treats security as a lifecycle concern covering data used to train, test and operate AI systems. Apply that same lifecycle view to privacy, ownership and operational control.

Separate location, ownership, access and use

Four questions need separate answers:

1. Where is the service and information located? Identify every relevant processing, storage, transit and backup location.
2. Who owns or controls each component? Record the customer, hosting provider, model provider, software supplier, sub-processors and support parties.
3. Who can access it? Include users, administrators, service identities, supplier staff, support teams and lawful-access conditions.
4. How can the information be used? Confirm whether it can be retained, reviewed, used for service improvement, used to train models, shared with another party or included in diagnostics.

An onshore data centre may still involve an offshore provider, support team, parent company or model service. The Government Chief Digital Officer's Cloud Jurisdictional Risk guidance notes that jurisdictional risks may span several locations and can remain in onshore data centres owned by offshore vendors. The guidance is written for government agencies, but the distinction between location and control is useful for other organisations assessing their own circumstances.

Design identity and permissions end to end

Private infrastructure does not correct weak identity or excessive access. Define how people and services authenticate, how roles are assigned, how access is approved and removed, and how privileged activity is reviewed.

Test permissions through the complete path. A person who cannot open a source document should not receive its content through search, a prompt, generated answer, preview, citation, export, log or workflow step. Include tenant boundaries, group changes, inherited permissions, service accounts, support access and emergency administration.

Use separate identities for services and integrations where practical. Give each one only the sources, tools and actions required for its task. Record secrets, token lifetimes, rotation, certificate ownership and the process for disabling access during an incident or supplier change.

Govern model choice and model data paths

A local model can reduce some external transfers, but it creates operational responsibilities for model provenance, files, updates, vulnerabilities, capacity, performance and support. An external model can reduce infrastructure work, but its service terms, retention, regional processing, sub-processors, provider access and change practices become part of the control boundary.

For every configured model, record:

• provider, model, version and endpoint;
• deployment location and network path;
• permitted information classes and use cases;
• prompt, context, output and log retention;
• whether customer content can train or improve a model;
• safety, quality and failure limits;
• change notifications and version-retirement terms; and
• an owner, review date and replacement route.

Keep the model decision separate from the application and workflow where possible. This makes it easier to route different tasks to approved models, change a provider under control and test whether a local option is suitable without redesigning the complete service.

Treat knowledge stores and derived data as governed information

Enterprise AI commonly creates searchable text, chunks, embeddings, indexes, classifications and summaries from source information. These are part of the information environment even when they are technically derived. Define their classification, ownership, permissions, retention, backup, deletion and rebuilding process.

Preserve the relationship to the source so that people can verify an answer, identify the owner and version, and respond when a source changes. Test additions, updates, permission changes and deletion. Removing a source file is not enough if an older index, cache, backup or workflow output continues to expose its content.

Decide who monitors ingestion failures, stale material, duplicates and conflicting sources. Private AI depends on reliable information operations as well as infrastructure controls.

Control workflows, integrations and actions

An AI assistant that only drafts text has a different risk profile from an agent or workflow that calls APIs, changes records, sends messages or starts another process. Map each integration, allowed operation, credential, input, output and failure path.

Keep tool permissions narrow and separate read from write access. Validate data before it reaches another system. Set approval points where judgement, irreversible action or material impact requires a person. A human approval step must give the approver enough source evidence, context, authority and time to make a real decision.

Test attempts to exceed the permitted purpose, including misleading instructions in documents, excessive requests, indirect prompt injection, unavailable services and partial execution. Define what stops, what can retry, what needs manual recovery and what evidence is retained.

Assign operational ownership

A private deployment transfers responsibilities rather than removing them. Agree who owns:

• infrastructure, operating systems, containers and platform updates;
• model files, endpoints, capacity and performance;
• identities, permissions, secrets and certificates;
• source ingestion, indexes and data quality;
• workflow versions, prompts, integrations and approvals;
• logging, monitoring, alerting and evidence retention;
• backup, recovery, continuity and manual fallback;
• vulnerability management, incidents and communications; and
• user support, supplier escalation and change approval.

For shared-responsibility services, document the boundary between the customer, application team, infrastructure provider, model provider and any managed-service partner. Avoid gaps where each party assumes another one is monitoring or patching a component.

The NCSC's secure AI deployment guidance covers protection of data and AI systems during deployment and operation. Apply the guidance according to the use, architecture and threat profile rather than treating on-premises or private-cloud deployment as a security outcome.

Monitor the service, not only the servers

Infrastructure availability does not show whether an AI service remains within its approved purpose. Monitor the conditions that matter to the operating model:

• authentication and denied or unusual access;
• source, permission and ingestion failures;
• changes in model, provider, configuration and workflow versions;
• retrieval quality, unsupported answers and unsafe outputs;
• approval, rejection, override and exception patterns;
• integration failures and unintended actions;
• capacity, latency, cost and dependency availability; and
• incidents, complaints, corrections and repeated user workarounds.

Set thresholds and response ownership according to the impact of the use. Keep enough evidence to investigate a result without retaining sensitive content longer than necessary. Reassess when the purpose, users, data, model, supplier, workflow, autonomy or integration changes materially.

The voluntary NIST AI Risk Management Framework Core treats governance as continuous across the AI lifecycle and links technical risk management to organisational policies and operations. That lifecycle approach is useful regardless of where the service is hosted.

Assess suppliers and the complete supply chain

AI supply chains can include data, models, libraries, application code, infrastructure, hardware, external APIs and support services. Maintain an inventory of material components and the evidence used to trust them.

The NCSC's AI and machine-learning supply-chain guidance recommends combining general cyber-security practices with controls for AI data, models, software, infrastructure and third-party services. Review provenance, security updates, vulnerability response, dependency changes and supplier access throughout the lifecycle.

Contracts and service descriptions should address data use, sub-processors, locations, support access, incident notification, audit evidence, changes, deletion, export and termination. A private label has little value if the organisation cannot obtain the evidence needed to verify those conditions.

Address privacy, jurisdiction and data sovereignty

When personal information is involved, complete and maintain a Privacy Impact Assessment and map collection, purpose, accuracy, access, use, disclosure, retention and correction. The Office of the Privacy Commissioner's AI guidance applies the Information Privacy Principles across the stages of building and using AI tools.

Check whether an offshore arrangement is a disclosure, an agent relationship or another permitted arrangement under the Privacy Act 2020. The Commissioner's Information Privacy Principle 12 guidance explains the rules for disclosing personal information outside New Zealand. Obtain qualified advice for the actual service and contract rather than assuming a New Zealand region resolves the question.

Where Māori data, iwi, hapū, whānau, Māori organisations or material effects on Māori communities are involved, identify appropriate Māori governance and engagement. Location, ownership, access, control and purpose all affect data-sovereignty considerations.

Plan portability, continuity and exit

Private AI should not create an unexamined dependency on one model, supplier, administrator or infrastructure pattern. Decide how the organisation will continue essential work if a model endpoint, licence, cloud service, local server, integration or supplier becomes unavailable.

Test whether the organisation can export its source mappings, configuration, prompts, workflow definitions, indexes where appropriate, logs and decision records in usable forms. Record how a replacement environment would be built, how data would be deleted from the old one and how completion would be verified.

Maintain a manual or reduced-service route where the impact requires it. Portability is an operating capability only when responsibilities, formats, dependencies, time, cost and tests are understood.

The evidence that supports a private-AI claim

A practical evidence pack can include:

• the approved purpose, scope and risk assessment;
• a current architecture and complete data-flow diagram;
• information classifications and a derived-data inventory;
• identity, permission, tenant and privileged-access design;
• a model and endpoint register with data-use terms;
• supplier, sub-processor and jurisdiction assessments;
• privacy and data-sovereignty assessments;
• workflow, integration, tool and human-approval design;
• security, quality, permission and failure-path test results;
• the operating, monitoring, incident and support model;
• backup, continuity, portability and exit test evidence; and
• approvals, exceptions, owners and review dates.

The depth of evidence should reflect the impact and complexity of the use. The aim is to make the real boundary understandable and reviewable, not to create paperwork that hides uncertainty.

Questions for a private-AI decision

• Can we trace every path by which source content, prompts, outputs and logs leave the customer boundary?
• Can each user, service and supplier access only the information and actions required?
• Can we prove how local and external models use, retain and change customer information?
• Are indexes, embeddings, caches and backups governed with the source information?
• Who operates, monitors, patches, supports and changes every layer?
• Which jurisdictions, parent companies, sub-processors and lawful-access conditions apply?
• Can people verify results and intervene before material actions?
• Can the service fail safely and can essential work continue?
• Can we change the model, hosting pattern or supplier without losing control of information and records?
• What evidence will show that the boundary still holds after the next change?

Where Pūnaha fits

Pūnaha supports customer-managed deployment or Pūnaha Cloud, with Windows and Linux support subject to the implementation architecture. Supported model choices include local GGUF execution through llama.cpp and configured connections to OpenAI or compatible endpoints, Azure OpenAI, Anthropic and Google Gemini. Model deployments are configured separately from the workflows that use them.

Pūnaha also provides tenant-aware knowledge stores, semantic and hybrid retrieval, versioned workflows, optional human approval, application logs, audit information and workflow execution history. These capabilities can support a controlled design, but the identity, network, source permission, model data path, monitoring, retention, support and operational responsibilities still need to be agreed for each implementation.

The Pūnaha technical-buyer guide records current limits. The site does not present native Amazon Bedrock requests, complete OIDC sign-in, local MFA, image-only-PDF OCR, active object-storage delivery or cloud-monitoring delivery as ready capabilities. It also makes no broad air-gap claim. These boundaries must remain visible during evaluation.

Hot Desk provides product-neutral AI governance and implementation consulting. A recommendation may involve Pūnaha, another product, improvement of the client's current environment or no product implementation. If Pūnaha is selected, its dedicated product team is responsible for product implementation, configuration and support.

Continue the discussion

Use the New Zealand AI governance checklist, read how to evaluate enterprise AI search with your own knowledge, explore cyber security and assurance support, or discuss a private-AI architecture and operating-model assessment.

About this insight

This article is published by Hot Desk Consultancy Services Limited as general information. It does not certify a service as private, secure, compliant, sovereign or air-gapped and is not legal, privacy, security, architecture or procurement advice. Confirm the current requirements and complete design that apply to your organisation and use.