What makes an AI workflow governed and operable?

Governance has to reach the workflow

Putting an approved AI model inside a sequence of steps does not govern the resulting workflow. The workflow may also retrieve organisational knowledge, transform information, call another system, wait for a person, send a message or change a business record. Each part can affect the result.

For this article, a governed AI workflow has clear decision rights, limits, responsibilities and evidence. An operable AI workflow can be run, monitored, supported, changed, stopped and recovered within an agreed service model. Both conditions matter. A well-documented experiment can fail in daily operation, while a reliable automation can still make decisions the organisation has not approved.

Define the purpose and decision boundary

Start with the business need rather than a model or workflow tool. Record:

• the problem or opportunity;
• the intended users and people who may be affected;
• the information and systems in scope;
• the decisions, recommendations, content or actions the workflow may produce;
• what the workflow must never do;
• the expected value and acceptable operating cost;
• the relevant obligations, policies and risk tolerance; and
• the accountable business owner and approval authority.

Define where the workflow advises, where it decides and where it acts. A drafting assistant, a recommendation that a person can reject and an automated action against a customer record need different controls. The level of governance should reflect the impact, reversibility, sensitivity, scale and degree of autonomy.

The voluntary NIST AI Risk Management Framework Core connects intended purpose, context, human oversight, testing, production monitoring, incident response and change management. Its govern, map, measure and manage functions provide a useful lifecycle structure rather than a one-time approval gate.

Map the workflow as a complete system

Document the path from trigger to final outcome. Include:

• the event, schedule or user request that starts the workflow;
• input sources, classifications, permissions and validation;
• transformations, temporary data and derived information;
• retrieval, prompts, models, rules and thresholds;
• human tasks, approvals, rejections and escalation;
• tools, integrations, credentials and allowed operations;
• outputs, notifications, updates and downstream processes;
• logs, evidence, retention and reporting; and
• failure, cancellation, timeout, recovery and closure states.

Record the owner and control at each boundary. A workflow diagram that shows only the successful path is incomplete. The design must also explain what happens when data is missing, a model is unavailable, a permission changes, a person does not respond or an external action partly completes.

Separate process, decisions and AI behaviour

Do not ask a model to rediscover a known business rule on every run. Approval limits, calculations, eligibility thresholds, routing rules and mandatory checks should remain explicit where they can be represented deterministically. The AI component can then handle the work that genuinely benefits from interpretation, classification, retrieval or generation.

Preserve the distinction between a repeatable process, an explicit decision and adaptive casework. A process defines ordered work. A decision model defines rules and outcomes. Casework responds to changing information and professional judgement. An AI step may support any of these, but it does not make them the same form of work.

For every step, identify whether the behaviour is deterministic, probabilistic, human or external. This makes the control and test method clearer. A calculation can be tested against exact expected results. A generated answer needs representative evaluation and limits. A human decision needs authority, information and an accountable record. An external action needs permission, confirmation and recovery.

Control inputs, knowledge and context

A workflow can only be governed when its inputs are understood. Define which users and systems can supply information, which classifications are permitted, how data is validated and what happens when required material is missing or conflicting.

For retrieved organisational knowledge, preserve source identity, ownership, version and access conditions. Test whether additions, changes, permission updates and deletions reach the workflow correctly. Treat extracted text, chunks, indexes, embeddings, summaries and caches as governed information with defined retention and rebuilding arrangements.

Keep instructions and content separate where possible. A document retrieved as evidence should not silently redefine the workflow or expand a tool's authority. Test misleading instructions, prompt injection, malformed content, oversized inputs and attempts to obtain information or actions outside the approved purpose.

Bound models, tools and integrations

Register each model, endpoint, tool and integration used by the workflow. Record its purpose, owner, provider, version, data path, credentials, allowed operations, retention conditions, failure behaviour and replacement route.

Apply the least authority needed for each task. Separate read and write access. Restrict tools to approved operations and destinations. Validate model output before using it as an API parameter, query, message, record update or instruction to another service.

Do not treat a model response as authority. The workflow definition, explicit rules, permissions and approved decision points should determine what can happen next. Where a tool can create material or irreversible effects, use confirmation, approval, transaction or compensation controls suitable for that action.

Make human judgement meaningful

Human involvement is not established by adding an approval button. The person needs enough time, authority, source evidence, context and explanation to assess the proposed outcome. They must be able to approve, reject, correct, return, escalate or stop the work, and the workflow must record the decision and reason where appropriate.

Place human review according to the use and risk. Some low-impact steps may continue automatically within approved limits. Higher-impact, uncertain, sensitive or irreversible actions may need review before execution. Define what happens when an approver is unavailable, conflicted, outside their authority or repeatedly overrides the system.

The Office of the Privacy Commissioner's generative-AI guidance expects human review before an organisation acts on generative-AI output where personal information is involved. The Public Service AI Framework also calls for accountable human oversight through the AI lifecycle. That Framework is intended for New Zealand Public Service decision-makers and practitioners and is encouraged rather than binding by itself.

Use explicit execution states

An operable workflow needs states that people and systems can understand. Depending on the use, these can include draft, queued, running, waiting for data, waiting for approval, approved, rejected, failed, cancelled, recovering and completed.

Define who or what can move the workflow between states, the information required, the time limit and the resulting action. Distinguish a technical success from a business completion. A model may return a valid response while the business task remains incomplete because evidence is missing, an approval has expired or a downstream update failed.

Give operators a safe way to inspect a run, identify the current state and decide whether to resume, retry, correct, cancel or escalate it.

Make retries and recovery safe

Failures will occur across models, networks, source systems, people and external services. Decide which steps can retry automatically, how often, under what conditions and with what delay. A retry must not create a second payment, duplicate record, repeated notification or conflicting decision.

Use stable run and action identifiers where appropriate so that a repeated request can be recognised. Record whether a partially completed action can be reversed, compensated, corrected manually or only escalated. Preserve enough evidence to distinguish a new request from a retry and to reconstruct what completed before the failure.

Set timeouts and expiry rules for data, model responses and approvals. Define a manual or reduced-service route where the business impact requires continuity.

Version the complete workflow

A workflow version is wider than its diagram. The controlled definition can include:

• steps, routes, states and decision rules;
• prompts, templates and output schemas;
• model, provider, endpoint and parameter choices;
• retrieval configuration and knowledge-source mappings;
• tools, integrations and permission scopes;
• approval roles, thresholds and escalation paths;
• validation, monitoring and alert conditions; and
• retention, incident and recovery settings.

Record who proposed, tested, reviewed and approved a change, when it became effective and how it can be withdrawn or replaced. Preserve the version used for each material run. A later prompt or model update should not silently rewrite the evidence for an earlier decision.

Test the complete service

Component tests are necessary but do not prove that the complete workflow works. Test representative end-to-end scenarios using realistic users, permissions, information, volumes and operating conditions.

Include successful cases and deliberate failure paths:

• missing, conflicting, stale and malformed inputs;
• restricted information and permission changes;
• unsupported, unsafe or irrelevant model output;
• prompt injection and attempts to exceed tool authority;
• model, retrieval, network and integration failure;
• timeout, duplicate trigger and partial completion;
• approval, rejection, correction and escalation;
• operator absence, support handover and recovery; and
• version change, rollback and retirement.

Set acceptance criteria before reviewing the results. Keep separate evidence for accuracy, privacy, security, fairness, accessibility, business completion, recoverability and operating fit. A combined score can hide a mandatory failure.

Preserve traceability without uncontrolled retention

For material runs, record enough to explain what happened. This may include the workflow and model version, source references, key inputs, rules applied, generated output, tool calls, approvals, state changes, exceptions and final outcome.

Traceability does not require retaining every sensitive prompt or output indefinitely. Classify the evidence, minimise unnecessary content, protect access and set retention and deletion according to purpose and obligations. Logs can create new copies of personal, confidential or security-sensitive information and need the same care as other records.

Design an appropriate route for correction, complaint, review or appeal where people may be affected. Link the run evidence to the responsible owner and the action taken in response.

Monitor outcomes and operating health

Server uptime is only one signal. Monitor conditions that show whether the workflow remains within its approved purpose and operating limits:

• input quality, source availability and permission failures;
• retrieval gaps, unsupported outputs and exception patterns;
• approval, rejection, override and escalation rates;
• tool errors, duplicate actions and incomplete business outcomes;
• model, prompt, provider and workflow-version changes;
• latency, capacity, dependency availability and cost;
• privacy, security, fairness and safety events; and
• user feedback, complaints, corrections and workarounds.

Agree thresholds, review frequency and response ownership according to the impact of the use. Look for changes in behaviour after a new model, source, user group, process, rule or integration is introduced. Monitoring must lead to a decision route, not only a dashboard.

Set stop, incident and change routes

Define who can pause new runs, stop an active workflow, disable a model or tool, move to manual handling and authorise a restart. Operators need current contact paths and enough authority to act when an agreed limit is exceeded.

Connect workflow incidents to security, privacy, records, legal, operational and customer-response processes. Record triage, containment, affected runs, correction, communication, recovery and lessons. The New Zealand National Cyber Security Centre's secure AI deployment guidance covers protecting data and AI systems and responding to malicious activity. Its scope is particularly relevant to organisations operating externally developed AI systems on premises or in private cloud environments.

Reassess the workflow when its purpose, users, data, autonomy, model, provider, tools, impact or operating environment changes materially. Do not let a release process reduce governance to a technical deployment approval.

Assign the operating model

Named responsibilities can include:

• the accountable business owner;
• the workflow and process owner;
• information and knowledge owners;
• model, prompt and evaluation ownership;
• integration and credential ownership;
• platform operation and monitoring;
• privacy, security, records and assurance review;
• human approvers and escalation authorities;
• service support and supplier coordination; and
• incident, continuity and change approval.

Document the boundary between the customer, consultants, product team, infrastructure provider, model provider and other suppliers. Agree support hours, response routes, maintenance, evidence access and handover. Avoid gaps where each party assumes another one owns a failed run or an emerging risk.

The minimum workflow evidence pack

A practical evidence pack can include:

• approved purpose, scope, owner and decision boundary;
• workflow, data-flow, state and integration diagrams;
• information classifications and permission design;
• model, tool, supplier and endpoint register;
• rules, prompts, schemas and human-oversight design;
• risk, privacy, security and impact assessments;
• representative test cases, acceptance criteria and results;
• version, release, approval and change records;
• run history, decision evidence and retention rules;
• monitoring, alert, support and escalation design;
• incident, recovery, continuity and stop procedures; and
• exceptions, residual risks, owners and review dates.

The depth should reflect the impact and complexity of the workflow. The purpose is to make the service reviewable and supportable, not to create documents that conceal an uncertain design.

Questions for a workflow decision

• Is the purpose clear enough to decide what the workflow must never do?
• Can we trace each input, model, rule, person, tool and action?
• Are deterministic rules kept explicit instead of delegated to a model?
• Can each user and service access only the information and actions required?
• Does human review have the evidence, authority and alternatives needed to be meaningful?
• Can a retry occur without duplicating a material action?
• Can we identify the complete version used for a previous run?
• Have we tested permissions, failure paths, partial completion and recovery?
• Can operators see whether the business outcome completed, not only whether the software ran?
• Who can stop the workflow and move the work to a safe alternative?
• What evidence will show that the workflow remains within its approved purpose after a change?

Where Pūnaha fits

Pūnaha provides visual workflow steps for prompts, retrieval, models, human approval and merging results. Developer-configured backend nodes include conditions, generic REST requests, read-only MySQL access and web search. Workflows support validation, versioning, publication, parallel stages, durable execution and execution history.

Human approval can pause a workflow until a person approves or rejects, and automatic continuation remains available where it is suitable and governed. Application logs, audit information and workflow execution history support inspection. These capabilities can support a governed design, but they do not define the customer's purpose, decision rights, information permissions, acceptance criteria, monitoring, incident response or operating responsibilities.

The Pūnaha technical-buyer guide keeps current boundaries visible. Authentication, network access and integration permissions require implementation design. Complete OIDC sign-in, local MFA and cloud-monitoring delivery are not presented as ready capabilities. No workflow tool, including Pūnaha, makes an unassessed workflow governed or operable by itself.

Hot Desk provides product-neutral AI governance and implementation consulting. Advice may involve the client's current environment, another product, Pūnaha or no product implementation. If Pūnaha is selected, its dedicated product team is responsible for product implementation, configuration and support.

Continue the discussion

Use the New Zealand AI governance checklist, review the private-AI operating model, explore business-process and automation advisory, or discuss a governed AI workflow assessment.

About this insight

This article is published by Hot Desk Consultancy Services Limited as general information. It does not certify a workflow as governed, operable, accurate, safe, secure, private or compliant and is not legal, privacy, security, architecture, procurement or operational advice. Confirm the current requirements and complete design that apply to your organisation and use.