What we mean by AI governance, security and safety depends on what we are building, where we use it and who can change the outcome.

Put a model builder, a CISO and a business leader in a room and ask whether an AI system is safe.

The model builder may describe evaluations of dangerous capabilities. The CISO may ask what data the system can reach and whose credentials it uses. The business leader may want to know whether it can approve the wrong payment.

Everyone has heard the question. Each is answering a different version of it.

We use governance, security and safety as though attaching “AI” gives us a shared definition. Then we discover we are discussing different systems and consequences.

Those of us who have spent more than thirty years in cybersecurity have seen versions of this before. A new technology arrives, familiar responsibilities move, and we spend a while arguing about which team owns the new nouns. Eventually, someone asks who can actually prevent the failure.

I would like us to get to that question sooner.

First, let us move AI out of the room. There is a raccoon in the kitchen.

The raccoon has not accepted your policy

A wild animal has entered your home. What do our three words mean here?

Safety concerns the encounter: people, pets and the animal could be injured. A frightened animal and a frightened homeowner can produce a terrible collaboration.

Security, in the household sense, concerns the boundary: how did it get inside, and what prevents another visit? Think openings in the building or access to food. The animal does not need malicious intent for the boundary to matter.

Governance concerns authority and responsibility: who decides what happens, who has the competence to act, and which local authorities or rules apply?

“Secure the house” and “make the encounter safe” describe different jobs. Repairing the entry point does little for the person currently sharing a kitchen with the visitor. Resolving the encounter leaves the entry point to address.

Now change the context. Put the same animal in a wildlife rehabilitation facility. Containment may protect the animal as well as the people outside. Authorized handling becomes part of the work. Governance includes decisions about care and eventual release.

Same animal. Different purpose, boundaries and responsibilities.

The words remain recognizable. The effective controls change.

A laboratory makes the distinction more precise

In laboratory work, biosafety concerns preventing unintended exposure to biological agents or their accidental release. Biosecurity concerns protecting and controlling biological materials and related information against unauthorized access, loss, theft and misuse. WHO distinguishes these concerns in its biorisk terminology.

Imagine a laboratory studying a pathogen. A spill that exposes a technician raises a biosafety concern. Someone taking material for an unauthorized purpose raises a biosecurity concern. Decisions about whether the research should proceed, under which conditions, and with whose oversight are governance questions.

But even here, “accident versus attacker” is only a starting point. WHO’s 2024 laboratory biosecurity guidance includes accidental unauthorized access and loss as well as deliberate misuse. A missing sample does not wait for us to establish intent before becoming a control problem.

A locked door and a containment procedure can protect different things. Both may be essential. Neither answers whether the institution should have authorized the experiment in the first place.

Everyone can follow an approved procedure whose risks were poorly understood or accepted by the wrong person. Governance needs evidence from both safety and security.

The manufacturer does not operate the hospital

Consider a hypothetical connected infusion pump.

The manufacturer controls design choices, software behavior and product updates. The hospital controls deployment, local configuration, network access, maintenance and the clinical workflow. A clinician uses the device in the care of a particular patient.

A defect that delivers the wrong dose creates a safety problem. An unauthorized change to dosing instructions creates a security problem with a safety consequence. A hospital deploying the device without clear responsibility for maintenance and updates creates a governance problem that can contribute to both.

The FDA connects medical-device cybersecurity to device performance and patient safety, with responsibilities spanning manufacturers and healthcare organizations.

The patient does not care which vocabulary we use while investigating. They need the device and the care process to work together.

A manufacturer cannot determine every bedside decision. A hospital cannot repair every design defect through staff training. A clinician cannot personally audit the firmware before each use.

Responsibilities follow what each can control. The handoffs need owners too.

Bring AI back into the room.

The model vendor does not run accounts payable

The hospital gives us the closest guide. The manufacturer supplies a product, someone puts it to work, and people rely on the result.

Imagine a payment agent. An AI vendor supplies the model. A practitioner builds the application. An enterprise connects its supplier records and payment system.

An invoice arrives. The agent selects the wrong supplier from two nearly identical names. A finance employee catches the mistake before money moves.

Then someone lets the agent submit payments directly.

Same model. The mistake now has a bank receipt.

Safety means keeping that plausible supplier match from becoming a harmful payment. The builder and finance team need a way to catch ambiguity. The vendor’s model evaluations cannot settle which supplier this business owes.

Security means protecting the records and payment authority. If an invoice tells the agent to ignore its rules and send money elsewhere, its author has not become the agent’s manager. The builder and enterprise must enforce that boundary.

Governance means deciding who can approve automatic payments, under what conditions, and who can stop them. Connecting a tool does not grant the business authority to use it.

The vendor has work at its end too: deciding what to release, protecting its service and evaluating harmful behavior and foreseeable misuse. Like the laboratory, AI safety includes concerns about what someone might deliberately do with a useful capability.

The enterprise buys a model but helps create the system around it. The finance employee needs enough evidence to check a result. The supplier waiting for payment needs a way to dispute a mistake.

The manufacturer does not choose the treatment. The model vendor does not run accounts payable. Both owe the people downstream a product whose limits they can understand and work with.

Thirty years of cybersecurity should make these handoffs familiar. Our access controls help establish who may pay. Finance helps establish whether they should.

Give the words a job

For your next AI project, pick one consequential action the system can take. Bring the people responsible for the model service, the application and the business process into the same conversation. Where the vendor cannot participate directly, bring its documentation and a clear path for unresolved questions.

Walk through one ordinary success, one mistake and one attempt at misuse. Follow each from input to outcome, including what happens to the person affected. Then write down three things:

  • Governance: Who authorizes this use, accepts the remaining risk and can stop it?
  • Security: Which information and authority must be protected, where is that protection enforced, and who owns it?
  • Safety: What harm remains possible through use or foreseeable misuse, what catches it, and who verifies that the safeguard works?

Give each answer a named owner and evidence. If your answer is “the vendor handles that,” establish exactly what the vendor handles and what your builder or enterprise must supply. If it is “a human checks,” show what that person sees and how they know when to intervene.

For the payment agent, that might mean a finance owner approving the automation, an enforced transaction limit owned by the payment-service team, and an ambiguity check tested jointly by finance and the builder. Agree which changes trigger another review.

Now “AI risk” has become something someone can work on Tuesday morning.

The value of these terms comes from the decisions, protections and evidence they help us organize. A locked door does not approve an experiment. A reliable pump does not choose the right treatment. A protected model does not establish that a payment is appropriate.

My concern is the space between a vendor’s assurance, an enterprise’s approval and a user’s assumption that someone has checked.

Walk through that space together. Give it an owner before a failure gives it a name.

Mahalo for reading!

Aloha –TK