A citizen receives a denial of a licensing application and requests, in writing, the justification for the decision. The response deadline starts running, the case file is located, and then the uncomfortable question comes up within the team itself: part of the initial screening was done by an artificial intelligence feature embedded in the document management system contracted in 2023. No one can say which version of the tool generated the analysis, what data was sent to it, or whether anyone reviewed the result before it became an official act.
The scene repeats itself in government agencies and departments, and it repeats itself just as naturally in manufacturers, health insurers, law firms, and distributors, because the entry mechanism is identical in every case. AI arrived through an update to software the organization was already paying for, often enabled by default, without going through a committee, without a new contract, and without a single line of record about how it came to influence the work of the people making decisions.
The debate about model quality, however legitimate, diverts attention from the risk that actually reaches the bottom line. When a result is challenged, what the organization needs to present is the chain of evidence, that is, the trail linking the request, the data used, the version of the tool, and the human signature that turned a suggestion into a decision. Without that trail, liability quietly shifts from the vendor to whoever used the result.
How liability shifts without anyone deciding it
ISACA research, in the 2024 AI Pulse Poll, indicated that only 15% of the organizations surveyed had a formal, comprehensive policy on the use of artificial intelligence, while use by employees was already reported as widespread. The gap between actual adoption and formal governance creates an administrative gray zone, in which the person who asks a generative assistant to draft an opinion records the request nowhere, simply because no procedure requires recording it. The tool does the work, the text is edited, and the document enters the workflow looking like entirely human output.
Software contracts tend to be explicit on one point that few managers read carefully: AI output may contain inaccuracies and it is up to the customer to validate it before use. That clause, combined with the absence of internal records, produces a predictable outcome when a formal challenge arises. The vendor demonstrates that it gave notice, the organization cannot demonstrate that it reviewed, and the burden of explanation falls entirely on whoever signed the final decision.
What to do before someone asks
The first move is inexpensive and revealing, and it consists of building an inventory of what is already in use, area by area, including AI features embedded in systems contracted for another purpose. The question that unlocks this assessment is operational before it is technical: on which tasks does the team already ask the machine for help in order to decide faster? An AI exposure assessment turns loose impressions into an objective list, and the list is usually longer than the executive team imagines.
With the inventory in hand, classify each use by the consequence of an error, not by volume of use, because a tool used ten times a year to analyze proposals carries more weight than one used a thousand times to summarize internal meetings. For the highest-consequence decisions, require mandatory human review and adopt a minimum traceability record with five fields: who made the request, which tool and version, what data was used, what the original output was, and who approved it. Five fields fit in a simple spreadsheet and cover most of what an auditor, a client, or a regulatory body typically asks for.
Contracts and a review cycle complete the design. Require written answers from vendors about the destination and retention of the data sent, about liability for model error, and about advance notice when AI features are changed or enabled by default. A quarterly two-hour review, led by a named owner, keeps the inventory current without consuming a small team's calendar, and it is at that cadence that governance and compliance stop being a project and become routine.
Five questions every manager should ask
1. Can your organization list, precisely, which AI tools are being used by area, including those embedded in systems you already pay for? 2. Which decisions in your operation are already influenced by AI and, of those, which could be formally challenged by a client, a citizen, a regulatory body, or an auditor? 3. If someone requested the full justification for a decision supported by AI, how long would it take to gather the evidence, and would it even exist? 4. Who, by name and title, is ultimately accountable for each class of automated decision, and is that accountability written down or merely assumed? 5. Do your contracts with software vendors define what happens to your data, who answers for model error, and how you are notified when AI functionality changes?
Can your organization list which AI tools are in use by area?
Most organizations answer this question with a short, incomplete list, because they see only the subscriptions explicitly contracted for AI. The real volume appears when someone reviews the management, customer service, email, and document systems, where suggestion, summarization, and classification features have been built into products already in use.
Ask each area manager for a named list within fifteen days, including the task being supported and the name of the person using it. Until that list exists, any AI policy remains a statement of intent, with no reach over what actually happens in practice, and the executive team makes decisions based on a picture that does not match the operation.
Which decisions in your operation are already influenced by AI and could be challenged?
The screening criterion is simple and uncomfortable: identify which outcomes affect someone outside the organization and could trigger a complaint, an appeal, or a lawsuit. Proposal analysis, queue prioritization, document classification, drafting of official responses, and vendor evaluation concentrate this type of exposure in practically any sector.
An institution that ranks service requests by an automated score needs to be able to explain the criterion case by case, with the same clarity it would demand from an employee who made the decision manually. Applying this test to the five or six most sensitive decisions usually reveals where human review needs to be formalized first, and where it can wait without relevant risk.
How long would it take to assemble the complete evidence for an AI-supported decision?
It is worth running the exercise as a real simulation, choosing a concrete case from the previous quarter and timing how long the team takes to reconstruct the path to the decision. The result of that simulation says more about governance maturity than any written policy, because it measures capability rather than intention.
When the evidence does not exist, the conclusion is immediately actionable: implement the minimum record before expanding use of the tool to new areas. Recording five fields per sensitive decision costs minutes per case, an incomparably smaller price than reconstructing a history under the pressure of a deadline, an audit, or public exposure.
Who is ultimately responsible for each class of automated decision at your company?
Diffuse responsibility is the default state when no one assigns it formally, and the practical effect shows up at the moment of a challenge, with departments pointing at one another while the response deadline runs down. Naming one person per class of decision eliminates that ambiguity and creates a clear point of contact for auditors and clients.
The appointment also changes the behavior of whoever uses the tool, because an identified owner tends to demand criteria, records, and limits before approving. In smaller organizations, this role can be taken on by an operations manager, provided the assignment is written down, communicated, and reviewed whenever staffing changes.
Do your contracts define who is liable for model error and for changes in functionality?
Three points deserve immediate verification in every relevant software contract: the destination and retention of the data sent to the tool, liability in the event of model error, and the obligation to give advance notice when AI features are changed or enabled by default. The absence of any one of them means the organization is absorbing a risk it has not priced.
Contract renewal is the natural window to negotiate these terms, and mature vendors respond in writing without resistance, because they already document the capabilities and limitations of their features. Comparing the answers of two or three vendors to the same set of questions quickly reveals which of them will treat you as a partner when something goes wrong.
Frequently asked questions
What is a defensible decision supported by artificial intelligence?
A defensible decision is one whose origin can be fully reconstructed: who requested the analysis, which tool and version was used, which data fed the request, what the original output was, and who reviewed and approved it before it was applied. Model quality matters, but it is the existence of this trail that makes it possible to answer a client, an auditor, or a regulatory body. Without it, the organization assumes responsibility for a process it cannot explain.
What minimum records ensure traceability in corporate use of AI?
Five fields per sensitive decision cover most practical requirements: requester, tool and version, data used, original output generated, and human approver with name and job title. This record can live in a spreadsheet or in the workflow system itself, with no need for additional software investment. The critical point is to apply it to the decisions with the greatest consequences, and not to every use of AI in the organization.
Who is legally liable for an error made by a contracted AI tool?
In most software contracts, the vendor states that the output may contain inaccuracies and assigns the client the duty to validate it before use. In practice, this means that the organization that applied the result is liable to the affected party, absent a specific clause to the contrary. Reviewing liability for model error, data handling, and notice of functionality changes is therefore a contract management task before it is a technical one.
To map where artificial intelligence already influences decisions in your operation and what is missing to make them defensible, schedule a no-obligation Strategic IT Assessment with Zamak Technologies at www.zamakt.com/contactus.