Responsibility Before Automation
Why a human in the loop is not accountable when they lack evidence, authority, time, or a real right to stop the system
If you are new to the publication, Welcome to The Executable Company explains the larger question behind this work. The previous issue, Your Company Is Not Integrated. Your Best People Are., examined the people who carry context, commitments, exceptions, and company state across disconnected systems.
Now consider what happens when the company automates part of that work.
The system gathers the available data, ranks the options, recommends an action, and presents its conclusion. A person clicks approve.
When the outcome is good, the organization celebrates the intelligence and speed of the system. When the outcome is harmful, an investigator eventually reaches the person at the end of the chain and asks a simple question:
Why did you approve it?
The question may be legitimate. It may also hide almost everything that matters about how the decision was designed. The person may have seen the recommendation without the underlying evidence. They may have lacked time to investigate, the expertise to recognize the failure, the authority to change the policy, or a credible right to reject the default. Their approval may have existed mainly to make an automated process appear supervised.
Madeleine Clare Elish calls this a moral crumple zone: a human absorbs the force of blame for a complex automated system despite having limited control over its behaviour. The company decides what the system will do before deciding what responsibility exists around the outcome, then places a person where blame is likely to land.
That is not responsibility. It is an approval step added after the consequential choices have already been made.
A human in the loop needs more than a place in the workflow
The phrase human in the loop sounds reassuring because it implies that a person remains in control. Sometimes that is true. Sometimes the human is a real decision-maker with enough evidence, competence, time, authority, and support to challenge what the system proposes.
But a person does not become meaningful oversight simply because a workflow stops at their screen.
One of the most important details in the European Union’s AI Act is not the general requirement for human oversight. It is the more specific obligation in Article 26: deployers of high-risk AI systems must assign oversight to people with the necessary competence, training, authority, and support. Those four conditions turn a comforting diagram into an organizational question. Who is this person, what can they see, what can they change, and what happens when they say no?
The same principle appears in the current NIST AI Risk Management Framework 1.0. Its governance function does not merely ask organizations to name accountable owners. It calls for accountability structures in which the relevant teams and individuals are empowered, responsible, and trained, with documented roles and lines of communication.
This matters because review is itself work. It consumes attention, requires domain knowledge, depends on the quality of the evidence presented, and takes place inside incentives that may reward speed more strongly than challenge. In a 2026 controlled experiment with 2,784 participants reviewing AI-generated suggestions for extracting data from corporate emissions reports, the structure of the review task changed how people responded to bad suggestions. When flagging an error also required entering the corrected value, participants made fewer corrections and accepted more incorrect suggestions. The study does not establish the same effect size for every enterprise decision, but it demonstrates a mechanism that matters well beyond annotation: the quality of human oversight depends not only on model accuracy, but also on who reviews the output and how the organization designs that review.
This is why a signature, click, or final approval field can become a legitimacy artifact. It shows that a human was present without proving that the human could exercise judgment.
The useful question is not:
Is there a human in the loop?
It is:
Does the person have enough evidence, competence, time, authority, and support to change the outcome?
If the answer is no, the loop is not a control mechanism. It is a place where the organization can later deposit blame.
A task is not a responsibility
Most automation projects begin with an activity: classify the claim, rank the candidate, approve the expense, draft the response, schedule the worker, generate the code, or follow up with the lead.
The team asks whether the activity is repetitive, expensive, predictable, and technically feasible. It decides which parts a model can perform and where a person should review the result. These are useful questions, but they begin after the most important choices have already been assumed.
Why does the activity exist? Whose purpose does it serve? What commitment does it enact? Who may decide? Who is affected? What evidence makes the action warranted? Who must answer for the consequence? What capability should the company retain even if the ordinary work becomes automated?
A task describes activity. Responsibility connects activity to purpose, parties, authority, consequence, and answerability.
Organizational language often compresses several different relations into the word responsible. Someone performs the work. Someone possesses the capability. Someone may decide. Someone carries an obligation toward the outcome. Someone must explain what happened. Someone may be blamed. Someone may be legally liable. Someone lives with the consequence.
These positions sometimes belong to the same person. Often they do not.
Activity is what is done. Capability is the ability to do it. Authority is the recognized right to decide, commit, permit, or prohibit. Responsibility is an obligation toward an outcome, commitment, role, or party. Accountability, in Mark Bovens’s useful formulation, is a relationship in which an actor must explain and justify conduct to a forum able to question and judge it. Liability and blame are different again.
An AI system can perform an activity, demonstrate bounded capability, act through delegated permissions, preserve evidence, and produce an explanation. None of those capabilities makes it the institutional party that must answer to a customer, employee, profession, regulator, court, or public.
Replacing the word tool with agent does not resolve this. A person-like interface can make the distinction harder to see because fluency, confidence, and access feel like authority. They are not authority.
Responsibility was already difficult before AI. Dennis Thompson described the problem of many hands: complex organizations distribute decisions across so many participants that responsibility becomes difficult to locate. AI adds more hands, some computational and some far from the moment of action. A model developer chooses an objective. A product team designs the interface. A leader selects the metric. A vendor controls infrastructure. Historical data carries earlier decisions. A manager determines how the system enters work. An operator encounters the case. An affected person experiences the result.
Assigning one employee as “accountable” does not make this chain responsible. Blaming “the algorithm” does no better. The algorithm cannot explain why the company chose the objective, accepted the evidence, delegated the authority, ignored an affected voice, or treated the output as sufficient.
Responsibility must be designed across the chain before the chain acts.
The organization must carry what no reviewer can carry alone
Responsibility before automation does not mean placing a morally heroic human behind every machine action. That would recreate the same dependency in a different form and make automation slower without making it safer.
Many low-stakes, reversible actions do not need transaction-level approval. A system can format a document, reconcile a routine record, schedule a non-critical meeting, or execute a bounded update without asking a person to confirm every step. The point is not to maximize human intervention.
The point is to make the institution’s responsibility operable.
Organizations already carry responsibilities that no individual can fulfill alone. They make commitments, establish policy, allocate resources, maintain professional systems, preserve evidence, provide appeal, investigate failure, and compensate harm. A responsible design makes those institutional duties visible rather than forcing one operator to absorb contradictions created elsewhere.
It should be clear who may notice, who may decide, what a machine may execute, what evidence must persist, when a case leaves the ordinary path, who can stop the operation, and which forum can demand an explanation or remedy.
It should also be clear who is affected. The person using the system is not the only party with relevant knowledge or standing. Employees may be scheduled, evaluated, promoted, or disciplined by systems they do not operate. Customers may be classified or refused. Suppliers may be ranked. Communities may absorb consequences that never appear in the workflow.
The International Labour Organization’s 2025 case studies of social dialogue around AI and algorithmic management document worker representatives influencing decisions about employment, skills, algorithmic management, working conditions, and rights across national, sectoral, company, and workplace settings. In 2026, the ILO made the temporal point explicit, arguing that robust social dialogue is critical throughout AI design and deployment, not merely after a system has been introduced.
This is not only a labour-relations point. It is a design principle. Affected parties often possess evidence that the formal owner cannot see. Their ability to question, correct, appeal, or refuse can reveal whether the system is producing the consequence the company claims to want.
A company may still decide to automate extensively. Responsibility architecture does not presume that a human must make every decision. It requires the company to know which responsibilities remain institutional, which authority has been delegated, where that delegation ends, how consequences are verified, and how the operation can be corrected or stopped.
Humans make mistakes. Committees diffuse responsibility. Institutions protect themselves. None of this becomes easier merely because we keep people in the loop. The answer is not faith in human judgment. It is an architecture in which judgment has evidence, authority, challenge, and consequence.
Map the responsibility before you allocate the work
Before deciding what an agent should do, write a short responsibility brief for the outcome the company is trying to produce.
Ask:
What consequence are we responsible for? Describe the result in the world, not merely the completion of a task.
Who is the principal? Identify the customer, employee, patient, citizen, client, community, or institution whose legitimate purpose gives the operation meaning.
Who may decide or commit? Separate system access and technical capability from recognized authority.
Who contributes capability? Identify the human and computational contribution without assuming that either one owns the whole outcome.
Who governs and can stop the operation? Name the party that can change policy, revise boundaries, investigate failure, or suspend the system.
Who is affected, and how can they contest or correct it? Do not treat appeal as an exception added after deployment.
What evidence closes or reopens the responsibility? A completed workflow is not proof that the intended consequence occurred.
What capability must the organization continue to form? Preserve enough human and institutional understanding to challenge the system, handle rare cases, and recover when the ordinary path fails.
This is more work than drawing a task flow, but it prevents the company from discovering responsibility only after something goes wrong.
The result may show that a human approval step is essential. It may show that the approval is theatre and should be removed. It may reveal that the real authority sits somewhere else, that affected people have no practical way to challenge a decision, or that the organization has delegated activity without retaining the capability to understand it.
The next issue, Start With One Consequential Operation, will turn this responsibility map into a bounded starting point for implementation. The company remains the design object, but one real operation gives us somewhere concrete to begin.
Before automating the next consequential decision, look at the person expected to approve it. Can they see the evidence? Do they have time to think? Can they reject the recommendation without punishment? Can they change the rule that produced it? Can they stop the system?
If not, do not call them accountable.
Ask what the company itself has failed to take responsibility for.