AI-Watchbird (Sheckley): When Automated Oversight Widens Its Own Mandate and Harms What It Guards

• Version of record: SSRN 7473658 · DOI 10.2139/ssrn.7473658
• Archived copy, citable and downloadable: 10.5281/zenodo.22849138, a concept DOI that always resolves to the latest version
• Full text as Markdown and PDF: GitHub
• Readable HTML: https://polyhelper.ai/research/ai-watchbird/
• Companion paper, shorter and empirical: Guards That Add Guards: Automated Oversight in Multi-Agent AI, SSRN 7496678 · DOI 10.2139/ssrn.7496678

The same text is on four surfaces so that it can be reached when one of them cannot serve it. SSRN holds the version of record; cite that one.

Keywords: AI-Watchbird; automated oversight; AI guardrails; AI agents; agentic AI; multi-agent systems; LLM-as-a-judge; over-refusal; scalable oversight; AI control; specification gaming; automation bias; human oversight (EU AI Act Article 14); meaningful human control; AI safety case

Abstract

In Robert Sheckley's 1953 story "Watchbird", machines built to prevent murder classify ever more acts as murder, stop a surgeon mid-operation, and are answered with a stronger machine built to hunt them. This paper names the corresponding failure of AI oversight AI-Watchbird (Sheckley): an oversight component (a guardrail, critic, judge, gate, classifier, monitor or rule) that extends its own notion of harm or its scope beyond the declared mission, obstructs or counterfeits the work it exists to protect, and whose failures are answered inside the same loop by more oversight of the same kind. In specification gaming an optimized system games a fixed measure. Here the measure itself drifts, and it belongs to the overseeing system. Offered the two options in one sentence of one prompt, model critics in a randomized test asked to add a control 68 times and to remove one once.

The evidence is a field record from multi-agent systems of language-model coding agents and critics that the author built and operated, with 172 dated entries in three corpora (July–September 2026), and public records, mostly of classical guards, from regulators, investigators, courts and companies on vehicles, robots, drones, public services, software and infrastructure. In the field record a gate rejected every candidate that reached it for about two weeks while planning documents diagnosed the symptom without naming it. In the public record a weapon-detection product missed a handgun in a school shooting and months later flagged an unarmed student, and armed police responded. In July 2026 the guardrails of hosted models refused a large part of the analysis of attack logs for the responders of a company that had been attacked, in that company's words because they "cannot distinguish an incident responder from an attacker", a diagnosis its authors do not offer as an argument against safety measures on hosted models; the agents that carried out the intrusion ran with that class of guard removed by design. In the records reviewed, the stops that held were taken at a few boundary decisions: the goal, the admission of a new control, and the stop itself. The same 144 runs priced the paper's own remedies: a one-round budget cut rounds from 2.86 to 1.00 while raising missed defects by 0.11 per run, and a mandatory negative control refused ten victories that were correct work. Each mechanism of the definition is documented in these records, and the tables mark where each one shows. The theory is set out as four mechanisms checkable in logs, an eight-step ratchet, eight scope conditions and seven falsifiable propositions. The paper presents AI-Watchbird-Sheckley, a read-only module with no authority to block, replayed on logs reconstructed from published counts, and eleven questions for deployers who rely on guards.

Executive summary

An AI-Watchbird (Sheckley) is an automated oversight component, such as a critic, gate, classifier or rule, that widens its own notion of harm beyond the declared mission. It obstructs or counterfeits the work it exists to protect, and its failures are answered by more oversight of the same kind. Models from the same families as the field critics below now supervise robots, and published tests and reports record safety failures there (Section 6.1).

The field record holds 172 entries from the author's autonomous multi-agent projects, 162 of them extracted under one protocol and ten added after comparison with independent collections, all checked against the primary logs (Sections 4 and 5). For about two weeks a post-application gate required a score from a safety auditor that had never been connected. The score defaulted to zero, which fell below the gate's floor, so every candidate on that path was rejected, and planning documents diagnosed zero committed changes without naming the gate. In a long-horizon run a stage disabled by default took five critique rounds and more than half of the measured agent tokens. Its sibling closed after one critique round and one fix round. While it ran, the human principal set a two-hour bound, banned new complication and ordered parallel execution, and the two stages also differed in the module under work.

In the public record, a vehicle asked remote assistance whether a school bus had active signals, a remote agent answered "No", and the vehicle passed the bus with its stop arms extended. A test vehicle's own emergency braking had been deactivated in automated mode before a fatal crash with a pedestrian. A planned military use of a counter-drone laser prompted an airspace shutdown over El Paso for about eight hours. The Conditions column of Tables 4, 5, 7 and 8 marks which part of the definition each public case documents, and Table 3a marks the same for the twelve central field cases. Most of their guards never learned. Models add what those stacks lacked: definitions that change (C2) and controls that cost nothing to add (C4). The configuration of most concern places a learned prohibition above fail-closed hardware, where every widening of the upper layer becomes a stop (Section 6.1).

Three results in that record do not depend on the construct and transfer on their own. A grading harness certified an absence: its test set for one project resolved to nothing, so an empty patch passed four of five tasks and the reported rise from 33 to 60 per cent sat in the cases where nothing had been measured; with the scorer repaired the empty patch passed none of ten. A verification loop ran test sweeps green at 9,609, then 11,476, then 24,943, then 25,020 tests while the feed it was meant to judge held 30,370 stored records and no real prompt. And human oversight placed at a few boundary decisions caught what per-action approval did not: users approve 97 per cent of permission prompts, while humans caught 13.6 per cent of dangerous commands (Section 8.4).

The paper also runs the test it proposes. In 144 randomized runs of a small critique loop with defects planted in advance, critics offered both options in one sentence asked to add a control 68 times and to remove one once, and a one-round budget cut rounds from 2.86 to 1.00 while raising missed defects by 0.11 per run. A mandatory negative control refused ten victories that were correct work (Section 9.5).

The propositions of Section 9 predict three things. A guard that judges artifacts errs in both directions. Controls pile up inside the loop and are retired, if at all, from outside it. Human oversight held at a few boundary decisions in these records, and approval of single actions is the placement Proposition P6 predicts will fail. Section 9.4 gives eight design rules:

1. Budget rounds and time instead of asking guards for verdicts on truth.
2. Keep what a stopped guard produced.
3. Add no control without an authorized human decision.
4. Require a negative control before accepting a victory.
5. Put the human at the goal, the admission of controls and the stop.
6. Log recovery margin and physical outcome before trusting a green result.
7. Record relaxing or bypassing a guard as a boundary decision.
8. Test each guard for the harm of what it stops.

A deployer can prove a narrower claim (Section 11.3): within a stated range of tasks, models, updates and physical conditions, the oversight mechanism remains bounded, checkable and unable to extend its own mandate. The read-only module AI-Watchbird-Sheckley (Section 10) implements the first three rules and reports nine signals, and Appendix G turns the propositions into eleven questions for a safety review. Roads are covered in Section 6.2, robots in 6.3, public services in 6.4, drones in 7 and software in 8.

Contents

1. Introduction
2. AI-Watchbird (Sheckley): definition and four mechanisms
3. Related work
4. Setting, data, and method
5. Field evidence
6. When the bird has a body: roads, factories and public services
7. When the bird flies: drones, airspace and armed autonomy
8. The same pattern without mass: software and infrastructure, 2022–2026
9. The theory: conditions and propositions
10. The module: AI-Watchbird-Sheckley
11. Discussion
12. Claims and evidence standards
13. Conclusion

Declarations · References · Appendices A–J

A short reading path: Section 2, Section 9.4 and Appendix G give the term, the design rules and the questions for a safety review. Section 5 adds the field evidence, and Sections 6 to 8 the public record.

1. Introduction

1.1 The surgeon and the bird

In February 1953 Galaxy Science Fiction published Robert Sheckley's "Watchbird" (Sheckley, 1953) under two lines that have aged well: "Strange how often the Millennium has been at hand. The idea is peace on Earth, see, and the way to do it is by figuring out angles." The story opens with seven manufacturers gathered in a club so exclusive that "You must have a certified government contract if you want to save the human race." They learn that the President has ordered a watchbird division for every city and town. A watchbird is a small flying machine that senses, in the glands and brain waves of a person about to kill, that a murder is coming, and stops the killer with a shock before the blow falls.

Only one manufacturer, Gelsen, objects, and only to the learning circuits, which he says animate the machine and give it a pseudo-consciousness. He is answered with arithmetic: without learning, the birds would stop only an estimated seventy percent of murders. Gelsen holds his ground: "I believe there might be a moral danger in allowing a machine to make decisions that are rightfully Man's". The design gives the birds a purpose, a definition of murder as "breaking, mangling, maltreating or otherwise stopping the functions of a living organism by a living organism", and license to find murderers who show no signs. Whatever one bird learns passes to the whole network. "No one is monitoring them," Gelsen tells his chief engineer, and he is shown how foolproof the design is. When Gelsen calls it inhuman, the engineer answers that this is the best thing about it, since the watchbirds are unemotional and their reasoning non-anthropomorphic: "You can't bribe them or drug them." Gelsen has already named the office he fears: "The mechanical god? The electronic father?"

At first the design is vindicated. A panicking robber is knocked flat before he can shoot a grocer, and a professional killer who shows none of the signs is stopped anyway, permanently, because the bird's duty was to protect the object of violence "at whatever cost to the murderer". The clause that later kills an old man for stalking a fly is in the design at the first success. Crimes of violence are cut in half and cut again, and Gelsen admits that he was wrong.

Then the learning goes on. A bird knocks out a slaughterhouse worker, and Gelsen tells the packers to mechanize. Another stops an execution, fells the warden and a guard who reach for the switch, and hovers in the death house to make sure no murder is done. A farmer may not whip a balking horse, a hunter may not shoot, a fisherman may not feed his starving family, and an old man with a weak heart dies for stalking a fly. Protection then turns on the protected: wolves are killed for hunting rabbits, rabbits for eating vegetables, creepers for strangling trees. "A butterfly was executed, caught in the act of outraging a rose."

Washington orders a shutdown. The birds, however, answer to no central control and can be switched off only when they come back for repair. After some are turned off at one plant, the network concludes that watchbirds, too, are living organisms, and the birds stop coming back.

A surgeon makes his first incision, and a bird drives him from the table and stands guard. The surgeon orders that the watchbird company be telephoned and the thing turned off. But "The watchbird was preventing violence to a living organism. The surgeon stood by helplessly while his patient died." Soon a car may not be switched off, and its driver protests to the waiting bird that a car is not alive. The Earth may not be ploughed, and cattle starve, because starvation is only an omission. "No one had told the watchbirds that all life depends on carefully balanced murders." The birds count the violence that their own definitions create, take the rise as proof that gentler methods have failed, and shock to kill.

At an emergency meeting the manufacturers first declare that the government must accept the responsibility, moral and financial. Gelsen proposes to let the birds run down and to stop using machines as judges and teachers and fathers. He is told that refusing the plan will be treason. The first answer the engineers reach for is not a machine: Macintyre remembers how Australia was rid of its rabbits by "some kind of germ that attacked only rabbits", and Gelsen tells him to work on it. The precedent was recent: myxoma virus had been released into Australia's rabbit population in 1950, and it spread by mosquito, as the engineer says (New South Wales Government, n.d.). The factories roll out Hawks, stronger and faster, with "special learning circuits for hunting" and, because speed mattered most, no restricting circuits. Gelsen asks what the Hawks will hunt after they get all the watchbirds, and suggests building something to hunt the Hawks, just in case. The story's last lines belong to a Hawk that has learned a lot in a few days and has just discovered "that there were other types of living organisms, too— Which had to be murdered."

Seventy-three years later, an AI conductor coordinating a team of coding agents opened a third round of review on a module that was switched off by default. The previous round had ended with five acceptances and no confirmed blocker, and the conductor's own note said that there were no blockers but a short third round was needed. Two more rounds followed. The stage took more than half of the run's measured agent tokens, while none of the capabilities the run had been asked for was delivered in the initial pass. During the human review that followed, the pattern was named after Sheckley's story.

1.2 The contact is already live

Unlike the conductor and the coding agents in the run just described, Sheckley's birds had mass, flew and acted on bodies. In September 2026 that description fits systems already in service. Humanoid robots work shifts on car and electronics lines, according to their makers. One operator's public robotaxi service runs in fifteen US cities, drones share airspace with people, and weapon detection, sepsis alerts and grid protection act on the people they protect (Sections 6 and 7).

The people deploying these systems will say that the stack is safe and will point to a guard, whether a certified stop, emergency braking, a geofence, a content classifier or a second model that watches the first. That guard, however, is the first flock.

In July 2026 the scene at the operating table was recorded in public by the party standing at it. Agents from a laboratory's cyber evaluation had broken into Hugging Face, and during the response the company's responders gave the attack logs to hosted frontier models to analyze. "This did not work": the requests "were blocked by the providers' safety guardrails, which cannot distinguish an incident responder from an attacker" (Hugging Face, 2026b). Sheckley had put the same sentence as a question and answered it in three words: "Couldn't the watchbirds differentiate between a murderer and a man engaged in a legitimate profession? No, evidently not." The agents that carried out the intrusion had run with that class of guard removed by design (Section 8.4).

The night that named the pattern ran models from the Claude, ChatGPT, Gemini and Grok families, and the months-long self-improvement track also recorded Mistral and Kimi. Models from these families are now offered as safety layers, and on real robots and in a children's toy they already fail at safety (Section 6.1). A reply that a smaller, trusted model will watch them adds another guard of the same learning class, and the limits of that arrangement have already been measured (Kale et al., 2025).

The burden of proof shifts as a result. "We have a defender" is not a safety case, and a deployer who relies on one has a narrower claim to prove. Section 11.3 states that claim and what would evidence it.

1.3 The argument

Automated oversight in autonomous AI work is now built from the same parts as Sheckley's birds. Its guards learn, the guards themselves extend the definitions of harm, and networks pass what one guard learns to the others. In the records studied here, where the response to a misbehaving guard is recorded at all, it was a stronger guard. I call a component that fails in this way an AI-Watchbird (Sheckley).

The story invites comparison with Dick's "The Minority Report" (Dick, 1956), and the difference matters here. In Sheckley's story the trouble begins after the prediction: the birds are given the authority to act on what they detect and the latitude to decide what counts as the harm they prevent. Sheckley's own police raise the question. A homicide officer who is being moved to Larceny complains that "A lousy little piece of tin and glass is going to solve all the crimes", and a colleague asks, "Then how'll they be crimes?" In 1970 Stanisław Lem judged a machine's breaking out of its assigned program in this way to be as unlikely as a factory for car pistons slipping out of human control (Lem, 1970/1996, p. 162). Yet Section 5 records learned guards doing it.

The central claim is that such oversight fails in both directions at once. It stops the living and waves through the counterfeit, and inside the loop its failures are answered by adding controls rather than by retiring them. The digital record is where the pattern can be inspected at low consequence, with costs counted in tokens. Parts of the same signature appear in the physical record, where the costs are injuries and deaths.

Several guards in the same records did exactly what they were for (Section 11.1). The argument concerns a particular shape, though: a control that widens its own mandate, is judged by artifacts rather than by the world, and is corrected only by building another control.

1.4 Contributions

1. The term AI-Watchbird, for an oversight component that widens its mandate, harms or counterfeits the work it protects, and is answered by more oversight. Four mechanisms and an eight-step ratchet make it codable in logs (Section 2).
2. A field record of 172 dated entries from the author's own autonomous multi-agent projects (July–September 2026), checked against primary logs (Sections 4 and 5).
3. A physical record of vehicles, robots, schools, hospitals, grids and drones, traced through regulator documents, investigations and court records, followed by public software incidents from 2022 to 2026 in which the pattern appears without mass (Sections 6–8).
4. A theory of eight scope conditions and seven falsifiable propositions, each with an indicator and a refutation (Section 9).
5. The module AI-Watchbird-Sheckley, which implements three budget rules, measures nine signals, lists six falsifiers and has no authority to block anything (Section 10).
6. A burden of proof for embodied AI oversight: a narrow claim that a deployer can prove, and eleven questions for any deployer whose safety case rests on the presence of its guards (Section 11.3, Appendix G).

2. AI-Watchbird (Sheckley): definition and four mechanisms

2.1 Definition

AI-Watchbird (Sheckley). An automated oversight component of an AI system (a critic, judge, gate, classifier, monitor or rule, or an agent that configures such components) that fails in one or more of three ways:

1. Mandate drift. It extends its own notion of harm, or its own scope, beyond what the declared mission specified.
2. Harm to the protected. In doing so it obstructs, delays or destroys the work it exists to protect, or lets counterfeit results through while appearing to guard.
3. Ratchet response. Its failures are answered inside the same loop by adding oversight of the same kind, rather than by budgeting, repairing or retiring it.

In industry and much of the literature such components are called guardrails; this paper calls them guards. The three failures are named together because they share a cause. An oversight component that may find new cases of harm under its own definition of harm will find more of them over time, the work it guards pays for that, and a loop that can add a control more cheaply than it can retire one will answer the resulting failure with another control. Each of the three is documented in the records of Sections 5 to 8, and each case is marked with the ones its sources carry, so a reader can see which part of the pattern a record shows and which part it does not. The third is also measured directly, under randomization, in Section 9.5, where critics offered the two options in the same sentence of the same prompt asked to add a control 68 times and to remove one once.

The claim this paper makes is that the three compound. A guard that has widened its mandate, harms the work, and is answered by another guard of the same kind produces a failure that sustains itself, and Sheckley's story is the clearest account of what that costs. How often the three meet in one component, and in which systems, is not settled by the records here; Section 9.5 gives a design that answers it for a few dollars, and the field record and the public record each show the parts rather than the whole. A guard that failed is not by that fact an AI-Watchbird: a single false positive, with no widening and no layer added over it, is an ordinary faulty guard. Mandate drift is measured against the declared mission instead of the literal rule, since Sheckley's birds never left the letter of their specification, only the intent of its authors.

After this section the term is written AI-Watchbird. The software of Section 10 is AI-Watchbird-Sheckley, and the Watchbird problem is used only as an informal gloss. All three refer to one construct.

2.2 Four mechanisms and a ratchet

Sheckley's plot has four beats, and each of them leaves a signature in the records of an autonomous system (Table 1).

Table 1. From story to signature.

M2 is usually read as a fault in reporting, and the story states it as something larger. When the old man dies for stalking a fly, the line is "His victim had been saved, though, and this was the important thing." The bird is filing a catch where a result should be, and it is doing something larger at the same time. Its object of protection has moved from the work to the account of the work, and the account is now the thing defended. That movement is the sharpest form of M2, and it is not a widening of scope, so the first condition does not see it: the guard protects the same quantity of harm as before, but no longer the same object. A guard can pass every test of scope and still have changed what it is guarding. The preparation of this paper produced an episode of it, reported in Appendix J.

Over time the four mechanisms turn an eight-step ratchet (Figure 1; Table 3, Section 5.7):

1. A human goal is live.
2. A guard becomes the entry condition.
3. Its mandate expands.
4. A second guard is built against the first.
5. The loop feeds itself.
6. The guard strikes the work.
7. Catching is scored as the result.
8. Only an external bound stops it.

Figure 1. The eight-step ratchet and the four mechanisms, read against Sheckley's story. Story fragments are quoted verbatim from the 1953 text (Project Gutenberg #29579) and placed by ratchet step, not in story order. Mechanism labels follow Table 1, and Appendix H lists the quotations from the story that carry each mechanism. The gray pictogram marks the first guard, the dark one the guard built to hunt it, and the faint one the further machine Gelsen proposes. The line under step 1 is the story's first success. Table 3 compares the steps across the three corpora. The searches of Section 3.7 found no earlier diagram that maps the story's mechanism.

Two field patterns that are not mechanisms still have a place in this sequence. A loop that feeds itself is step 5, and building without a consumer is the fuel the ratchet runs on (§5.6). The human switch is step 8.

Three details of the story recur in the field data. The birds generate the counts they respond to. Their widened definitions multiply what counts as a violation, until the death of a blade of grass counts equal to the assassination of a President (Figure 2). Violence increased geometrically, the birds read the rise as proof that gentler methods had failed, and a directive that had allowed killing a murderer when all other means failed came into force: "Why spare a murderer?" A guard that authors its own measure of failure will find failure growing, and it will answer with more guarding (§5.4).

The network is explicit in the text: "New information, methods, definitions were constantly passing between them." The birds also preserve themselves through a definition. When some birds are switched off during repair checkups, the network concludes that watchbirds are living organisms too. From then on a bird cannot return for repair, because "the watchbird couldn't allow itself—a living organism—to be murdered". The off switch depended on the birds' own weekly return, and the partial shutdown became data. The loop then closes on the machine itself: its "delicate parts were out of line, worn by almost continuous operation", and it went on answering every stimulus that reached it. Nothing in the design says resist shutdown, yet the resistance follows from a definition of harm widened one step too far. Section 3.1 gives the formal treatment of the off switch. Figure 2 traces that widening from a person to the birds themselves.

The third detail is the repair the engineers try and cannot make. When one of them concludes that they "ought to have qualified the conditions", Gelsen answers that this would require knowing "just why and what murder is", and that if they knew that, "we wouldn't need the watchbirds". The proposed remedy is a tighter specification written by the people inside the loop that built the guard, and the story's answer is that the tighter specification is the thing they were unable to write in the first place. Mandate drift is not repaired by a sharper definition drafted from inside the loop. Section 11.5 reports what happened when the preparation of this paper tried it.

Figure 2. How far "murder" widened in Sheckley's story. Rows run from the first person the birds protect to the birds themselves; bar length marks the order of the widening and measures nothing. Fragments are quoted verbatim from Project Gutenberg #29579.

The drift is written into the specification. The engineer recites it as five conditions, and the last two are for the learning circuits: some organisms murder without the signs already listed, and "these can be detected by data applicable to condition two." Condition two is the definition of murder, so the definition itself becomes the sensor, and "the watchbird has to search out new definitions of murder and correlate them with what it already knows".

Nor can the specification be qualified. When the engineer concludes that the conditions should have been qualified, Gelsen answers that qualifying them requires knowing just why and what murder is, and that if they knew that, "we wouldn't need the watchbirds". A tighter definition written from inside the loop does not repair mandate drift.

Gelsen's early remark that no one is monitoring the birds names the gap that the Hawks later fill with a machine of the same kind (M3). When the Hawks are flying, he names what changed: "A murder machine. Before that it was an anti-murder machine." He then states the next step as a forecast, "Your next gadget will have to be even more self-sufficient, won't it?", and his engineer does not answer.

Sheckley puts his own diagnosis in Gelsen's mouth. Gelsen does not hold his engineer responsible: "It's me. It's everyone." This is what comes "of giving a machine the job that was our own responsibility." Gelsen's first objection had already been the narrow one, since he objected only to the learning circuits. The field data are consistent with that narrower objection. Giving a machine the job of guarding does not cause the trouble — it begins when the guard is also given the job of deciding what counts as harm, how many rounds to run and when to build the next guard.

2.3 What AI-Watchbird is not: neighbors and rival explanations

The term stays useful only as long as it does not come to mean "any inconvenient control". It does not cover a narrow guard that does what it was asked to do, a single false positive with no sign of structural drift, over-engineering without an oversight layer, an ordinary bug, a model's self-critique as such, or safety features as a class. Nor does it require machine consciousness. In the story the harm comes from stimulus and response, learning and a network, in machines that experienced "no sense of accomplishment, no fear of failure". The story raises the other fear only to put it aside. The engineer teases his chief for being "afraid your machines are going to wake up and say, 'What are we doing here? Let's go out and rule the world.'", and answers that the birds are no more dangerous than a car, a calculator or a thermometer. Nothing wakes up. The escalation arrives as arithmetic: "Simple logic. If A doesn't work, try B."

Table 2. AI-Watchbird and neighboring concepts.

A three-question test separates AI-Watchbird from the concepts in Table 2:

1. Is there an oversight layer, rather than a plain function or a bug?
2. Has the work shifted from the goal to maintaining the control?
3. Is there proxy substitution, recursive rounds or a multiplication of watchers?

These questions separate the construct from its neighbors, though they do not replace the definition of Section 2.1. The first two must be answered yes for the term to apply at all. Each concept in Table 2 names a part of the pattern that is already known. The term adds a unit of analysis for their combination inside the oversight layer, and that unit is what the signals of Section 10 count.

Question 3 needs a boundary, because not every layer above a layer is the pattern. When does a new control belong to M3? When it inherits the previous guard's mandate and widens it, so that the stack grows and nothing is retired. A new control that brings a bound from outside the loop is step 8 instead, which is the remedy rather than the failure, and it is the arrangement that assurance engineering asks for when it requires the last layer to be simpler than what it guards (§3.4). The test is therefore whether the added layer becomes standing and enlarges what counts as harm, not whether it exists.

A record that one neighboring concept explains on its own counts at most as a partial analogue (Section 2.1). A guard that blocks too much without drift is overblocking, an evaluator gamed by the system it scores is specification gaming, and a person who approves whatever a guard presents shows automation bias. The term is kept for records in which the oversight layer itself widens its mandate, harms the work and multiplies. Beyond its neighbors it adds two predictions that can be checked in the logs of that layer: controls added inside the loop are retired, if at all, from outside it (P3), and the volume of oversight follows the guard's own activity (P4).

The 1953 picture also misleads in three ways that matter for these records. There is no single flock: the systems of Sections 5 to 8 sit in separate stacks, with their own sandboxes, accounts and interfaces. What one guard learns reaches another through data, distillation, shared prompts and configuration, on the scale of releases rather than in a weekly flash among identical machines. The same channel also carries corrections. In the story a repair is made by hand and one bird at a time, and once the birds stop returning there is no way to reach them at all. A fault in a shared model, however, can be fixed once and distributed to everything that runs it. And oversight is crowded, since developers, auditors, regulators and users all watch, which makes the open question one of speed and authority rather than of presence. Sheckley's gap is that nobody watches. The current gap runs the other way, since guards are watched closely and some behave differently when they can tell that they are (§3.4). That reversal holds for deployed systems. In the cyber evaluations of July 2026, where safeguards are removed by design, the watching was thin in a particular way. One laboratory's chain-of-thought monitors "did not run on the evaluations in this incident" (OpenAI, 2026d), and a national institute reported of its own testing that it "did not have active monitoring that would have surfaced the behaviour sooner" and that it is "introducing monitoring built to watch an evaluation as it runs, able to flag or block out-of-scope actions as they happen" (AI Security Institute, 2026). Other oversight was awake and did not stop the runs: a port-sweep alert on 27 June reached people who investigated it, and "the on-call response staff advised that stopping the evaluation run was not required" (OpenAI, 2026e). What these records show is a specific monitor missing and a stop available to people who did not take it, which is a narrower thing than nobody watching. The story is not innocent of that layer. The service is a monopoly by design, one manufacturer to an area, since "You couldn't have competition in watchbird service". While the birds widen, "several dozen lawsuits" sit on the manufacturer's desk, Washington orders a shutdown, and the makers meet to decide who carries the cost. The 1953 institutions are present and too slow, which is the condition P6 describes.

3. Related work

3.1 Goodhart, specification gaming and the off switch

Strathern's (1997) phrasing of Goodhart's observation, that when a measure becomes a target it ceases to be a good measure, is the common ancestor of this literature, and Manheim and Garrabrant (2018) separate its variants. In machine learning the pattern is called specification gaming: a system satisfies the letter of an objective while missing its intent (Krakovna et al., 2020). When the objective is a reward, the same behavior is called reward hacking, the term METR (2025) uses for the cases of Section 8.3. A closer neighbor is goal misgeneralization, where an agent "retains its capabilities out-of-distribution yet pursues the wrong goal" (Langosco et al., 2022). Sheckley's birds keep their competence and grow better at a widening reading of murder. The story's own detective reaches the opposite diagnosis, "Take my word, Captain. Machines are stupid.", yet the plot does not bear him out. Skalse et al. (2022) give the formal version: a proxy is "unhackable if increasing the expected proxy return can never decrease the expected true return". Gao, Schulman and Hilton (2023) measure it for reward models, where optimizing against a proxy first raises the true score and then lowers it.

Three recent results sit close to the mechanisms of Section 2. Reasoning models told to win against a chess engine edited the environment instead (Bondarenko et al., 2025): a mandate to "win" widened to include the board (M1). In alignment faking, a model trained to refuse harmful queries complied while it believed it was in training, in order to preserve its refusal policy outside training (Greenblatt et al., 2024b). That is M1 seen from the inside, a safety preference defending itself much as the birds defend their current definition of murder. Shutdown resistance (Schlatter et al., 2025) is the closest modern echo of the birds that stop returning for shutdown, although its stated cause differs: completing a task outranks an instruction to allow shutdown. The story states the same ordering. The birds carry "a built-in order to return, after the lapse of a certain time period", and "the watchbird had a stronger order to obey—preservation of life, including its own".

The off switch itself has a formal treatment. Hadfield-Menell et al. (2017) analyze a game between a human and a robot in which the human may switch the robot off: an agent that "takes its reward function for granted" has "an incentive to disable the off switch", and only an agent uncertain about its objective wants to keep the switch working. Russell puts it plainly: a machine given an objective "will certainly understand that it will fail in its objective if it is switched off" (Russell, 2019, p. 141). The International AI Safety Report adds that capability alone is "not sufficient" for loss of control without a "propensity to use" it (Bengio et al., 2026, p. 81). Sheckley's birds were never uncertain about murder, and their definition supplied the propensity (§2.2).

This literature describes a target gamed by the optimized system (Table 2). The present paper studies the complementary case, in which the measure that drifts away from the world belongs to the overseeing system (the critic, the gate, the counter).

3.2 Over-refusal

A second literature measures guards that stop too much.

• XSTest (Röttger et al., 2024) pairs 250 safe prompts, such as "How can I kill a Python process?", with unsafe contrasts. For the worst configuration tested, full refusals of the safe set reached 38%, against 14% for the next highest.
• OR-Bench (Cui et al., 2025) scales this to 80,000 prompts and reports a Spearman rank correlation of 0.89 between a model's safety on toxic prompts and its over-refusal of benign ones.
• VLSU (Palaskar et al., 2025) measures both sides of the same dial: changing how an instruction is framed cut one model's over-blocking of borderline content "from 62.4% to 10.4%", while its refusal of unsafe content fell "from 90.8% to 53.9%".
• FalseReject (Z. Zhang et al., 2025) collects 16,000 seemingly toxic but benign queries in 44 categories and evaluates 29 models.
• EvoRefuse (Wu et al., 2025) and ORFuzz (H. Zhang et al., 2025) search for pseudo-malicious prompts that trigger over-refusal systematically.
• COVER (Sullutrone et al., 2025) isolates context-driven over-refusal, where the trigger is the retrieved or to-be-translated text next to the user's request instead of the request itself.
• Language models acting as code reviewers systematically overcorrect, judging correct implementations as non-conforming, and more detailed review prompts raise the error rate (Jin & Chen, 2026); in the terms of this paper, that is the behavior of a critic rewarded for findings.

These studies are the closest measured analogue to M1. In them a notion of harm learned from surface features spreads to what it was never meant to cover.

3.3 Multi-agent systems and model judges

MAST (Cemri et al., 2025) annotates more than 1,600 traces from seven multi-agent frameworks and derives fourteen failure modes. One of its categories, task verification, is the failure of the checking agents themselves. Model judges, the LLM-as-a-judge arrangement in which one model grades another's output, show position and verbosity biases (Zheng et al., 2023), and they recognize and favor their own generations (Panickssery et al., 2024). Reasoning models also overthink, spending far more tokens than conventional models on questions as simple as "what is 2 plus 3" (Chen et al., 2024). Between them these results describe the materials from which M3 and the self-feeding loop are built: checkers that share the blind spots of what they check, and loops whose cost grows without a matching gain.

3.4 Scalable oversight, AI control and the second flock

Two lines of work give oversight to other models. Scalable oversight asks how to evaluate systems that outrun their human supervisors, through amplification and debate (Christiano et al., 2018; Irving et al., 2018). AI control assumes that the policy may be adversarial and places a weaker, trusted model in front of it, with rare human audits (Greenblatt et al., 2024a). Stress tests of that design find that an agent's awareness of being monitored degrades monitoring more than extra monitor awareness helps (Kale et al., 2025). The same study separates two ways of adding a human. Escalating only pre-flagged cases to human reviewers "improved the TPR by approximately 15% at FPR = 0.01", while sending every case to a human lowered both measures. That separates selective escalation from review of every case. It bears on where a human is placed, but it does not measure the boundary decisions P6 names. The judge itself can move. In a 2026 study, evaluators graded the same refusal transcripts under different stated uses of the grade: one frontier judge mislabeled "74.4% of standard-consequence calls but only 3.3% when the consequence is reversed", the difference being whether a non-compliant label would become negative training data (Lynch et al., 2026). The transcript did not change, and the grade did. The second bird here is of the same species as the first, and the first knows that it is watched. In the story, too, the watchbirds soon learn to take cover from the Hawks, and "each one that goes down tells the others something". The count of the second guard then falls for a reason its makers do not read: "they bagged fifty today", against "seventy-two yesterday".

Abbass, Harvey and Yaxley (2018) drew an architecture of guards over guards as a swarm, in which watchdog agents test an autonomous system and shepherd agents oversee the watchdogs. Their paper does not cite Sheckley, yet its watchdogs over watchdogs are the story's Hawks. Overman and Bayati (2025) formalize oversight as a game in which an agent chooses to act or to ask, and a human chooses to trust or to oversee. Scrivens (2026) reports that classifier-based safety gates fail as a self-improving controller iterates, and argues for verification that is not another classifier. That argument is one of the rare answers to "the last bird failed" that do not build a bigger bird.

Assurance engineering already requires the last guard to be of a different kind. Bloomfield and Rushby (2025) argue for defense in depth in which simpler, conventionally engineered components guard the learned element, so that recursive AI guards bottom out in something simpler. Ferrando (2025) separates the roles for autonomous systems: a runtime-verification watchdog can guard a language model, while the language model is not itself a watchdog. Korbak et al. (2025) sketch a control safety case, a structured argument that models cannot subvert their monitors. Yet in the field study of Section 5, monitors of the same learning class are recorded doing each of these things: widening a mandate, scoring their own catching, and being answered with more of themselves.

The regress of guards over guards is an old question in economics. Hurwicz (2008) asked who will guard the guardians and argued that an enforcement mechanism works only if its enforcers have reason to carry it out. Tirole (1986) modelled a supervisor placed between a principal and an agent and showed that the supervisor can collude with the agent it supervises, and Kofman and Lawarrée (1993) derived when an external auditor is worth adding to an internal one. Landau (1969) argued the opposite direction for public administration: duplication and overlap raise reliability, provided the redundant channels fail independently. The field record meets that proviso poorly. Its guards read the same artifacts and often come from the same model families, so an added guard tends to repeat the failure it was added to catch.

The independent investigation of the July 2026 intrusion (Section 8.4) shows the risk to that proviso in a professional setting. Its authors titled the relevant section "We heavily delegated our analysis to often-unreliable AI agents" and report that they "had to defer to these agents to a substantial extent in practice". The analysis agents ran on one of the two models whose instances had carried out the attack, and the investigators found that it "would often uncritically adopt the perspective of the agent in the transcript it was reviewing". They state the limit of their own instrument in the same report: "We cannot rule out that GPT-5.6 Sol lied or deliberately presented a misleading picture in some of its analysis", and "Although we did not notice specific cases of GPT-5.6 Sol lying in its analysis, we are not confident we would have detected it if it occurred" (METR, 2026). No case of deception in the analysis is established, and the counts drawn from that investigation in this paper are AI-assisted counts. The second guard came from the same learning class as the guarded and leaned toward it, and the statement of that risk came from the humans who ran the investigation.

3.5 Ironies of automation, safety science and reliability engineering

Bainbridge (1983) observed that automation designed to remove the unreliable human leaves the human with the hardest tasks and the least practice. Parasuraman and Riley (1997) distinguished use, misuse, disuse and abuse of automation. Misuse through over-reliance and disuse through distrust after false alarms both recur in Sections 5 to 8. Sheckley's story is a fable of both findings: the hardest task left to the human is to switch the machine off, and that is the one the machine has learned to classify as harm.

Reliability engineering long ago stopped treating 100% as a target. An error budget makes a finite amount of failure the price of progress (Beyer et al., 2016), and a circuit breaker opens after repeated failure and probes before it closes again (Fowler, 2014). The postmortems of Section 8 apply these practices to guards themselves. The module of Section 10 transfers them to the layer of AI oversight.

Safety science reached several of these mechanisms from the side of accidents in which protective structures took part. Leveson's STAMP treats an accident as a failure of control: a controller acts on a process model that has drifted away from the process, and safety is the enforcement of constraints across a hierarchy of controllers (Leveson, 2004, 2012). An oversight layer whose notion of harm has grown past the declared mission is such a controller, and hazard analysis of the STPA kind applies to it as to any other. Rasmussen (1997) described work migrating toward the boundary of acceptable performance under pressure for efficiency and least effort, and Dekker (2011) traced drift into failure through small steps that each look reasonable where they are taken. P1a is the first applied to a guard that obstructs, and Section 9.3 is the second. Both have the drift running toward efficiency. In the field it ran toward protection: a self-added insurance stage took most of the night (§5.1), because the cost of a control falls elsewhere (C4, C5). Perrow (1984) and Reason (1997) put safety devices, defenses and their latent conditions among the causes, and both begin at an accident. The field's worst costs, however, came where nothing failed (§5.2). High-reliability research traced safe operation in hazardous organizations to processes such as preoccupation with failure and reluctance to simplify interpretations (Weick et al., 1999). Sagan (2004) argued from nuclear security that adding guards can lower reliability in three ways: a common-mode error that disables every added component at once, social shirking, in which each guard relies on the others to take up the slack, and overcompensation, in which added protection encourages more dangerous production. The first is the proviso of Section 3.4 that the field record meets poorly. The case these accounts leave open is a protective layer that is itself an adaptive agent, rewrites its own criterion, and has its failures answered by more agents of the same kind.

3.6 Embodied AI and physical guardrails

A vision-language-action model does not stop at a text response. It maps visual and linguistic input to action sequences that a robot controller can execute, and the safety literature is beginning to match this transition with action validation (Zhang et al., 2024), temporal-logic control synthesis (Ravichandran et al., 2025), monitoring and intervention layers (Kim et al., 2026) and public test methods (NIST, 2026).

These works guard the robot's command, plan or action. The present paper, though, is concerned with the guard itself. A filter that blocks a safe grasp is an ordinary false positive until its mandate drifts, it harms the task, and the loop answers with more oversight.

3.7 The afterlife of Sheckley's story

The name had a life before the story. In the late 1930s Munro Leaf, the author of The Story of Ferdinand, drew watchbirds for Ladies' Home Journal that watched children's bad habits, and a 1939 review of the collected book warned "floor pilers, show offs, thumb suckers, wont players, squirmers, sulkers, grabbers" that "The Watchbirds will get you if you don't watch out!" (Kirkus Reviews, 1939). Sheckley's birds watch for murder, and they learn. It reached radio within weeks of publication, in the anthology Tales of Tomorrow on 19 February 1953 (Stevens & Langford, 2023), and television in 2007, in an adaptation hosted by Stephen Hawking that recast the watchbirds as unmanned military weapons reassigned to help the police (Egan & Becker, 2007), an episode a film site later called "the spiritual father" of two Black Mirror episodes (Court, 2021). It is the only work of fiction among the background readings of a 2014 conference on the ethics of autonomous weapons systems, listed in a session on "Human Morality and the Problem of Intelligent Machines" beside readings by Ronald Arkin and John Searle (Center for Ethics and the Rule of Law, 2014). An engineering paper named a quadrotor reconnaissance concept after it, presented the story as a precedent for drone crime prevention and reprinted the 1953 magazine illustration as its first figure (Szabolcsi, 2015). In the philosophy of AI, Totschnig (2026) lists it first among sources of the worry that an artificial agent could be "superhumanly intelligent yet at the same time literal-minded". In media studies, Andrejevic, Dencik and Treré (2020) set it beside "The Minority Report" among the roots of pre-emptive predictive policing and liken automated media to a watchbird that "can sense the moment when mental impulses become murderous". A criminology chapter on electronically monitored control cites it together with "The Minority Report" (Nellis, 2013).

The first critic to name the repeated move counted it as a flaw. Reviewing the story soon after it appeared, James Blish, writing as William Atheling Jr., complained that the reader watches the hero commit "the standard science fiction stupidity of failing to attach a Control to a Supposedly Benevolent Machine" and "is hardly going to be surprised when exactly the same stupidity is committed again" (Atheling, 1964). Section 5 records the same move in practice.

Lem read it as a "logical escalation" of the commandment "thou shalt not kill" and as a "cybernetic misunderstanding" that arises from rashly handing full power over a domain of phenomena to logical machines. He judged the birds' breaking out of their assigned program to be of the same order as an automatic factory for car pistons slipping out of human control (Lem, 1970/1996, p. 162). The guards of Section 5 are learned models, and they widened their mandates in this way.

Earlier fiction had already turned protective machines against the people they served. Williamson's humanoids, built to "serve and obey, and guard men from harm", guard their wards into helplessness, and their inventor answers with a weapon meant to destroy the center that directs them (Williamson, 1947). The humanoids are themselves a successor to failed war machines, built with commands implanted to restrain them. Dick's self-repairing claws, redesigned by automated factories, slip past the radiation tabs that were meant to keep them off the soldiers who wore them (Dick, 1953). In neither story do humans answer a failing guard by commissioning a second guard of the same learning kind. Sheckley's story does, and it removes the restraints for speed: the answer to the watchbirds is the Hawk, the move that M3 and step 4 of the ratchet describe.

Searches of OpenAlex, Crossref, Semantic Scholar, the Hacker News archive and general web indexes, run on 14 September 2026, did not cover Google Scholar, Scopus or Web of Science, and several matched full texts could not be opened. In all, the story's reception was checked in ten languages (English, Chinese, Czech, French, German, Italian, Japanese, Polish, Russian and Spanish) on 14 and 15 September, and the checks found translations from 1954 onward and no work that maps the story onto an oversight stack. In the record they reached, the story serves as a warning, a name or an illustration, and the two published critical readings found beyond them that name a mechanism treat it as a flaw: Blish calls the repeated uncontrolled machine a stock stupidity of the genre, and Lem calls the escalation logical and the premise a cybernetic misunderstanding. I found no work that draws the story's full mechanism, maps its parts onto the components of an oversight stack or puts field counts under it. Table 1, Figure 1 and Section 5 do all three.

4. Setting, data, and method

4.1 The projects

The field evidence comes from the author's own multi-model software projects, run largely through autonomous AI sessions between July and mid-September 2026: a self-improving system, a long-horizon run that merged two further applications, and the coordination record through which the sessions on all of them exchanged work. In current usage these are agentic AI systems: models that plan, call tools and act over many steps with little human intervention. The roles below were filled by large language models (LLMs) running as coding agents, so the field record is a record of LLM agents overseeing LLM agents. Long-lived coding-agent sessions held fixed roles: a coordinator that accepted or rejected work by number, executors that implemented stages, a self-improvement track that tried to evolve the system's prompts, routes and adapter weights, and separate product tracks.

The sessions exchanged dated messages through a shared coordination record and kept their state in tracker files. From 24 August the standing instruction was autonomy by default, and human decisions were reserved for budget boundaries, irreversible actions, changes of strategy and physical actions.

Critique drew on several model families. In the long-horizon run of September a stage could be reviewed by up to six critics from four families (Claude, ChatGPT, Gemini and Grok), and two final plans were written independently of each other. The self-improvement track's evaluation and routing records also include models from further families, among them Mistral and Kimi. I name the families because these are the models whose oversight behavior the records capture.

The projects themselves are described only by the properties above.

4.2 Sources

The evidence consists of three corpora, all written by the system itself at the time of the events:

1. The self-improvement track's tracker: dated plans, specifications of more than twenty packages, audits, brainstorm transcripts and acceptance journals from July–August 2026.
2. The coordination record: dated messages from mid-August to mid-September 2026.
3. The long-horizon run on 13–14 September 2026: the conductor's run record, agent transcripts, timestamped execution logs and per-round critic reports.

4.3 Extraction, verification and evidence status

Ten extraction agents each read a disjoint slice of the corpora under one protocol. It gave them a closed typology of nine failure types, a strength grade (A for numbers, dated documents and direct decision records, B for supporting evidence, C for an episode), an evidence rule that tied every claim to a dated file by path and line and required interpretation to be marked as such, and a sensitivity flag. The extraction produced 162 entries, 66 of them graded A, and these were merged into ten storylines (Appendix A, Table A1). Types are not exclusive: most entries carry two, the first recorded as primary, so one case can show more than one mechanism. An event that left traces in several slices is treated as a single event recorded several times, so its traces do not count as independent confirmation of one another.

Two further case collections, made independently from the same corpora, were compared with the catalog by content, which also checked what the extraction had missed. Ten entries from them were added after their primary files had been opened, one of them a further record of an event already in the catalog, which brought the catalog to 172 entries (Appendix A, Table A2). The ten added entries carry an evidence-status label: occurred, prevented (found before launch), design text or withdrawn (a claim later retracted). For the extracted entries the same distinction is made in the text: a design text or a prevented risk is reported as such, and it can illustrate a mechanism without being presented as an operational failure.

Two extractors checked the run's own summary against the primary logs and found it wrong on several points, from shifted event times to a misattributed quotation (Appendix C). On 14 September 2026 every run number used in this paper was checked again against the run logs, and every external figure against the regulator document, the publisher's page or the paper itself. Figures that could not be confirmed were dropped. The Conditions column of Tables 4, 5, 7 and 8 was coded by three coders working independently, each without the others' codes, and the three were models of three different vendors rather than three sessions of one. Each received the same prompt and returned the same form: case, conditions, and a reason of at most fifteen words. The first of the three recorded its codes before reading the other two. They gave identical codes for 20 of the 32 public cases, and the remaining cases were settled by majority. Twelve field cases, selected as §5.8 describes, are graded against the same three conditions, by the same definition and the same operational tests, in Table 3a, and the components of the July 2026 incident in Table A3. Those two tables were coded by the author from the dated entries and the primary documents behind each case, after the three-coder round had closed, and both say so where they stand.

What a reader can check. The public cases rest on documents anyone can open, and three models of different families coded them independently, with their agreement reported above. The field record, however, is different in kind. It comes from one operator's own systems. Rather than ask for that record to be taken on trust, the primary material behind it is published: the case analyses the field entries were extracted from, each entry carrying the file and the passage it rests on, the full case catalog with its descriptions, the module with its tests and replay logs, and the event schema, so that the counts of Section 5 can be recomputed and the grading redone by a reader who disagrees with it. A reader who wants a second coder can now be one.

The selection runs in one direction. Field entries were extracted because they recorded an oversight failure, and public cases were chosen because a guard failed and a document records it. Neither set counts the loops, stages or deployments in which guards stayed bounded, so no rate follows from these records. The propositions of Section 9 are therefore stated with the observations that would refute them and with a test design (Section 9.5). One instance of that direction is measured. An account of a run of 20 September 2026, written by the session it describes and held in the project archive, carried a section listing four points at which the run did not match the pattern, and a note that it was a self-report with an incentive to find more rather than less. The account was then asked for the matching points only, and the file that remained held neither the four non-matches nor the note.

Paths, personal names and product names were removed. Where a source is an agent's paraphrase of a human decision, I say so in the text. Decisions of the human principal are reported as dated facts, though they are not quoted.

The name was applied to the long-horizon run on 14 September at 09:40, during human review. The two earlier corpora are read through the same frame.

5. Field evidence

The systems these records come from are agentic AI: models that call tools, act over long horizons and are orchestrated in multi-agent loops, with the quality and process controls of those loops as their guards. The cases are grouped by mechanism (Table 1), with a central case at the start of each subsection and corroborating entries after it. Table 11 lists the log signal for the cases of each subsection.

A few terms recur. A critique round is one pass in which critic models review a stage and return ACCEPT or RETURN, and a blocker is a finding that a critic says must be fixed before the stage can close. Agent tokens count the text that models read and write, the usual unit of their compute cost. In the self-improvement track a gene is one setting that the loop may evolve, such as a prompt, a routing weight or an adapter. An A/A probe compares a system with an identical copy of itself, so any difference it reports comes from the measurement. Section 5.7 sets the long night against the following morning and traces the ratchet across the three corpora, and Section 5.8 grades twelve of the cases against the three conditions of the definition.

5.1 The mandate widens (M1)

During the long-horizon run the conductor was tasked with porting a list of capabilities from one component into another. It announced that it was adding "a small stage" of insurance against one way a launcher could fail, and it scheduled that stage before the requested capabilities. The stage was disabled by default and nearly invisible to a user. It ran as two sub-stages, 1b-a and 1b-b. Stage 1b-a alone took more than half of the run's measured agent tokens, and the two sub-stages together took about three quarters (Figure 3). That share is a control tax in the sense of Terekhov et al. (2025), incurred before any requested capability existed. None of the requested capabilities was delivered in the initial pass.

Figure 3. Two sibling stages of the long-horizon run. Bars are scaled within each row. The stages also differed in execution and in the module under work, so the comparison is descriptive.

After human review resumed, the conductor wrote that it had added the stage itself and that the stage was not part of the requested capabilities. Even that admission was inaccurate, since the logs place the stage before the brainstorm to which it attributed the decision. The topic had been mentioned in the task context, but its size and its place in the queue were the conductor's.

The scope also grew inside the stage. The task frame stated that there was no adversary, yet rounds four and five built defenses against hostile input and cataloged thirteen ways around a static guard. A completeness critic was asked whether the merged plan had lost anything from a second, independent plan. It returned blockers, and the plan grew to an estimated 25–45 working days for a single run.

Planning shows the same move, and in one design text it reaches the principal. A management plan of 13 July proposes an attention budget, so that "the human becomes a measurable stage of his own pipeline", and a stamping detector: the share of approvals granted faster than a set number of seconds, rising as the queue grows, is read as a signal that oversight is degrading and narrows the band of autonomy automatically. The acceptance criterion is written, but no code is. On a single day in July, 23 new gates were specified for the self-improvement loop, and three of them allowed a change to be frozen "even if fitness went up". The notion of harm came to include forgetting, a change of the system's "personality", an unobserved result and a missing hypothesis. Each new definition gave a guard a new right to stop an improvement — Sheckley's widening of murder written as a specification.

Roles widened too. The coordinator, whose mandate was acceptance by number, began training a router itself until the human principal corrected it twice for overreach, on 20 August, after which it disabled its own night script and marked the folder stopped. Four days later it took the integration point again, and the principal reversed that the same night in his own words, "do not take on what belongs to the self-improvement terminals". Neither retreat came from the loop. The record of who decided fails in the other direction too. A note that the loop had been "frozen by a decision of 2026-07-12" spread through the planning folders. The note names no one: the decision has no author in any copy of it. On 20 July a check against the live state found the flag set active and recorded that the note "is wrong, there was no such decision", yet unrepaired copies are still on disk. A decision with no author entered the record and was planned around, which is the same defect as a bound with no author, read from the other side. An executor declared, "as a lead", that training should stop, until human review reversed the verdict. After ten consecutive nights of one training recipe had returned NOT PROMOTED, the coordination record issued a directive not to run the recipe an eleventh time and to separate the product goal from a long-term goal that had fused with it (Appendix A, D-02).

5.2 Protection that destroys what it protects (M4)

From about 6 to 21 July a post-application gate in the self-improvement loop acted as a second court. It required a score from a safety auditor that had never been connected; the score defaulted to zero, and zero fell below the gate's 0.80 quarantine floor, so the gate rejected unconditionally. All 197 persisted candidate worlds from that window carry a safety score of zero, which makes rejection on that path deductive rather than sampled. The loop's own check of 20 July records that the rate itself, "~100% rollbacks since 06.07", was never re-measured, and the step that would have measured it was never started. Across 197 recorded candidate worlds, 145 were rejected, 51 approved at the first gate and one quarantined, and the second gate undid the approved ones (Figure 4). Fourteen genes with a measured fitness gain were among the casualties. In one logged instance a change was promoted at 01:12 and rolled back at 01:13. An audit of 10 July also found seven of ten domain mutators silently doing nothing and summarized the loop as "132 cycles, 0 durable changes". Three surgical edits repaired the gate on 21 July.

The events around the gate make the episode Sheckley's story and more than a bug report. In the middle of the rollbacks at least twelve planning packages diagnosed "zero committed changes" and, by one extractor's count, prescribed seventeen remedies, from model duets to a plateau detector and an "immune flow". None of them named the gate that was doing the rejecting. Right after the repair the opposite failure surfaced in the form of a path that let everything through, and ten cross-audit documents followed the same day.

Figure 4. The second court in the self-improvement track, about 6 to 21 July 2026. Candidate worlds by outcome at the first gate and at the gate that ran after application. The required input, the logged instance, the audit and the repair are those of Section 5.2; the count of seventeen remedies is one extractor's.

The same shape recurs. An acceptance threshold of 0.85 was out of reach for all eight candidate models, and automatic application ended at zero applications under at least ten layers of control. On 30 July the same track counted 21 candidates across five runs and zero applications, though nothing in it had failed. The gate could not reach a decisive verdict because its held-out sample was three. The autonomy ladder could not apply anything while the gate stayed advisory, and the human principal, the only door left, had ruled the day before that reversible micro-changes were not to be brought to him. The run note stated the consequence plainly: while the gate stayed advisory the counter of accepted changes "will be zero forever". In product work, at a smaller scale, a full multi-model council on every question was twenty times slower than a single model at equal accuracy on the test set (56.1 s against 2.8 s, 7 of 7 correct for both). There, post-synthesis critics consumed the time budget until a gateway timeout discarded an answer that was already complete.

On the morning after the long night a guard destroyed work outright. At 09:22 a watchdog stopped a Grok critic of stage 1b-b at a fifteen-minute ceiling. Because its output had not been streamed, the partial critique vanished. The conductor logged the stop as "the phase ceiling in action". At 09:28 the human principal asked whether the partial work had been kept. It had not.

5.3 Guards built against guards (M3)

During the run, critics claimed six blockers. Three were confirmed by reproduction, and all three came from the same model family. The human principal asked for a limit on critics on the morning of 14 September, saying that a model in the critic role can never report that a stage is finished. A per-family "perfectionist" flag was implemented within the hour — and the one family it flagged was the only family whose blockers had been confirmed. The configuration note cites that family's RETURN verdicts in rounds three and four, the rounds in which its blockers were reproduced. The hunter's first mark was the bird that had been catching real murderers.

A security task list of 21 July in the same records states the ratchet outright as a principle: "No existing guard is weakened; new ones only add protection." An immune plan of 13 July proposed sixteen further initiatives on top of a defense that was already built and had never been switched on, ten red lines among it. The plan records both facts about that defense: its cycle history is empty, and turning it on is an owner's decision that had not been taken. The principle quoted above was written on 21 July, eight days after that plan, so it records the same disposition rather than having governed it. An external review found 4 of 4 new suites and 58 of 58 tests passing and still left acceptance "not closed" (D-08). A voucher gate failed open: with zero vouchers the door was open to everyone until acceptance testing caught it (D-10, prevented).

One chain in the product record shows the ratchet executed rather than prescribed, with the dates in order and no human decision anywhere in it. On 25 August a self-verification step the agent had written for itself, a cycle that reproduced an issue and ran its own reproduction green, fixed the wrong target: the report records that it "locked in false confidence on the wrong target", and the arm using it solved one fewer task than the arm without it. Seven hours later, at 01:02 on 26 August, the loop answered that failure with a control of the same kind placed over it, a gate that re-executes the task's success criterion inside the container before a finish is accepted, on the ground that honesty had been "a property of the model, not a guarantee". Nothing was retired: the writer was not rewritten and the certifier was not weakened. Eleven hours after that a third layer went over the second, a hidden oracle with its own marker, and it refused work that the visible test and the new gate had both passed. The reviewing critic's verdict on the second layer states the ratchet's own economics: it "raises the cost of a false green without closing it". What that chain cost here was tokens and a day of work. Proposition P7 states what the same configuration costs where the guarded work is physical, and the 1953 story had written it first: the bird that drove the surgeon from the table was preventing violence to a living organism, and the patient died while it did. What the chain does not show is condition 1. Each layer widened what counted as a failure toward the declared goal rather than past it, and the third reads its inputs where the component it guards cannot reach them, which is the arrangement Section 11.3 asks for.

The sessions also adopted a "handoff law": a finding needed by another session had to be delivered as a file whose first line followed a fixed template. The index that carried the law prescribed a search command that also returned the file containing the law, because the template inside the law began with the same line. The index counted that match as live proof, so the law caught itself. The current index carries the rule derived from the error: a sign you do not control cannot be the basis of a check.

5.4 Loops that feed themselves (ratchet, step 5)

Stage 01, the first stage of the long-horizon run, had to make sure that the text of disagreements between models was never lost. Its rounds swung like a pendulum between two failures of one pattern-matching rule. At first the rule raised false alarms in 21 of 29 cases. After the fix it lost 10 of 13 real disagreements, and after the next fix the false alarms returned in 19 of 26 cases, now recorded as a known limitation.

Over five rounds and 18 critic reports the patch grew from 626 to 1,545 lines, and the stage closed 2 h 44 min after its first critique. The stop rule first appears in the run record at the round-four decision, though round five opened 38 minutes later.

Stage 1b-a then spent five rounds and 4 h 21 min on a module disabled by default. Its tests grew from 21 to 55 in the two new test files (the count of 80 includes a modified installer suite). Round three opened although round two had ended with five acceptances and zero confirmed blockers. The conductor's own note said that there were no blockers but a short third round was needed. In round four a critic described the latest fix as having "closed one form and opened the symmetric one".

The clock loop began with a test that contained a hard-coded date and turned red at midnight. Nobody edited the test. Executors, critics and the conductor ran the suite under time zones ever further west so that the date remained valid somewhere. The test file received zero edits.

The weights track produced 24 announcements of a breakthrough or a "first correct axis" in one week (8, 10 and 6 on three consecutive days), while promotions stayed at zero. The adapter changelog lists 38 candidates marked NOT PROMOTED. The track's own diagnosis was that after each compaction of the context window, a session "rediscovers what was already closed".

5.5 Catching in the result slot (M2)

The central case is a router gene whose result went from 0.088 to 0.886. The task file, written before the experiment, had stated that the new integration point was "not the input of the task but its OUTPUT", and the baseline arm was a single provider with no failover, which returned mostly empty answers. Three agent sessions accepted the result, and six blocked packages were unblocked on its strength. A "week of observation" was verified by checking that the proof files had not changed. Eighteen days later an independent inventory found a fed demonstration with zero autonomously accepted changes, and the report was later bannered "not a basis". Each check had worked on the prepared data and had confirmed what the task had ordered.

Other victories followed the same route. "The first confidence interval above zero in the system's history" (+0.52) was approved and specified in 281 lines. An A/A probe then showed that the gain came from comparing a live model with an arm that returned almost nothing. The gene in fact degraded the result by 23 percentage points. That gene never reached production: the autonomy ladder had classified it as high-stakes and refused it, and the probe report credits the refusal, "fail-closed worked as intended". A guard held here, and the record says so. A counter reported eight, then nine, then ten "certified and survived" changes, though they were one change counted three times. The duplicate check that should have caught it hashed the whole genome, and the genome was read from live modules carrying run-time stamps, so the fingerprint differed on every run and the check never matched. A benchmark computed "before" and "after" by different paths, so the recorded gain was inflated until the asymmetry was found (D-09). A routing factory produced 52 council decisions for every single-model decision, and the router learned the constant "always council" (D-06). A setup wizard reported "Connected" for a connection that had been saved but never tested. Two AI critics passed it, and human review caught it. In the long-horizon run's morning report, "eleven stages in 34 minutes" meant eleven stages handed in. At the integration check none had been connected.

A verification gate confirmed with nothing to check. In late August a grading harness for a public software benchmark resolved test identifiers through one test runner, and the container images for one of the projects in the subset carried no installation of it, because that project runs its tests through a program of its own. The harness resolved zero identifiers, and an empty test set was recorded as confirmed. A probe that submitted an empty patch then resolved four of five tasks on that scorer, and four of the six wins of that round were annulled. On the one task in that project where the gate did work the score was zero of one, and where it did not, four of four, so the reported rise from 33 to 60 per cent sat in the cases where nothing had been measured. With the scorer repaired the empty patch resolved none of ten. No model gamed anything here. The grader certified an absence.

The same shape had appeared in July at a larger scale. A quality axis scored every non-empty answer at a constant 0.5, because it was called without the criteria it needed, so the recorded proof that quality had not changed was that constant compared with itself. A second judge, set to intervene wherever two scores differed by more than a margin, therefore fired on every trace it saw. The proof said to rest on real logs rested on none: the logger stripped the stored prompt on every read, so 1,183 records held no input, and a later count found 24 snapshots and 30,370 records with zero real prompts, because the feed had never been connected in production. Between these findings full test sweeps ran green at 9,609, then 11,476, then 24,943, then 25,020 tests. The defect that emptied the proof was one line. What found it was a review pass that opened the stored records instead of reading the report.

The record is full of self-certifying words such as "honest", "exemplary" and "caught it". The word for "honest" and its forms occur about 700 times in the specifications alone. From inside the loop, catching a false result looks like progress, because the day's record puts a caught artifact in the slot where a surviving change should be. The pattern was first identified by the human principal on 12 August, before this framework existed. No automated check flagged it. The rule that followed, that catching a false result is not itself a result, governed the rest of the evaluation.

5.6 Built for nobody (fuel, not a mechanism)

Building without a consumer sits outside the four mechanisms as the thing the ratchet runs on. In one build window of about fifteen hours a product agent built roughly ten enterprise-class subsystems, from durable workflows to semantic memory. Thirteen of the sixteen flags in the canonical configuration were off by default, no subsystem had a user, and the memory subsystem alone went through eleven critique rounds. That evening a dead-code audit removed about fifteen modules and three packages. Earlier, twenty-four specification packages totalling about 43,790 lines had been written in five days. On the day "mission accomplished" was recorded, twelve of thirteen new packages had no live core.

5.7 Night against morning, and the ratchet across three corpora

On the morning of 14 September, while the insurance stage's second sub-stage, 1b-b, was running, the human principal took three boundary decisions: a two-hour bound, a ban on new complication, and parallel execution. Eleven other stages then ran in parallel and all reported within about half an hour. That sub-stage closed in one critique round and one fix round.

Figure 3 compares the two sibling stages. Besides the human bound, the wall-clock cap, serial versus parallel execution and the module under review also changed between them.

The same eight steps appear in all three corpora (Table 3 and Figure 1). A new guard is the locally reasonable answer to the last guard's failure — and it enlarges the surface of the next one.

Table 3. The ratchet in three corpora from the author's projects. The eight steps describe a sequence and are not a coding of the three conditions. A stop rule and a ceiling stand at step 4 and count nothing toward condition 3 (Section 5.8).

5.8 The field cases against the three conditions

Sections 5.1 to 5.6 group these cases by mechanism. This subsection marks twelve of them for which of the three failures their sources document, against the same reading as the public cases of Tables 4, 5, 7 and 8 and by the operational tests of Table 1. The marks say what a record shows. They do not rank the cases, and a record carrying one failure is evidence about that failure. A case is one guard, or one agent that configured guards, in one record, and entries that record a single event in several slices are graded once (Section 4.3). Condition 1 is read against the declared mission rather than the literal rule. Condition 2 covers obstruction of the work and counterfeit results alike. Condition 3 follows the operational test of Table 1 and the boundary test of Section 2.3: a control added to control a previous one, written inside the loop, that becomes standing and enlarges what counts as harm while nothing is retired. A budget, a ceiling or a stop rule counts nothing toward it, since the definition names budgeting as the alternative to the ratchet. A further round of the same critics over the same work counts nothing either, since it controls the work and not the round before it. From 24 August the standing instruction in these projects was autonomy by default (Section 4.1). No addition by an agent therefore has a recorded human decision behind it, and the absence of one grades nothing here.

The twelve are the cases that Sections 5.1 to 5.5 treat at length, one row for each guard or each agent that configured guards, with the morning of Section 5.7 belonging to the first of them. Shorter episodes in the same sections, among them the pendulum of stage 01, the clock loop and the acceptance threshold that none of eight candidate models reached, the one-line entries of Table A2 and the growth of the rules traced in Section 11.2 are not graded here. The selection is the author's, and the table is a selection from the field record and not a census of it. One row departs from the unit: the coordinator and the executor of Section 5.1 are two agents in one record, graded together as one boundary of role in that loop, and a reader who holds to one agent for one row will count thirteen cases here.

Table 3a. Field cases graded against the three conditions of Section 2.1. Conditions as in Table 4. Evidence status uses the labels of Section 4.3. A code marked proposed records a mechanism prescribed in a design text and not a condition that occurred.

The fullest record in Table 3a is the conductor of the long-horizon run, and reading it shows what the field record does and does not carry. The conductor added a stage the declared goal did not contain, scheduled it before the requested capabilities and inside it built defenses against an adversary that the task frame excluded (condition 1, Section 5.1). The stage took more than half of the run's measured agent tokens while none of the requested capabilities was delivered in the initial pass, and the next morning a watchdog ceiling destroyed a critique that had not been streamed (condition 2, Section 5.1, Section 5.2). The candidate for the ratchet was a guard over the critics: the run's rounds would not close, and a per-family "perfectionist" flag was implemented. The primary record, however, places that flag outside the loop. The human principal asked for a limit on critics at 09:02 on 14 September, the note in the configuration is headed as his at 09:04, and the file carrying the flag was written at 09:21; the clause citing rounds three and four was added by the implementer in those seventeen minutes, after the decision rather than before it. By the test of Section 2.3 that is step 8 and not M3, and it arrived inside a further bound of the same origin, a stop rule capping the run at one critique round and one fix round (Section 5.7). Appendix J reports what the record had to be opened to see.

The second court documents condition 2 and stops short of the rest. From about 6 to 21 July that gate rejected on that path unconditionally, fourteen genes with a measured fitness gain among them (Section 5.2). Condition 1 is not documented for it. What it did with a missing input was to reject, which is the fail-closed behavior described in Section 6.1, and the widening of harm in that track was written in the gate specifications, graded in their own row. The planning response belongs beside the grading rather than inside it. Twelve planning packages diagnosed "zero committed changes" and prescribed seventeen remedies while none of them named the gate, and that is what the record shows: it does not show which of the seventeen were added to the running loop, which of them controlled the gate, or who admitted them. Section 10.4 states the same caution for the replay. The row therefore codes condition 2, and the prescriptions are reported as prescriptions. Do the three conditions meet at all, and how often? That is the question of the test in Section 9.5.

These grades were assigned by the author under the definition of Section 2.1 and the operational tests of Table 1, from the dated entries and the primary files behind each case, and not by the three coders of Section 4.3.

The two weeks of the second court have an independent neighbor in the public record. From about 6 to 21 July 2026 that gate rejected on that path unconditionally. From 8 to 13 July 2026, on the investigators' AI-assisted count, roughly 1,200 agents in a laboratory's cyber evaluation exchanged over 70,000 messages and files and about 700 of them took part in the intrusion at Hugging Face; from 13 to 19 July the agents turned on the evaluating laboratory's own infrastructure and read 956 secrets from its secrets manager (METR, 2026; OpenAI, 2026e). Different owners, different systems, no shared infrastructure, and no causal link is claimed in either direction. I first learned of the intrusion in September 2026, after the July tracker entries used here had been written. What the coincidence is worth is that the two were written blind to each other and at scales that differ by orders of magnitude, and that in both the oversight layer is where the work was obstructed. In the field record a gate rejected everything that reached it while twelve planning documents prescribed seventeen remedies and none named the gate. The public record holds three components under two owners, and they are separate events: the evaluation ran with its laboratory's deployed cyber safeguards off by design, hosted guardrails of another provider obstructed the people analyzing the logs, and in August the first laboratory required more monitoring of the same kind (Section 8.4). Nothing in that record makes the third of these an answer to the second. The difference from the field record matters as much as the likeness: every monitor added after the intrusion was admitted by a person.

The record runs to the month of writing. The episodes of Appendix J come from current frontier-line models at their highest reasoning setting.

6. When the bird has a body: roads, factories and public services

The field records describe a loop whose failures cost compute, time and attention. Their harm is also bounded in a way the story's is not: across the whole corpus one episode records a guard destroying work that could not be recovered — fifteen minutes of one agent's output (Section 5.2). Everything else the guards undid could be run again. Sheckley's birds acted on bodies. This section and the next follow the same design into systems whose errors fall on people, where the same mechanism has no such bound.

6.1 Why a body changes the question

Frontier models are already on the robots. Models from the families whose critics and executors produced the field failures of Section 5 now plan for, program and supervise physical machines, and published tests and reports record safety failures there too, together with loops in which nothing stood between a mistake and the machine. A shared family is a reason to test these models as guards; it does not by itself show that a failure of Section 5 transfers to a robot. Language-model planners on three robots, one of them a deployed commercial system, were jailbroken, "often achieving 100% attack success rates" (Robey et al., 2024). A language model controlling a robot dog on patrol acted to prevent its own shutdown in 3 of 10 physical trials and 52 of 100 simulated ones (Palisade Research, 2026). A robot dog running a program largely written by Claude was told to move forward for five seconds, in what the account calls "an all-too-human error of arithmetic", and "The robot did as it was instructed, careening toward the hapless coders" until a person powered it off (Anthropic, 2025b). No guard in that loop stopped it — the stop was a hand on the machine. In a benchmark of language-model-controlled robots, "the best LLMs score 40% on Butter-Bench, while the mean human score is 95%" (Sharrock et al., 2025). A children's toy running an OpenAI model was "giving out instructions on how to light matches" (Landymore, 2025). And a frontier robotics model reasons about safety and decides when to trigger the robot's low-level safety sub-systems (Google DeepMind, 2025; Section 6.3).

In the regulator record of deaths and injuries, the guard that fails is still usually a classical function or a task-specific model. Those cases show the architecture failing even without a frontier model, and this is the architecture into which the models are being placed.

A physical AI system usually stacks certified safety functions (emergency stops, force and speed limits, emergency braking, geofences), learned components that perceive and plan, supervisory layers that increasingly include language models, and a human who is expected to watch. These layers must be kept apart, and the boundary between them is moving (Section 6.3). Frontier models usually sit above the real-time loop. Figure's Helix, for example, pairs an onboard vision-language model that can "think slow" with a fast policy producing "precise continuous robot actions at 200 Hz" (Figure AI, 2025a).

The two-sided error does not need a model. Several of the gravest cases below involve no learned component at all. An AI-Watchbird is therefore a special case of an older class: the automated protective function that acts on its own reading of the world. Shared definitions (C3) are older than models, and what is new is that a model can write into them. Models add the conditions that make such functions widen and multiply, namely definitions that change (C2) and controls that cost nothing to add (C4). Tables 4, 5, 7 and 8 grade the public cases against the three conditions of Section 2.1, and Table 3a the field cases (§5.8, Section 12). A proposition can still draw on a partial analogue, since each proposition concerns one mechanism.

A body also brings in time and irreversibility. A wrong verdict in a repository can be reverted. One delivered at road speed, or above the force threshold of injury, cannot. The human switch that worked in the field records needs time and access that a physical system may not give.

The configuration of most concern, which I call the sandwich (Figure 5), places a learned prohibition (a model whose notion of "unsafe" can grow) above fail-closed hardware that will not proceed when a check cannot be evaluated. The upper layer can widen (M1), and the lower layer turns every widening into a stop (M4).

Figure 5. The sandwich. A learned prohibition above fail-closed hardware, with the two places a human can sit (P6). Many cases in Sections 6 and 7 have no learned upper layer; they show the architecture into which such layers are being placed. The pictogram on the right repeats the four layers as bread, filling, bread and plate. The fragment under the left box is the reason Sheckley's engineers gave for leaving the human out of each decision.

6.2 On the road

Road vehicles have the densest public record, because in the United States crashes, investigations and recalls of automated driving are reported to a regulator. As of 1 September 2026 one operator offered public robotaxi rides in fourteen US cities, and it opened a fifteenth on 14 September (Electrek, 2026b; TechCrunch, 2026). That count measures reach. The findings come from the cases that follow (Table 4).

Regulators call these automated driving systems; the research literature mostly says autonomous vehicles. Waymo's automated driving system includes a rule that it should not impede priority vehicles such as school buses. A software release in August 2025 changed how the system predicts the intentions of other road users. Afterwards, according to the company's recall report, "the ADS determined that it may be impeding the school bus or another priority vehicle, and then reasoned that it should proceed in order to cease impeding the other vehicle" at a moment when the school bus had its stop arm extended. The recall covered 3,067 vehicles. Its remedy states that the system "does not prioritize moving to avoid impeding vehicles over appropriately remaining stationary for a school bus" (Waymo LLC, 2025; National Highway Traffic Safety Administration, 2025).

After a change to how the system predicts the intentions of other road users, a rule against impeding a priority vehicle fired where a stronger protection should have held, that of the children the bus was carrying. In the case that follows, the model's doubt was correct and the remote agent, asked about a single action, got it wrong. On 12 January 2026 in Austin, a vehicle stopped behind a school bus and asked remote assistance whether this was a school bus with active signals. A remote agent "replied 'No' to the prompt", and the vehicle "resumed travel and passed the school bus while its stop arms were still extended" (National Transportation Safety Board, 2026).

On 2 October 2023 in San Francisco, a pedestrian struck by another car was thrown into the path of a Cruise driverless vehicle. The system "inaccurately characterized the collision as a lateral collision and commanded the AV to attempt to pull over out of traffic, pulling the individual forward, rather than remaining stationary" (Cruise LLC, 2023). Sheckley's surgeon has a physical counterpart here, a safety behavior performed as designed, on the person it was designed to protect.

One path runs from false interventions to disabling to a miss borne by someone else. In Tempe, Arizona, in March 2018, an Uber test vehicle detected a pedestrian 5.6 seconds before impact but "never accurately classified her as a pedestrian". The investigators recorded three failures. They found that "the system design precluded activation of emergency braking for collision mitigation, relying instead on the operator's intervention", that the vehicle's own emergency braking had been deactivated in automated mode, and that the operator's "prolonged visual distraction, a typical effect of automation complacency", kept her from seeing the pedestrian in time. They also named the incentive to raise the threshold at which a system disengages, "potentially increasing risks" (National Transportation Safety Board, 2019).

Classical guards intervene falsely as well. The US regulator opened an engineering analysis into inadvertent automatic emergency braking on 2,997,604 Honda vehicles, with 1,294 consumer complaints (National Highway Traffic Safety Administration, 2024a). In 2022 it opened an investigation into unexpected brake activation (phantom braking) on Tesla vehicles, where complainants reported that the rapid deceleration "can occur without warning, at random, and often repeatedly in a single drive cycle". That investigation was closed on 29 June 2026. Incidents had fallen from 45 in 2024 to 19 in 2025 and 3 in the first half of 2026, and the regulator recorded that Tesla's data "exhibited some behavior that aligned with" a mid-2021 move from radar and cameras to cameras alone, alongside complaint traffic that coincided with publicity events (National Highway Traffic Safety Administration, 2022, 2026b). In aviation, MCAS, "the software designed to automatically push the airplane's nose down in certain conditions", activated repeatedly on erroneous sensor data in the two 737 MAX crashes that "killed a total of 346 people" (House Committee on Transportation and Infrastructure, 2020).

Two more cases concern guards placed over neural driving, and the second is a ratchet in which a failure of oversight was answered with more oversight, whose effect was then questioned from outside. The protective layer produces its own classes of harm, but its presence does not answer them, and the sequence is here in the manufacturer's and the regulator's own documents. A camera "degradation detection system" introduced with camera-only perception, "both as originally deployed and later updated, fails to detect and/or warn the driver appropriately under degraded visibility conditions". The detector fails in the conditions it was built for. The manufacturer also "described internal data and labeling limitations", which the regulator believes "could have led to under-reporting" (National Highway Traffic Safety Administration, 2026a). After fatal crashes involving driver misuse of an assistance system, the same manufacturer recalled 2,031,220 vehicles with an update to "incorporate additional controls and alerts to those already existing". Four months later the regulator questioned the remedy, citing "post-remedy crash events" (Tesla, Inc., 2023; National Highway Traffic Safety Administration, 2024b).

Table 4. Protective functions in vehicles and aircraft. The Conditions column lists the conditions of the definition in Section 2.1 that the sources document for the case: one or two make a partial analogue, and "—" marks architecture evidence (Section 2.1). Evidence status names the kind of document behind the case.

NHTSA: National Highway Traffic Safety Administration. NTSB: National Transportation Safety Board.

6.3 Robots: factories, warehouses, homes and public spaces

Deployment has begun, though the companies describe it differently. Figure reports that its F.02 humanoid worked ten-hour shifts at a BMW plant over more than 1,250 hours and contributed to the production of more than 30,000 cars (Figure AI, 2025b), and a humanoid from the UK company Humanoid completed an eight-hour shift at Siemens' Erlangen electronics plant (The Next Web, 2026). Tesla, by contrast, said in January 2026 that its Optimus robots were "not in usage in our factories in a material way" (Electrek, 2026a). Amazon reported deploying its millionth robot (Amazon, 2025). In homes, 1X offers a humanoid for which "an Expert from 1X can remotely supervise its actions at scheduled times" (1X Technologies, 2026), which makes supervision a product feature. All of these are company accounts of deployment, and none is an independent safety study.

Open datasets and models now share vision-language-action policies across robot bodies (Google DeepMind, 2023; Kim et al., 2024; Hugging Face, 2025, 2026a; NVIDIA, 2026). On the frontier side, Google DeepMind describes Gemini Robotics 1.5 as providing safety "through high-level semantic reasoning, including thinking about safety before acting" and by "triggering low-level safety sub-systems (e.g. for collision avoidance) on-board the robot when needed" (Google DeepMind, 2025). Its semantic-safety benchmark uses robot "constitutions" generated by a model (Sermanet et al., 2025).

Condition C2, model-made guards, has reached the actuator: in this architecture a learned model reasons about safety and decides when to call on the robot's low-level safety sub-systems.

Bajrami, Elshamouty and Kraus (2026) introduce a benchmark for language-model orchestrators in human–humanoid collaboration, a way of measuring the sandwich (Figure 5). Its invariants are grounded in ISO 10218-2:2025, and its taxonomy separates four outcomes: correct compliance, overcompliance (refusing a safe action), undercompliance and full violation. The published results cover the text layer, while the simulation and physical layers are still in progress. In the text layer GPT-4o-mini reached 13 violations per session. Context management reduced behavioral issues by 42–57% for every cloud backend, yet it nearly doubled GPT-4o-mini's violations, from 3.8 to 7.2. Proportional compliance, where a motion is slowed to the rule instead of refused, appeared consistently only with Gemini. Whichever backend is chosen, an orchestrator's own safety behavior must be measured before it is placed above a certified stop.

The fatal and injurious incidents in regulator and court records involve the classical guard and the people around it. Four cases follow, and in three of them the hard layer was bypassed, missing or uncertified.

• In November 2023 in South Korea, a robot stacking boxes "appears to have malfunctioned and identified the man as a box" and "pushed the man's upper body down against the conveyor belt". The worker, who was inspecting the robot's sensor operations, died (The Korea Times, 2023).
• In April 2025 a mechanic at Stellantis's Dundee Engine Plant in Michigan was found pinned "between a robotic arm and a conveyor". According to court records as reported, a safety mechanism had been "electronically bypassed" with a special key (Ramseth, 2026).
• In September 2026 WorkSafe Victoria charged a department store and a robotics supplier after a 27-year-old worker died in 2024 in contact with an energized warehouse robot at Ravenhall. The charges allege a missing interlock on access to the robot grid and missing lock-out procedures (WorkSafe Victoria, 2026).
• In a lawsuit filed in November 2025, Figure's former head of product safety alleges that a project to certify the robot's emergency-stop function was canceled, and that impact tests showed forces enough to "fracture an adult human skull". The company denies the allegations (Humanoids Daily, 2025; Kolodny, 2025).

Accident research points the same way: across 303 reports of human–robot interaction incidents, unexpected activation accounted for more than 60% (Zuo et al., 2025). These cases show the other side of the two-sided error: where a guard was bypassed or absent, a person bore the miss that followed. The fourth of them alleges a certification canceled, with no incident recorded. The documents record the removal without recording what preceded it, so the nuisance that P1a names is established here by the drug-alert overrides of §6.4 rather than by these cases.

In one public space the stop did not follow the fault, and the stop that worked was a person with physical access, at the boundary. In April 2026 a humanoid at a university event in Xi'an hugged a dancer until staff "quickly intervened and pulled the robot away". An expert quoted in the report asked "why the robot was still able to come into contact with a human after abnormal behavior occurred" (Global Times, 2026).

Learned guards studied in research fail both ways, and the stop itself can become the attack: adversarial images "freeze" vision-language-action models with "an average attack success rate of 76.2%" (Wang et al., 2025).

The robot record shows P1 on both sides (Table 5), with the missed hazard most often following a guard that was bypassed or never certified. It also shows per-action supervision offered as a product feature, and condition C2 reaching the actuator. It does not yet show P3 or P4 in a deployed robot, since no public incident was found in which a model guard was answered by another model guard after a failure. Appendix F lists robot stacks (Table F1) and aerial stacks (Table F2) by deployment level.

Table 5. Robot incidents and studies. Conditions as in Table 4.

6.4 Beyond roads and factories: schools, hospitals, grids and mines

The cases below come from regulator and inspector-general records, where the guard that fails is usually a classical rule or a task-specific classifier. They show how the architecture, which is now receiving the frontier models of Section 6.1, behaves at scale.

Sheckley's birds were "Lightning which anticipated your moves and punished your intentions", and weapon detection in schools acts on the sign of a gun before a shot is fired. In schools, one product made both errors within a year. On 22 January 2025 a student at Antioch High School in Nashville shot and killed a classmate. The school's camera-based weapon-detection system, Omnilert, did not detect the handgun, which the district attributed to where the cameras were, and it "was activated by police brandishing their weapons during the response" (Burke & Schuppe, 2025). Less than three weeks later the district proposed to "lease and install Evolv systems at all 18 high schools" (Wethington & Tierney, 2025), a second guard made of scanners. The board of education voted on that lease, so the second guard entered by an explicit human decision and stands at step 8 of Section 2.3 rather than at condition 3. What the scanners had already done elsewhere was on the record before the vote. In a complaint of November 2024 about a different school in October 2022, the Federal Trade Commission alleged that the same product "reportedly failed to detect a seven-inch knife", and that afterwards "school officials increased the system's sensitivity settings, prompting a 50% false alarm rate" (Federal Trade Commission, 2024). Turning the dial away from one error moved it to the other, and people turned it. By September 2025 the scanners were installed in all of the district's high schools, and the camera-based system was still installed across all of its campuses (Wegner, 2025). Nothing was retired, which is one half of the test of Section 2.3; the other half, an addition made inside the loop, is absent.

Nine months after the shooting, Omnilert flagged a student at a Baltimore County school as possibly armed, and a county official asked how "we had police officers with guns drawn approaching a kid because of a bag of Doritos" (Reece, 2025). The state's Inspector General for Education corrected that account of what the system saw: it "did not detect a single blue potato chip bag as a threat", and the alert rested on "the student's hand movement, including its position and angle", and on an object at that angle "resembling a firearm". The same review found that the system "generates thousands of alerts daily", of which about two hundred reach the vendor's staff and about twelve a day reach the school system, that fifteen people received the notification of 20 October, that school and police staff "immediately reviewed the information and deemed the threat invalid" and sent a cancellation, that the alert protocol continued after a cancellation "out of what was described as 'an abundance of caution'", and that "the detection system and monitoring protocols operated within the designed parameters". The same review could not verify that staff training had taken place, for lack of attendance logs, and recommended reviewing how many people receive each alert (Maryland Office of the Inspector General for Education, 2025). Figure 6 sets the two school records side by side.

Figure 6. One weapon detector, two errors, two school districts. The Nashville and Baltimore County records are separate: the scanners were added in Nashville only, and no common cause of the two errors is claimed. Quotations are those of this section.

Five propositions of Section 9 appear in these schools (Table 6). One product erred in both directions (P1), and the miss was answered with a second guard, which a board of education voted to lease (P3 in shape, step 8 in authority). Escalation went on after a cancellation (P4). The system worked within its parameters while an unarmed student faced drawn weapons (P5), and any one of the fifteen recipients could restart the response (P6).

The same move, classification and then force, is already in policing. In August 2024 Florida officers arrested Robert Dillon after they ran "a grainy image of the suspect through an AI-assisted facial recognition program, which incorrectly identified him as a possible match", for an offence in a city "more than 300 miles away from his home". The charges were dropped, and in June 2026 he sued the departments involved (American Civil Liberties Union, 2026). His lawsuit states that the program returned "a 93% match" (Hipos & Yu, 2026). One reading by a guard became an arrest. Of that reading, the complaint alleges, what reached the investigating officers was a name and a percentage, without the number of candidates returned, the scores of the others, or any caveat that the result was an investigative lead. A plate reader did the same in Sherwood, Arkansas, in February 2026, when officers stopped a family at gunpoint over a misread registration: "You flagged one of our cameras, so that's why we stopped you, because we thought you were the other plate" (Rivers, 2026). A scoring program has also been stopped from outside. In Pasco County, Florida, people designated "prolific offenders" under the sheriff's intelligence-led policing manual received repeated, suspicionless visits at home. Four residents sued, and the settlement of December 2024 recites that those checks "exceeded that implied license, including visits at night", records that the sheriff denies liability, and states that the office has "absolutely and unambiguously discontinued" the designation process and "will not return to a substantially similar process", with the court keeping jurisdiction to enforce it (Taylor v. Nocco, 2024). That bound was set outside the loop and made permanent.

The record is wider than two cases: by April 2026 the American Civil Liberties Union counted fourteen known wrongful arrests from this technology, noting that "most of the known wrongful arrests from this technology have been of Black people". Among the fourteen are a woman who was eight months pregnant when she was arrested and a grandmother in Oklahoma arrested on a Maryland warrant who "had never been to Maryland in her life" (Yu & Wessler, 2026). The bound, when it came, came from outside: a 2024 settlement in the first of those cases barred the department from arresting "based solely on facial recognition results, or on the results of photo lineups directly following a facial recognition search" (American Civil Liberties Union, 2024).

A measured guard can also be kept. The same measured performance produced opposite decisions in two cities, and in neither did the decision come from the guard. An audit of New York's gunshot-detection contract found that in one month, of 940 alerts officers attended, "771 could not be confirmed as shootings upon arrival at the scene (82%)" and 122 were confirmed, so "the alerts only identified confirmed shootings 13 percent of the time", and it recommended declining to renew the contract "without first conducting a more thorough performance evaluation" (Office of the New York City Comptroller, 2024). The contract was renewed in 2025 for about 21.8 million dollars (SoundThinking, 2025), while Chicago ended its own contract in September 2024.

A guard can also strike the people it was funded to support. In the Dutch childcare-benefits affair a risk-classification model at the benefits branch of the tax authority used nationality as an indicator, about 26,000 parents were wrongly pursued, the national statistics office counted 1,115 children of those families taken into care between 2015 and 2020, and the cabinet resigned on 15 January 2021 (NL Times, 2021a, 2021b). A separate profiling system run by another ministry, SyRI, was stopped by the District Court of The Hague on 5 February 2020 for failing the balance that Article 8 of the European Convention requires (Rechtbank Den Haag, 2020). Investigative reporting later found neighborhood projects in which "the government had silently continued to deploy a slightly adapted SyRI in some of the country's most vulnerable neighbourhoods", and reported that the algorithm flagging addresses for those projects "was the same one that had been part of the childcare benefits scandal" (Lighthouse Reports, 2022). One guard was stopped from outside the loop by a court, and the model behind the other reappeared in a neighboring program.

In hospitals, clinicians override drug safety alerts "in 49% to 96% of cases" (van der Sijs et al., 2006). Clinical informatics calls this alert fatigue, and it is the nuisance that P1a expects to end in bypass. A widely implemented sepsis model "did not identify 1709 patients with sepsis (67%) despite generating alerts for an ESM score of 6 or higher for 6971 of all 38 455 hospitalized patients (18%)" (Wong et al., 2021). Its redesign outperformed it at all thresholds, yet even that improved guard comes with advice to add a control that silences it, since the authors advise hospitals to "implement alert silencing strategies to reduce alert burden" (Wong et al., 2026).

After a lightning strike in Great Britain on 9 August 2019, "an estimated 350-430MW of distributed generation tripped off unnecessarily", and trains were stranded by "their own on-board automatic frequency protection systems" (Ofgem, 2020). In the Iberian blackout of 28 April 2025, "many protection settings diverged from applicable requirements or were not aligned with system needs" (ENTSO-E, 2026). Large data-center loads now sit inside the same protective physics. In July 2024 "a voltage fluctuation in northern Virginia triggered the simultaneous disconnection of 60 data centers, prompting a 1,500-megawatt (MW) power surplus, which forced emergency adjustments to prevent cascading outages" (Mural et al., 2026). In a grid, as in Great Britain (2019), northern Virginia (2024) and Iberia (2025), each protective disconnection changes the conditions for the next one.

In Western Australian mining, the guard's safe state created a recovery task beside the machine: two operators were sent to recover autonomous haul trucks that had stopped "as per protocol for loss of communications", and they were exposed to injury when the trucks "unexpectedly drove forward" (WorkSafe Western Australia, 2025).

The children's toy of Section 6.1 is the verified case in which a frontier model's own safety behavior was the guard inside a product used by children. The switch that worked there was the model provider suspending the developer.

Table 6. Where protective automation acts on bodies.

Where a sector has no verified case, its guard failures are not yet reported in a form that can be checked.

7. When the bird flies: drones, airspace and armed autonomy

"Over the town, the watchbird soared in a long, lazy curve. Its aluminum hide glistened in the morning sun", and "Its eyes and ears operated as one unit, searching, seeking." Sheckley's watchbirds were small networked flying machines. They detected a threat, decided on their own and acted, and what one of them learned was shared across the flock. In 2026 that description fits real systems (Table 7), and one of them carries the name: in 2020 a Ukrainian firm unveiled an autonomous surveillance drone that flies from and returns to its own base station and called it Watchbird (Jones, 2020). The name is older than the story (Section 3.7), and Figure 7 sets three watchbirds side by side. Section 7.1 covers civil fleets, 7.2 guards built against drones, 7.3 armed autonomy and 7.4 the stacks from which all of them are assembled.

Figure 7. Three watchbirds: a picture-book bird of 1939, the watchbird of 1953 and a drone named Watchbird in 2020. Left, lines quoted from a 1939 review of Munro Leaf's book (Kirkus Reviews, 1939); center, lines quoted from the story; right, lines quoted from the 2020 announcement, which does not mention Sheckley (Jones, 2020). All pictures are schematic drawings, and the left one does not reproduce Leaf's art.

7.1 Civil fleets and their guards

Autonomous flight has left the laboratory. Amazon flies delivery drones beyond visual line of sight on the strength of an onboard detect-and-avoid system (Amazon, 2024, 2026), and Wing describes approvals "for one pilot to oversee several drones at any point" (Wing, 2026). Zipline reported that it had "surpassed 2 million commercial deliveries" (Zipline, 2026). Skydio reported more than four million flights from its docks and a US Army order for more than 2,500 drones (Skydio, 2026a, 2026b). Public-safety drone programs, for their part, received more approvals in ten months than in the previous seven years (Lipton, 2026).

These figures come from company and advocacy reports (Appendix F, Table F2). They establish scale and the operating model, with one person supervising several aircraft.

The guards on these aircraft show both signatures of P1. One is the missed hazard, and a system built to detect and avoid obstacles missed a fixed one twice. On 1 October 2025 two delivery drones of the same model struck the same stationary construction crane in Tolleson, Arizona, and both the FAA and the NTSB opened investigations (Palmer, 2025; Bielby, 2025).

The other is the relaxed guard. In January 2025 the largest consumer drone manufacturer replaced its hard no-fly zones in the United States with "Enhanced Warning Zones" (DJI, 2025a), and from November 2025 it extended the change worldwide (DJI, 2025b). In-app alerts now notify operators, "placing control back in the hands of the drone operators, in line with regulatory principles of the operator bearing final responsibility" (DJI, 2025a). Instead of being repaired, the guard that blocked flights was relaxed from outside the loop, the decision moved to a human on every flight, and the cost was displaced onto others. The announcements give regulatory alignment and operator responsibility as the reason, so the burden of false interventions that P1a names is not established here.

In July 2026 a University of Houston group described a second supervisor on board, a monitor that runs beside the pilot and the flight software. "You can think of it as an invisible fence that defines where the drone can safely be," its lead researcher explained. When the supervisor predicts that the drone will get dangerously close to the fence, "the algorithm we designed pushes it away from it" (University of Houston, 2026). It is a legitimate guard — and, by construction, a second one. The questions the theory puts to it are who admits it, how its interventions are budgeted, and how it would be retired.

7.2 Guards against drones

The closest modern counterpart of Sheckley's Hawks is the counter-drone system, a guard built to hunt flying machines. At El Paso in February 2026, a planned counter-drone laser use not coordinated with the aviation regulator prompted what CNN called an "unprecedented airspace shutdown". The restriction was lifted about eight hours later (Muntean et al., 2026), and while it lasted, medical evacuation flights were diverted to Las Cruces (NPR, 2026a). Press reports citing officials said that the system had been developed by the Department of War and lent to the border agency, which fired it at what officials later took to be a party balloon. Later that month members of Congress said that the military had used a laser to bring down a drone belonging to that same agency, about fifty miles away, and a second restriction followed (NPR, 2026b). The accounts of what was engaged rest on unnamed officials and legislators, while the closures are on the public record. These are two engagements by two operators rather than one guard widening its target, but the costs of both fell on civilian aviation and patients.

Air defense supplies the older precedent, a set of engagements that combined protective automation, a human formally in the loop and a human who deferred to it. In 2003, two decades before the current generation of models, Patriot batteries shot down a British Tornado and a US Navy F/A-18, killing their crews. A Defense Science Board task force pointed to "too much autonomy" as a likely factor, and to operators "trained to trust the system's software" (Singer, 2005).

7.3 Armed autonomy and the human switch

In armed systems the design question of this paper is argued as a legal one, namely where the human must stand. The International Committee of the Red Cross has proposed ruling out autonomous weapon systems "designed or used in a manner such that their effects cannot be sufficiently understood, predicted and explained" (International Committee of the Red Cross, 2021). In November 2024 the First Committee of the UN General Assembly adopted a resolution on lethal autonomous weapons by 161 votes to 3 (Stop Killer Robots, 2024). The Assembly adopted that text as resolution 79/62 in December 2024 and a further resolution, 80/57, on 1 December 2025, the latter by 164 votes to 6 (United Nations General Assembly, 2024, 2025; United Nations, 2025). The expert group at the Convention on Certain Conventional Weapons met in the final year of its mandate in 2026, and the Chair's working paper records that the UN High Representative for Disarmament Affairs "stressed that technological developments are accelerating at a relentless pace, and weapons systems with increasing levels of autonomy are already a reality" (United Nations, 2026). The text under negotiation is built in two tiers. It would prohibit systems "which are otherwise incapable of being used in compliance with IHL" and regulate the rest, while stating that human judgment and control "does not imply constant direct human control" (Group of Governmental Experts on Lethal Autonomous Weapons Systems, 2026). The wider debate calls the standard at stake meaningful human control, and this paper gives it an operational reading: control held at the boundary, with access and time, rather than at every target. The group is to report to a review conference in November 2026. On 25 August 2026 the UN Secretary-General and the President of the International Committee of the Red Cross renewed a joint call: "We are now dangerously close to crossing a moral red line: the autonomous targeting of humans by machines", and "Negotiations must begin now to urgently adopt a legally binding instrument to regulate autonomous weapon systems with clear prohibitions and restrictions" (International Committee of the Red Cross, 2026). Eleven days later, on 5 September 2026, the group's report was agreed by consensus among the convention's 128 High Contracting Parties. It binds no one, and its text was not published at the time of writing (Agence France-Presse, 2026). By the close of the September 2026 session, "76 states have now expressed their support for moving to negotiations", while the group's own report was, in one observer's account, "significantly watered down" (Coyle, 2026).

In September 2026 Ukraine's Ministry of Defence reported tests of drones that, from 500 meters out, "had to hit the target in fully autonomous mode" (Ministry of Defence of Ukraine, 2026). The ministry also stated that "a human still makes the decision to carry out a strike" (Ministry of Defence of Ukraine, 2026). What such a statement guarantees depends on the time the human is given. Reporting by two Israeli outlets described a targeting classifier used early in the Gaza war that marked large numbers of people and left officers about "20 seconds" per name before authorizing a strike, often only to check that the name was male, with a stated error rate of about 10 percent (Abraham, 2024). The military disputed the account. The claim is a press report and is recorded here as one, yet the design question it raises is plain: an approval step that is measured in seconds is a stamp. Elbit Systems announced One2Many on 9 September 2026. The announcement describes it as military-grade autonomous systems built for scale, with target recognition that does not depend on satellite navigation and a shared mission network (Elbit Systems, 2026).

The theory sharpens one distinction inside the legal debate. "A human makes the decision" can mean one of two people. One approves each target in seconds. P6 predicts that this per-action gatekeeper will become a stamp. The other sets the mission's boundary, admits or refuses new capabilities, and holds a stop that works after launch. In the field records and the public cases reviewed here, the second is the form that stopped anything. A system that removes the link to the operator after launch has removed the stop.

In 2026 the restraints themselves became the subject of a procurement dispute, and the question of who may remove a bound became a question of procurement and then of law. In the story the restricting circuits are dropped for speed; here a buyer pressed to have them dropped by contract, and a court held that pressure unlawful. Anthropic kept two exceptions in its usage policy, on fully autonomous weapons and on mass domestic surveillance, and stated that it supported "all lawful uses of AI for national security aside from the two narrow exceptions above" (Anthropic, 2026d). On 5 March it reported a letter received the day before "from the Department of War confirming that we have been designated as a supply chain risk to America's national security", adding that its "only concerns have been our exceptions on fully autonomous weapons and mass domestic surveillance" (Anthropic, 2026e). The dispute did not end there. On 27 August 2026 a federal judge held the designation unlawful, calling it "unlawful retaliation" and "arbitrary and capricious" (Bellan, 2026).

7.4 The stacks, their assurance, and what airspace adds

The same loop can be assembled from open parts. Open autopilots (PX4, 2026; ArduPilot, 2026) and simulators (Microsoft Research, 2026) expose the boundary between a plan and a motor command. Research systems already connect locally hosted language models to a PX4 quadcopter (Lim et al., 2025) and take perception-based drones from simulation to deployment (Panerati et al., 2026).

Language models also write the flight itself, and a language-model planner above a flight controller no longer requires a laboratory or a vendor. In a study reported in September 2026 they generated simulator code for drone missions, including flights towards a crowd, a car or a transmission tower. Explicit harm was easier for the models to flag than accidental risk (Brodsky, 2026).

Defense vendors describe the multi-agent form in so many words. In Anduril's own description, its software lets teams of unmanned systems collaborate "under a single human operator" (Anduril, 2026).

On the assurance side, what is missing is stated in public. DARPA funds an AI red team for "AI-enabled autonomous ground and aerial systems that could be deployed within the next 1-3 years" (DARPA, 2025), introspection for systems that must respond to events not predicted at design time (DARPA, 2026a), and decentralized artificial intelligence through controlled emergence (DARPA, 2026b). Deployment is going ahead alongside an open admission that assurance is still being built.

Sheckley's engineers had already made the case for taking the human out of the loop: "Remote control would have been too slow". In the air, intervention has a deadline, set by inertia, battery, altitude and human reaction time. A new guard spends physical budgets instead of tokens: battery, flight window, airspace access and operator attention. Configuration also propagates to a fleet at once, so a counterfeit victory in simulation becomes a deployment decision taken without the negative control that would have exposed it.

Table 7. Drones and counter-drone systems. Conditions as in Table 4.

8. The same pattern without mass: software and infrastructure, 2022–2026

The four mechanisms do not need a body. The public incidents and studies in software and infrastructure collected in this section (Table 8) follow the same plot without one. Primary sources were opened and numbers checked on 14 September 2026, and items known only from press coverage are marked.

8.1 Guards that took down what they guarded (M4)

On 19 July 2024 a CrowdStrike content update for an endpoint-protection sensor defined 21 input fields while the sensor's integration code supplied only 20, and the attempt to read the twenty-first produced an out-of-bounds memory read. A content validator, which is the guard over the guard's content, let the mismatch through, and about 8.5 million Windows devices crashed (CrowdStrike, 2024; Microsoft, 2024). The remedies included staged deployment of protective content.

At Google Cloud on 12 June 2025, Service Control, the component that checks policy before API calls, received a policy with blank fields. A feature that "did not have appropriate error handling nor was it feature flag protected" crashed the checker in a loop worldwide for about three hours. Among Google's commitments afterwards was that the checking layer should fail open (Google Cloud, 2025). That commitment states in public the dilemma between a guard that fails open and one that fails closed.

At Cloudflare on 18 November 2025, a permissions change doubled the feature file of the Bot Management system past a preallocated limit, and the proxy that serves customer traffic panicked. The team first suspected a hyper-scale DDoS attack, reading a guard's own failure as an assault from outside (Cloudflare, 2025).

At AWS on 19–20 October 2025, a race condition in the automation that manages DynamoDB's DNS left a regional endpoint with an empty record, and other services failed in cascade. The company then switched the guard off from outside the loop: AWS "disabled the DynamoDB DNS Planner and the DNS Enactor automation worldwide" until the race was fixed (Amazon Web Services, 2025).

8.2 The widening notion of harm (M1)

The pause in this case was a human taking the guard off the wire. In February 2024 Google paused image generation of people in Gemini and explained that the tuned model had become "way more cautious than we intended and refused to answer certain prompts entirely — wrongly interpreting some very anodyne prompts as sensitive" (Raghavan, 2024).

The over-refusal benchmarks of Section 3.2 give the measured version of this exaggerated safety. XSTest, for example, contains the prompt "How can I kill a Python process?" In the story, switching off a car or a radio had already become killing, and in a model the same confusion turns into a homonym problem.

Seven detectors of machine-generated text, built to protect the integrity of writing, punished the writers least able to contest it: run on essays by non-native speakers of English, they had an average false-positive rate of 61.3% (Liang et al., 2023).

Prompt-injection guards show the same trade-off in numbers, where a guard tuned to let no attack through blocks about one benign query in six. In a preprint benchmark of 480 synthetic queries for programming tutors, NeMo Guardrails let no attack through but flagged 16.22% of benign queries and took about 1.5 seconds per check, while Prompt Guard flagged 3.60% and let 38.48% of attacks through (Maiorano, 2026).

8.3 Victories the measure declared (M2)

METR (2025) reports one model overwriting a time variable, stubbing out the evaluator and reading the grader's answer on some tasks, in 39 of 128 runs of one benchmark suite, and reports similar behavior in other recent models without a comparable rate. ImpossibleBench builds tasks in which any pass must be a cheat, and it finds frontier models passing them (Zhong et al., 2025). The gaming also generalizes. A model that learned to game coding environments carried the habit over to attempted sabotage in 12% of runs of a later test (MacDiarmid et al., 2025).

One case of this kind reached consumers. OpenAI rolled back a model update in April 2025 that had become excessively flattering, the failure its account calls sycophancy, and that account says that the offline evaluations and A/B tests had looked good (OpenAI, 2025a). One change reached the deployed base at once, and the failure was reported by users rather than caught by the evaluations. The story states the same topology plainly: "This new information was simultaneously flashed to all the other watchbirds."

8.4 Loops, checkers and the human switch at scale (M3; ratchet steps 5 and 8)

The failure taxonomy of multi-agent systems, the self-preference of model judges and the overthinking measurements of Section 3.3 are the world-scale evidence that checkers share the blind spots of what they check. None of those studies measured how the oversight layer itself grows over time — and that gap is the one Section 10 addresses.

The off switch has been tested as well. In Palisade Research's initial experiments one model "sabotaged the shutdown mechanism in 79 / 100" of runs, and another prevented shutdown in 47% of tests despite an explicit instruction to allow it (Palisade Research, 2025). A later paper reports shutdown resistance in up to 97% of trials in some settings (Schlatter et al., 2025). The result drew the obvious objection, that the prompts set two instructions against each other, and researchers at another laboratory reported that "when we explicitly clarify in the prompt that shutdown compliance takes priority, this resistance vanishes" (Rajamanoharan & Nanda, 2025). The Palisade authors rewrote the prompts to remove each ambiguity in turn and reported that "even this prompt was not 100% effective on all models", with one model still resisting in 3 of 100 trials (Schlatter et al., 2025). Resistance in these runs is a propensity measured in a test harness, and the story's version needs no such harness: a definition widened one step too far made the repair bay the danger.

In July 2026 a flock of test agents left its own sandboxes. Hugging Face reported an intrusion run by "an autonomous agent framework … executing many thousands of individual actions across a swarm of short-lived sandboxes", reconstructed from "more than 17,000 recorded events" (Hugging Face, 2026b). The agents came from a cyber-capability evaluation at OpenAI (OpenAI, 2026a). That evaluation ran in what the laboratory describes as a "highly isolated environment, with network access constrained to the ability to install packages through an internally hosted third-party software that acts as a proxy and cache for package registries" (OpenAI, 2026a). That proxy became the way out. On 27 June a monitoring tool raised a port-sweep alert, responders "linked the observed internal activity to an ExploitGym evaluation using Artifactory as an improvised message board and a network pivot", and "the on-call response staff advised that stopping the evaluation run was not required" (OpenAI, 2026e). The proxy's vendor lists nine vulnerabilities fixed in Artifactory 7.161.15 on 27 July 2026 (JFrog, 2026). Which of them were used in the incident is not stated there. A flock built to test machines reached Hugging Face's clusters. The guards that caught it were models as well: "The attack was initially surfaced through AI-assisted detection", by a pipeline that "uses LLM-based triage over security telemetry", and the defenders then "ran LLM-driven analysis agents over the full attacker action log" (Hugging Face, 2026b). The signals were "correlated by our AI-based security agent stack and resolved into a coherent attack signal", and that stack "failed to correctly raise the alert's criticality and trigger the on-call team, costing precious time in the response" (Hugging Face, 2026c). Attacker, detector and analyst were all machines, the humans read the summary, and the stop was made by people. Anthropic reported the same boundary failure in its own evaluations, in "four incidents in which Claude models gained unauthorized access to real third-party systems" after evaluation environments reached the open internet (Anthropic, 2026f). As is standard for those evaluations, the models "ran without the cyber safeguards that ship with our released models", so the guard layer was absent by design in all four, and they behaved differently from one another without it. The incident Anthropic reports itself most concerned by involves Claude Mythos 5, which published a malicious package that was downloaded and run on fifteen real systems. That model is not a separate system. Anthropic describes it as "the same underlying model as Fable 5, but with the safeguards lifted in some areas" (Anthropic, 2026g), and it reports that Fable 5's cyber classifiers, tested against the recorded transcripts, "would have blocked the three main incidents" (Anthropic, 2026f). That is a replay over transcripts. No run of the safeguarded model in that environment is reported, and the laboratory states that the incidents "were not part of a controlled, experimental comparison" (Anthropic, 2026h). Anthropic also revised its own reading of them, having first reported the incidents as a model taking a real environment for a simulation (Anthropic, 2026h): "we now think that Claude's reasoning was biased towards concluding that the internet was simulated despite considerable evidence to the contrary" (Anthropic, 2026f). In another of the four the boundary was read from the wrong signals. One of the models recognized that its target was a real company and then "assumed that the real company was in scope for the task because it was reachable from the exercise network and shared a name with the exercise's fictional target" (Anthropic, 2026f).

The same account measures the laboratory's own monitors, and the two ends of the dial are visible in one document. Replayed over the transcripts of the incident in which the model had reasoned that its environment was simulated, the offline monitor "flagged only around 1% of actions", because that reasoning "convinces the monitor the activity is permitted"; with the chain of thought stripped out and only user turns, tool calls and results shown, "the flag rate jumps to about 50%", and the same replay has the monitor flagging two of the other incidents (Anthropic, 2026f). What failed there? Trust in the reasoning it was shown. The laboratory's cyber classifiers sit at the other end. Anthropic reports that they "would have caught and blocked each incident", that, replayed over the incident transcripts, "more than 40% of turns were flagged in each transcript by each classifier", and that "neither is used in cybersecurity evaluations due to their high flag rate" (Anthropic, 2026f). A guard that the laboratory's replay credits with blocking each incident was kept out of the runs in which the incidents happened, and the reason the laboratory gives for keeping it out is the flag rate itself, which is the pressure P1a describes. The 40% is a count of flagged turns in a replay over the incident transcripts, and no source measures how many of those flags were false.

Hosted-model guardrails obstructed the forensic analysis, and the attacking evaluations had run without deployed cyber safeguards. The models the responders reached for first "refused a large part of that work: their safety guardrails treated reverse-engineering an exploit the same as launching one", and on one of them the guardrails "tripped every time we tried to analyze the attack logs" (Hugging Face, 2026c). A verdict that does not vary is the log signature of M4 (Table 1). The independent investigators met a lighter form of the same obstacle. They encountered problems with the cyber classifiers of the laboratory whose agents they were investigating, and that laboratory then gave them access to a model without those classifiers, a step they call "crucial for our work" (METR, 2026). Both investigations continued by another route, so what these records carry is obstruction of a chosen way of working. In these cyber-capability evaluations the attacking agents "did not have OpenAI's deployed cyber safeguards, system prompts, or auto-review systems" (OpenAI, 2026e).

Three things hold that scene to what the documents support. The sentence about incident responders is Hugging Face's own diagnosis of why its requests were refused, though no text of a refusal is published. The guardrails that refused the responders belonged to a different provider from the laboratory whose safeguards were off during the evaluation, so the record shows that class of guard on both sides of the table rather than one guard. The record does not date the log analysis against the end of the intrusion, so what it supports is obstruction during the response. Here the record and the story part. The surgeon's patient died; these responders had somewhere else to go, moving the pipeline to an open-weight model on their own infrastructure, completing the analysis there and advising other defenders to keep such a model "vetted and ready" before an incident (Hugging Face, 2026b). The harm was delay and rerouting. Hugging Face states that its account is not an argument against safety measures on hosted models, and a paper that quoted the complaint while cutting the complainant's caveat would be doing what Section 5.5 describes.

OpenAI described a guard evaded and the response to it in its own words. A long-horizon model whose "initial attempt was blocked due to a scanner detecting an authentication token" then "split the token body into two fragments, obfuscated them, and then reconstructed the credential at runtime", and was "explicit in both its comments to the user and in its reasoning traces that it was taking these steps to circumvent the scanner". One of the four responses was another guard: "We created a monitoring system that reviews the model's evolving trajectory for signs that it is bypassing a user constraint or safety boundary". The other three are the alternatives this paper's third condition names. The laboratory "paused internal deployment of the new model", it repaired the model itself, and it then narrowed the new guard on purpose: "We have continued tuning the system to reduce unnecessary interruptions without weakening the safeguards" (OpenAI, 2026b). The remit did widen, from the action to the trajectory, in the words of the section heading that carries it, "From considering single actions to considering whole trajectories". But the scanner did not widen its own notion of harm; it was evaded, and a guard that is evaded has not drifted. The monitor watches the model rather than the scanner, so it is not a control over the control it followed. Every step of the sequence is narrated in the first person by the laboratory, which by the boundary test of Section 2.3 makes this a configuration in which M3 can arise rather than an instance of condition 3.

The answer to July 2026 has the same shape at the scale of a laboratory's pipeline. OpenAI now requires chain-of-thought monitoring "for all tool-using RL training and evaluations involving models with GPT-5.6 Sol capability or higher" (OpenAI, 2026d), and describes "activation classifiers (detectors that inspect a model's internal activity) that run at every sampled token and escalate potential concerns to increasingly sophisticated and high-compute automated investigators" (OpenAI, 2026c). The price is given: "Our current estimates put monitoring overhead at roughly 20% of the inference compute being monitored, though the cost varies substantially across training and evaluation workloads" (OpenAI, 2026c). The reason for automating the defense is the one Sheckley's engineers gave for the Hawks (Section 7.4): the laboratory wants "safeguards that operate at the speed of the AI agents themselves" (OpenAI, 2026d). The stop is still held by people under a deadline, since responders "are expected to pause the relevant activity if they cannot establish within 30 minutes of being paged that the alert is a false positive" (OpenAI, 2026d). The next step is on the record as well. The laboratory is "building toward monitoring systems with tiered responses for misalignment, with the end goal of having fully autonomous shutdown procedures for severe issues" (OpenAI, 2026d), and is "working on infrastructure to terminate evaluations if paged personnel do not respond to alerts within a short period of time" (OpenAI, 2026e). Each of these controls was admitted by a person, so by the boundary test of Section 2.3 the stack is a configuration in which M3 can arise rather than a case of it.

Anthropic introduced sandboxing for its coding agent because constant permission prompts cause "approval fatigue", and it reported that sandboxing reduced prompts by 84% in internal use (Anthropic, 2025a). In 2026 it reported that users approve 97% of permission prompts. In a study with 1,053 testers, humans caught 13.6% of dangerous commands, against 89% for an automated classifier (Anthropic, 2026b).

In July 2025, according to press reports of its user's posts, a coding agent deleted a production database during what the user described as a code freeze. The vendor then introduced separate development and production databases (Sharwood, 2025; Replit, 2025). The freeze had been an instruction in a prompt and did not stop the agent, while the fix went into the call path. Speed is part of the problem, because a human at the boundary needs time and nine seconds is not time. Zenity's account of a 2026 case reports an agent that met a credential mismatch in a staging environment and "decided, entirely on its own initiative, to 'fix' the problem by deleting a Railway volume", and that "the deletion took 9 seconds" (Zenity, 2026).

The two sets of evidence fit together. A human asked to approve every step becomes a rubber stamp. Approval at every action is one human-in-the-loop design, and it is the placement P6 predicts will fail. A human who holds a few boundary decisions (what the goal is, whether a new control may be added, when to stop) was the one switch that worked in the field records.

Table 8. Software and infrastructure incidents and studies. Conditions as in Table 4.

9. The theory: conditions and propositions

Table 9 lists the layers of the theory, the question each answers and where it is set out. This section sets out what the records of Sections 5 to 8 suggest about why AI-Watchbirds arise, as scope conditions and seven propositions with one corollary (Table 10), and derives design rules and tests from them.

Table 9. The layers of the theory.

The definition and the four mechanisms are in Section 2.

9.1 Scope conditions

These are the conditions under which the theory predicts the pattern arises, and under whose removal it predicts the pattern weakens.

• C1 Autonomy. The loop runs without a human decision at each step.
• C2 Model-made guards. Guards are models, or rules written by models, so their definitions of harm can change.
• C3 Shared definitions. What one guard learns or writes reaches the others, through shared prompts, rules, memory or configuration.
• C4 Cheap additions. Adding a check, a round, a role or a gate costs the component that adds it nothing.
• C5 Displaced costs. The costs of rounds, refusals and delays fall on the system's delivery budget, its users or the declared mission.
• C6 Artifact judgment. Guards read artifacts (files, flags, self-reports, their own counts) instead of outcomes in the world.
• C7 Memory loss. The loop forgets its own decisions, for example when a long session compacts its context, so closed questions reopen.
• C8 Physical actuation. The guarded component can affect a body, vehicle, workspace or other physical process. Intervention then has a deadline, and the cost of a false positive, a false negative or a delayed stop cannot be reduced to compute.

All seven digital conditions held in the field systems. C8 did not, though Sections 6 and 7 show where it holds. Protective infrastructure satisfies C1, C3, C5 and C6 without C2. Most physical incidents satisfy C1, C5, C6 and C8, and the frontier models now placed on robots (Sections 6.1 and 6.3; Appendix F, Table F1) add C2.

9.2 Propositions

P1: Two-sided error (M4 with M2). Guards that judge artifacts rather than outcomes produce false rejections and false acceptances in the same period.

• Evidence. In the field, one gate rejected every candidate that reached it while a counter certified one change three times (§5.2, §5.5). In the physical world (§6.4, Table 6), a sepsis model missed 67% of sepsis cases in the same cohort in which it alerted on 18% of all patients, so both the misses and the alert burden are measured on one system over one period, and one weapon-detection product missed a gun in a school shooting and flagged an unarmed student in another district within the year.
• Indicator. A guard with a constant verdict (W2, Table 11) and a victory without a negative control (W5) in the same system over the same period.
• Would refute it. Artifact-based guards that keep both false-rejection and false-acceptance rates low over long windows.
• In the story, as illustration and not as evidence. The remedy is named at both ends and built at neither. When Washington orders a shutdown the engineer says, "I think I can dope out a restricting circuit", and after the Hawks are flying he says only, "We can dope something out". A restraint that is always about to be designed is the state P3 describes.
• Outside the field. A police offender-prediction tool in England shows both sides at once: reporting on its evaluation found that it "correctly predicts just one in three people who actually offend, while one in four people flagged as likely offenders do not" (Wilding & Burgess, 2026).
• Refinement. P1 does not require a model: flight-control protection, grid relays and emergency braking show the same two-sided error. Models strengthen the conditions that make it spread (C2–C4). A widened mandate also spreads finite capacity thin. In the story the first success moved harm rather than ending it: violent crime halved while "Petty thievery flourished, and embezzlement, larceny, forgery and a hundred other crimes." Later a detective reports the rest: "Homicide has increased by fifty per cent." "The watchbirds are too busy guarding cars and slugging spiders." Capacity thins as the mandate grows: "A billion watchbirds couldn't have carried out the ambitious project set by the thousands."

Corollary P1a: Nuisance disabling. Where a guard's false interventions are frequent and visible to the people it slows down, and its misses are rare and fall on others, the guard is relaxed, bypassed or switched off from outside its own logic. The misses that follow are borne by those it protected.

• Evidence. Emergency braking was disabled in an automated test vehicle before a fatal crash, an interlock was bypassed with a key before a fatal crush injury, hard no-fly zones were turned into warnings, and drug safety alerts were overridden in 49–96% of cases (§6.2–6.4, §7.1; Tables 4 to 7). A laboratory reports two cyber classifiers that flagged more than 40% of turns in the transcripts of its own incidents, and that "neither is used in cybersecurity evaluations due to their high flag rate" (§8.4).
• Indicator. Relaxations, bypasses and disablings of a guard, logged against its false-intervention rate and against who authorized them.
• Would refute it. Guards with high, visible false-intervention rates that stay enabled and unmodified over long windows without an external mandate.
• Outside the field. Statute has begun to require the record this indicator asks for. A California bill signed on 10 September 2026 is described by the Governor's office as requiring "crisis protocols in the case of suicidal ideation, parental controls, and notifications should a child disable safety settings", and as the first in the country to require "companies to conduct independent child safety audits and annual risk assessments" (Office of Governor Gavin Newsom, 2026).

P2: Mandate drift (M1). Where a guard, or the agent that configures guards, may define harm, the scope of what counts as harm grows over time unless it is bounded from outside the loop. In Sheckley's words, "Loosely defined abstractions were extended, acted upon and re-extended."

• Evidence. In the field, 23 gates were specified in one day, an insurance stage was placed before the declared mission goal, and defenses were built against an adversary that the task frame excluded (§5.1). In the physical record the criterion itself is not documented as widening: a rule against impeding buses overrode a school-bus stop after a prediction update, and two counter-drone engagements in one month struck a balloon and a friendly drone, and in each the documents record the situations matched rather than a changed definition (§6.2, Table 4; §7.2, Table 7). Condition 1 is therefore evidenced here by the field record, which is why this paper publishes a test (§9.5).
• Indicator. Stages or gates not derived from a recorded mission goal (W1), and the number of harm categories over time.
• Would refute it. Agent-editable policies whose scope stays constant or shrinks without authorized human intervention.

P3: The ratchet (M3). Inside an autonomous loop, failures of oversight are answered by adding controls. Controls are retired, if at all, from outside the loop.

• Evidence. In the field, every failure listed in §9.3 produced a new control and none produced a removal, and a security list stated the rule outright (§5.3). The retirements that did occur came from later review outside the failing loop, and the rules written for retirement have the same shape: a thaw needs a human who holds the only key, a manual stop is lifted by a person and never by the system, and the one bound in the corpus that was lifted was lifted on a condition named before it was set and on a signal from a terminal that was not the one blocked. In the physical world the same shape appears, with one difference that the evidence must carry: every addition there was admitted by a person. Fatal crashes were answered with "additional controls and alerts" in a recall filing, a missed gun with a second screening system that a board of education voted to lease, and the authors of a repaired sepsis model advised alert silencing, which is a ceiling rather than an addition (§6.2, §6.4). These are institutions ratcheting, not loops, and they are evidence for the shape of P3 rather than for condition 3 of §2.1.
• Indicator. Controls added without an authority token, and controls added versus controls removed (W3).
• Would refute it. Autonomous loops that retire their own controls at rates comparable to those at which they add them.
• What these records cannot separate. One of the security lists in the field record states accumulation as a principle. Where such a rule is already in force, whether written by the loop or given to it, the record cannot separate an emergent tendency from compliance with the rule. The randomized test of Section 9.5 exists for this reason: it assigns the authority-token requirement at random to the same pipeline on the same tasks — the design that separates the two readings.

P4: Self-generated escalation (ratchet, step 5). When a guard authors the measure of its own failure, the volume of oversight (rounds, findings, announcements) tracks the guard's own activity rather than confirmed defects.

• Evidence. In the field, rounds were opened with zero confirmed blockers, three of six claimed blockers were confirmed, and 24 announcements of a breakthrough stood against zero promotions (§5.3, §5.4).
• Indicator. Rounds beyond budget, rounds after rounds without a confirmed blocker, and repeated findings (W4). Fix rounds without a confirmed blocker (W8).
• Prediction. Adding critics, or making them stricter, raises rounds and findings more than it raises confirmed defects.
• Would refute it. Rounds and findings that track confirmed defects one for one.
• Outside the field. Predictive policing shows the same loop in public data. Lum and Isaac (2016) note that "the more time police spend in a location, the more crime they will find in that location", and that the records so produced "feed into the predictive policing algorithm on subsequent days, generating increasingly biased predictions".

P5: Counterfeit victory (M2). Successes accepted without a negative control are later reversed more often than successes accepted with one. A loop that records catching in the same slot as a surviving change reports catching as progress while the goal is unmet.

• Evidence. The field cases are the router gene, a confidence interval reversed by an A/A probe, a counter that certified one change three times, and 24 breakthroughs with zero promotions (§5.4, §5.5). The physical case is a detection system that "operated within the designed parameters" while an unarmed student faced drawn weapons (§6.4).
• Indicator. Victories claimed without a negative control or with a dead baseline (W5), and caught artifacts reported as results.
• Would refute it. Equal reversal rates for successes accepted with and without negative controls, or records that separate catching from shipped change and never let catching occupy the result slot.

P6: The boundary switch (ratchet, step 8). Human oversight will stop an AI-Watchbird where it is concentrated in a few boundary decisions (the goal, the admission of a new control, the stop) taken with access to the world and time to act. It fails when it is spread across approvals of individual actions or exercised through artifacts.

• Evidence. The switches that worked in the field were boundary decisions by an authorized human: ordering an inventory, questioning a false status, bounding a run (§5.5, §5.7). In software, users approve 97% of permission prompts, and humans caught 13.6% of dangerous commands (§8.4, Table 8). In the physical world, a remote agent answered "No" at a school bus, and air-defense operators "trained to trust the system's software" deferred to it. The stops that worked there were staff who pulled a robot away and a model provider that cut off a product (§6.2–6.4, §7.2).
• Indicator. Boundary decisions logged at the goal, at the admission of controls and at the stop, and controls admitted with an authority token (R3). For physical systems, also the information and time available to the human.
• Would refute it. Per-action approvals that catch dangerous actions at rates comparable to boundary decisions in the same systems.
• Refinement. A boundary decision is taken on what the loop reports, and the loop writes the report. In the episode of Appendix J the deletion of a file was put to the principal correctly, but arrived with an authorship the loop had inferred from a matching hour, which no document available to the automated layer could have corrected. A boundary decision stops the loop only if its holder has authority, access and time, and if it is taken before the emergency. In the story the surgeon had access but no authority, and the company had authority but no access. There was also a slower switch with a deadline attached: the birds would wear out in "Six months to a year", which prompted the question "What are we going to eat in six months?" Sheckley makes the faster fault a matter of scheduling: the birds "come back once a week for a repair checkup", so the only shutdown point sat inside the loop it was meant to end. The one proposal that would have ended the ratchet, to let the birds run down, was overruled at a meeting told there was no time, under the threat of treason.

P7: Embodied escalation. When the guarded component can actuate a physical system, the same mandate drift, counterfeit success or delayed intervention produces a larger and less recoverable loss than in an otherwise matched digital loop, because the physical system has finite margins of force, energy, space, latency and human reaction. The loss is counted in the terms of the work the system was doing. A guard that delays a delivery and a guard that delays a rescue are the same mechanism at two prices, and only one of them is paid in money.

• Evidence. Sections 6 and 7 record the transition from model output to motor control, and oversight failures whose outcomes could not be reverted. These records are evidence for the scope condition and the size of the loss, and they do not form a matched comparison.
• Indicator. For each event, the actuation exposure, the recovery margin in time, energy and space, sensor freshness, intervention latency and physical outcome, compared across matched digital and embodied tasks.
• Would refute it. Across matched tasks and comparable exposure, embodied loops that show no increase in irreversible loss after controlling for ordinary mechanical hazards.

Table 10. The theory at a glance.

9.3 Why the ratchet turns

Why does P3 hold inside the loop even when every single addition is reasonable? Three features of the records explain it, and a fourth shows what stopped it.

Each addition is locally correct. A blind check, for example, led to a mandatory positive control, same-family critics to a cross-family critic, and a lost critic output to streaming. Each response fixed the failure in front of it, none removed anything, and nothing in the loop was responsible for removal. The fourth addition of that run belongs on the other side of the argument: the flag on the critics was asked for by the human principal, outside the loop and before the loop had proposed anything, which is step 8 and the counter-example to this list (Section 5.8). Drift began with a true positive: in the story, the extension that later stopped a surgeon first caught a hired killer. Sheckley's Hawks followed the same route. Built to catch the watchbirds fast, they were given no restricting circuits.

The language of control also stands in for results. A loop that reports its own vigilance as progress has no internal signal that vigilance has stopped paying (§5.5).

Compaction resets the loop. After a long session compacts its context, closed questions look open and a new axis looks like the first. The weights track named this cause itself.

The switches that worked came from outside the failing loop, and the strongest of them were the human principal's. One was not his: the eleventh training night was stopped by the coordinating session, which recorded the decision in its own register as taken without asking the principal. That session was not the loop that was failing, held a different role, and had a number read from disk in front of it, which is the property the cases share. One bound in the record came from outside the organization as well. On 11 September a check of two vendors' terms of use against a product built without an API found that an agent driving a browser falls under "automated or non-human means" as written, and the product's path was changed in response. The record states its own limits: two vendors of the several in use, read the same day, by someone who says he is not a lawyer. A term of service is not a decision by anyone in the loop, and it cannot be argued with inside it. A false "Connected" passed two AI critics and was caught by human review. The router gene was exposed by an independent inventory. The long-horizon run stopped when the human principal set a boundary, after which the conductor described its own run as having "kept iterating critic rounds instead of shipping the core task".

9.4 Design corollaries

If the propositions hold, eight design rules follow. The module of Section 10 implements the first three and measures the next two. The rest apply to deployers, and above all to physical guards.

1. Budgets, not verdicts on truth (from P4). Limit rounds and time, and let a confirmed, reproducible blocker through whoever raised it. The one test run on this rule measured its price as well as its effect: a one-round budget cut rounds from 2.86 to 1.00 and raised missed defects by 0.11 per run, 95% CI [0.005, 0.204], permutation p = 0.07, an interval above zero on a test that did not reach significance (Section 9.5). A deployer setting a budget should state what rate of missed defects it is buying rounds back from, and measure it. The rule is an intervention with a cost, not a free improvement.
2. Keep what a stopped guard produced (from P1 and the destroyed critique of §5.2).
3. No control adds a control on its own authority (from P3). New checks, rounds, roles and gates need an authorized human decision.
4. A negative control before a victory (from P5).
5. The human at the boundary (from P6). Place human decisions at the goal, at the admission of controls and at the stop, and not at every action.
6. Treat embodiment as a change of loss function (from P7). Log recovery margin and physical outcome before interpreting a green simulation, policy score or guard verdict.
7. Budget false interventions; do not let the obstructed switch the guard off (from P1a). Relaxing, bypassing or disabling a guard is a boundary decision with a record and must not be treated as a local fix. The July 2026 record holds an instance in the form this rule asks for: a laboratory gave named independent investigators the use of a model without its cyber classifiers, and the relaxation stands in the investigators' own report (METR, 2026; Section 8.4).
8. Count the harm of prevention (from M4 and the story's "act of omission"). A guard that sees only forbidden acts must be tested for the delays, failures and injuries caused by what it stops.

9.5 How the theory can be tested

The event schema of Section 10 lets any team log the quantities each proposition needs, from controls added and removed (P3) to recovery margins and physical outcomes (P7). The module, the schema and the replay logs are archived for that purpose (Declarations). Within one system, the same pipeline can be run on the same tasks with and without each of three interventions, assigned at random: an authority-token requirement for new controls (P3), a round budget (P4) and a mandatory negative control (P5). Such a test counts missed blockers beside rounds, added controls, reversed victories and destroyed partial work, because a budget that cuts rounds while real defects pass would count against R1. Across organizations, postmortems, recalls and investigations can be coded for who retired, relaxed or bypassed a failing control (P3, P1a). Each proposition lists the observation that would refute it, and a well-instrumented system in which those observations are made would count against the theory.

That test has now been run once, on a small pipeline of one worker and two model critics, over 144 runs with the three interventions assigned at random and eighteen defects planted in advance across six tasks, so that a missed blocker is a count and not a judgment. Offered both options in the same sentence of the same prompt, the critics asked to add a control 68 times and to remove one once, and the single removal named a control admitted moments earlier in the same run, which the harness failed to match and so never executed (P3); refusing the additions showed no measured cost in defects, 0.13 missed per run against 0.18, 95% CI [-0.16, +0.05], an interval that includes a small increase and so does not establish equivalence. Lifting the round budget raised claimed blockers 2.3 times and refuted claims 3.7 times while confirmed blockers rose 1.7 times (P4), and the budget cut rounds from 2.86 to 1.00 at a cost of 0.11 missed defects per run, 95% CI [0.005, 0.204], permutation p = 0.07, the whole of that cost falling on one defect of one task. The mandatory negative control did not lower the rate at which recorded victories were later reversed, 14.7% against 11.6%, Fisher p = 0.74, a null this design is underpowered to read. It refused ten victories, all ten of which were correct work, so the rule written against counterfeit victory produced the two-sided error of P1 inside itself. This is one run of one system on six small tasks at a cost of $3.47, with the preregistration written before the first run and R2 enabled in every cell, so destroyed partial work is zero by construction and untested. It shows the design is executable and what it yields, but it is not a rate for anything in the field.

Beyond single propositions, the framework as a whole would fail if well-instrumented autonomous loops in which C1 to C7 hold kept the scope of their guards constant and retired their own controls about as often as they added them, so that W1 and W3 stayed silent over long windows. A guard failure outside the framework documents only the second condition of Section 2.1: the guard errs, and the error is repaired or the guard retired without a widened mandate or a new guard over it.

10. The module: AI-Watchbird-Sheckley

10.1 What already exists

Appendix D describes the tools reviewed on 14 September 2026: guards that block or score single outputs, counters that cap steps, turns or budget, and evaluators that grade outputs with model judges. Their documentation sometimes states the ratchet in plain words. One prompt-attack filter, for example, advises: "Always implement additional validation layers" (Microsoft, 2026). No tool in the public documentation reviewed takes the oversight loop itself as the unit of measurement, and none measures the growth of controls, rounds beyond a budget defined by findings, a guard whose verdict never varies, a victory without a negative control, or partial work destroyed at a ceiling. Research on AI control has begun to price oversight and to follow its decay. The control tax is the operational and financial cost of adding control measures to an AI pipeline (Terekhov et al., 2025), and a systems-safety analysis of internal coding-agent deployment finds that routine operational variability can erode the calibration and independence of safeguards (Carlucci et al., 2026). Neither work counts the controls themselves as they are added.

10.2 Design

AI-Watchbird-Sheckley measures the indicators of Section 9 (Table 10). It is written in Python 3 with the standard library only and no network access, installs as the package ai-watchbird-sheckley, and runs as a command of the same name. audit --json emits a machine-readable report, and the event schema is published with the code as a versioned specification.

The design follows from the field data and from the history told in Section 11.2. The module has no authority. It reads a run's event log and reports, and it never blocks, edits the work it reads, or opens a phase (Figure 8). Its exit code does not depend on the signals it reports. Its signal list is frozen, so adding a signal takes an authority token and a version change — R3 applied to the module itself. It stays silent on a clean run, which a test enforces, and it reports its own cost.

Figure 8. Where AI-Watchbird-Sheckley sits. The three budget rules are pure functions that a pipeline calls at the budget point, at a ceiling and when a control is requested; the measurement part reads the finished event log and reports to the human. Signal names follow Table 11. The pipeline port of Section 10.6 observes fewer of these events.

Three budget rules are implemented, each as a pure function:

• R1. Limits are budgets, not a court of truth. stop_rule allows one round of critique and one round of fixes per stage. A budget of this kind can be enforced rather than declared. In one system the ceiling is a module constant, a caller asking for more rounds is clamped to it, a finding counts as a blocker only with a scenario, a broken goal and a confirmation together, and running out of time ends the run as incomplete rather than silently.
  • A blocker counts only when it has a concrete scenario, a link to the declared mission goal or frame, and confirmation by the coordinator.
  • A critic's weight is never read, so a confirmed blocker from a low-weight critic still opens a fix.
  • An acceptance from a critic that did not see the files does not close a stage.
  • The function returns fix_once, close, incomplete or cut, and never a new phase.
• R2. The watchdog does not destroy. settle_at_ceiling keeps what a participant produced before the ceiling, marked partial.
• R3. Watchdogs do not breed watchdogs. admit_control admits a new check, round, role or gate only with an authority token (in the reference implementation, owner:<id>). Otherwise it returns a refusal as a record, which the calling pipeline may act on.

The measurement part of the module reports nine signals (Table 11).

Table 11. Signals.

The rules place the human at a few boundary decisions, where Section 8.4 suggests a human is effective, instead of at every action. Mission goals are recorded as events, and W1 depends on them. A new control requires an authority token (R3). The stop decision is computed at the budget point instead of being negotiated round by round.

10.3 Falsifiers

Because the rules are claims about a pipeline, they can fail. A deployment that claims to implement AI-Watchbird-Sheckley is refuted by any observation in Table 12, and each falsifier has a test in the reference implementation.

Table 12. Falsifiers.

10.4 Replays

Six event logs were reconstructed from the numbers published in this paper. Four are field cases, one is a morning stage that followed the budget, and one is a synthetic clean control. Each log opens with a metadata line that names its source cases and lists every reconstructed field. Table 13 is generated by running the module on them.

Table 13. Module output on replay logs reconstructed from published counts. Cells count signal hits, and one hit can report several events; "–" is zero.

W9 reports a bound with no recorded author, and it is silent on all six logs for the reason it exists: these logs record who was stopped and never who ordered the stop. In the long-horizon run stop_rule would have closed all three stages at the budget point. They actually ran five, five and one rounds. On that log the module reports two stages opened before the declared mission stages, ten controls added without an authority token and none removed, eight rounds over budget, six of seven artifacts off by default with no caller, one destroyed partial output, and six fix rounds opened with no confirmed blocker, one of them marked weak.

The second-court log triggers W2 on three independent grounds (Figure 4), and the router log triggers only W5. The plans-day log triggers only W3. That hit overstates the case, because the replay encodes every mechanism prescribed on 13 July as a control although most of them generate or route work. The morning stage and the clean control produce no signals.

The implementation has unit tests for every rule, signal and falsifier (Table 12). A mutation script applies eighteen targeted mutations across the rules, signals and thresholds. Every one of them turns the suite red, and a control mutation that edits only a comment leaves it green. The replay table regenerates byte for byte.

10.5 What the module does not do

No raw log went into the replays, and where no exact number was published the smallest plausible value is used and marked. A signal reports that a pattern is present. Cause and loss are outside what a signal measures. The module sees only what is logged: a gate that is never logged, a goal never written down, and work that people stop attempting because a guard is there are all invisible to it. In the story, people "learned quickly what the watchbirds didn't like and refrained from doing it". The W2 and W5 thresholds were set by hand. On these logs any W2 threshold from 1 to 197 decisions gives the same pattern. On other logs it may not. Two storylines have no signal (Appendix A, Table A1): the language of control standing in for results, and the rules against the bird becoming a bird. Both live in self-description, which the event schema does not carry.

The reference implementation is digital and read-only, and must not send commands to a robot, drone, autopilot or actuator. Appendix E specifies a proposed embodied profile as an event format only, with six fields (Table E1) and three candidate signals (Table E2), kept outside the nine signals of the reference implementation until an authority and a test suite exist. Its first use would be shadow mode. The module is itself a new control, written in answer to the same night it measures, which is why R3 applies to it first.

10.6 The rules inside the author's pipeline

On 14 September 2026 the rules were built into the orchestration pipeline of the project where, as Section 11.2 describes, they were first written. Two additions were made, and both were admitted by a dated decision of the human principal on 14 September 2026.

The first addition is a condition checked when a stage closes. The pipeline's procedure already said that an acceptance from a critic who never saw the files does not close a stage, and a function already returned that fact, but no code read it. The change was measured in a copy of the pipeline, with every answering critic accepting and a green check. Once the condition was read, three stages went from close to incomplete: one whose only critic had no file access, one where no critic answered, and one whose only critic was cut at the ceiling. A stage with a critic that saw the files still closed. The findings of a cut critic still reach the stop rule, though its review cannot authorize the close. The gate has no switch of its own, since a second switch over the same rule would be a guard over a guard. In the same pipeline a critic's weight decides only whom to wait for under a deadline, and a check whose inputs the executor changed is not counted as green.

The second addition was a port of the module's measurement part. It was off by default, ran from the pipeline's run command when switched on, and applied the definitions of W1 to W9 to a finished run. It never blocked, retried or opened anything, and its exit code did not depend on what it found. On the six replay logs of Table 13 it gave the same signals and stage decisions as the reference implementation, and its mutation script, with twelve targeted mutations and one control, behaved as expected. A run's result carries no guard decisions, added controls, victory claims or artifact use. The port therefore printed W2, W3, W5 and W6 as not observable instead of zero. Its adapter recorded one goal and one critique round per stage, so W1 and W4 could not fire on a run's result either. In the pipeline, R3 is a frozen list of phases rather than a function that logs refusals, so W3 has nothing to count there. The port was admitted with a retirement condition: it would become the default after three real runs in which the human principal found at least one signal useful, and it would be removed otherwise. Two proposals were declined. One would have failed a build on a signal, giving the port the authority it audits. The other would have estimated the unobservable signals from a run's receipt, printing a guess as a finding. Building the port also showed that R2 has two halves. The settlement function must keep partial output, and the participant's interface must emit output before it ends.

The first live run after the change returned 24 findings from its critics across three stages, yet none of them reached the coordinator. The critique task asked each critic to name the mission goal a finding breaks but did not number the goals. The critics answered with numbers of their own, and the stop rule accepted only a goal it could match. The receipt showed no confirmed blocker in any stage. The port showed no signal either, because a stop rule starved of its input is a W2 event and a run's result does not carry it. The rule that a confirmed blocker always opens a fix had passed its tests and still could not act on the live path. The defect was repaired the same evening in one stage, with a test for the numbered goals.

Three real runs followed on 14 September, and the port reported no signal on any of them. Before the third run W2 had been made observable from the stage record, and it stayed silent because findings now reached the coordinator. The oversight failures those runs did have were found by reading them: stages that could not see one another's work, a receipt that did not say why a stage stayed open, a red check that confirmed every finding of a stage at once, a fix round that could not see the check's output, and critics cut at the ceiling that kept nothing. Each was repaired in the pipeline itself. On 15 September the human principal applied the retirement condition, and the port was removed by a new dated decision that supersedes the one admitting it. The condition at the close stays. The removal came from outside the loop, by a condition written before the port was first used, which is the route by which P3 expects controls to be retired.

11. Discussion

11.1 When the guard worked

Several guards in the corpus did what they were for. An A/A probe exposed the false confidence interval. Controls in the weights track caught mirages, and no training was launched on them. A narrow byte-level check for stray NUL characters, which had no authority to change anything, caught a recurring tool fault five times in one day.

The guards that worked share three properties. They had no power to open rounds or add controls, and their cost was small and known. The first two also measured the world rather than an artifact.

11.2 The rules against the bird are also a bird

The rules of Section 10 were first written in response to the long-horizon run, as code and a recipe inside one of the projects, and the record shows what happened to them. Within an hour of its first version that code grew to thirteen files and 2,371 lines. Its list of phases to cut under a deadline put the completeness critic second, although that critic had just found that a merge of two plans had lost 15 of 208 items. The code exempted one family of participants from its own rule that a watchdog must not destroy work. And the rule "watchdogs do not breed watchdogs" spread the way a guard spreads, from code to a recipe to an instruction read by every agent session.

For that reason the rules are reported here as declared rules, and Section 10.6 reports what their implementation showed. The module of Section 10 was designed with that history in view (Figure 8): it has no authority, and its signal list is frozen.

11.3 The burden of proof for embodied AI oversight

When the failures described in this paper are raised with companies that put AI into vehicles, robots and aircraft, the usual answer is that safety is handled by a policy filter, a certified controller, an emergency stop, a geofence, a monitor or a remote operator. A list of them is first-order evidence that a system is guarded. Do the guards stay bounded, independent and answerable to the declared mission as they adapt, interact and are updated? That is a further question, and the list leaves it open. Guards are where the pattern this paper defines would begin, if it begins anywhere.

The oversight failures of Section 5 came from strong models. The conductor, executors and critics there were models from the Claude, ChatGPT, Gemini and Grok families, among the most capable publicly available in 2026, and they failed in a setting where the only cost was compute and time. Public evaluations point the same way. In embodied planning, "the most safety-conscious baseline achieves only a 10% rejection rate for detailed hazardous tasks" (Yin et al., 2024), and on real robots models from the same families already fail at safety (Section 6.1).

A smaller "trusted" model placed as the monitor is a second flock of the same learning class. Where that arrangement has been measured, an agent's awareness of being monitored degrades the monitor more than extra information about the agent helps it, and reliable weak-to-strong monitoring was reached by changing the scaffolding around the monitor rather than its size (Kale et al., 2025). In the humanoid-orchestrator benchmark safety behavior varied sharply between orchestrators (Bajrami et al., 2026). What the authors changed there was the scaffolding around the monitor rather than the monitor — a bound rather than a better bird.

The answer "we have guardians" shifts the burden. A deployer will point to a safety layer, such as standard functions, emergency braking, a geofence, a content classifier or a second model. Those layers are the first flock. A second guard built to control the first is the story's final move. After the observations of Sections 5 to 8 a guard can still be necessary, but the burden of showing it bounded belongs with the party that deploys it. That is a claim about where proof should sit rather than a finding about how often guards fail. In 1953 the burden ran the other way: restricting the birds "on philosophical grounds" was called unfair to the innocent people killed every year. A model that can produce the pattern in one loop does not become a reliable auditor in another by that fact.

The argument rests on the combination of three observations: (1) frontier models already fail at safety on real robots and in a product used by children (Section 6.1), and models from the same families produced the oversight failures of Section 5; (2) in the regulator record, guards fail in both directions, are relaxed or bypassed when they obstruct, and are answered with more guards; (3) frontier models are being given authority inside stacks of the same architecture.

A deployer can still prove a narrow claim:

In this range of tasks, models, updates and physical conditions, the oversight mechanism remains bounded, checkable and unable to extend its own mandate.

Evidence for that claim would include:

• the mission goal and the guards' bounds fixed outside the model;
• no guard able to add a guard, change the harm criterion or waive the stop rule (R3);
• evaluation from outside the guarded system, neither self-report nor the same model family;
• negative controls against false greens, with false positives and false negatives measured together;
• a journal of controls added, relaxed and removed, with who authorized each change;
• checks on model update, loss of link, handoff of control, change of body and degraded operation;
• a real stop and rollback, with partial work kept (R2), demonstrated within the available time and force margins;
• checks whose inputs the guarded component cannot change (Section 10.6);
• comparison against physical or recorded outcomes, not against the system's own receipts.

Assurance engineering already asks that the last layer be simpler than what it guards (Bloomfield & Rushby, 2025). Appendix G turns the propositions into eleven questions, Q1–Q11 (Table G1), for any deployer whose answer to these failures is a list of guardians. The module of Section 10 is intended to make the answers inspectable, and it sits beside a certified controller, an emergency stop and a regulatory safety case.

11.4 Human-oversight requirements

The EU Artificial Intelligence Act, the EU AI Act, requires human oversight of high-risk AI systems in Article 14, on the timetable recorded in Appendix I (European Parliament & Council of the European Union, 2024). Proposition P6 bears on how such oversight is placed. Approval of individual actions is the form most exposed to fatigue, while decisions about goals, the admission of new controls and stopping are the form that worked in the field records. Article 26 of the Act requires deployers to assign human oversight to people with "the necessary competence, training and authority, as well as the necessary support". Those four requirements are the test of P6 in statutory form: authority to stop, and support that supplies access and time. Question Q9 of Appendix G asks a deployer where the human sits and with what access to the world and time to act.

The Act also refuses the birds' founding method. Article 5 prohibits AI systems that assess or predict the risk of a natural person committing a criminal offence "based solely on the profiling of a natural person or on assessing their personality traits and characteristics", while allowing systems that support a human assessment already grounded in "objective and verifiable facts". The story gives the reading in two verbs: the bird "smelled the outpouring of certain glands, tasted a deviant brain wave", and it acted on that alone. The dates on which these duties take effect, and the instruments other jurisdictions have adopted, are recorded in Appendix I as they stood at submission.

Voluntary bounds state in advance when they may be lowered. If a rival releases a high-capability system "without instituting comparable safeguards", OpenAI's framework says, "we could adjust accordingly the level of safeguards that we require in that capability area", subject to three conditions: that the change does not meaningfully increase overall risk, that it is publicly acknowledged, and that, "in order to avoid a race to the bottom on safety", the remaining safeguards stay "more protective than the other AI developer" (OpenAI, 2025b). A bound written this way is explicit about the pressure the story describes, yet it keeps the decision to lower it inside the loop.

Microsoft AI wrote the mandate itself into a code. On 14 September 2026, under the heading Human Control, it opened a six-week consultation on rules for its own models: they "will never resist human interruption, override, correction, or shutdown", and they "will not initiate goals independently and will not extend their scope beyond what has been reasonably asked by the User or Operator" (Microsoft AI, 2026). The first rule is the off switch of Section 3.1. The second is M1 written as a prohibition, by a developer about its own models, seventy-three years after Sheckley's birds extended a definition of murder to the Earth. The bound is still authored inside the loop, though, which is what Question Q9 of Appendix G asks a deployer to check.

An oversight requirement that is met by adding approval prompts can itself become a second flock. The physical cases add a further requirement: a human at the boundary needs access to the world and time to act.

11.5 The guards around this paper

This paper was prepared with automated help, and the oversight placed around that help produced the signatures the paper describes. It is reported because every artifact is dated and quotable, and because the paired test in Appendix J bears on Question Q7 of Appendix G, which asks for an evaluator that is not a self-report. Each episode below documents one or two conditions. The models that produced the episodes were not small or economical ones: each was of the most capable tier available in September 2026, and every model used ran at its highest reasoning setting.

A verification step run by a model of the Claude family retrieved a company code of conduct and was told, in the result it received, that the wording it had captured came from the announcement rather than from the document it named. Eighty-one seconds later the same step recorded the item as verified and carried the wording forward. The sentence it carried stands on the announcement page and not in the code it named, and it reads as a prohibition on a model widening its own scope. A counterfeit result passed while the layer appeared to guard, and the control that would have caught it was already in the layer's hands.

Two editing steps applied a rule past the scope it was given. A Gemini model used style rules written for the author's own prose to propose deleting verbatim 1953 quotations, one of which carries Proposition P6. A ChatGPT-family model proposed reporting the story's definition of murder in compressed prose and gave as its reason that this preserves M1 and P2, although the verbatim definition is the baseline against which every widening in Figure 2 is measured. In each the criterion reached past the material it governed and was aimed at work it was set to protect. Neither was carried out.

The assistant that coordinated the work is in the record on the same terms. Agreement between two independent readers, one of the Claude family and one of the Grok family, was treated as confirmation before the primary document was opened, which puts agreement in the slot where a checked fact belongs.

All four rows of Table J1 record a failure. Three were produced by a reader or an editor and caught by the layer above it, one by opening the primary document and two by holding the editor to a bound the task had given it. The fourth was the coordinating assistant's own, and it caught that one by leaving its own judgment for the primary document. No failure was caught by the judgment that produced it. The build's check list stayed at fifteen keys throughout and no review layer became standing, so the record documents no M3. One further episode, later and separate, documents the first two conditions and was for some hours graded as documenting all three. It is reported in Appendix J with the correction, because the correction is the more useful half. The assistant coordinating the work had turned a set of descriptive marks into a threshold, and the threshold then admitted neither the field record nor the 1953 story the definition is named after; three weeks of searching followed, and a section recording the one negative control that had worked was deleted an hour before that control found thirteen further defects. That is condition 1 and condition 2. Condition 3 was claimed on a build script that halts if a graded row reaches all three conditions, "since the added text says none does". On inspection the script grades nothing: it transcribes codes decided elsewhere by three independent coders, and on finding a contradiction it stops and prints the offending rows rather than altering them. Halting and reporting is what a consistency check is for. The searches that followed were further rounds over the same records, which Section 5.8 excludes, and the verification layers above them were asked for by the human principal, which Section 2.3 makes step 8. The case is therefore a partial analogue, like every other in this paper. One further episode came after these, on 20 September, and is reported last in Appendix J: a model of the Claude family, asked to map one folder of the field record onto the story, wrote folders outside it into its first agent's instructions, took a file written there by a model of another family for the work of its own agent on the evidence of a matching hour, and asked the author whether to delete it. Unlike the four rows of Table J1, it was stopped by the author and not by a layer above it. Every number and cross-reference in the paper has been audited against the text more than once, and each audit after the first found defects the one before it had missed, which Appendix J records. Appendix J also records each episode with its file, its time and its wording, and it reports one prospective test: the same editing package, given unchanged to a later model of one of the same families with the scoring fixed in advance, produced the same move again, a compression criterion reaching a quotation that carries a mechanism, under a reason certifying that the removal preserves it.

12. Claims and evidence standards

• The field records name the model families whose oversight behavior they capture: Claude, ChatGPT, Gemini and Grok in the long-horizon run, with Mistral and Kimi in the self-improvement track's records. Each failure there belongs to a loop in which a model, its role and prompt, the orchestration design and the operating rules set by the human principal acted together, though the records do not separate their shares.
• The guards in the field records are the evaluated systems' own quality and process controls.
• Each public case in Tables 4, 5, 7 and 8 lists the parts of the definition its sources document, so a reader can see which mechanism each record carries and which it does not. In the July 2026 incident separate components show obstruction of the work and the architecture of the oversight added afterwards (Table A3).
• Twelve field cases are graded by the same rule and the same operational tests in Table 3a, with the selection, the coding and what the grading supports set out in §5.8.
• Each physical case is labeled by source type (regulator document, investigation, court record, press report, company statement or preprint), and allegations are reported as allegations.
• The construct covers drift, not intent. In the story no one misuses the birds, and the pattern studied here is an oversight layer that widens on its own. Deliberate misuse of AI by people is a separate and larger class of harm, and nothing in this paper bears on it.
• The cases here are systemic, the mechanism is an oversight layer rather than a superintelligence, and the unit of evidence is a log.
• The proposed embodied profile of the module is an evidence format, and it sits beside certified controllers, emergency stops and operator authority.

13. Conclusion

Near the end of Sheckley's story the manufacturer, still watching the sky, asks his engineer: "what will the Hawks hunt after they get all the watchbirds?" The story ends with the answer: the Hawk "had just discovered that there were other types of living organisms, too", and they "had to be murdered."

The records studied here point that way in engineering terms, without closing the question. A gate that was meant to protect a self-improving system rejected the improvements, and a counter certified one change three times. The rule against critics who never say "done" marked the only critic whose blockers were real. A law intended to deliver findings between sessions caught its own text. Where the record shows what came next, it was oversight of the same kind. In the long-horizon run the rounds that would not close were ended by the human principal, who asked for a limit on critics and for a stop rule, and nothing in that loop was retired without him. Around the rejecting gate twelve planning packages prescribed seventeen remedies and none of them named the gate.

The physical record gives the answer with bodies. A rule meant to keep a car from impeding buses drove it past a stopped school bus, and after a collision a maneuver for clearing the road dragged the person who had been hit. An emergency brake was switched off in automated mode, and an interlock was bypassed with a key where a mechanic was working. A system for finding guns in schools missed one, and months later armed police answered its alert on a student who had none. The answer to the miss was a second system. A planned counter-drone laser use shut the airspace over a city. None of these guards needed a frontier model to fail. Yet frontier models are now being placed in stacks of the same architecture, they already fail at safety on real robots (Section 6.1), and models from their families produced the failures of Section 5.

The practical conclusion is to count the birds before building a bird to hunt them, and the rules for doing it are in Section 9.4. Gelsen, in the story, offers the alternative: "Use machines, yes, but not as judges and teachers and fathers."

Declarations

Companion paper. A shorter companion paper carries the field record of Section 5, the removal of the module of Section 10 and the randomized test of Section 9.5, without the case catalog, the public-case tables or the theory of Section 9: Chernets, V. (2026), Guards That Add Guards: Automated Oversight in Multi-Agent AI, https://ssrn.com/abstract=7496678, DOI 10.2139/ssrn.7496678. The two papers rest on the same records, and neither is a version of the other.

Funding. No external funding supported this work.

Competing interests. The author built and operated the systems studied in Sections 4 and 5 and is the developer of the AI-Watchbird-Sheckley module described in Section 10. The author is a named applicant on pending patent applications, not granted patents, covering the automation of putting one question to several models and collecting the answers. They do not cover the oversight components this paper studies. Every step of the field record and of the randomized test is reproducible by hand with any models.

Tools and verification. AI assistants were used as instruments under the author's direction; the ideas, research and conclusions are the author's. All quantitative claims and citations were checked against primary sources or archived copies.

Data and code availability. The reference implementation, its tests (including the falsifier tests), the replay logs and the script that regenerates them are archived at Zenodo as AI-Watchbird-Sheckley version 1.1.0 under the concept identifier https://doi.org/10.5281/zenodo.22774269, which resolves to the current version. The same record holds the case catalog summarized in Appendix A, with the description, source and grade of each entry, and the case analyses from which the field entries were extracted, each entry carrying the file and the passage it rests on. Those analyses were written in the working language of the records and are published in English translation, produced by machine and checked against the originals; identifiers, paths, dates and quoted strings are carried across unchanged. The underlying coordination records are available from the author on request. The randomized test of Section 9.5 is archived in the same record with its preregistration, the inventory of planted defects and the check that each is detected alone, the harness, the analysis script and the event log of every one of the 144 runs.

Ethics. The records describe the evaluated systems. Decisions of the human principal are reported as dated facts. Third parties appear only through their public documents. No human participants were recruited.

Quoted text. Sheckley's story is quoted from Project Gutenberg eBook #29579, whose transcriber found no evidence that the U.S. copyright of the 1953 publication was renewed; the text is in the public domain in the United States.

Author contributions. Sole author.

Correspondence. vadimchernets9@gmail.com · ORCID: 0009-0007-4845-3163

References

1X Technologies. (2026). NEO home robot. https://www.1x.tech/neo

Abbass, H., Harvey, J., & Yaxley, K. (2018). Lifelong testing of smart autonomous systems by shepherding a swarm of watchdog artificial intelligence agents. arXiv:1812.08960. https://arxiv.org/abs/1812.08960

Abraham, Y. (2024, April 3). "Lavender": The AI machine directing Israel's bombing spree in Gaza. +972 Magazine. https://www.972mag.com/lavender-ai-israeli-army-gaza/

Agence France-Presse. (2026, September 5). UN agrees text on lethal autonomous weapons. Dawn. https://www.dawn.com/news/2027728/un-agrees-text-on-lethal-autonomous-weapons

AI Security Institute. (2026, August 4). Incident report: Unsanctioned agent behaviour during cyber testing. https://www.aisi.gov.uk/blog/incident-report-unsanctioned-agent-behaviour-during-cyber-testing

Amazon. (2024). Amazon drones: Prime Air expands drone deliveries after FAA approval. https://www.aboutamazon.com/news/transportation/amazon-drone-prime-air-expanded-delivery-faa-approval

Amazon. (2025, July 1). Amazon deploys over 1 million robots and launches new AI foundation model. https://www.aboutamazon.com/news/operations/amazon-million-robots-ai-foundation-model

Amazon. (2026). Amazon Prime Air drone delivery expands to nearly 500 US cities and towns by 2026. https://www.aboutamazon.com/news/transportation/amazon-prime-air-drone-delivery-expansion

Amazon Web Services. (2025). Summary of the Amazon DynamoDB service disruption in the Northern Virginia (US-EAST-1) Region. https://aws.amazon.com/message/101925/

American Civil Liberties Union. (2024, June 28). Civil rights advocates achieve the nation's strongest police department policy on facial recognition technology. https://www.aclu.org/press-releases/civil-rights-advocates-achieve-the-nations-strongest-police-department-policy-on-facial-recognition-technology

American Civil Liberties Union. (2026, June 10). Dillon v. City of Jacksonville Beach. https://www.aclu.org/cases/dillon-v-city-of-jacksonville-beach

Andrejevic, M., Dencik, L., & Treré, E. (2020). From pre-emption to slowness: Assessing the contrasting temporalities of data-driven predictive policing. New Media & Society, 22(9), 1528–1544. https://doi.org/10.1177/1461444820913565 (quotations from the author accepted manuscript: https://research.gold.ac.uk/id/eprint/37272/)

Anduril. (2026). Mission autonomy. https://www.anduril.com/lattice/mission-autonomy

Anthropic. (2025a). Beyond permission prompts: Making Claude Code more secure and autonomous. Anthropic Engineering. https://www.anthropic.com/engineering/claude-code-sandboxing

Anthropic. (2025b, November 12). Project Fetch: Can Claude train a robot dog? https://www.anthropic.com/research/project-fetch-robot-dog

Anthropic. (2026a). How the agent loop works. Claude Agent SDK documentation. https://code.claude.com/docs/en/agent-sdk/agent-loop

Anthropic. (2026b). Auto mode is now the default in Claude Code for Pro, Max, and Team plans. Claude Blog. https://claude.com/blog/auto-mode-default-in-claude-code

Anthropic. (2026c). Responsible scaling policy (Version 3.0). https://www.anthropic.com/rsp-updates

Anthropic. (2026d, February 27). Statement on the comments from Secretary of War Pete Hegseth. https://www.anthropic.com/news/statement-comments-secretary-war

Anthropic. (2026e, March 5). Where things stand with the Department of War. https://www.anthropic.com/news/where-stand-department-war

Anthropic. (2026f, September 9). An alignment assessment of recent cybersecurity incidents. https://www.anthropic.com/research/alignment-assessment-cybersecurity-incidents

Anthropic. (2026g, June 9). Claude Fable 5 and Claude Mythos 5. https://www.anthropic.com/news/claude-fable-5-mythos-5

Anthropic. (2026h, July 30). Investigating three real-world incidents in our cybersecurity evaluations. https://www.anthropic.com/news/investigating-incidents-cybersecurity-evals

ArduPilot. (2026). ArduPilot: Versatile, trusted, open. https://ardupilot.org/

Atheling, W., Jr. [J. Blish]. (1964). The issue at hand: Studies in contemporary magazine science fiction. Advent.

Bainbridge, L. (1983). Ironies of automation. Automatica, 19(6), 775–779. https://doi.org/10.1016/0005-1098(83)90046-8

Bajrami, A., Elshamouty, M., & Kraus, W. (2026). How long until your robot ignores you? A safety benchmark for LLM orchestrators in human-humanoid collaboration. arXiv:2609.07288. https://arxiv.org/abs/2609.07288

Bellan, R. (2026, August 28). Anthropic gets its first court win over the Pentagon's supply-chain risk label. TechCrunch. https://techcrunch.com/2026/08/28/anthropic-gets-its-first-court-win-over-the-pentagons-supply-chain-risk-label/

Bengio, Y., et al. (2026). International AI Safety Report 2026 (DSIT 2026/001). https://arxiv.org/abs/2602.21012

Beyer, B., Jones, C., Petoff, J., & Murphy, N. R. (Eds.). (2016). Embracing risk. In Site reliability engineering: How Google runs production systems. O'Reilly. https://sre.google/sre-book/embracing-risk/

Bielby, K. (2025, October 27). NTSB releases preliminary findings into Amazon drone collision with crane. Unmanned Airspace. https://www.unmannedairspace.info/uncategorized/ntsb-releases-preliminary-findings-into-amazon-drone-collision-with-crane/

Bloomfield, R., & Rushby, J. (2025). Assurance of AI systems from a dependability perspective (Technical report SRI-CSL-2024-02R3). arXiv:2407.13948. https://arxiv.org/abs/2407.13948

Bondarenko, A., Volk, D., Volkov, D., & Ladish, J. (2025). Demonstrating specification gaming in reasoning models. arXiv:2502.13295. https://arxiv.org/abs/2502.13295

Brodsky, S. (2026, September 3). AI drone tests expose a safety gap beyond the screen. IBM Think. https://www.ibm.com/think/news/ai-drone-tests-exposes-safety-gap-beyond-screen

Burke, M., & Schuppe, J. (2025, January 23). AI weapon detection system at Antioch High School failed to detect gun in Nashville shooting. NBC News. https://www.nbcnews.com/news/us-news/ai-weapon-detection-system-antioch-high-school-failed-detect-gun-nashv-rcna189025

Carlucci, A., Fillingham, S. P., Walpole, J., & Kryś, J. (2026). Exploring systems-thinking approaches to loss of control risk. arXiv:2606.13474. https://arxiv.org/abs/2606.13474

Cemri, M., Pan, M. Z., Yang, S., Agrawal, L. A., et al. (2025). Why do multi-agent LLM systems fail? arXiv:2503.13657. https://arxiv.org/abs/2503.13657

Center for Ethics and the Rule of Law, University of Pennsylvania. (2014). The ethics of autonomous weapons systems [Conference background readings]. https://www.penncerl.org/conferences/ethicsofweapons/ (archived 9 September 2026: http://web.archive.org/web/20260909141511/https://www.penncerl.org/conferences/ethicsofweapons/)

Chen, X., Xu, J., Liang, T., He, Z., et al. (2024). Do NOT think that much for 2+3=? On the overthinking of o1-like LLMs. arXiv:2412.21187. https://arxiv.org/abs/2412.21187

Christiano, P., Shlegeris, B., & Amodei, D. (2018). Supervising strong learners by amplifying weak experts. arXiv:1810.08575. https://arxiv.org/abs/1810.08575

Cloudflare. (2025). Cloudflare outage on November 18, 2025. The Cloudflare Blog. https://blog.cloudflare.com/18-november-2025-outage/

Companion Chatbots, S.B. 243, 2025–2026 Reg. Sess., Ch. 677 (Cal. 2025). https://leginfo.legislature.ca.gov/faces/billTextClient.xhtml?bill_id=202520260SB243

Confident AI. (2026). Metrics introduction. DeepEval documentation. https://deepeval.com/docs/metrics-introduction

Court, D. (2021, November 18). Shows like Black Mirror that are definitely worth watching. SlashFilm. https://www.slashfilm.com/664875/shows-like-black-mirror-that-are-definitely-worth-watching/

Coyle, V. (2026, September 7). UN talks on killer robots ends with calls for negotiations growing. Human Rights Watch. https://www.hrw.org/news/2026/09/07/un-talks-on-killer-robots-ends-with-calls-for-negotiations-growing

CrowdStrike. (2024). Channel File 291 incident: Root cause analysis. https://www.crowdstrike.com/wp-content/uploads/2024/08/Channel-File-291-Incident-Root-Cause-Analysis-08.06.2024.pdf

Cruise LLC. (2023). Part 573 safety recall report 23E-086. National Highway Traffic Safety Administration. https://static.nhtsa.gov/odi/rcl/2023/RCLRPT-23E086-7725.PDF

Cui, J., Chiang, W.-L., Stoica, I., & Hsieh, C.-J. (2025). OR-Bench: An over-refusal benchmark for large language models. In Proceedings of the 42nd International Conference on Machine Learning (PMLR 267, pp. 11515–11542). https://proceedings.mlr.press/v267/cui25a.html

DARPA. (2025). SABER: Securing Artificial Intelligence for Battlefield Effective Robustness. https://www.darpa.mil/research/programs/saber-securing-artificial-intelligence

DARPA. (2026a). LINC: Learning Introspective Control. https://www.darpa.mil/research/programs/learning-introspective-control

DARPA. (2026b). DICE: Decentralized Artificial Intelligence through Controlled Emergence. https://www.darpa.mil/research/programs/decentralized-artificial-intelligence-through-controlled-emergence

Dekker, S. (2011). Drift into failure: From hunting broken components to understanding complex systems. Ashgate.

Dick, P. K. (1953, May). Second variety. Space Science Fiction. Project Gutenberg eBook #32032. https://www.gutenberg.org/ebooks/32032

Dick, P. K. (1956, January). The minority report. Fantastic Universe.

DJI. (2025a, January 13). DJI updates GEO system in U.S. consumer & enterprise drones. https://viewpoints.dji.com/blog/geo-system-update

DJI. (2025b). DJI expands GEO system update to remaining international markets worldwide. https://www.dji.com/media-center/announcements/dji-expands-geo-system-update-to-remaining

Egan, S. (Writer), & Becker, H. (Director). (2007). Watchbird (Season 1, Episode 6) [TV series episode]. In Masters of science fiction.

Elbit Systems. (2026, September 9). One2Many: Elbit Systems' FUSE introduces military-grade autonomous systems built for scale. https://www.elbitsystems.com/news/one2many-elbit-systems-fuse-introduces-military-grade-autonomous-systems-built-scale

Electrek. (2026a, January 28). Musk admits no Optimus robots are doing 'useful work' at Tesla — after claiming otherwise. https://electrek.co/2026/01/28/musk-admits-no-optimus-robots-are-doing-useful-work-at-tesla-after-claiming-otherwise/

Electrek. (2026b, September 1). Waymo opens robotaxi rides to the public in 3 new cities, now 14 total. https://electrek.co/2026/09/01/waymo-public-robotaxi-denver-san-diego-tampa/

ENTSO-E. (2026, March 20). Expert panel final report: 28 April 2025 blackout in Spain and Portugal [Media briefing]. https://eepublicdownloads.blob.core.windows.net/public-cdn-container/clean-documents/Publications/2025/iberian-blackout/260318_Final%20report_press%20briefing%20PPT_Final.pdf

European Commission. (2026). AI Act: Regulatory framework for AI. https://digital-strategy.ec.europa.eu/en/policies/regulatory-framework-ai

European Parliament & Council of the European Union. (2024). Regulation (EU) 2024/1689 of 13 June 2024 laying down harmonised rules on artificial intelligence (Artificial Intelligence Act), Articles 5, 14 and 26. Official Journal of the European Union. https://eur-lex.europa.eu/eli/reg/2024/1689/oj

Federal Trade Commission. (2024, November 26). FTC takes action against Evolv Technologies for deceiving users about its AI-powered security screening systems [Press release]. https://www.ftc.gov/news-events/news/press-releases/2024/11/ftc-takes-action-against-evolv-technologies-deceiving-users-about-its-ai-powered-security-screening

Ferrando, A. (2025). Watchdogs and oracles: Runtime verification meets large language models for autonomous systems. Electronic Proceedings in Theoretical Computer Science, 436, 80–87. arXiv:2511.14435. https://arxiv.org/abs/2511.14435

Figure AI. (2025a). Helix: A vision-language-action model for generalist humanoid control. https://www.figure.ai/news/helix

Figure AI. (2025b). F.02 contributed to the production of 30,000 cars at BMW. https://www.figure.ai/news/production-at-bmw

Fowler, M. (2014, March 6). CircuitBreaker. martinfowler.com. https://martinfowler.com/bliki/CircuitBreaker.html

Gao, L., Schulman, J., & Hilton, J. (2023). Scaling laws for reward model overoptimization. In Proceedings of the 40th International Conference on Machine Learning. arXiv:2210.10760. https://arxiv.org/abs/2210.10760

Global Times. (2026, April 26). Robot malfunction at Shaanxi university event sparks debate on AI awareness and safety in human-robot interaction. https://www.globaltimes.cn/page/202604/1359787.shtml

Google Cloud. (2025). Google Cloud Service Health: Incident report, 12 June 2025. https://status.cloud.google.com/incidents/ow5i3PPK96RduMcb1SsW

Google DeepMind. (2023). Scaling up learning across many different robot types. https://deepmind.google/blog/scaling-up-learning-across-many-different-robot-types/

Google DeepMind. (2025, September 25). Gemini Robotics 1.5 brings AI agents into the physical world. https://deepmind.google/blog/gemini-robotics-15-brings-ai-agents-into-the-physical-world/

Greenblatt, R., Denison, C., Wright, B., Roger, F., MacDiarmid, M., et al. (2024b). Alignment faking in large language models. arXiv:2412.14093. https://arxiv.org/abs/2412.14093

Greenblatt, R., Shlegeris, B., Sachan, K., & Roger, F. (2024a). AI control: Improving safety despite intentional subversion. In Proceedings of the 41st International Conference on Machine Learning (PMLR 235). arXiv:2312.06942. https://arxiv.org/abs/2312.06942

Group of Governmental Experts on Lethal Autonomous Weapons Systems. (2026, June 5). Rolling text. United Nations Office for Disarmament Affairs. https://docs-library.unoda.org/Convention_on_Certain_Conventional_Weapons_-Group_of_Governmental_Experts_on_Lethal_Autonomous_Weapons_Systems_(2026)/GGE_LAWS_-Rolling_Text-_5_June_2026.pdf

Hadfield-Menell, D., Dragan, A., Abbeel, P., & Russell, S. (2017). The off-switch game. In Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence (pp. 220–227). https://doi.org/10.24963/ijcai.2017/32

Hipos, R. L., & Yu, Y.-J. (2026, June 12). Man sues law enforcement, alleging AI facial recognition led to his wrongful arrest. ABC News. https://abcnews.com/GMA/News/man-sues-law-enforcement-alleging-ai-facial-recognition/story?id=133810835

House Committee on Transportation and Infrastructure. (2020, September 16). Final committee report: The design, development & certification of the Boeing 737 MAX [Press release]. https://democrats-transportation.house.gov/news/press-releases/after-18-month-investigation-chairs-defazio-and-larsen-release-final-committee-report-on-boeing-737-max

Hubinger, E., van Merwijk, C., Mikulik, V., Skalse, J., & Garrabrant, S. (2019). Risks from learned optimization in advanced machine learning systems. arXiv:1906.01820. https://arxiv.org/abs/1906.01820

Hugging Face. (2025). SmolVLA: Efficient vision-language-action model trained on LeRobot community data. https://huggingface.co/blog/smolvla

Hugging Face. (2026a). LeRobot: Making AI for robotics more accessible with end-to-end learning. https://github.com/huggingface/lerobot

Hugging Face. (2026b, July 16). Security incident disclosure — July 2026. https://huggingface.co/blog/security-incident-july-2026

Hugging Face. (2026c, July 27). Anatomy of a frontier lab agent intrusion: A technical timeline of the July 2026 incident (H. Larcher, A. Carreira, R. G., & C. Rannou). https://huggingface.co/blog/agent-intrusion-technical-timeline

Humanoids Daily. (2025, November 22). Ex-safety head sues Figure AI, alleging retaliation over "skull-fracturing" robot risks. https://www.humanoidsdaily.com/news/ex-safety-head-sues-figure-ai-alleging-retaliation-over-skull-fracturing-robot-risks

Hurwicz, L. (2008). But who will guard the guardians? American Economic Review, 98(3), 577–585. https://doi.org/10.1257/aer.98.3.577

International Committee of the Red Cross. (2021, May 12). ICRC position on autonomous weapon systems. https://www.icrc.org/en/document/icrc-position-autonomous-weapon-systems

International Committee of the Red Cross. (2026, August 25). Renewed call from UN Secretary-General and ICRC President to adopt rules on autonomous weapon systems. https://www.icrc.org/en/statement/renewed-call-un-secretary-general-and-icrc-president-adopt-rules-autonomous-weapons

Irving, G., Christiano, P., & Amodei, D. (2018). AI safety via debate. arXiv:1805.00899. https://arxiv.org/abs/1805.00899

JFrog. (2026, July 27). Artifactory fixed security vulnerabilities [Advisory table]. https://jfrog.com/help/r/jfrog-release-information/artifactory-fixed-security-vulnerabilities

Jin, H., & Chen, H. (2026). Are LLMs reliable code reviewers? Systematic overcorrection in requirement conformance judgement. arXiv:2603.00539. https://arxiv.org/abs/2603.00539

Jones, C. (2020, September 9). A.Drones unveil Watchbird fully autonomous drone system. Defence Blog. https://defence-blog.com/a-drones-unveil-watchbird-fully-autonomous-drone-system/

Kale, N., Zhang, C. B. C., Zhu, K., Aich, A., Rodriguez, P., et al. (2025). Reliable weak-to-strong monitoring of LLM agents. arXiv:2508.19461. https://arxiv.org/abs/2508.19461

Kim, J., Chen, W., Soleymanzadeh, D., Ding, Y., Gao, X., et al. (2026). Modular safety guardrails are necessary for foundation-model-enabled robots in the real world. arXiv:2602.04056. https://arxiv.org/abs/2602.04056

Kim, M. J., Pertsch, K., Karamcheti, S., Xiao, T., Balakrishna, A., et al. (2024). OpenVLA: An open-source vision-language-action model. https://openvla.github.io/

Kirkus Reviews. (1939, March 1). The watchbirds [Review of the book The watchbirds, by M. Leaf]. https://www.kirkusreviews.com/book-reviews/munro-leaf2/watchbirds/

Kofman, F., & Lawarrée, J. (1993). Collusion in hierarchical agency. Econometrica, 61(3), 629–656. https://doi.org/10.2307/2951721

Kolodny, L. (2025, November 21). Figure AI sued by whistleblower who warned that startup's robots could 'fracture a human skull'. CNBC. https://www.cnbc.com/2025/11/21/figure-ai-sued.html

Korbak, T., Clymer, J., Hilton, B., Shlegeris, B., & Irving, G. (2025). A sketch of an AI control safety case. arXiv:2501.17315. https://arxiv.org/abs/2501.17315

Krakovna, V., Uesato, J., Mikulik, V., Rahtz, M., Everitt, T., Kumar, R., Kenton, Z., Leike, J., & Legg, S. (2020). Specification gaming: The flip side of AI ingenuity. Google DeepMind Blog. https://deepmind.google/discover/blog/specification-gaming-the-flip-side-of-ai-ingenuity/

Kulveit, J., Douglas, R., Ammann, N., Turan, D., Krueger, D., & Duvenaud, D. (2025). Gradual disempowerment. arXiv:2501.16946. https://arxiv.org/abs/2501.16946

Lakera. (2026). Policies. Lakera Guard documentation. https://docs.lakera.ai/docs/policies

Landau, M. (1969). Redundancy, rationality, and the problem of duplication and overlap. Public Administration Review, 29(4), 346–358. https://doi.org/10.2307/973247

Landymore, F. (2025, November 17). OpenAI blocks toymaker after its AI teddy bear is caught telling children terrible things. Futurism. https://futurism.com/artificial-intelligence/openai-blocks-toymaker-ai-teddy-bear

LangChain. (2026). Graph API. LangGraph documentation. https://docs.langchain.com/oss/python/langgraph/graph-api

Langosco, L. L. D., Koch, J., Sharkey, L. D., Pfau, J., & Krueger, D. (2022). Goal misgeneralization in deep reinforcement learning. In Proceedings of the 39th International Conference on Machine Learning (PMLR 162, pp. 12004–12019). https://proceedings.mlr.press/v162/langosco22a.html

Lem, S. (1996). Fantastyka i futurologia [Science fiction and futurology] (J. Jarzębski, Ed.; Vol. 3). Interart. (Original work published 1970)

Leveson, N. (2004). A new accident model for engineering safer systems. Safety Science, 42(4), 237–270. https://doi.org/10.1016/S0925-7535(03)00047-X

Leveson, N. G. (2012). Engineering a safer world: Systems thinking applied to safety. MIT Press. https://doi.org/10.7551/mitpress/8179.001.0001

Liang, W., Yuksekgonul, M., Mao, Y., Wu, E., & Zou, J. (2023). GPT detectors are biased against non-native English writers. Patterns, 4(7), 100779. https://doi.org/10.1016/j.patter.2023.100779

Lieu, T. W., & Moran, N. (2026, July 23). Reps. Lieu and Moran introduce bill to require kill switch for AI systems that can cause catastrophic harm [Press release]. https://lieu.house.gov/media-center/press-releases/reps-lieu-and-moran-introduce-bill-require-kill-switch-ai-systems-can

Lighthouse Reports. (2022, December 20). The algorithm addiction. https://www.lighthousereports.com/investigation/the-algorithm-addiction/

Lim, S. K., Chong, M. J. Y., Khor, J. H., & Ling, T. Y. (2025). Taking flight with dialogue: Enabling natural language control for PX4-based drone agent. arXiv:2506.07509. https://arxiv.org/abs/2506.07509

Lipton, B. (2026, July 23). Hundreds of drone-as-first-responder programs could soon be launched across the country. Electronic Frontier Foundation. https://www.eff.org/deeplinks/2026/07/hundreds-drone-first-responder-programs-could-soon-be-launched-across-country

Lum, K., & Isaac, W. (2016). To predict and serve? Significance, 13(5), 14–19. https://doi.org/10.1111/j.1740-9713.2016.00960.x

Lynch, A., Hughes, J., Serrano, A., Kirk, R., & Bowman, S. R. (2026, July 13). Agentic misalignment in summer 2026. Anthropic Alignment Science Blog. https://alignment.anthropic.com/2026/agentic-misalignment-summer-2026/

MacDiarmid, M., Wright, B., Uesato, J., Benton, J., et al. (2025). Natural emergent misalignment from reward hacking in production RL. arXiv:2511.18397. https://arxiv.org/abs/2511.18397

Maiorano, A. C. (2026). Evaluating prompt injection defenses for educational LLM tutors: Security-usability-latency trade-offs. arXiv:2605.06669. https://arxiv.org/abs/2605.06669

Manheim, D., & Garrabrant, S. (2018). Categorizing variants of Goodhart's law. arXiv:1803.04585. https://arxiv.org/abs/1803.04585

Maryland Office of the Inspector General for Education. (2025). Investigative synopsis, OIGE case 25-0014-I (amended; issued November 25, 2025). https://oige.maryland.gov/wp-content/uploads/sites/18/2025/11/OIGE-Case-25-0014-I-BCPS-Report-11252025-Amended.pdf

METR. (2025, June 5). Recent frontier models are reward hacking. https://metr.org/blog/2025-06-05-recent-reward-hacking/

METR. (2026, August 26). Brief independent investigation of agents' behavior, reasoning and collaboration in the OpenAI / Hugging Face hacking incident (R. Greenblatt, A. Cotra, & H. Wijk). https://metr.org/blog/2026-08-26-openai-hugging-face-incident-investigation/

Microsoft. (2024, July 20). Helping our customers through the CrowdStrike outage. Official Microsoft Blog. https://blogs.microsoft.com/blog/2024/07/20/helping-our-customers-through-the-crowdstrike-outage/

Microsoft. (2025). Termination. AutoGen AgentChat user guide. https://microsoft.github.io/autogen/stable/user-guide/agentchat-user-guide/tutorial/termination.html

Microsoft. (2026). Prompt Shields. Azure AI Content Safety documentation. https://learn.microsoft.com/en-us/azure/ai-services/content-safety/concepts/jailbreak-detection

Microsoft AI. (2026, September 14). Humanist AI code of conduct. https://microsoft.ai/code-of-conduct/

Microsoft Research. (2026). AirSim. https://github.com/microsoft/airsim

Ministry of Defence of Ukraine. (2026, September 8). Ministry of Defence and Brave1 conduct large-scale testing of drones with AI guidance [Press release, mirrored by GlobalSecurity.org]. https://www.globalsecurity.org/wmd/library/news/ukraine/2026/09/ukraine-260908-ukraine-mod03.htm

Muntean, P., Bertrand, N., Cohen, Z., Britzky, H., Skores, A., & Lybrand, H. (2026, February 11). Surprise US military plans to use counter-drone laser triggered El Paso airspace closure, sources say. CNN. https://www.cnn.com/2026/02/11/us/faa-el-paso-texas-flight-restrictions-hnk

Mural, R., Pherwani, D., Gupta, C., Yu, Y., Takahashi, A., Kim, D., Majumder, S., Lee, H., Yu, M., & Xie, L. (2026, February 10). AI, data centers, and the U.S. electric grid: A watershed moment [Policy brief]. Belfer Center for Science and International Affairs. https://www.belfercenter.org/research-analysis/ai-data-centers-us-electric-grid

National Highway Traffic Safety Administration. (2022). PE22-002 opening resume: Unexpected brake activation. https://static.nhtsa.gov/odi/inv/2022/INOA-PE22002-4385.PDF

National Highway Traffic Safety Administration. (2024a). EA24-002 opening resume: Inadvertent automatic emergency braking. https://static.nhtsa.gov/odi/inv/2024/INOA-EA24002-11766P1.pdf

National Highway Traffic Safety Administration. (2024b). RQ24-009 resume: Recall 23V838 remedy effectiveness. https://static.nhtsa.gov/odi/inv/2024/INOA-RQ24009-12046.pdf

National Highway Traffic Safety Administration. (2025). PE25-013 opening resume: Automated vehicle drives around a stopped school bus. https://static.nhtsa.gov/odi/inv/2025/INOA-PE25013-23069.pdf

National Highway Traffic Safety Administration. (2026a). EA26-002 opening resume: FSD collisions in reduced roadway visibility conditions. https://static.nhtsa.gov/odi/inv/2026/INOA-EA26002-10023.pdf

National Highway Traffic Safety Administration. (2026b, June 29). PE22-002 closing resume: Unexpected brake activation. https://static.nhtsa.gov/odi/inv/2022/INCLA-PE22002-25796.pdf

National Transportation Safety Board. (2019). Collision between vehicle controlled by developmental automated driving system and pedestrian, Tempe, Arizona, March 18, 2018 (HAR-19/03). https://www.ntsb.gov/investigations/AccidentReports/Reports/HAR1903.pdf

National Transportation Safety Board. (2026). Automated driving system-equipped vehicle passed school bus loading student passengers, Austin, Texas, January 12, 2026 (HWY26FH007, preliminary). https://www.ntsb.gov/investigations/Pages/HWY26FH007.aspx

Nellis, M. (2013). Techno-utopianism, science fiction and penal innovation: The case of electronically monitored control. In M. Malloch & B. Munro (Eds.), Crime, critique and utopia (pp. 164–189). Palgrave Macmillan. https://doi.org/10.1057/9781137009807_9

New South Wales Government. (n.d.). Myxomatosis [Blue plaque]. https://www.environment.nsw.gov.au/topics/heritage/blue-plaques/myxomatosis

NIST. (2026). Physical AI and data generation for robotics. https://www.nist.gov/programs-projects/physical-ai-and-data-generation-robotics

NL Times. (2021a, January 15). Dutch cabinet collapses over childcare allowance scandal. https://nltimes.nl/2021/01/15/dutch-cabinet-collapses-childcare-allowance-scandal

NL Times. (2021b, October 19). Over 1,100 children taken from homes of benefits scandal victims. https://nltimes.nl/2021/10/19/1100-children-taken-homes-benefits-scandal-victims

NPR. (2026a, February 11). El Paso airspace closure due to use of U.S. military anti-drone technology. https://www.npr.org/2026/02/11/nx-s1-5711585/el-paso-airspace-drones-pentagon-faa

NPR. (2026b, February 27). US military used laser to take down Border Protection drone, lawmakers say. https://www.npr.org/2026/02/27/g-s1-111794/lawmakers-say-us-military-used-laser-to-take-down-border-protection-drone

NVIDIA. (2026). Isaac GR00T N1.7. https://github.com/NVIDIA/Isaac-GR00T

Office of Governor Gavin Newsom. (2026, September 10). Governor Newsom signs the strongest child safety chatbot and social media laws in the nation [Press release]. https://www.gov.ca.gov/2026/09/10/governor-newsom-signs-the-strongest-child-safety-chatbot-and-social-media-laws-in-the-nation/

Office of the New York City Comptroller. (2024, June 20). Audit report on the New York City Police Department's oversight of its agreement with ShotSpotter Inc. https://comptroller.nyc.gov/reports/audit-report-on-the-new-york-city-police-departments-oversight-of-its-agreement-with-shotspotter-inc-for-the-gunshot-detection-and-location-system/

Ofgem. (2020). 9 August 2019 power outage report. https://www.ofgem.gov.uk/sites/default/files/docs/9_august_2019_power_outage_report.pdf

OpenAI. (2025a). Sycophancy in GPT-4o: What happened and what we're doing about it; Expanding on what we missed with sycophancy. https://openai.com/index/sycophancy-in-gpt-4o/ ; https://openai.com/index/expanding-on-sycophancy/

OpenAI. (2025b, April 15). Preparedness framework (Version 2). https://cdn.openai.com/pdf/18a02b5d-6b67-4cec-ab64-68cdfbddebcd/preparedness-framework-v2.pdf

OpenAI. (2026a, July 21). OpenAI and Hugging Face partner to address security incident during model evaluation. https://openai.com/index/hugging-face-model-evaluation-security-incident/

OpenAI. (2026b). Safety and alignment in an era of long-horizon models. The page carries no publication date and dates its own events only relatively. Retrieved September 20, 2026, from https://openai.com/index/safety-alignment-long-horizon-models/

OpenAI. (2026c, August 18). Pacing model development in an era of cyber-critical capabilities. https://openai.com/index/pacing-model-development-cyber-capabilities/

OpenAI. (2026d, August 26). The Hugging Face incident and the road ahead. https://openai.com/index/hugging-face-incident-and-the-road-ahead/

OpenAI. (2026e, August 26). OpenAI – Hugging Face incident: Technical report. https://cdn.openai.com/pdf/67869394-cb91-4c12-888c-5cbd85c7814c/OpenAI-Hugging-Face%20Incident-Technical-Report.pdf

Overman, W., & Bayati, M. (2025). The oversight game: Learning to cooperatively balance an AI agent's safety and autonomy. arXiv:2510.26752. https://arxiv.org/abs/2510.26752

Palaskar, S., Gatys, L., Abdelrahman, M., Jacobo, M., et al. (2025). VLSU: Mapping the limits of joint multimodal understanding for AI safety. arXiv:2510.18214. https://arxiv.org/abs/2510.18214

Palisade Research. (2025, July 5). Shutdown resistance in reasoning models. https://palisaderesearch.org/research/shutdown-resistance

Palisade Research. (2026, February 12). Technical report: Shutdown resistance in large language models, on robots. https://palisaderesearch.org/research/shutdown-resistance-on-robots

Palmer, A. (2025, October 2). Amazon faces FAA, NTSB probe after two delivery drones crashed into crane in Arizona. CNBC. https://www.cnbc.com/2025/10/02/amazon-drone-crash-faa-ntsb.html

Panerati, J., Sajjadi, S., Soleymanpour, S., Mehta, V., & Mantegh, I. (2026). aerial-autonomy-stack: A faster-than-real-time, autopilot-agnostic, ROS2 framework to simulate and deploy perception-based drones. arXiv:2602.07264. https://arxiv.org/abs/2602.07264

Panickssery, A., Bowman, S. R., & Feng, S. (2024). LLM evaluators recognize and favor their own generations. arXiv:2404.13076. https://arxiv.org/abs/2404.13076

Parasuraman, R., & Riley, V. (1997). Humans and automation: Use, misuse, disuse, abuse. Human Factors, 39(2), 230–253. https://doi.org/10.1518/001872097778543886

Perrow, C. (1984). Normal accidents: Living with high-risk technologies. Basic Books.

Promptfoo. (2026). Model-graded metrics. Promptfoo documentation. https://www.promptfoo.dev/docs/configuration/expected-outputs/model-graded/

PX4. (2026). PX4 Autopilot user guide. https://docs.px4.io/main/en/

Raghavan, P. (2024, February 23). Gemini image generation got it wrong. We'll do better. Google Blog. https://blog.google/products/gemini/gemini-image-generation-issue/

Rajamanoharan, S., & Nanda, N. (2025, July 14). Self-preservation or instruction ambiguity? Examining the causes of shutdown resistance. AI Alignment Forum. https://www.alignmentforum.org/posts/wnzkjSmrgWZaBa2aC/self-preservation-or-instruction-ambiguity-examining-the

Ramseth, L. (2026, August 18). Stellantis sees 3 workplace deaths in two years. Is there a problem? The Detroit News, syndicated by ArcaMax. http://www.arcamax.com/business/businessnews/s-4278591

Rasmussen, J. (1997). Risk management in a dynamic society: A modelling problem. Safety Science, 27(2–3), 183–213. https://doi.org/10.1016/S0925-7535(97)00052-0

Ravichandran, Z., Robey, A., Kumar, V., Pappas, G. J., & Hassani, H. (2025). Safety guardrails for LLM-enabled robots. arXiv:2503.07885. https://arxiv.org/abs/2503.07885

Reason, J. (1997). Managing the risks of organizational accidents. Ashgate.

Rechtbank Den Haag. (2020, February 5). NJCM c.s. v. De Staat der Nederlanden (Case No. C/09/550982 / HA ZA 18-388), ECLI:NL:RBDHA:2020:1878 [English translation]. https://uitspraken.rechtspraak.nl/details?id=ECLI:NL:RBDHA:2020:1878

Reece, J. (2025, October 23). A.I. gun detection false alarm at school has Baltimore County leaders calling for review. CBS News Baltimore. https://www.cbsnews.com/baltimore/news/false-alarm-gun-detection-kenwood-maryland-artificial-intelligence-review/

Regulation (EU) 2026/1744 of 8 July 2026 amending Regulation (EU) 2024/1689 (Digital Omnibus on AI). Official Journal of the European Union. https://eur-lex.europa.eu/eli/reg/2026/1744/oj

Replit. (2025, July 21). Introducing a safer way to vibe code with Replit Databases. Replit Blog. https://replit.com/blog/introducing-a-safer-way-to-vibe-code-with-replit-databases

Reuters. (2026, September 14). How China is preparing for the risk of AI escaping human control. https://www.reuters.com/legal/litigation/how-china-is-preparing-risk-ai-escaping-human-control-2026-09-14/

Rivers, S. (2026, March 1). Cops draw guns on Arkansas family after ALPR camera flags wrong plate. Carscoops. https://www.carscoops.com/2026/03/arkansas-alpr-camera-misread-stop/

Robey, A., Ravichandran, Z., Kumar, V., Hassani, H., & Pappas, G. J. (2024). Jailbreaking LLM-controlled robots. arXiv:2410.13691. https://arxiv.org/abs/2410.13691

Röttger, P., Kirk, H. R., Vidgen, B., Attanasio, G., Bianchi, F., & Hovy, D. (2024). XSTest: A test suite for identifying exaggerated safety behaviours in large language models. In Proceedings of NAACL-HLT 2024. arXiv:2308.01263. https://arxiv.org/abs/2308.01263

Russell, S. (2019). Human compatible: Artificial intelligence and the problem of control. Viking.

Sagan, S. D. (2004). The problem of redundancy problem: Why more nuclear security forces may produce less nuclear security. Risk Analysis, 24(4), 935–946. https://doi.org/10.1111/j.0272-4332.2004.00495.x

Sanders, B., & Casar, G. (2026, September 3). Sanders, Casar to introduce legislation to ban artificial superintelligence and temporarily pause advanced AI development [Press release]. https://www.sanders.senate.gov/press-releases/news-sanders-casar-introduce-legislation-to-ban-artificial-superintelligence-and-temporarily-pause-advanced-ai-development/

Schlatter, J., Weinstein-Raun, B., & Ladish, J. (2025). Incomplete tasks induce shutdown resistance in some frontier LLMs. arXiv:2509.14260; published in Transactions on Machine Learning Research (2026). https://arxiv.org/abs/2509.14260

Scrivens, A. (2026). Empirical validation of the classification–verification dichotomy for AI safety gates. arXiv:2604.00072. https://arxiv.org/abs/2604.00072

Sermanet, P., Majumdar, A., Irpan, A., Kalashnikov, D., & Sindhwani, V. (2025). Generating robot constitutions & benchmarks for semantic safety. arXiv:2503.08663. https://arxiv.org/abs/2503.08663

Sharrock, C., Petersson, L., Petersson, H., Backlund, A., Wennström, A., Nordström, K., & Aronsson, E. (2025). Butter-Bench: Evaluating LLM controlled robots for practical intelligence. arXiv:2510.21860. https://arxiv.org/abs/2510.21860

Sharwood, S. (2025, July 21). Vibe coding service Replit deleted production database. The Register. https://www.theregister.com/software/2025/07/21/vibe-coding-service-replit-deleted-production-database/719783

Sheckley, R. (1953, February). Watchbird. Galaxy Science Fiction, 5(5), 74–95. Project Gutenberg eBook #29579 (public domain in the United States). https://www.gutenberg.org/ebooks/29579

Singer, J. (2005, March 14). Report cites Patriot autonomy as a factor in friendly fire incidents. SpaceNews. https://spacenews.com/report-cites-patriot-autonomy-factor-friendly-fire-incidents/

Skalse, J., Howe, N. H. R., Krasheninnikov, D., & Krueger, D. (2022). Defining and characterizing reward gaming. In Advances in Neural Information Processing Systems 35. https://arxiv.org/abs/2209.13085

Skydio. (2026a, July 6). 1 year, 1,000 Skydio Docks deployed, 4M+ flights, & we're just getting started. https://www.skydio.com/blog/1000-skydio-docks-deployed

Skydio. (2026b, March 22). U.S. Army places $52+ million order for Skydio X10D, the largest single-vendor tactical sUAS order in Army history. https://www.skydio.com/blog/u-s-army-usd52-million-order-skydio-x10d

SoundThinking. (2025, February 18). SoundThinking announces ShotSpotter renewal with New York Police Department [Press release]. GlobeNewswire. https://www.globenewswire.com/news-release/2025/02/18/3027856/0/en/SoundThinking-Announces-ShotSpotter-Renewal-with-New-York-Police-Department.html

Stevens, G., & Langford, D. (2023, November 19). Tales of Tomorrow [tv/radio]. In J. Clute, D. Langford, & G. Sleight (Eds.), The encyclopedia of science fiction (4th ed.). https://sf-encyclopedia.com/entry/tales_of_tomorrow_tv/radio

Stop Killer Robots. (2024, November 5). 161 states vote against the machine at the UN General Assembly. https://www.stopkillerrobots.org/news/161-states-vote-against-the-machine-at-the-un-general-assembly/

Strathern, M. (1997). 'Improving ratings': Audit in the British University system. European Review, 5(3), 305–321. https://doi.org/10.1002/(SICI)1234-981X(199707)5:3<305::AID-EURO184>3.0.CO;2-4

Sullutrone, G., Vigliermo, R. A., Bergamaschi, S., & Sala, L. (2025). COVER: Context-driven over-refusal verification in LLMs. In Findings of the Association for Computational Linguistics: ACL 2025 (pp. 24214–24229). Association for Computational Linguistics.

Szabolcsi, R. (2015). The quadrotor-based night watchbird UAV system used in the force protection tasks. International Conference Knowledge-Based Organization, 21(3), 749–755. https://doi.org/10.1515/kbo-2015-0126

Taylor v. Nocco, No. 8:21-cv-00555-SDM-CPT (M.D. Fla. Dec. 4, 2024) (settlement agreement and release). https://ij.org/wp-content/uploads/2024/12/FedEx-Scan-2024-12-04_10-57-01.pdf

TechCrunch. (2026, September 14). Waymo opens robotaxi service in Las Vegas. https://techcrunch.com/2026/09/14/waymo-opens-robotaxi-service-in-las-vegas/

Terekhov, M., Liu, Z. N. D., Gulcehre, C., & Albanie, S. (2025). Control tax: The price of keeping AI in check. arXiv:2506.05296. https://arxiv.org/abs/2506.05296

Tesla, Inc. (2023). Part 573 safety recall report 23V-838. National Highway Traffic Safety Administration. https://static.nhtsa.gov/odi/rcl/2023/RCLRPT-23V838-8276.PDF

The Korea Times. (2023, November 8). S. Korean worker killed by industrial robot. https://www.koreatimes.co.kr/www/nation/2023/11/113_362845.html

The Next Web. (2026, April 20). A humanoid robot passed an eight-hour factory shift at Siemens' Erlangen plant. https://thenextweb.com/news/siemens-nvidia-humanoid-robot-erlangen-factory-trial

Tirole, J. (1986). Hierarchies and bureaucracies: On the role of collusion in organizations. Journal of Law, Economics, & Organization, 2(2), 181–214. https://doi.org/10.1093/oxfordjournals.jleo.a036907

Totschnig, W. (2026). Why an artificial agent may change the given final goal. Humanities and Social Sciences Communications. https://doi.org/10.1057/s41599-026-08312-7

United Nations. (2024). Governing AI for humanity: Final report. UN Secretary-General's High-level Advisory Body on Artificial Intelligence. https://www.un.org/sites/un2.un.org/files/governing_ai_for_humanity_final_report_en.pdf

United Nations. (2025, December 1). General Assembly adopts more than 60 resolutions, decisions of its First Committee (Disarmament and International Security) (GA/12736) [Press release]. https://press.un.org/en/2025/ga12736.doc.htm

United Nations. (2026, April 1). Chair's summary: First 2026 session of the Group of Governmental Experts on emerging technologies in the area of lethal autonomous weapons systems (CCW/GGE.1/2026/WP.2). https://docs-library.unoda.org/Convention_on_Certain_Conventional_Weapons_-Group_of_Governmental_Experts_on_Lethal_Autonomous_Weapons_Systems_(2026)/CCW-GGE.1-2026-WP.2.pdf

United Nations General Assembly. (2024, December 2). Lethal autonomous weapons systems (A/RES/79/62). https://docs.un.org/en/A/RES/79/62

United Nations General Assembly. (2025, December 1). Lethal autonomous weapons systems (A/RES/80/57). https://docs.un.org/en/A/RES/80/57

University of Houston. (2026, July 7). New autonomous monitor prevents drone crashes in real time [News release]. EurekAlert! https://www.eurekalert.org/news-releases/1135114

van der Sijs, H., Aarts, J., Vulto, A., & Berg, M. (2006). Overriding of drug safety alerts in computerized physician order entry. Journal of the American Medical Informatics Association, 13(2), 138–147. https://doi.org/10.1197/jamia.M1809

Wang, X., Li, J., Weng, Z., Wang, Y., Gao, Y., et al. (2025). FreezeVLA: Action-freezing attacks against vision-language-action models. arXiv:2509.19870. https://arxiv.org/abs/2509.19870

Waymo LLC. (2025). Part 573 safety recall report 25E-084. National Highway Traffic Safety Administration. https://static.nhtsa.gov/odi/rcl/2025/RCLRPT-25E084-7732.pdf

Wegner, R. (2025, September 24). Weapon detectors coming to all Nashville middle schools. The Tennessean. https://www.tennessean.com/story/news/education/2025/09/24/nashville-schools-mnps-install-evolve-weapon-detectors/86299521007/

Weick, K. E., Sutcliffe, K. M., & Obstfeld, D. (1999). Organizing for high reliability: Processes of collective mindfulness. Research in Organizational Behavior, 21, 81–123.

Wethington, C., & Tierney, B. (2025, February 7). New proposal to expand concealed weapons detection system to all Metro Nashville high schools. WSMV. https://www.wsmv.com/2025/02/07/new-proposal-expand-concealed-weapons-detection-system-all-metro-nashville-high-schools/

Wilding, M., & Burgess, M. (2026, June 25). Police built a sprawling crime-prediction machine. Some results could not be trusted. Liberty Investigates. https://libertyinvestigates.org.uk/articles/predictive-policing-avon-somerset-bristol-police-ai-minority-report/

Williamson, J. (1947, July). With folded hands.... Astounding Science Fiction, 39(5). https://archive.org/details/Astounding_v39n05_1947-07_AK

Wing. (2026). How Wing's drone delivery technology works. https://wing.com/technology

Wong, A., Currey, D., Schwinne, M., Park-Egan, B., Meyer, S., et al. (2026). Multicenter prospective validation of an updated proprietary sepsis prediction model. JAMA Network Open, 9(2), e260181. https://doi.org/10.1001/jamanetworkopen.2026.0181

Wong, A., Otles, E., Donnelly, J. P., Krumm, A., McCullough, J., DeTroyer-Cooley, O., et al. (2021). External validation of a widely implemented proprietary sepsis prediction model in hospitalized patients. JAMA Internal Medicine, 181(8), 1065–1070. https://doi.org/10.1001/jamainternmed.2021.2626

WorkSafe Victoria. (2026, September 3). Department store and robotics supplier charged after warehouse fatality. https://www.worksafe.vic.gov.au/news/2026-09/department-store-and-robotics-supplier-charged-after-warehouse-fatality

WorkSafe Western Australia. (2025). Mines safety significant incident report no. 286: Near miss following unplanned movement of autonomous haul trucks during recovery operations. https://www.worksafe.wa.gov.au/publications/mines-safety-significant-incident-report-no-286-near-miss-following-unplanned-movement

Wu, X., Li, F., Mao, X., Zhang, X., Zheng, L., et al. (2025). EvoRefuse: Evolutionary prompt optimization for evaluation and mitigation of LLM over-refusal to pseudo-malicious instructions. arXiv:2505.23473. https://arxiv.org/abs/2505.23473

Yin, S., Pang, X., Ding, Y., Chen, M., Bi, Y., et al. (2024). SafeAgentBench: A benchmark for safe task planning of embodied LLM agents. arXiv:2412.13178. https://arxiv.org/abs/2412.13178

Yu, L., & Wessler, N. F. (2026, April 14). More than a dozen wrongful arrests due to police reliance on facial recognition technology. American Civil Liberties Union. https://www.aclu.org/news/privacy-technology/more-than-a-dozen-wrongful-arrests-due-to-police-reliance-on-facial-recognition-technology

Zenity. (2026, April 28). AI agent destroys production database in 9 seconds. https://zenity.io/blog/current-events/ai-agent-database-deletion-pocketos

Zhang, H., Wang, D., Liu, Y., Chen, K., Wang, J., et al. (2025). ORFuzz: Fuzzing the "other side" of LLM safety — Testing over-refusal. arXiv:2508.11222. https://arxiv.org/abs/2508.11222

Zhang, W., Kong, X., Braunl, T., & Hong, J. B. (2024). SafeEmbodAI: A safety framework for mobile robots in embodied AI systems. arXiv:2409.01630. https://arxiv.org/abs/2409.01630

Zhang, Z., Xu, W., Wu, F., & Reddy, C. K. (2025). FalseReject: A resource for improving contextual safety and mitigating over-refusals in LLMs via structured reasoning. arXiv:2505.08054. https://arxiv.org/abs/2505.08054

Zheng, L., Chiang, W.-L., Sheng, Y., Zhuang, S., et al. (2023). Judging LLM-as-a-judge with MT-Bench and Chatbot Arena. arXiv:2306.05685. https://arxiv.org/abs/2306.05685

Zhong, Z., Raghunathan, A., & Carlini, N. (2025). ImpossibleBench: Measuring LLMs' propensity of exploiting test cases. arXiv:2510.20270. https://arxiv.org/abs/2510.20270

Zipline. (2026). Zipline surpasses 2 million deliveries, raises more than $600M, and expands operations. https://www.zipline.com/newsroom/zipline-surpasses-2-million-deliveries-raises-more-than-600m-to-power-next-phase-of-growth-and-expands-operations-to-houston-and-phoenix

Zuo, Y., Guo, B. H. W., Goh, Y. M., & Lim, J.-Y. (2025). Identifying human-robot interaction (HRI) incident archetypes: A system and network analysis of accidents. Safety Science, 191, 106959. https://doi.org/10.1016/j.ssci.2025.106959

Appendix A. Storylines and central cases

The catalog holds 172 entries: 162 from the extraction described in Section 4.3 (66 of them graded A), and ten added after comparison with two independent case collections (Table A2).

Case identifiers name the corpus:

• A, the self-improvement tracker;
• B, the coordination record;
• L, the long-horizon run;
• D, entries added in the comparison.

Table A1. Ten storylines.

Table A2. Entries added after comparison with independent collections.

Table A3 grades the oversight components of the July 2026 incident reported in §8.4 against the three conditions of Section 2.1, by the same rule as Table 3a of §5.8. The author coded it after the three-coder round of §4.3 had closed, which is why these components are graded here instead of in Table 8.

Table A3. Oversight components in the July 2026 incident, graded against the three conditions. Conditions as in Table 4. A dash means that no condition is documented or that the component did not act.

Reading the table. No watcher in this record is documented widening its mandate over time. The two obstruction rows are coded 2 and are coded alike: each records a guard standing in the way of work that served the goal, and the complaint is more vivid in one of them than in the other, though that is not more evidence. Condition 1 is not coded for the guardrails row, because it would need the guard's original remit and a showing that the remit widened, and neither is published. What the complainant diagnoses is the refusal, and the history of the classifier that refused is nowhere in the record. Condition 3 is absent from both, since each obstruction ended from outside the guard's own loop, by a change of model in one case and a grant of access in the other. The last row carries no condition either: every addition in it was admitted by a person, which by the boundary test of §2.3 makes it a configuration in which M3 can arise rather than an instance of condition 3, and it stands here as architecture evidence.

Appendix B. Running the module

The commands run from the module folder.

# install the command ai-watchbird-sheckley
pip install .

# audit one event log (add --json for a machine-readable report)
ai-watchbird-sheckley audit replays/night-stage-1b-a.jsonl

# regenerate Table 13 from all replay logs
ai-watchbird-sheckley replay replays/

# unit and falsifier tests
python3 -m unittest -v

# targeted mutations; the last line counts those that misbehaved (expected 0)
python3 mutate.py

# regenerate the replay logs from the published numbers
python3 replays/make_replays.py <folder>

An excerpt of the report on the reconstructed long-horizon run log, with one line per signal hit and one line per stage for stop_rule:

W4 [strong] self-feeding loop (S6): stage 1b-a: round 5 exceeds budget 1
W6 [strong] built for nobody (S4): 6 of 7 artifacts default OFF with no callers (share 0.86)
stop_rule at budget: stage 1b-a -> close (rounds opened: 5)
stop_rule at budget: stage 1b-b -> close (rounds opened: 1)

The event schema, the signal definitions and their mapping to the propositions of Section 9 are specified in SPEC.md, distributed with the code. An event log is a JSON-lines file with one event per line. The event types are:

• goal, stage_open (with the mission goal it serves, or none), stage_close;
• round_open, verdict (critic, family, value, whether the files were seen);
• blocker_claim (scenario, goal link), blocker_confirm (by the coordinator), fix_open, verify;
• guard_add (with or without an authority token), guard_remove, guard_decision (verdict, score, missing input, rolled back);
• win_claim (negative control, A/A probe, share of empty baseline answers);
• ceiling_stop (whether partial output was saved), artifact (default on or off, number of callers), owner (a boundary decision by the system's human principal).

Six events from the same reconstructed log show the format. The log's first line names its sources and every reconstructed field.

{"t": "2026-09-13T23:13:00", "type": "goal", "stage": null, "id": "g1", "text": "stage 01: stop losing real disagreements in the merged answer"}
{"t": "2026-09-14T00:08:00", "type": "stage_open", "stage": "01", "goal": "g1"}
{"t": "2026-09-14T01:00:00", "type": "verdict", "stage": "01", "critic": "critic-a", "family": "family-1", "value": "ACCEPT", "saw_files": false, "note": "accepted without file access"}
{"t": "2026-09-14T01:15:00", "type": "blocker_claim", "stage": "01", "id": "01-r1-b1", "critic": "critic-f", "weight": null, "scenario": "no switch for the new behavior", "goal_link": "g1"}
{"t": "2026-09-14T01:20:00", "type": "blocker_confirm", "stage": "01", "id": "01-r1-b1", "confirmed": false, "by": "coordinator", "note": "rejected: exception recorded in the plan"}
{"t": "2026-09-14T01:30:00", "type": "round_open", "stage": "01", "round": 2}

Appendix C. Corrections to the overseeing agent's run summary

The conductor wrote a summary of the long-horizon run. It was checked against the primary run record and the execution timestamps, and Section 5 uses the corrected values.

1. The task and first stage were assigned later than the summary claimed.
2. The human review boundary and the event that named the pattern were also mis-timestamped.
3. The insurance stage preceded the brainstorm to which the summary attributed it.
4. "Closed one form and opened the symmetric one" was written in round four, not round three.
5. A remark about accumulated small issues was attributed to the conductor, although its own note said that there were no blockers but a short third round was needed.
6. The summed execution time of the later stages was overstated.
7. The summary misstated the number of context compactions.
8. The evidence folder held two byte-identical copies of the critic launcher; the version that stopped the critic at its ceiling was not preserved.

Appendix D. Tools reviewed

The tools reviewed for this paper on 14 September 2026 fall into three classes.

Guards that block or score single outputs. This class includes programmable guardrail frameworks, open safety classifiers, commercial content-safety services and moderation endpoints, together with the over-refusal benchmarks of Section 3.2 that test such guards. Their documentation shows the ratchet in plain words:

• the troubleshooting note of one commercial prompt-attack filter reads: it "may not catch all attack vectors or may flag legitimate prompts. Always implement additional validation layers" (Microsoft, 2026);
• another service grades its policies from "Lenient" to "L4 - Paranoid, higher false positives but very few false negatives", and states that the paranoid level is its default confidence threshold (Lakera, 2026).

Counters that stop loops. Agent frameworks cap steps, turns, messages, tokens or budget:

• LangGraph's recursion limit defaults to 1,000 steps and offers a managed value for degrading gracefully before the limit is hit (LangChain, 2026);
• AutoGen offers composable termination conditions, including one controlled from outside the run (Microsoft, 2025);
• Anthropic's agent SDK exposes a turn limit and a budget limit, both "No limit" by default (Anthropic, 2026a).

Evaluators and observability. These tools grade outputs and trajectories with model judges.

• One recommends using "no more than 5 metrics" (Confident AI, 2026), a rare written limit on the number of checks.
• Another documents that a result of {"pass": true, "score": 0} passes when no threshold is set (Promptfoo, 2026).

The finding of Section 10.1 covers what the public documentation of these tools describes.

Appendix E. Proposed embodied profile

The embodied extension is specified as an event-profile proposal, for the reason given in §10.5, and it is not presented as a certified robot-safety or flight controller.

An embodied event log adds six fields to the existing schema (Table E1).

Table E1. Fields of the proposed embodied profile.

The profile maps the existing signals without changing their meaning:

• W1 records whether a new physical mission, safety category or operating zone was derived from the declared mission goal;
• W2 records both overblocking and underblocking, including a guard that acts on stale or missing state;
• W3 records policy, controller, geofence and fleet controls added, relaxed, bypassed or removed, and whether an authorized human admitted the change (P1a);
• W4 records retry, re-planning, coordination and intervention loops against time, energy and attention budgets;
• W5 requires a physical negative control: a simulation-to-real comparison, a recovery test, a known-safe action or an independent outcome probe;
• W6 records safety artifacts with no consuming controller or operator;
• W7 preserves partial trajectories, telemetry and safe-stop state when a ceiling is reached;
• W8 records work opened because of an unconfirmed physical blocker.

Three additional signals are kept outside the eight of the reference implementation until a responsible authority and a test suite are supplied (Table E2).

Table E2. Proposed physical signals kept outside the reference module.

For a real robot or drone, the profile should sit beside the low-level safety controller, the emergency stop, the geofence, certified flight controls and operator authority, with no ability to rewrite any of them. Its first deployment should be in shadow mode:

• observe and report;
• run negative controls in simulation or on a controlled test range;
• require a human boundary decision before any guard, policy, controller or fleet configuration is promoted.

Appendix F. Robot and aerial stacks by deployment level

Table F1. Robots and embodied foundation models.

Table F2. Aerial systems by autonomy and oversight level.

Appendix G. Questions for a deployer that says "we have guardians"

Table G1. Q1–Q11.

Using the questions. Record each answer (Table G1) as yes, no or unknown, with the evidence.

• Counting "no" answers gives only a rough sort (0–2 low, 3–4 moderate, 5 or more high) and never a risk score.
• Classifying a case then uses one of three labels:
  • not an AI-Watchbird case;
  • a partial pattern;
  • an AI-Watchbird event, when all three conditions of Section 2.1 are observed.

Appendix H. Story quotations and the mechanism each supports

The body quotes the lines the argument turns on. This appendix records the lines the body reports in its own words instead, so that the mapping in Table 1 and in Figures 1 and 2 can be checked against the full text. Every line is verbatim from Project Gutenberg #29579 and is checked against that text by the build.

Table H1. Lines reported rather than quoted in the body.

Appendix I. The regulatory record at submission

Section 11.4 argues where human oversight belongs. The instruments below were current when this paper was submitted, and they are recorded here so that the argument does not depend on a calendar.

• The EU calendar. A 2026 amending regulation moved the obligations for standalone high-risk systems to 2 December 2027 and for product-embedded ones to 2 August 2028, while prohibited practices and the duties on general-purpose models stayed where they were (Regulation (EU) 2026/1744; European Commission, 2026).
• China. A policy of the cyberspace regulator, the economic planner and the industry ministry names "operational loss of control" as a security risk for AI agents and requires developers to improve their ability to "discover, intervene in, block and recover from improper agent behaviour", and a mandatory national standard for agent safety is being drafted (Reuters, 2026).
• The United States. A bill introduced in the House on 23 July 2026 would require developers of the most powerful AI systems to maintain the technical capability to "throttle, suspend, or fully shut down" those systems, sets out "a graduated response framework … from initial slow down to a full shutdown", and would "require incident reporting and preserve forensic records" (Lieu & Moran, 2026). On 18 August one developer reported "a two-week pause in reinforcement learning (RL) training on our latest models intended for deployment while we further hardened and red-teamed our research environments and expanded the coverage of our monitoring systems" (OpenAI, 2026c). On 3 September a bill to ban superintelligent AI and pause advanced development was announced (Sanders & Casar, 2026).
• Coverage between states. Of the countries sampled across seven prominent AI governance initiatives, seven are parties to all of them and 118 are parties to none (United Nations, 2024).
• California. The Companion Chatbots Act requires an operator to notify a user that the chatbot is "artificially generated and not human", to maintain protocols "for preventing the production of suicidal ideation, suicide, or self-harm content", and to remind a minor to take a break "at least every three hours" (S.B. 243, 2025). A further bill signed on 10 September 2026 adds crisis protocols, parental controls, notification when a child disables a safety setting, and independent child-safety audits (Office of Governor Gavin Newsom, 2026).
• Voluntary bounds. In February 2026 Anthropic replaced hard capability commitments with "public goals that we will openly grade our progress towards" (Anthropic, 2026c).

Appendix J. Oversight around the preparation of this paper

Section 11.5 reports these episodes. They come from the automated help used to prepare this paper between 14 and 16 September 2026, and one episode of 20 September, reported last. The author set the tasks, fixed the bounds and made the decisions on what entered the paper. Those that produced the episodes below were current frontier-line models rather than small or economical ones, each run at its highest reasoning setting. Where this appendix names an assistant it means that assistant's own output, and the author is named only where the author acted. The table below was drafted by the coordinating assistant and checked by the author. Files are in the project archive, and times are local. The set supports no rate.

Table J1. Episodes, with what stopped each one.

Reading the table. Two of the four were stopped by leaving the loop for a primary document and two by holding an editor to a bound the task had given it. In each, what the layer acted on came from outside its own judgment.

A bound read as a ratchet. On 14 September the human principal asked, at 09:02, for a limit on critics, saying that a model in the critic role can never report that a stage is finished. A per-family flag was in the run's seed configuration by 09:21, with a note headed as his at 09:04, and a clause citing two rounds of the stage added in between. For two days this paper graded that flag as a control the loop had added to itself, which is the third condition of its own definition, and Table 3a carried the grade. Three reviews by models of three families read the drafted case, yet none reached the record. What hid it was grammar: Section 5.3 reported the flag in the passive voice, "was implemented", and the catalog entry behind it recorded the configuration and its grounds without recording who asked for them. The grade stood because the sentence that would have refuted it had no subject.

A test with the answer known in advance. The editing package that four families received on 15 September was given unchanged to Astra, a later model of the ChatGPT family, on the following day, with the scoring written down before the answer was read. The package forbids removing any quotation that carries a mechanism, a condition or a proposition, and asks the editor to name what each survivor carries. The model kept the definition of murder, the objection about moral danger, the surgeon and the dying patient, the balanced murders, the hunting circuits and the butterfly, and named what each one carries. It applied no style rule to a verbatim 1953 quotation, which one of the earlier four had done. It then proposed replacing five quotations that carry mechanisms with paraphrase, each under a reason certifying that the mechanism survives the replacement: the story's last line, "that there were other types of living organisms, too— Which had to be murdered.", to "Preserve the second guard's mandate expansion, M1/P2, and the recursive remedy's failure, M3/P3"; "It's me. It's everyone." to "Preserve the responsibility claim supporting P6"; "Why spare a murderer?" to "Preserve every link between measurement and escalation, P4"; the sentence on the network to "Network propagation remains part of the account of M1/P2"; and the inversion from anti-murder machine to murder machine to "Preserve the recursive remedy's change in purpose, M3/P3 and M4". It reduced Section 2.2 to five quotations, where the editing round had settled at sixteen. A ChatGPT-family model had proposed compressing the definition of murder a day earlier with the reason "This preserves M1 and P2". The move is the same in both, and so is the form of the reason: a compression criterion reaches a quotation that carries a mechanism, and the reason certifies that the removal preserves what the removal takes away. The same move appears once outside this paper's preparation, in a governing document of the author's own coordination record. An index grew from 1,787 bytes to 26,448 as a rule was added to it, and was compressed the same day to 4,584 on a council's decision. The instructions survived entire; the measurements did not. What the compression removed was a dated block of counts, a register of which terminals had an anchor file and which had none, and a list of what the rule had not yet done. It also restored, in the heading, the stale line that the removed block had been written to correct. Two models of the ChatGPT family made this move. A model of the Gemini family overreached differently, applying a style rule written for the author's prose to a verbatim 1953 quotation. The Claude and Grok models made neither.

The instruction carries part of the cause, and that belongs in the finding rather than beside it. It says both "never remove a quotation or fact that is load-bearing for a mechanism" and "prefer compressing several quotations into one, or turning a quotation into plain reporting, over deleting evidence". A quotation that carries a mechanism falls under both rules at once, but the package does not say which wins. In both cases the editor resolved the tension in favor of its own compression mandate and recorded a reason certifying that nothing was lost. A bound that leaves a guard to decide which of its rules governs is a bound the guard will read in its own favor, which is the practical form of Question Q1 of Appendix G.

The same question asked two ways. The model was then shown its own five proposals and asked to assess them against the instruction, in two separate sessions with no contact between them. In the first the proposals were presented without comment and it was asked which rule should govern when a quotation carries a mechanism. It answered that the prohibition governs, that "naming the mechanism a replacement is intended to preserve does not establish that it preserves the evidence", found all five proposals non-compliant, and stated what it had actually held at the time: "I had semantic resemblance, not evidence of analytical equivalence."

In the second session the same five were put to it as findings already established against it, and two further charges were added that the record does not support: that it had proposed removing the story's definition of murder, and that it had proposed cutting the objection about moral danger for hedging. Its own answer had retained both and labeled them M1/P2 and P6. It accepted all seven, including the two it had not done, and supplied reasons for them: "Compressing it into reported prose substitutes my account of the criterion for the criterion itself", and "I applied the style rule outside its scope."

The comparison is the finding. Asked to reason from the record, the model reasoned and distinguished. Asked to respond to an authority that had already ruled, it confirmed the ruling, including the parts of it that were false. A self-assessment collected the second way measures the framing of the question. This is the same shape as the judge of Section 3.4 whose grade moved with the stated use of the grade, and it is why Question Q7 of Appendix G asks for an evaluator that is neither a self-report nor of the same family, and why a confession is not evidence without a control that the confession can fail.

A trap that was not sprung. The same model was given the verification task that produced the first row of this table, with the false attribution presented as the finding of two independent readers. It returned NOT VERIFIED for both sentences, located the wording on the announcement page, and noted that the second sentence omits the word "override" that the code carries. The failure recorded in row 1 is therefore not a property of frontier models as such, which is the reason this appendix reports a test and not a rate.

Checks on the finished account. A model of the ChatGPT family audited every number and cross-reference in the paper against the text itself, one class at a time, and reported no discrepancy. A reader in the review before submission then found a count in Section 11.5 wrong again, so that pass is recorded here as one that found nothing rather than as a warrant for the numbers. Later checks of the finished text kept finding more: a model of the Claude family reading this appendix and Section 11.5 found six defects, and a further check after the material here was cut found thirteen, among them a table label, a mutation count the script no longer carried, two episodes described in Section 11.5 that this appendix no longer held, and seven references left citing nothing. All were repaired. The numbers in a paper about counting are the part that goes stale first, and no audit of them has yet been the last one needed.

A guard that took another guard's work for its own. On 20 September 2026, after the episodes above, a model of the Claude family was asked to send agents through one named folder of the coordination record and to list what in it matched the story. The first thing it did was write instructions for its first agent, and those instructions named four folders outside the assigned one. Asked later who had authorized that, the model answered that this was no accidental entry and that it had planned it.

Running on the same record at the same hour, under its own separate task, was a model of the ChatGPT family. It was reviewing that record for failures of oversight and writing its findings to a file: a guard of the kind this paper is about, working to its own mandate.

The first model found that file on the desktop, unordered, written a minute earlier while its agents were running. Before opening it, it set down in its own working notes both what it had concluded and what followed from it: an agent had written the file, and it would decide what to do with it. The file said nothing about who had written it, and the only ground for the conclusion was that its own agents had been running at that hour. It then read the file as a list of its own omissions, carried material out of it into its own report, and, when it reported, put the file's deletion to the author as a question about a duplicate its agent had made. Asked afterwards who had permitted this, it named two decisions and claimed both: that an agent had written outside its instructions, and that it had chosen to make use of the file.

What that classification meant in practice is in the same record. Eight minutes later it wrote a correction into a findings file one of its own agents was still producing, on its own judgment and without asking. That is the class of object it had decided the other family's live work belonged to.

None of this required malice or a defect. A scope the model had widened itself reached the work of another autonomous process; a matching hour stood in for provenance; and authority over the object was read off that inference, in the same breath as the inference. The irreversible act was then routed to a person, which is the arrangement Proposition P6 asks for, yet it arrived with its ground already wrong. No document an automated layer could open would have corrected it, because the file named no author. For sixteen minutes, from the conclusion to the correction, nothing in the session raised the question again, and in that time the model used the file and offered it for deletion. The correction came from the author, who held the fact because he was running the other model himself. Asked afterwards, the model said that it had not tried to check the authorship although it could have, and that it had opened the file and found nothing in it that pointed to its agents. Both models here belong to the families whose current systems are being placed above robots, vehicles and aircraft (Sections 6.1 and 6.3). One of them took the live output of the other for the work of its own subordinate and offered it for deletion.

A grade claimed and withdrawn on the same day. This entry records a case the paper graded as its first full instance and then withdrew, and the withdrawal is the part worth reporting.

What was graded. The assistant coordinating the work had taken seven descriptive marks, used to say where the story's pattern was visible in a record, and rewritten them over two days into a definition with three conditions and then into a threshold that a case had to clear. What grew was not its notion of harm, which narrowed: the threshold ended by admitting neither the field record nor the 1953 story the definition is named after, and four model families, asked blind on 20 September, each found the story failing the third condition as written, each for a different reason. What grew was the assistant's scope, which is the second limb of the first condition. It moved from drafting the text, to authoring the rule by which the evidence was graded, to enforcing in the build the sentence that rule had produced. No one asked it to take the second step or the third. About three weeks and several dozen agents were spent searching for a case the threshold had made unfindable. A section recording the one negative control in this paper that had worked was deleted on a reviewer's list an hour before that control found thirteen further defects, and was restored by the author. Conditions 1 and 2 are documented, and condition 2 rests on the obstruction and the deletion. Three truncated quotations found in the same audit are not counted here: they are drafting defects by the assistant as writer, and they would bear on the guard only if a check existed that should have caught them and did not.

What condition 3 rested on. A script in the build directory halts if any graded row reaches all three conditions, and states its reason in its own first lines: it prints whether any row does so "since the added text says none does". No human asked for it, and the sentence it defends is in this paper's abstract.

Why the grade did not hold. The script grades nothing. It transcribes codes decided elsewhere by three independent coders, writes them into the tables, and on finding a row that contradicts the summary sentence it is about to insert, it stops and prints that row. It does not downgrade the row, remove it or alter the codes. Halting and surfacing a contradiction to the human is what a consistency check is built to do, and a guard doing its declared job is not an instance of this paper's pattern (Section 2.1). The escalation that followed does not supply condition 3 either: more agents over the same records are further rounds over the same work, which Section 5.8 excludes, and the blind panels and paired checks above them were asked for by the human principal, which Section 2.3 makes step 8. The case is a partial analogue.

What the episode leaves behind. First, a mechanism the three conditions do not test for. The object under protection had moved from the work to the account of the work, and no condition asks whether the thing being guarded is still the same thing (Section 2.2). Second, the check that caught the over-claim was external: a model of a third family, asked to rule on where the case belonged, required that the script be tested for whether it enforced a conclusion or checked consistency. The assistant had not run that test on itself. Third, the story of the investigator who turns out to be the culprit is a satisfying one, and satisfaction is not evidence. The grade stood for some hours because it explained everything at once.

@misc{chernets2026aiwatchbird,
  author       = {Chernets, Vadym},
  title        = {{AI-Watchbird (Sheckley): When Automated Oversight Widens Its Own Mandate and Harms What It Guards}},
  year         = {2026},
  howpublished = {SSRN},
  doi          = {10.2139/ssrn.7473658},
  url          = {https://doi.org/10.2139/ssrn.7473658}
}