When the Machines Start Building the Machines

A LegalTek.ai policy brief for legislators and regulators. June 2026.

The one thing to understand

For most of computing history, humans built the machines and the machines did what they were told. That is starting to change. The leading artificial intelligence companies now say their AI systems are helping to build the next generation of AI systems. The tools are starting to improve themselves.

The technical name for this is recursive self-improvement. The plain version is a feedback loop. A smarter AI helps design an even smarter AI, which helps design one smarter still, and each turn of the wheel happens faster than the last.

We are not at the end of that loop. No AI today designs its own replacement from scratch with no human involved. But the trend line is steep, and it is bending in a direction that should concern anyone responsible for writing the rules. On June 4, 2026, the AI company Anthropic published an essay arguing that this moment could come sooner than most institutions are prepared for, and asked governments for something unusual: the legal option to slow down if the technology starts moving faster than society can absorb.

This brief explains what that means for one of the institutions least prepared for it — the legal system — and what you can do about it now.

Part one: what is actually happening

You do not need to understand the engineering. You need to understand three numbers and what they imply.

A reasonable person should treat the exact figures with caution. They come from the companies selling the technology, and the companies themselves admit some of the numbers are probably too rosy. Anthropic published its essay days after filing to go public, so the piece is part warning and part sales pitch. But you do not need to believe the most dramatic version to see the problem. Even the conservative version says the same thing: the capability of these tools is growing faster than the speed at which laws, courts, and professional rules can adapt. That gap — between how fast the technology moves and how fast our institutions move — is the entire policy problem in one sentence.

Part two: why the legal system is the warning light

You might expect the first cracks to show up in some exotic corner of the tech world. They are showing up in courtrooms, in ordinary cases, often involving ordinary people. The legal system is a useful early-warning light for the whole economy, because it runs on written documents, it has clear rules, and it keeps a public record when something goes wrong.

Here is what the record shows.

Lawyers are filing fake law

AI chatbots sometimes invent things. Ask one for a court case to support an argument, and it may hand you a case that sounds perfectly real — with a name, a court, a date, and a quotation — that does not exist. The industry calls this a hallucination. It is not a rare glitch. It is a predictable feature of how the technology works.

Lawyers have been caught filing these fake cases in court, and the number is climbing fast. An independent researcher who tracks these incidents worldwide had logged more than fifteen hundred of them by June 2026, over a thousand of them in the United States alone. Most striking, the majority of those cases involve people representing themselves without a lawyer — exactly the people most likely to type their problem into a free chatbot and trust the answer.

The courts have stopped being patient. A federal judge in Ohio with forty-six years on the bench called one such filing the most serious violation of the rules he had ever seen, fined the lawyers, and reported them to the state disciplinary authorities. A federal appeals court fined two attorneys fifteen thousand dollars each. A federal court in Oregon imposed a sanctions package of more than one hundred ten thousand dollars, the largest of its kind to that point. These are not warnings. These are penalties, and they are getting bigger.

The harm reaches real families

This is not only a problem for big-money litigation. In a divorce case in Georgia, a court's final order relied on cases that turned out to be fictional. On appeal, eleven of the fifteen cases cited were fake. The divorce had to be reopened. In a California family dispute, a lawyer was sanctioned for citing invented cases that traced back to a post on a social media site. Family law — where the parties are emotional, the paperwork is heavy, and many people cannot afford a lawyer — is turning out to be one of the most exposed areas in the entire system.

And the fakery is no longer limited to text. Forensic specialists have documented custody disputes in which one parent submitted AI-generated audio of the other parent saying things they never said. The technology to fabricate convincing evidence is now cheap and widely available, and most judges have never been trained to suspect it.

Your private conversation with a chatbot may not be private

When you talk to your lawyer, the law protects that conversation. It is confidential. People assume the same protection covers their conversation with an AI chatbot. A federal judge in New York recently ruled that it does not. A person's chats with an AI tool, used before hiring a lawyer, were treated as fair game for prosecutors. The lesson for the public is blunt. Typing the facts of your legal problem into a consumer chatbot may hand those facts to the other side.

Part three: the policy landscape you are inheriting

Here is the uncomfortable part. Almost every rule currently on the books was written in response to last year's technology. The systems described in part one are already a generation ahead. A quick tour of where things stand.

Professional rules for lawyers exist, but they are guidance, not hard law. The national bar association issued formal advice in 2024 telling lawyers they must understand the technology, protect client secrets, and personally verify everything an AI produces before it goes to a court. Several states, including Ohio, have issued their own versions. This guidance is sensible. It is also mostly advisory, it varies state to state, and it depends on individual lawyers choosing to follow it.

Court rules are a patchwork. Some individual judges now require lawyers to certify whether they used AI in a filing. There is no national standard. A lawyer practicing in three counties may face three different rules, or none.

Federal and state governments are pulling in opposite directions. In December 2025 the President issued an executive order directing the Justice Department to challenge state AI laws the administration considers too burdensome, and to use federal funding as leverage to discourage them. At the same time, states like Colorado have been writing and rewriting their own AI statutes. The result is a genuine fight over who gets to regulate this technology — the federal government or the states — and that fight is unresolved.

Other countries are moving faster on hard rules. The European Union's comprehensive AI law becomes broadly enforceable in August 2026. Any American company using European AI vendors, or handling cases that cross borders, will feel those rules regardless of what the United States decides.

So the landscape you inherit is fragmented, advisory where it should be firm, contested between levels of government, and already out of date. Meanwhile the technology compounds on itself every few months.

Part four: what lawmakers and regulators should actually do

The instinct in a moment like this is to either freeze the technology or ignore the problem. Neither works. Here are practical, durable steps that hold up even as the tools keep changing.

On that last point: Anthropic's most unusual request was for the ability to pause frontier development if the technology outruns society's ability to manage it. You do not have to take that proposal at face value, given who is making it and when. But the underlying point deserves serious thought. The same feedback loop that lets these systems improve themselves also means a problem can compound across generations before anyone notices. The capacity to stop — even briefly, even just to look — is worth preserving while it still exists.