The shift from traditional software to AI is a shift from deterministic systems (where a specific input leads to a specific output) to indeterministic systems (where outputs are probabilistic and fluid). Almost every magical capability of AI is downstream of this indeterminism. But crucially, so are its most frustrating limitations.

If there is one fatal misunderstanding of AI today, it’s that we are engaging with this shift inadequately. “Indeterminism” has entered the lexicon, but usually only at a surface level. And because we are stuck on the surface, the loudest debates about AI have become incredibly boring.

Why the Extremes are Boring

Let’s look at the three loudest factions in the AI debate.

First, the AI doubters. They look at the unpredictable, indeterministic nature of large language models and declare it a failure. To them, a system that hallucinates cannot be trusted, and therefore cannot be useful. This is a boringly misguided example of confirmation bias. Humans are highly indeterministic—we forget things, we make math errors, we have bad days—yet we’ve muddled along reasonably well. How? By inventing deterministic tools to anchor us: long multiplication, checklists, standard operating procedures, etc. The doubter assumes you can’t extract value from an unpredictable system when you need reliability. History proves otherwise.

Second, the AI doomers. They also view indeterminism as a critical failure, but in the opposite direction. They are painfully aware of the immense power of AI systems and assume that this power is inherently uncontrollable. While this makes for a more gripping narrative than the doubters’ view, it strips away human agency. We’d have only one option left, don’t create powerful AI. Setting aside whether it is even possible to perpetually prevent its creation, this fatalism leaves no room for a practical conversation about how to retain control.

Finally, the radical accelerationists. They acknowledge the wild nature of AI but fall prey to a blind optimism, assuming a purely indeterministic system will somehow self-regulate and perfectly align with our needs. This is just as boring. The need for control is not irrational, nor is control a given. If control is achievable, it will demand a deliberate, concerted effort, requiring understanding every tool to engineer that control.

If you want to find interesting conversations, look for the solution seekers.

The Solution Seekers: Layers and Workflows

The most compelling builders today are those who reject both absolute pessimism and absolute optimism. They recognize that solutions aren’t singular or total. The most promising path is layers and workflows that mix and join determinism and indeterminism.

Think about how we manage high-stakes reasoning in the physical world—like in an intensive care unit or the cockpit of a commercial jet. We don’t rely entirely on the raw, in-the-moment reasoning of a doctor or pilot; human reasoning is brilliant but fluid, prone to fatigue, distraction, and variance. But we also don’t rely entirely on rigid, unyielding flowcharts, because a flowchart cannot reason through a novel, complex anomaly.

Instead, we design workflows that rely on both. We build strict, deterministic protocols—mandatory checklists, hard limits on medication dosages, automated collision warnings—to create a safe, predictable framework. Inside that framework, we rely on the judgement of a doctor or pilot to handle context, nuance, and problem-solving. Protocols enforce absolute boundaries; experts provide reasoning. Frameworks change, doctors update their own based on their learning, with debate and review, inside another layered framework.

This is the architecture of the AI future. AI will dominate the next generation of software, but it will not render deterministic code obsolete. Instead, code is how protocols are encoded. Those route, authorize, evaluate, and constrain indeterministic AI actors. Control points written in deterministic code will provide the necessary mechanisms to enforce rules, isolate agency, and supply safety. AI will be called upon within those specific boundaries to reason, interpret intent, and adapt to the messy reality of the user.

The Myth of the Developers Demise

This need for control has profound implications for how software is built. Recently, the term “vibe coding” has emerged to describe the practice of building software through natural language interactions with AI. A maximalist subgroup makes an extreme claim that with vibe coding, developers are obsolete and users will prompt their own custom software into existence on the fly.

This misses the fundamental purpose of a developer. A developer’s job is not to write code; a developer’s job is to remove effort for the user. Developing is ultimately not about producing code, but about producing reusable, accessible capabilities for users. An accessible capability is one that requires the least effort to access, and a reusable one is one that can be applied to multiple situations. Code is just the mechanism.

When developers create software, they establish guardrails, conventions, and reusable patterns. Sometimes, a user wants absolute flexibility, and a fluid AI companion is perfect. But often, a user wants rigid reliability. They want to press a button and know exactly what will happen. It’s easy to forget, amidst the explosion of AI capabilities, that rigidness has immense value.

It’s tempting to view recent advancements as a single evolutionary timeline—assuming we are moving from hand-written code, to AI-assisted code, to a future where code is entirely replaced by just in time reasoning of AI agents. That is a mistake, over-extending a trend. Committed code, generated, reviewed, tested and committed as stable will exist in abundance. Just in time generated code, executed in a protected sandbox will also be used abundantly.

The use of models and instructions, reasoned upon just in time, shifts the balance point between flexibility and rigidity, but it won’t abandon code nor the developer.

A Shared Experience: Taming the Machine

For users, future software interfaces will be a mix of structured and natural. Learning to navigate the difference between them will be a vital modern skill.

Structured interfaces (buttons, menus, traditional apps) sit atop deterministic systems. You can trust them to follow a plan. However, that plan was written by a developer. If the developer didn’t anticipate your specific need, the software becomes frustrating. You are forced to learn its non-intuitive logic.

Natural interfaces (chatbots, voice agents) sit on top of indeterministic systems. They can do things developers never anticipated and can interpret your unique intent. But they make assumptions. Using an AI interface is like ordering from a waiter at a restaurant. You need to develop an instinct for how your communication might be misinterpreted. You need to know when the system will ask a clarifying follow-up question (”soup or salad?”), and when you need to be proactively rigid and structured in your commands (”hold the mustard”). Make a mistake here, and you end up with a mustard-covered sandwich. Everyone then has to start over from scratch, and someone has to pay for the waste.

Interestingly, the people building the software are going through the exact same transition.

Developers are increasingly using natural language to write code. For a brief moment, this felt like magic without rules—just type what you want, and the machine builds it. But developers are quickly realizing that an AI coding assistant is just as indeterministic as a chatbot. If they aren’t careful, they end up with the equivalent of a “mustard-covered sandwich” deep in their codebase.

Because of this, we are watching a new kind of structure reemerge in software development. Developers aren’t abandoning natural language, but they are scaffolding it. They are learning when to let the AI riff creatively, and when to enforce strict, deterministic tests to verify the AI’s output. The developer’s job is evolving from writing rigid rules by hand to managing the chaotic intelligence that writes them, locking its best outputs into place so they can be relied upon tomorrow.

Conclusion

For decades, our relationship with computers was fundamentally one-sided: humans had to learn to speak like machines. We memorized menus, learned strict syntax, and clicked exact sequences of buttons. We were forced to be rigid operators of deterministic systems.

AI flips this dynamic, but it introduces a new burden. The era of the comprehensive user manual is over, because you cannot write a complete manual for a probabilistic system. Its capabilities are discovered through interaction, not documented in a spec sheet.

This is why understanding the architecture beneath your feet is no longer just a concern for software engineers. It is a vital literacy for everyone.

If you are an everyday user, recognizing whether you are interacting with a deterministic system or an AI agent changes how you engage. The caution you apply to inputs and outputs should shift. For deterministic systems you should provide what is required and just what is required. For AI systems consider where elaboration yields better results, and vagueness leads to guesswork. Unless you need guesswork, avoid triggering that path.

If you are trying to predict where the industry is going, looking for these architectural layers is the only way to cut through the boring extremes of blind hype and cynical doom.

And if you are a builder—whether you are writing thousands of lines of code or just stringing together a few tools to solve a daily problem—understanding this duality is your ultimate advantage. The future of technology isn’t about choosing between the rigid reliability of the past and the creative chaos of the future. It’s about learning to bolt them together.

One thing that is still being missed, is why the long term favors the defenders. One of the reasons is that fewer defects is good for defenders in a generally absolute way. But another one relates to costs and benefits. Read that economist article linked and the final statements, suggesting that defenders will have to pay highly to discover defects.

Now reflect, that this isn’t new, it has always been expensive to discover defects. The risk that products like Mythos bring is that they lower the cost of discovering defects to exploit. The solution is to raise that cost. That might tempt you to suggest you should rewind the clock, and never invent Mythos. That’s not a solution though as eventually attackers would invent something similar, and you would then lose any control and advantage from the defenders being the first with access.

Instead the solution is that you find as many easy defects as you can and fix them. The first 100 defects might cost $20,000 / defect to discover. The next 100 might be $40,000 per, etc. Along the way you end up with defensive layers that are more and more reinforcing, and the cost for attackers to discover defects goes up, especially if they have less sophisticated tools, and/or have to spend a lot to first illicitly gain access to tools. When Mythos is publicly released you can generally assume providers will increase their attempts to find and ban users with ill intent. Those protections create costs for attackers, such that if a defender can find a defect for $20,000, an attacker might need $100,000. The attacker’s main advantage is they just need one, but as unpatched defects become more rare and harder to find that advantage tends to shift toward favoring the larger aggregate budgets of defenders.

The defenders have a strong advantage in terms of money. Where they struggle is in organization, because they have a much harder organizational problem to solve. The hard part about being a defender is getting changes deployed everywhere quickly. Once attackers find a defect, they can try and use it everywhere. If they find it first, that works in a lot of places. If they find it second, it’s dependent on how organized the deployment process is.

And this is why the long term economics favor the defender. Statistically, most defects are found first by defenders, due to larger budgets. As the period between discoveries gets longer, the chances that attackers have really good targets declines. That lowers their cost/benefit, which probably also lowers their actual budget. Criminals invest in things that make (them) money, not ones that lose it.

In Part 1 of this series, we explored how AI is fundamentally altering software control through the lenses of determinism and scope. We traced the journey from passive, strictly bounded chatbots to the threshold of active agents—AI systems capable of autonomous, multi-step planning. But what happens when these indeterminate systems are given broader scope and powerful tools? The consequences ripple outward, reshaping not just the security of our infrastructure, but the shape of our workflows and emotional relationship to work. To understand the recursive systems of tomorrow, we must dive into the agent ecosystem itself.

The Agent Ecosystem

Agents represent a significant shift in control, trading linear human prompting for continuous indeterministic planning.

To understand how these agents operate, we must briefly consider tools. Agents use tools to accomplish their plans. Tools can be anything, and which tools an agent is provided with define its constraints. You can provide an agent instructions, cautions, and directives through its prompt and context, but like anything in an agent, it’s indeterminate.

A tool might be as basic and low-risk as retrieving a specific account balance, where the boundaries are tight and predictable. It might be as broad as searching gigantic repositories like the entire internet or an organization’s internal files, which escalates risk by exposing the agent to untrusted data or sensitive information.

A broader path still is the ability to create and execute computer code, introducing severe risk if left unchecked. That might initially seem like it loses all constraints, allowing the agent to perform unanticipated or dangerous actions. However, code can be executed in a sandbox that limits how it communicates and what data it can access. Assuming the sandbox is secure—which requires careful planning, inspection, and testing—restricting communication to untrusted sites prevents data exfiltration or external control. Just as critical is controlling the credentials provided to the sandbox. Strictly limiting credentials restricts the agent’s ability to update records or access systems outside the purview of its current authorized activity. Together, these boundaries provide the necessary mechanism to constrain this high-risk capability.

Tool use isn’t restricted to retrieving information, either; it can allow changing information, which can trigger further actions. This is an area that requires much more caution, doubly so for writes and actions that are irreversible. Beyond that obvious observation, two other dimensions enter in. First, since an agent’s plan is indeterminate, the ability for a designer to remove the risk that it performs actions in unanticipated ways is vastly more complex than when working with a deterministic plan. Second, we must account for prompt-injection—the risk that something an agent has read can influence its choices, resulting in actions desired by an attacker rather than the user or designer. There are protections against this type of attack, but it would be foolish to consider them foolproof.

With that foundational understanding of how agents act on the world, we can observe this frontier opening up across escalating levels of scope:

Standalone AI Agents

Unlike a chatbot that waits for a prompt, a standalone agent is given a high-level objective, allowed to indeterministically generate its own step-by-step plan, and execute it using available tools (like searching the web or scraping data). While the planning is continuous and autonomous, the agent still typically operates within a relatively bounded scope, restricted by specific APIs to prevent runaway consequences.

Like chatbots, there are standalone agents from OpenAI, Claude, Google, and others. In fact, most chatbots have silently become agents1, though still with many constraints.

In time, these standalone agents may have more and more autonomy. But from the perspective of this framework, the fundamental aspect of taking user input, deriving a plan through an indeterministic process, and executing that plan won’t re-enter the realm of determinism until it invokes a tool2.

Agents Embedded in Applications

Moving beyond the simple “embedded AI node” discussed in Part 1 involves agents operating continuously alongside users within a shared software environment. Consider a complex data analysis platform: the human user might explicitly invoke deterministic tools to filter data, while an embedded agent operates in the background, autonomously invoking its own set of analytical tools to highlight anomalies. The application becomes a hybrid ecosystem where human indeterminism and agent indeterminism collaborate in real-time bounded by the application’s guardrails.

Agents embedded in applications have an advantage over agents called by other agents: the input data is controlled by the calling application. Still, remember that the applications agents are embedded in may themselves be working with dynamic data. A data analysis platform has many data sources; are they all vetted and invulnerable to an injection attack?

Another common example today is agents embedded into development workflows. They can reason about code, look for security issues or defects, generate fixes, and submit them as pull-requests for developers to review, effectively combining the code-generation function with the embedding function.

Agents Using Agents

An agent can become a tool used by other agents. To think about why this is valuable, you want to first understand that agents generally have a few components. At their core, they create plans via the GenAI model. They generally have some sort of instruction (or persona) file. They’ll also have access rights or boundaries associated with a tool list.

The instruction file is the interesting part here, as it provides the reason for a distinct agent. In the simplest version, it might describe a persona (“you are an insurance adjuster”), but this relies on the GenAI model to blindly decide how an adjuster behaves. If you’re building an agent, you want more control. A detailed instruction file can’t shift you to a fully deterministic world (if you want that, you should write code), but it can reduce variability.

Another aspect that can remove ambiguity is placing restrictions on inputs and input sources. If an agent expects to receive user input, it has to be fully prepared for anything. If it’s called from an application, those expectations are more constrained.

Agent input expectations fall into two categories: expectations formed around successfully fulfilling its goal under valid usage, and expectations formed around avoiding taking action on behalf of an attack. These two have some non-overlapping aspects. If an input is suspected of being for the purpose of an attack, there’s no need to try to do anything other than quit and refuse action. But the consequences of allowing an attack are generally far more severe. On the other hand, failing to successfully complete an action is less severe, but it’s less acceptable to give up because of uncertainty.

If we had wanted certainty, and accepted inaction for uncertainty, we should have written a traditional application, not an agent. In many ways, the creation of agents with sophisticated instruction files is a type of meta-programming that never coalesces into a deterministic form. While we could use vibe-coding to generate an application, creating an instruction file for an agent has a similar outcome, except designers never get the chance to validate the plan for each agent execution. We might restore some of that validation through a human-in-the-loop workflow, but the agent designer won’t be in the loop unless they are also the user.

Furthermore, when input comes from another agent, the expectations on input are not very clear. It’s not as unclear as coming from an untrusted user, but since the user of the calling agent might be less than fully trusted, we have to consider the possibility that a malicious user could cause the calling agent to pass dangerous inputs to the called agent. Depending on the design, that might be difficult, but proving it’s impossible is a high bar without some deterministic system in the path.

Agents Building Applications

At the apex of the framework we have agents building applications. Instead of a human using an AI tool to write code, an autonomous agent—or a multi-agent framework—is given the broad scope to architect, write, test, and deploy entire applications. Operating within bleeding-edge, emerging ecosystems like Steve Yegge’s concept of Gas Town, an overarching agent might autonomously spawn specialized “code worker” sub-agents to solve specific architectural problems. This introduces the reality of deep recursion: AI systems dynamically writing, testing, and deploying new AI systems at machine speed.

Agents Building Agents

An alternate apex is agents building other agents. While both scenarios rely on deep recursion where at least one level is indeterminate, the agents-building-agents path stores its recursive plans in natural language, rather than a programming language. Tools like Claude Cowork and Claude Code are bordering on this. Technically, they’ve always been capable of it, as a developer can create recursion somewhat trivially.

The barrier here has generally been security. It’s rather easy to say, “Agent A calls Agent B to create Agent C, which can call Agent B” (look, I just did it!). The hard part is whether that’s a good idea. Projects like OpenClaw push this further. When agents build other agent skills or update through tools like Moltbook, they are acting at this highly complex, deeply recursive layer. OpenClaw has some security controls, but not enough to prevent many users from making significant mistakes.

Another example here that illustrates the movement from applications to agents, is Gas Town in Gas City. Gas Town, the original multi-agent orchestration system for Claude Code, GitHub Copilot, and other AI agents, was an application. When Yegge wrote Gas City, a “orchestration-builder SDK for multi-agent systems”, Gas Town became “code free”, and instead a bundle of prompts and skills.

Ripple Effects: Cybersecurity

One constant that comes from more complex systems is greater challenges at securing them. All else being equal, indeterministic systems, either fully or wholly, are more complicated than fully deterministic ones. Security is traditionally about protecting deterministic plans from indeterminate actors (human hackers). Each layer adds complexity as well. Attackers will have new tools and will use them against the highest value targets that have weak points.

Fortunately, all this does not come without some benefits, both inside and outside the realm of security. Inside the realm of security, access to dynamic systems like agents allow for faster responses. As I’ve explored previously, this means security can’t wait—it will force not just an overdue commitment to defense, but complete organizational changes.

The highest value targets know this. They will rapidly adopt new defensive techniques, patching weak points faster than ever before. Where we will see more successful attacks is against more moderate value targets. Some operate efficiently and will adapt, but others will face an “adapt or fail” pressure cooker. They will suddenly find themselves defending against highly sophisticated, indeterminate automated attacks.

Overall, however, this is a narrative of optimism. As Dario Amodei notes, “the offense-defense balance may be more tractable in cyber, where there is at least some hope that defense could keep up with (and even ideally outpace) AI attack if we invest in it properly.” Security must simply shift toward robust bounding and sandboxing of environments, rather than assuming the predictability of the software operating within them.

Automation and Workflow Change

This shift in control—from human-driven applications to autonomous, recursive agents—isn’t happening just for the sake of technological novelty. Ultimately, the goal is, and has always been, automation. AI changes opportunities for automation by lowering automation costs that were previously prohibitive.

Classification and ML

Machine Learning (ML) is an AI technique that achieved broad use earlier than Generative AI. Classification and prediction tasks were the core use cases. It’s generally less well known than Generative AI because those use cases fit into embedded AI workflows that have less direct user interaction. But that doesn’t mean they haven’t been effective. Generative AI has some overlap, but it’s useful to note ML is not obsolete—it will continue to dominate specific classification tasks where the trade-offs favor highly optimized, low-compute execution.

But GenAI is shifting the math for automation’s long tail where engineering effort is the limiting factor. Traditional ML models require a significant engineering investment to train. While that engineering could theoretically be automated, doing so would bring you back to using GenAI to generate the code. When a general-purpose GenAI model can perform a task at equal quality without that upfront engineering time, it opens up a new option for countless use cases that were never practical to tackle with traditional ML. AI provides the structure to finally capture and automate the tacit knowledge we previously had to rely on humans to execute.

Workflow Change

This, and the other uses of Generative AI, allows a deeper decomposition of workflows. In prior methodologies, it was too expensive to capture the output of specific, granular steps. Those steps were done “in the head” of human workers, existing only as “tacit knowledge.” A workflow that may have produced better results might have been avoided because the human cost of data preparation or classification was too high.

While it’s possible to replicate the same workflows, trade-offs have shifted. Consider a nurse who has learned a new symptom for a patient. That nurse may lack the depth of medical knowledge or the patient’s full history, so may not be able to do more than record that information until the patient’s doctor can review it. But an AI system can reanalyze a patient’s information nearly instantly. It can recategorize data, make new recommendations, or provide the nurse with the relevant medical information and patient history. This could allow next steps that both improve efficiency, but also improve outcomes. Maybe it’s an extra test, or an extra question, or a life-saving reaction.

Because AI lowers the cost of executing small, indeterminate tasks, we can now decompose workflows further. What workflow is optimal depends heavily on hand-off costs. In the example of the nurse, the hand-off costs to the doctor were the impediment. Human to human or human to machine hand-offs are expensive compared to machine-to-machine. When tasks shift from human dependent to machine dependent, a reorganization of workflow makes sense. A particular flow which was used to avoid handoffs may no longer be necessary. More importantly, those handoffs that remain, will have higher relevance than before, and optimizing for them, rather than those that are no longer needed, takes priority.

Initially, we should expect to see pilots, trials, and first iterations operate within existing workflows. Changing workflows requires planning, training, and is hard to reverse or do incrementally. As such, it follows in later iterations. But many of the largest gains are realized with those iterations.

The Human Side

Workflow change can also have a significant impact on satisfaction amongst workers. Hand-offs can be the most frustrating type of work, depending on your personality type. Human to machine hand-offs become frustrating when flexibility is lacking, and you feel like your task is to fit a round-peg into a square hole. Human to human hand-offs can sometimes be enriched by the personal interaction, but they also expose you to misaligned goals, competing priorities and personality conflicts. Personal interactions are a lot more reliably fun when you get to pick the individuals and circumstances.

Worst of all, machine to human hand-offs can create the impression that you’re serving the machine, not the inverse. All hand-offs can have that effect somewhat, but it’s especially hard if there’s an endless list of machine generated work. It helps to detach from the “end” and focus on the progress here, but when an organization turns that into a metric, ruthlessly gamifies it, and fails to consider the impacts, it requires extreme stoicism to avoid burnout.

It’s important to remember the human side with workflow change. Ruthless metrics fail in the long term. Leaders should watch for that, and avoid allowing short-term goals to overwhelm long-term health. It’s not always clear that this fulfills the “bottom-line” if that’s financial performance. It’s always clear it’s better for broader goals than financial, but even for financial goals, the benefits are likely there, though harder to see.

Conclusion

The era of software as purely rigid, deterministic planning is ending. In its place is the rapidly expanding Agent Ecosystem. By integrating indeterministic models into our autonomous systems, and allowing agents to build other agents, we are trading perfect predictability for unprecedented scale and capability.

As the scope of these systems increases, our primary job shifts from writing static instructions to managing boundaries. We must build robust technical sandboxes to protect our cybersecurity, and we must build equally robust organizational boundaries to protect human workers from the burnout of endless machine-to-human hand-offs. We have to design systems that serve humans, not the other way around.

The question is no longer just “What can the chatbot say?” The real questions are: How much scope are we willing to give to indeterminate plans? How will we effectively bound the recursive systems of tomorrow? And, ultimately, how do we bound ourselves?

To the earlier question, you wouldn’t be alone if your first answer was a user-facing chat application—ChatGPT, Gemini, or Claude. Hundreds of millions, maybe billions of users have tried one of these1. It’s a great starting point to understand the current state of AI. You can probe with questions, get answers, and evaluate the output. 30 minutes of that will demonstrate more than any other 30-minute investment. That said, forming your entire impression of AI based solely on chatbots misses deeper shifts.

A second common answer is a robot from a science fiction movie—a human-like, but fundamentally alien, being. While creative, this vision represents what AI might be, not what it is today. Thinking about Sci-Fi AI often does more to help us understand human nature than actual machine learning. Take any part of it too literally, and it will make you less informed.

Understanding AI’s true impact requires closing the gap between simple chatbots and science fiction robots. We can do that by looking at AI through the lens of software engineering and control. Specifically, we can treat the control mechanism of software as a system of planning, governed by two critical axes: Determinism and Scope.

The Mechanics of Control: Determinism and Scope

Historically, software has been built on predictability. When a human developer writes traditional code, they are creating a deterministic plan. Once it passes through quality checks, validations, and testing, it becomes a solid, rigid set of instructions. While bugs exist, the system is fundamentally designed to execute the same way every time.

Humans, on the other hand, are inherently indeterministic; you never know exactly how a user will approach a problem, what strategies they will employ, or how they might adapt their plans on the fly. Generative AI models2—the underlying engines powering the familiar chat tools like ChatGPT and Gemini mentioned earlier—share this indeterminism. When a GenAI model creates a plan or a response, it is probabilistic. It will not necessarily produce the same output twice. That said, if there is a single correct answer that has been heavily reinforced during its training, the model will predominantly provide that specific answer. This consistency occurs because the statistical weight leans overwhelmingly in that direction, not because the system’s inherent flexibility has been mechanically removed.

The second axis of control is the execution environment, or its scope. An environment can be strictly bounded, meaning the actor (human or AI) has very limited tools and access. A user in a simple data-entry application cannot do much other than enter data. Conversely, an environment can have broad scope, featuring wide-ranging access to tools with compounding effects, such as command-line execution, file system access, or the ability to write and deploy new code outside of a sandbox.

By analyzing AI through these axes—how predictable a plan is, and how bounded its environment is—a clear narrative of evolution emerges.

What is Indeterminism?

What you identify as the “AI” in these applications is called a foundation model. This model generates responses to your inputs based on its “training.” Training is a process where the model is incrementally updated to make its responses match an ideal. You can think of the first training pass as an attempt to create a model that could rewrite the entire internet with as little inconsistency as possible. Initially, the model might predict a word because it looks like a piece of content it just saw. But as it processes more information, it encounters conflicts. By forcing the model to resolve millions of overlapping conflicts, it learns the underlying rules of how concepts connect. Later, a second training pass is applied to be much more specific about what is considered a “good” or “bad” way to respond to a human user.

Output from AI models is indeterminate. There are two factors that cause this. The first, unconquerable aspect is that the relationship between input and output is too complex to reason through. You actually can get a model to produce the exact same output for the exact same input if you adjust its “temperature” to zero. However, this doesn’t mean the model is truly predictable, because even minute changes in the input can put the model on completely different paths, even at zero temperature. The second factor is that when you see a model used in practice, the temperature is generally set greater than zero. Zero temperature tends to be boring, less creative, and less insightful without necessarily being more accurate—it just enforces a stronger consistency between input and output. But since the complexity of that relationship makes strict prediction impossible anyway, the value of zero temperature is limited, and the output remains, in all practical senses, indeterminate.

But indeterminate isn’t the same as random; it has a direction. With evaluation you can find a probability, and those probabilities can be high. But indeterminate also entails the chance for novelty, including surprise. To some degree, we might say indeterminism reflects a limitation of the user or designer’s ability to predict outcomes. But it’s not a limitation reflective of a lazy user or designer; it reflects a level of complexity no amount of attention can fully address. You can fight it a bit, gain some control and understanding, but if your expectation is full control, you’re using the wrong toolbox, wasting your time, and will ultimately fail.

Indeterminism is something you tame, not control. A tame agent is something that works with you. Tame things have many benefits, but also bear caution. A tame horse has advantages a car does not. If you fall asleep on a horse (and don’t fall off), the horse is very unlikely to jump off a cliff. A traditional, non-autonomous car doesn’t behave that way—it’s very deterministic at driving straight, whether that straight path leads down the road, into a wall, or over a cliff.

But a tame horse can still kick you. It’s far less likely than a wild horse, but if you approach it wrong or scare it, there’s no horse so well-trained that a kick becomes impossible. That’s part of the tradeoff of working with something indeterminate. A car with its engine off is going to behave like any other 2,000+ lb. hunk of metal on wheels, governed entirely by physics. Even in motion, while there are a few exceptions like engine or brake failures, it’s all just physics in the end.

Most software applications are designed to be determinate. A developer reasoned out what output a particular input should create, and planned this carefully. The plan of these applications is encoded into a language, and translated into machine code.

Understanding this shift to a probabilistic nature is crucial, because many choices we make will be founded on the seeking of a balance between dynamism and trust that intersects with that fundamental property of models.

The Thin Layer: AI as an Application

Most users have started to understand what GenAI is, and what its capabilities are, by using it as an application. What’s interesting about this is that these first applications started as very thin layers over the core internal generative AI model, so users have experienced the technology at nearly its most basic. It’s been a while since a novel computing technique has been exposed with so few extra layers.

In a formal sense, “AI as an application” means the primary interface is directly to a GenAI model. There are a few wrapper elements—identifying who you are, moving data back and forth, and providing some presentation of returned data—but mostly, it’s a wrapper. You send inputs, it sends outputs, and you directly converse back and forth.

In our framework, this is an indeterminate system operating within a strictly bounded scope. The user and the model are primarily in control. The text, images, or files you provide get fed to the AI, and its probabilistic responses are safely constrained by the application’s sandbox. Safeguards exist that neither can override, but within those boundaries, the direction of the interaction is controlled linearly by the human and the AI model.

Embedded Intelligence: AI within Applications

As software evolves, we are seeing a shift toward embedding AI directly into applications. While technically a chatbot is an example of this, it is highly useful to differentiate the two.

When a developer embeds GenAI into an existing application—for example, a GenAI-powered A/B testing engine that automatically generates and tests multiple variations of marketing copy to identify the best performer—the control dynamics shift. The overarching application remains a rigid, deterministic plan, but the AI represents a small, contained pocket of indeterminism.

With AI embedded in an application, the application is primarily still in control. When it uses an AI for a specific function, it cedes a small amount of control, but it has strict boundaries. The traditional software dictates exactly when the AI is called and where its output goes, strictly bounding its influence to specific micro-outcomes. There is immense potential left in this domain that the general public doesn’t recognize, simply because its precise use is entirely dependent on the creativity of application developers.

Prompt Engineering

An interesting detail about embedding is that prompt engineering becomes a critical function. Embedded GenAI needs a goal. With chatbots, the user can provide the goal. With embedded GenAI, if the input is from the user, it’s through things like filling forms or uploading documents. The embedded function should reliably reach a result that allows the application workflow to proceed.

Prompt engineering is the process of creating inputs to a GenAI model that perform better at achieving a goal than other inputs. Some parts of this are intuitive to anyone fluent in a language, like if you were instructing an actual assistant. Some parts are more particular to GenAI models, or even particular to specific GenAI models.

Technically, you can use prompt engineering when you use a chatbot, and you’ll get better results if you do. But there’s also an overhead in doing so, as you’re no longer expressing your simple intent, but working to make it fit a pattern. Model builders work to try and make prompt engineering less necessary for general user interactions, so some tricks are less important than they were in 2023. The most obvious parts will probably always be useful, like avoiding ambiguity when you have a clear intent in mind.

For embedding, the overhead of prompt engineering has a higher payback, so it makes sense to engage with it more deeply, and so developers do. Prompt-engineering is also used when creating custom chatbots3.

Another goal of prompt engineering is to constrain the output. Stronger, more consistent instructions produce more consistent outputs.

Embedding and Value

You might have noted that the A/B testing example earlier was a marketing example, and thus falls into the adversarial category of use I’ve talked about before. Marketing is often an early adopter of these embedded systems because the industry is driven by adversarial motives—constantly competing against others for user attention and clicks. But the true potential is much more inspiring. Consider an adaptive educational platform. The overarching application rigidly tracks a student’s progress, curriculum, and test scores (the deterministic plan). However, when the system detects a student struggling with a specific concept, it calls upon an embedded GenAI model to instantly generate a custom, interactive story explaining that exact concept using the student’s favorite hobbies as an analogy. The application remains fully in control, but it uses the AI’s indeterminate flexibility to provide a deeply personalized learning experience that hardcoded software never could.

Building Systems with a Life of Their Own

The control dynamic fundamentally fractures with the next pattern: using AI as a tool to build applications. So far, experiences with this have revolved around coding assistants and “vibe coding.” In some cases, it’s immediately clear this is different from the chatbot model because the AI is embedded within complex Integrated Development Environments (IDEs).

But what truly distinguishes this pattern isn’t the interface. Rather, it’s the output and how it is used. Business-focused tools like Claude Cowork or Amazon Quick are increasingly managing different inputs and outputs to help end-users pursue task-oriented goals, generating artifacts like documents, summaries, and presentations. But if that output is ephemeral—a static artifact used only to accomplish a quick, singular objective—it’s not building an application.

Building applications means building something that has a life of its own. It is the indeterministic generation of deterministic plans. The AI indeterministically generates a code script, which is then refined through human review, automated reasoning, and testing. Once committed, it becomes a static, deterministic plan.

The topic of control highlights the profound nature of this shift. When an application is built this way, there are two distinct phases of control. In the first phase, the developer is in control, sharing control with the generative AI model by delegating, authorizing, and reviewing. But the developer does not retain persistent control during the second phase. Once deployed, the built application itself takes control. If it runs on a server, the developer can turn it off or replace it, but that is supervisory control at best.

By building something with a life of its own, we take a foundational recursive step in software: using indeterminate AI to architect and generate the very deterministic logic that will govern computing moving forward.

From Chatbot to Agent

The user-facing AI chat application you engage with has already evolved to be more than a simple interface. Designers try to make this seamless for you, so you shouldn’t feel bad if you missed the change, but to understand the systems of tomorrow, we need to distinguish between a passive chatbot and an active agent.

When you interact with a standard chatbot, the control dynamic is strictly conversational and reactive. You provide a prompt, the underlying generative AI model probabilistically calculates a response, and then it stops. It relies entirely on you to drive the interaction forward step-by-step.

An agent, on the other hand, is an AI system designed to pursue a broader goal autonomously. Instead of just answering a single prompt, an agent takes an objective, indeterministically breaks it down into a multi-step plan, and uses available tools to execute that plan. It can observe the results of its own actions, correct its course, and continue working until the goal is met.

Understanding this shift from passive generation to active execution is crucial. By combining the probabilistic reasoning of foundation models with the ability to take independent action, we are actively moving away from simple chatbots and into the era of agents.

We have traced the evolution of AI from simple chat interfaces to embedded intelligence, and finally to the threshold of these autonomous agents. But recognizing this shift is only the first step. To truly understand where software is heading, we must examine the wild frontier of the agent ecosystem itself—how these agents use tools, how they interact with each other, and the cybersecurity implications of granting them broad scope. In Part 2 of this essay, we will dive into this ecosystem, exploring the deep recursion of agents building agents, and what this shift means for the ultimate goal of automation.

It is common to hear people argue that “technology creates new jobs,” usually pointing out that despite centuries of technological advancement, nearly everyone is employed today. Therefore, they argue, the fear that technology destroys jobs must be wrong. While it is true that technologies can both create and eliminate specific roles, framing the debate entirely around the technology misses the underlying engine. The real topic is the economy itself, which naturally seeks to create new jobs from available resources—the most limited of which is labor. Whether ATMs create more or less demand for bank tellers is simply not as important as we think.

Debates about automation often get stuck here. One side correctly argues that economies have historically created new jobs, but incorrectly attempts to prove this by claiming technologies always create new jobs. These are two different arguments. This minor framing issue does a lot of heavy lifting in keeping both sides from understanding each other. Once you look past the technology and focus on the economic engine, you can discuss the future of jobs in far more effective ways than debating the fate of bank tellers.

Why Does an Economy Create New Jobs?

It is easy to misunderstand how economies work if you view them through a lens of strict limits rather than dynamic balance. A certain mindset approaches every economic issue as a zero-sum game of apportionment—assuming there is a fixed number of jobs in the world, and introducing a new technology either adds to or subtracts from that finite pool.

It is not hard to see why this mindset takes hold; in a moment-to-moment sense, it appears true. At any given second, there are a fixed number of jobs. Eliminate 500,000 of them instantly, and you have 500,000 unemployed people. But economies are not static moments; they are moving systems that perpetually seek equilibrium. While external limits exist, the internal machinery of an economy is entirely dedicated to finding a balance between those limits and the limitless preferences and desires of the people within it.

That is why, in their default state, economies always create new jobs out of available labor. If there is an unmet desire among the population, and there is labor available to fulfill it, the economy will generate an opportunity to put that labor to work. This balancing act isn’t instantaneous. There is a “seeking” process to find a new equilibrium. Sometimes this process stalls, the economy malfunctions, and we experience high unemployment. But high unemployment is not the result of an absolute limit on total possible jobs; it is a breakdown in how quickly the economy adjusts to new parameters.

Will AI Create New Jobs?

Yes, it will. Will it create more than it eliminates? Probably not. But the economy will still create new jobs, and it isn’t dependent on AI to do so.

Consider software engineering. The number of computer programmers necessary to write and maintain a specific piece of software will likely go down due to AI. However, that doesn’t extinguish the societal desire for more software or better software. AI didn’t create those desires, but those human desires will inevitably create new jobs focused on building that better software.

Economies do not constrain the matching of human desire and available labor to a specific job description. Often, one job type is entirely replaced by another. As farming became more efficient with the advent of tractors and fertilizers, freed-up labor initially went back into farming to manage more land. Eventually, agricultural limits were reached, a different source of balance was invoked, and that freed-up labor transitioned into industry and manufacturing.

The same principle applies to AI. Until every need and desire of the human population is met, there will be pressure on the economy’s balancing forces to create jobs to meet them. The only absolute barrier to meeting that pressure is a lack of available labor. If AI ever becomes so universally capable that humanity literally has no more unmet needs or desires, we will have reached an incredibly unprecedented state. While I have previously explored how the economy might actually function if that AGI future arrives, history tells us that this kind of post-scarcity utopia is always further away than we imagine. In the meantime, the world gets more efficient without flipping into a topsy-turvy reality where the fundamental economic force of putting available labor to work ceases to exist.

What About the Short Term?

These macroeconomic forces dictate our long-term expectations: the economy will eventually balance out. But we live in the present, making it entirely reasonable to ask how the job market is changing right now and what to expect in the short-to-medium term.

Currently, I find the evidence of broad job market changes already caused by AI to be very weak. Conversely, I find the probability of broad future changes to be very strong. Most informed observers without ulterior motives tend to agree with this assessment. However, the prevailing public narrative has settled on the exact opposite amalgamation.

To the casual observer, the dominant narrative is that AI has already triggered widespread job losses and restructuring, but will ultimately fail to live up to its long-term hype due to inherent technical limitations. While this is just a prevailing vibe—and conversations often reveal more nuance—it is worth examining how illogical this combination of opinions is, and why it is so easily adopted.

Weighing the Evidence for Changes Already Occurred

The belief that AI has already upended the job market is easy to support because countless articles have delivered it as a concrete conclusion.

One form of article takes real statistics about a weakening job market—or specific sectors like tech—and correlates them directly with the release of AI products like ChatGPT. As a hypothesis, this is fine; as a conclusion, it is incredibly poor. It fails for two main reasons: it ignores major alternative economic forces, and it assumes a timeline of corporate reaction that defies reality.

Because an economy is about balance, if other substantial forces can explain job market shifts, the “AI did it” correlation becomes incredibly weak. When looking at the period since ChatGPT’s release in late 2022, we are swimming in alternate economic forces.

First, the COVID-19 pandemic created a profound shock. In-person jobs vanished and slowly recovered, while tech firms over-hired to meet the surging demand for remote work, supply chain management, and digital education. Executives extrapolated that temporary surge into permanent future demand. While some of that expected permanent shift was indeed realized, the world largely returned to a physical “normal.” As the most extreme growth expectations evaporated, it triggered a sharp, ongoing correction in tech employment.

Second, we experienced a severe inflation surge. While the exact interplay is complex, the relationship between inflation, interest rate hikes, and employment cooling is a foundational and uncontroversial economic reality. Finally, we are running the radical experiment of applying 1930s-style tariffs to a modern, globalized economy. Disentangling these three major, structural forces from the data to pinpoint AI as the primary culprit for recent layoffs is nearly impossible.

Furthermore, the timeline required for these correlation theories is implausibly fast. ChatGPT is released, and supposedly, jobs immediately begin to decline. No economic theory predicts immediate structural decline from a new tool. At a minimum, users must adopt the tool and prove its efficiency. Then, managers must recognize this efficiency, rewrite staffing plans, get approval, and execute layoffs. This takes quarters, if not years. Yet, the main correlational narratives point to tech job losses that actually began months before ChatGPT was even released.

This exact flawed logic was perfectly encapsulated in a viral graph that circulated widely, which Derek Thompson observed was often being shared with commentary declaring it the "scariest chart in the world". The chart accurately shows the S&P 500 rising while total job openings fall, with an ominous vertical line marking ChatGPT’s release right at the inflection point.

Derek Thompson

Is This the New ‘Scariest Chart in the World’?

In the last few days, I’ve seen the following chart bounce around my corner of the Internet, often with some commentary declaring it the scariest chart in the world. The graph seems to show that the release of ChatGPT and the ensuing AI boom cracked the US economy in two, crushing the workforce while lifting the stock market…

Read more

6 months ago · 496 likes · 22 comments · Derek Thompson

While the data points themselves are factually accurate, the suggested correlation is a classic case of confusing coincidence with causality. As a piece of media narrative, it is highly persuasive; as an economic argument, it is entirely unsupportable, especially considering the downward trend in job openings clearly begins before the AI tool was even available to the public.

It is entirely possible that some executives are acting on the ‘vibes’ of AI, restructuring their companies in anticipation of future gains we can’t yet see in the data. But in the present moment, those ‘vibes’ serve as the perfect smokescreen. Whether an executive is genuinely anticipating an AI revolution or simply needs to fix a bloated balance sheet, the public narrative sounds exactly the same. In reality, managers claiming AI-driven layoffs rarely have the data to back it up; rather, they have ulterior motives for cutting staff that they prefer to keep hidden.

Hyperscalers and major tech firms do not want to discuss how their cash flow is being squeezed by the massive capital expenditures required to build data centers and hoard Nvidia chips. Highlighting that reality invites investor scrutiny regarding the ultimate return on those investments. It is much easier to feed the market a narrative of “AI efficiency.”

Then you have executives who simply mismanaged their companies and need a convenient scapegoat for the necessary corrections. Add to this the universal motivation of any executive looking for a short-term stock boost: announcing headcount reductions under the guise of “doing more with less” is a brilliant Wall Street narrative. This practice of “AI-washing” layoffs—as seen with Jack Dorsey’s recent cuts at Block—avoids calamitous explanations like “we are losing customers” or “we are running out of money,” and actively excites investors in the short term, even if the cuts hollow out the company’s long-term capabilities.

Whether a company can actually do more with less will be tested in the future, not the present. The popular narrative assumes that newfound efficiency naturally dictates layoffs, but for a healthy company with opportunities to grow, turning efficiency into layoffs is hugely damaging. If a company suddenly needs fewer resources to maintain its current output, the logical move is to redeploy those resources to capture more market share or build new products. Layoffs generally only make sense if a company is correcting a past mistake (like rampant over-hiring) or if it has exhausted its growth options.

This reality exposes two fundamental flaws in the current public discourse. First, extrapolating the actions of these shrinking companies to the entire economy leaves no room for the story of companies that will use AI to expand. Second, it means these opportunistic, short-term cuts will eventually have to be reversed for companies that actually do have future potential. If they cut too deep today, service quality will degrade, feature releases will slow down, and competitors will steal market share. Eventually, they will be forced to reverse course and re-hire to regain their footing—having needlessly sacrificed their growth momentum for a temporary Wall Street bump. By then, however, the executive will have likely kept their job, exercised their stock options, and enjoyed the short-term bump from the AI narrative.

Another form of article driving the public narrative simply repeats these executives’ statements verbatim. They do so without examining the underlying data or considering the obvious financial incentives for executives to spin bad news (over-hiring or cash flow issues) into a forward-looking story of AI-driven efficiency.

The Delusion of Immediate Efficiency

Accepting these narratives uncritically builds a dangerous delusion: that AI has already unlocked massive efficiency gains, that the best use of those gains is shrinking the labor force, and that companies failing to do so are falling behind.

In reality, while AI has added efficiencies in specific pockets, we are mostly still in the learning and adoption phase. Any hours saved are frequently counterbalanced by training, integration, and implementation costs. Where true, systemic efficiency has been achieved, it is very recent and far from pervasive.

Because this shift is so nascent, we don’t yet have stories of mature firms using AI to successfully expand. Aside from new startups or companies explicitly selling AI infrastructure, the narrative is entirely dominated by contractionary stories—which, as established, are largely misdirection. Accepting these false contractionary tales severely distorts our perception of what the technology is actually doing to the economy.

Non-Linear Transitions

Furthermore, we must remember that stories about the impact of a specific technology are not comprehensive stories about the entire economy. If employment in a field like insurance claims processing genuinely contracts due to AI, the compensating job expansion will not necessarily be AI-related at all.

Freed-up labor might allow housing construction to expand, making homes more accessible and lowering costs. While expanding the housing industry requires more than just available labor (like zoning reform or lower interest rates), if those external limits are removed, the economy will naturally funnel available labor toward that unmet demand. This kind of non-linear adjustment is exactly what dynamic economies do. It is rarely instantaneous, and it is never without friction, but in its default state, the economy makes the adjustment.

A common objection here is the “skills mismatch”—the idea that a laid-off insurance claims processor isn’t going to suddenly start swinging a hammer. But an economy does not rely on perfect one-to-one transitions. Labor markets are highly dynamic, and indirect shifts do most of the heavy lifting. While some claims processors might actually enjoy learning a trade or already have prior experience in one, it is far more likely that their existing skills shift adjacently. An insurance adjuster might transition to a construction firm as a project manager, which in turn frees up the multitasking owner to spend more time actually building.

Even if only a small proportion of the labor force makes these types of lateral moves, the cascading effect absorbs vast amounts of economic change when all the different pathways are added up.

Conclusion

Ultimately, the story about the future of jobs is incomplete without an understanding of this economic dynamism. AI is a profound technological shift, and when the really significant changes it promises do start happening, that dynamic adjustment process will become more obvious.

But right now, separating the noise of the current moment from the signal of long-term economic behavior is crucial. If we believe media narratives driven by ulterior corporate motives, we will confuse ourselves with expectations that are neither complete nor correct. We must remember that AI is just a tool; the economy is the engine. As long as human desires remain unmet, the economic engine will continue to do what it has always done: take available labor and put it to work.

While these are undeniably critical issues, this focus has created a strategic void. The AI industry is aware of enterprise cybersecurity, and they are actively building tools to address it. However, the problem is being approached tactically, rather than strategically. Because they are not treating the defense of our digital infrastructure as a core, existential mission, a cohesive, industry-wide narrative has failed to materialize.

The hard truth for technology executives—CTOs, CISOs, and business leaders driving technology strategy—is this: the AI cavalry isn’t coming. At best you can hope for the AI industry, and the security industry, to work to sell tools. But for gaps that aren’t tool-shaped, it’s up to IT organizations to make this a strategic priority.

As I argued in Security Can’t Wait, advances in AI are drastically accelerating the attacker-defender cycle. Threat actors are already utilizing AI to automate vulnerability discovery and weaponize exploits at unprecedented speeds. Without an equally aggressive response, the segments of our defense lifecycle that remain manual and sluggish will fall hopelessly behind, handing attackers a permanent, dangerous advantage.

And right now, the weakest, most sluggish point of the defense lifecycle isn’t vulnerability identification. It is deployment.

Untangling the Past

The ability to deploy quickly is an element of great variance across the industry. Those variances have always been of importance, but the acceleration of AI-driven threats makes them a critical crux point. It’s tempting to assume this variance is simply the fault of the organizations that lag behind. But not only is that unhelpful, it’s also untrue. History often offers a better explanation, with only a moderate amount of fault left to place on the organizations suffering the ill effects. In my experience leading organizations through such changes, I find it’s best to leave that be and move on.

You might be unwilling to do so without understanding that past, so it helps to examine it. Additionally, even if you are ready to move forward, your organization may not be able to unless you can explain it to them. Some members of an organization lived through previous efforts to change, bear scars, and understand the reality. Others may have joined more recently and fail to understand why things are the way they are. A shared understanding is critical for an organization to work as one.

The most recent crux point for deployment was the adoption of “DevOps,” “Continuous Integration” (CI), and “Continuous Deployment” (CD). These paradigms are real, and their value is immense. Understanding them, however, is often clouded by layers of marketing jargon that have saturated software development for the last decade.

Make no mistake: the advent of DevOps, CI, and CD has been incredibly important. Even half-implementations, aligned with marketing that sold success before completion, have moved the needle. And the organizations that implemented them fully are now industry leaders in far more than just technology.

To appreciate why these changes left scars—and why implementations varied so wildly—we must look at the mechanical baseline they aimed to improve. Historically, software development and IT operations were strictly isolated. Development teams created software, generally working independently for months or even years, before handing the code off to the operations team to support and run in production. Because these teams had opposing incentives—developers were measured by feature delivery (progress), while operations were measured by system stability—introducing change was treated as an inherent threat. As a result, deployments were often massive, infrequent, and high-risk events.

DevOps emerged as a pragmatic and cultural approach to resolve this dysfunction. At its core, DevOps isn’t just a set of tools; it is a commitment to teamwork, communication, and shared goals. In its full realization, it requires unifying leadership to keep the two disciplines from pulling apart and devolving into political, rather than technical, management.

The Mechanics of Modernization

To support this cultural shift, the industry developed specific pipeline tooling designed to automate away the friction and reduce the stress that leads to organizational divergence:

• Automated Builds: In software development, code changes must be “packaged” into a build. Depending on the platform, this involves compiling human-readable code into machine-readable formats, resolving third-party dependencies, and packaging it into a deployable format.

• Validation and Testing: Beyond just compiling, a mature pipeline validates the code’s quality and executes automated tests. To make testing efficient, engineers test the smallest possible units of code (unit tests). This limits the scope of failures and uses less compute time. Inadequate testing can cause a pipeline that otherwise looks complete to produce poor results. Errors that reach production cause costly rollbacks, and the fear of repeating those errors slows everything else down.

• Continuous Integration (CI): Integration is the process of reconciling the simultaneous contributions of multiple developers into a cohesive system. CI extends the build process by making this integration a frequent, if not constant, event. By merging developers’ working copies several times a day, the complexity and risk associated with a final, massive merge are dramatically reduced. In the context of security, CI serves as a crucial enforcement point for the unified system. It is here that dependencies from multiple contributors are brought together, making it the primary stage for running deep, automated scanning tools against the combined application.

• Automated Deployments (CD): Once integrated, software cannot simply be pushed to users; safety constraints require it to be deployed to isolated test environments first. A true pipeline requires test environments that accurately simulate production. However, creating and supporting these duplicate environments is highly complex and the costs often become prohibitive.

Together, the premise of these mechanics was straightforward: mitigate risk by moving faster with tiny, highly automated, and easily reversible changes caught early by continuous feedback loops.

Deployment Divergence

However, as these concepts gained mainstream traction, a clear divergence emerged across the industry. It is tempting to think of organizations making the same technological choices simply by nature of being in the same industry—surely all banks are similarly modernized? In reality, there are significant deviations even within the same sectors. These divergences are shaped heavily by a company’s specific history: when they were formed, or when they attempted a prior wave of modernization.

Generally, organizations fell into one of three paths:

1. True Adoption: Many organizations successfully navigated this transformation. They did the hard work of aligning incentives under unified leadership and invested in comprehensive test environments, proving that modern, automated deployment is a highly achievable goal when backed by genuine commitment.

2. Watered-Down Adoption: Driven by vendor sales cycles and a management desire for painless wins, many organizations adopted the terminology without the substance. The genuinely far-reaching concepts were distorted to justify incremental tool purchases. Crucial but non-mandatory steps—like rigorous unit testing or maintaining accurate test environments—were skipped or done poorly in the name of expediency. Without true CI, integration remained sporadic. Teams bought the tools and declared victory, but failed to fundamentally change their deployment process or speed.

3. Stalled Implementation: Other organizations simply struggled to get momentum at all, weighed down by the sheer complexity and cost of entrenched legacy systems, such as monolithic applications and mainframes, which are notoriously difficult to integrate into modern CI/CD pipelines.

Why did so many organizations fall into the latter two camps? The root causes are deeply embedded in organizational dynamics. For years, technology teams have been caught in a tug-of-war between competing priorities. There is an unrelenting push to deliver short-term wins and new features, which inevitably drives the accumulation of technical debt. This is compounded by coordination issues between siloed teams, cost-cutting mandates, and general corporate politics.

The result of this divergence is that while excellent pipelines certainly exist, a significant portion of enterprises still grapple with brittle, sporadic deployment processes. They have automated the easy parts (like compiling) but left the hard parts (comprehensive testing and security scanning) as manual roadblocks. Without continuous, reliable feedback, deployments are batched, delayed, and risky.

This isn’t an indictment of current leadership; it is simply a realistic accounting of the accumulated friction of technical debt and conflicting priorities. But it is a reality we must acknowledge before we can move forward.

Clouded Perceptions: Restarting from Stalled and Watered-Down Adoptions takes additional effort to rebuild momentum because of terminology drift. It’s too easy to assume a shared commitment which turns out to represent different expectations. While you can’t erase the effect of the past, you can take the extra effort to communicate what is meant at each opportunity.

The Widening Gap and the Irony of Regulation

When an organization’s deployment pipeline is insufficient, the time it takes to patch a newly discovered vulnerability stretches from hours to weeks or months. Attackers have delays too, but depending on their delays—which might be accelerated by AI—is a risk.

We often look to regulation to force improvements in these areas, hoping compliance mandates will motivate continuous improvement. But here lies a painful irony: for the organization that has already fallen behind, regulation often creates extra friction. It introduces new audit gates and reporting requirements that further slow down the deployment process. Until the pressure is redirected toward a truly dramatic overhaul—with all the costs and commitment that entails—the effect of regulation is to slow defenders, leaving a wider gap attackers can exploit.

Assessing the Battlefield and Avoiding the Blame Game

If the mandate is to unblock these pipelines, technology executives must first assess their own relationship to the organization before demanding changes. Are you a new leader brought in with an explicit mandate to improve? Are you an established leader leveraging newly acquired influence? Or are you new to an organization where continuity, rather than disruption, was the stated goal?

Understanding this positioning is critical because diagnosing a lagging deployment pipeline often delivers bad news to teams who believe they are already doing their best. If delivered poorly, it forces the organization into a “fight or flight” response.

Crucially, executives must actively suppress the “blame game.” Blame is a destructive concept when fixing technical debt. Technical systems do not care who is at fault; they will succeed or fail independently. Seeking blame causes internal information sharing to become strategic and self-preserving, rather than solution-oriented. While identifying failures is necessary for strategic leadership changes, day-to-day technical modernization requires actively discouraging the blame game so teams can focus entirely on the fix.

Turning AI Inward

If current pipelines are too encumbered by historical debt to move at the speed of modern threats, they need priority. Yet, that priority is lacking and must be built. The AI and security industries are offering tools, but not implementation.

Technology-focused executives must take the driver’s seat. The DevOps playbook is well documented. But so are the impediments. New efforts and commitments are difficult. Past failures create inertia that needs unblocking.

Tools can’t solve this alone. What they can do is modify the impediments that held back implementation in the past. Those modifications create a compelling narrative to overcome inertia, and start new efforts and commitments to modernize deployment pipelines.

Consider the new opportunities to improve modernization efficiency and effectiveness faster using AI:

The Testing Burden

A robust deployment pipeline requires comprehensive automated testing, but developers notoriously loathe writing and maintaining tests. AI fundamentally changes this dynamic. If you have no tests, AI can scale up baseline coverage rapidly. If you have some tests, AI can identify and fill the gaps. More importantly, AI can monitor existing tests for brittleness, automatically suggesting refactoring or updates when underlying code changes. By removing the maintenance overhead, AI removes a primary excuse for failing pipelines.

Accelerating Legacy Transformation

Many deployment bottlenecks are rooted in legacy systems—like mainframes and monolithic applications—that were previously deemed too complex, expensive, or risky to modernize. AI transformation software is changing this calculus.

One methodology here is to reverse engineer specifications from an existing codebase. One significant challenge to modernizing any legacy system is understanding how that system should behave. There may be documentation, but it very likely has drift and inaccuracies that would undermine a transformation. A reverse engineering process is not likely to be hands-free, but a combination of AI and operators complement each other. That should make it possible to reverse engineer any existing codebase sufficiently to perform a quality transformation.

Testing comes into focus here again. Tests can be generated and employed on both the old and new source. Specifications assist in test generation. Tests assist in mechanical code transformation for core functions and methods. Tests and specifications also assist in larger-scale structural changes. Transformation strategies have traditionally relied on both of these, preferring small incremental updates when practical, and resorting to larger-scale rewrites strategically.

AI-driven transformation tools not only reduce effort via these steps, but improve accuracy and probability of success.

Resources:

• Gartner AI Augmented Code Modernization Tool Peer Insights

Shift-Left Security

AI-powered static analysis can be integrated directly into the developer workflow. This ensures that the code (and the AI-generated tests themselves) adhere to established security and quality standards before they ever reach the integration phase. Not only can this help avoid introducing new security issues, but it can raise confidence in the process of deploying fixes to known issues and newly discovered issues.

The quality of the tools you can integrate may be influenced by how modern and mainstream other parts of the stack are. COBOL and FORTRAN code won’t have the same level of support as Rust, Python, TypeScript, .NET, C or C++ code. While static analysis tools have existed for some time, the most developed tools in this space have evolved past simply flagging potential errors; they now utilize AI to drastically reduce false positives, understand the context of the codebase, and suggest specific, workable auto-fixes.

Resources:

• Gartner Application Security Testing Peer Insights

• G2 Best Static Application Security Testing Software

Securing the Pipeline

As pipelines become the engine of the enterprise, they become prime targets for attackers. Implementing highly effective but difficult security practices—such as least-privilege access for the pipeline itself—is complex to manage manually.

How this is done will depend on where your pipeline is implemented. AI tools can analyze code for access requirements, avoiding the admin needing to guess developers’ requirements. AI and conventional tools can analyze deployment patterns to determine used and unused privileges, which create a signal where to limit privileges.

Resources:

• Do Your CI/CD Pipelines Need Identities? Yes. (Cloud Security Alliance, 2025)

Disrupting Active Exploitation: An Essential Stopgap

While modernizing the deployment pipeline is the ultimate cure, technology executives must manage the immediate reality: vulnerabilities will exist in production while fixes navigate a sluggish pipeline. It would be irresponsible to omit AI’s capability as an ameliorative control during this window. AI-driven behavioral analytics and dynamic anomaly detection can be deployed defensively to disrupt the control and exploitation phases of an attack in real time. By identifying and isolating threat actors attempting to leverage unpatched systems, these tools buy the organization the critical time needed for pipeline improvements to take effect.

Implementation

AI tooling isn’t enough in the same way that DevOps tooling wasn’t enough. A plan is necessary, and that plan must engage with the culture of your organization. What type of modernization is needed? Why hasn’t it happened already? Will it require a full-scale transformation (mainframes/monoliths)? Is it about completing a watered-down adoption?

There are good sources on DevOps adoption (i.e., The DevOps Handbook), so I won’t try and repeat these in their entirety. Committing to completing adoption, and taking advantage of new opportunities that shorten or de-risk challenging aspects, is how to create your plan.

Conclusion: The Call to Arms

The acceleration of the cyber battlefield is a reality. The mandate for technology executives is clear: we must stop viewing AI solely as a threat to be mitigated or a product to be purchased, and start wielding it as an operational imperative. Accelerating our defenses requires accelerating our deployments. The tools are in our hands; it is time to use them.

Anthropic’s recent labor market analysis has improved understanding by analyzing “observed exposure”—shifting from theoretical feasibility to measuring how AI is actually being used across different occupations. This is a crucial step in understanding AI’s real-world footprint. However, a core assumption remains: if a task can be done twice as fast by AI, the required human labor spent on that task will decrease.

I suggest a deeper framework that confronts that assumption. A significant reason why AI capabilities will not translate into reduced working hours is that observed exposure fails to account for the dynamics of economic demand for tasks. Demand for tasks is not static. As a task progresses from theoretical capability, to observed exposure, to full exposure, dynamic responses should be expected.

The impact of AI does not depend solely on whether a machine can and is used effectively to do work, but on whether the demand for that work is Bounded, Unbounded, or Adversarial. How work is divided between those categories, how it’s packaged into jobs, and the dynamic interplay are critical to accurately predicting how AI adoption will change demand for work.

While I am only presenting the conceptual framework here, extending this to a quantitative analysis via task classification and applying it to datasets like the Anthropic Economic Index is the logical next step.

Tasks, Jobs, Outcomes, and Demand Dynamics

To understand labor impacts, I must separate the elements of work into tasks, outcomes, and jobs:

• Tasks are individual units of work executed to achieve a specific result.

• Outcomes are the overarching goals or results that a job seeks to achieve through the execution of tasks.

• Jobs are bundles of tasks organized and executed to deliver specific outcomes.

To this, I also add three categories of demand:

• Bounded Demand: Demand that has finite usefulness within related outcomes, and does not itself enable demand for new outcomes.

• Unbounded Demand: Demand with the potential for self-expansion by enabling demand for new outcomes. When scaled, efficiency completes entirely new outcomes with positive value. (Practically speaking, this does not demand a true lack of boundaries, just incredibly distant ones).

• Adversarial Demand: A non-bounded state1 driven by a zero-sum competition. When scaled, efficiency drives volume and complexity within the same adversarial outcome.

These three categories of demand are most readily applied to classify tasks, but we’ll find we also think of jobs in this way based on the type of tasks they hold. We can also see how outcomes are key to this, in differentiating unbounded and adversarial demand—separated by whether scaling them generates new outcomes or inflates existing outcomes.

The Dynamics of Efficiency Reallocation

Jobs tend to have a primary outcome (e.g., producing a car, solving a medical issue, resolving legal disputes). When that overarching outcome is adversarial, the underlying adversarial tasks act as an “efficiency sink.”

When AI automates routine, bounded tasks within that job, the worker does not simply work fewer hours. Instead, the time saved is reallocated into the adversarial tasks to maximize the overarching outcome. This dynamic maintains stable human labor hours despite high overall observed AI exposure.

Crucially, if all adversarial tasks are fully automated, the efficiency sink effect moves from labor hours to automation (compute/API) costs. But as long as a single adversarial task is left unautomated, it retains a significant portion of the efficiency sink effect, and total labor hours remain mostly unchanged.

Three Part Demand Framework

Occupational tasks can be categorized into three distinct economic demand buckets. Because jobs are organized to achieve specific outcomes, we observe task-level dynamics manifesting at the job level. When AI introduces efficiency gains, each bucket reacts differently:

Bounded (Satiated Demand)

• Reallocation Dynamic: There is no new work to reallocate. Faster outcome completion means work is finished earlier. Fewer workers are needed to maintain the same pace. Unless an unbounded or adversarial outcome indirectly generates new work, available jobs attached to these bounded outcomes will decline.

• Job Examples: Payroll Clerks, Data Entry Operators, and Technical Writers. Demand for Payroll Clerks is bounded by the number of companies, workers, and efficiency of the payroll process.

Unbounded Utility (The Infinite Backlog)

• Reallocation Dynamic: Faster production lowers costs. A backlog of demand reuses freed resources. Cost efficiencies sustain or expand fulfillment of potential demand. Time saved is used to produce more output, higher-quality outputs, or both.

• Reallocation Friction: Reallocation is not immediate. The demand backlog can become stuck for organizational, training, research, finance, or any other coordination issue.

• Job Examples: Computer Programmers, Scientific Researchers, and Healthcare Professionals. The backlog of desirable software, scientific discoveries, and medical care is never fully satiated.

True Adversarial (Zero-Sum Escalation)

• Reallocation Dynamic: Efficiency gains are weaponized to win adversarial outcomes. Time saved is reinvested into performing the task at a higher volume or complexity to maintain an edge over an opponent, scaling effort within the same outcome.

• Reallocation Friction: Escalation between parties can take time to emerge, and can be delayed or deferred by agreement, law, or practical obstacles.

• Escalation Attrition: Adversarial escalation eventually hits diminishing marginal returns. If AI allows lawyers to draft 10x the claims and counterclaims, they will do so to maintain an advantage. These claims and counterclaims add little or no extra utility to the justice system. If any, it was such fractional quantities that they would not have been pursued outside an adversarial system. Those dynamics don’t mean escalation isn’t subject to its own attrition where the next escalation not only fails to create social value, but fails to yield individual value.

• Attrition and Friction: The combination of escalation attrition and friction is another factor in delayed reallocation. The last layers of escalation yield the least individual value, which lowers any incentive to bypass frictions created by time, happenstance, law, agreement, ethical standards, or other factors.

• Job Examples: Lawyers (maximizing legal strategy), Salespeople (maximizing competitive wins), Marketers (battling for attention), and Cybersecurity Analysts (offensive vs. defensive escalation).

Note on Transitions: Tasks and jobs can shift categories. Customer Service Representatives are currently Bounded (dealing with a finite number of human interactions). However, if AI agents drop the cost of interacting with customer service to near zero, these outcomes could transition into Adversarial territory. While a customer-obsessed company does not view legitimate customers as adversaries, an open, zero-friction channel inevitably attracts malicious actors, automated fraud rings, and algorithmic social engineering at scale. Companies will be forced to deploy defensive corporate AI to filter this malicious volume, reallocating human CSRs to investigate and manage these complex, escalated algorithmic attacks.

The AI Labor Impact Matrix: “Three Sextants and One Half”

By mapping AI Exposure (High vs. Low) against the 3-Part Demand Framework, I create a 3x2 matrix for describing expected labor market behavior.

The Bottom Half (The Control Group)

• Low AI Exposure (across all demand types): Protected by physical friction, manual dexterity requirements, or strict regulatory roadblocks. This represents the status quo (e.g., physical trades, nursing).

The Top Three Sextants (High AI Exposure)

The highly exposed segment of the economy splits into three distinct zones, driven by different adoption incentives:

Sextant 1: The Efficiency Transition (High Exposure + Bounded)

• Early Influences: Early adoption is driven top-down by organizations seeking to realize the benefits of automation to reduce labor costs.

• Labor Impact: Measurable job displacement and hiring slowdowns. While this creates disruption for current workers, it represents an efficiency gain for the broader economy by freeing human capital from bounded tasks.

• Social Impact: Managing this shift requires robust social infrastructure. Social programs like unemployment insurance, retraining initiatives, and general social support must be central to navigating these impacts.

While disruption occurs across all sextants to some degree, societal resources and attention must be most heavily directed toward transitioning workers. Without support you lose both pre-disruption stability and the productive use of freed human capital. There is no social value in structural optimization if it doesn’t lead to new, more productive employment.

Sextant 2: The Infinite Frontier (High Exposure + Unbounded)

• Early Influences: Early adoption is driven by closeness to the technology industry.

• Labor Impact: Minimal displacement (subject to reallocation frictions), accompanied by productivity and objective output growth.

Sextant 3: The Arms Race (High Exposure + Adversarial)

• Early Influences: Early adoption is driven bottom-up by individuals with an aggressive, advantage-seeking demeanor, and others who are forced to adopt to survive a zero-sum game.

• Labor Impact: Minimal displacement (subject to reallocation frictions), task inflation, and potential for worker burnout.

Predicting Adoption Velocity: The Role of Worker Demeanor

While exposure metrics predict where AI can be used, analyzing worker demeanor helps predict how fast and by whom it will be adopted. Adoption is not solely a function of technological capability; it is deeply tied to the behavioral incentives and natural disposition of the workers in these roles.

• Status Quo Bias and Corporate Mandates (Top-Down): In Bounded roles, workers have little incentive to adopt a tool that finishes their workload faster, as doing so threatens their job security. Consequently, organic worker adoption in this sextant is heavily muted. AI integration here is almost entirely top-down, driven by management seeking cost reductions.

• Curiosity and Tech-Affinity (Bottom-Up): In Unbounded roles, early adoption has been driven by domain proximity and natural curiosity. Workers in these fields inherently value systems that reduce friction to build, solve, and create more efficiently. Top-down influences are secondary, but as a complement, create the fastest adoption.

• Advantage-Seeking Demeanor (Bottom-Up): In Adversarial jobs, workers are structurally incentivized to seek an edge. They actively test and implement AI independently because failing to do so means losing a deal or a case. Adoption is organic and aggressive. Top-down influences are secondary, and more about approval than directives.

The Prediction: AI tooling will proliferate fastest and most smoothly in Unbounded and Adversarial jobs (Sextants 2 and 3), driven by eager, self-motivated workers. In contrast, Bounded jobs (Sextant 1) will experience a delayed adoption curve, followed by an abrupt, disruptive shock as corporate mandates are enforced.

Societal Impact: Disruption and Outcome Completion

From a macroeconomic and social perspective, society will primarily be concerned with two consequences of this framework: the friction of job disruption and the new value generated from outcome completion.

The Reality of Job Disruption and Reallocation

Job disruption will occur most acutely within highly exposed, bounded jobs. It is important to clarify that absolute “zero displacement” in Unbounded and Adversarial jobs is merely a theoretical equilibrium. In reality, frictions in reallocating time and learning new AI-augmented workflows cause some temporary displacement in Unbounded and Adversarial jobs. The key difference is in Unbounded and Adversarial jobs final equilibrium is resistant to durable reduction.

Furthermore, as acute displacement occurs in the Bounded sextant, the freed human capital will likely reallocate toward Adversarial or Unbounded work, shifting the composition of the broader labor market.

Value Creation and Outcome Completion

For society to realize net-new value from AI, it must look primarily to the unbounded domain. This is where efficiency translates directly into outcome completion rather than zero-sum escalation. While early tech discourse heavily features Computer Programmers, the most profound societal benefits will emerge from fields like Science and Healthcare. The demand for scientific discovery, novel treatments, and improved patient care is effectively infinite. Scaling these tasks generates profound, concrete new outcomes for human well-being.

Conversely, AI usage in adversarial jobs toward adversarial outcomes, is largely irrelevant to net-new social value. Because it consumes resources that might have been put to better use in another domain, adversarial adoption can sometimes be a net negative, though it often balances out to neutral or slightly positive. For starters, existing human labor usage was a drag of its own.

Additionally deeper task completion within adversarial roles can result in slightly positive effects, but this is far from guaranteed. Adversarial roles are usually composed of a very valuable social good (justice for example), with an adversarial layer on top. The most significant realization of that social good comes at the first layers of engagement, and at most incrementally improves with more engagement. Even this is not a guaranteed conclusion though, there is no inherent reason additional investment cannot become extractive while failing to produce more social goods.

Critically, good planning around critical systems (like finance and law), can improve their social outputs and avoid their adversarial aspects amounting to net losses, but the opposite can be said of poor planning.

The Complex Social Dynamic of Art

Society will have a more complex relationship with certain adversarial domains, most notably the Arts. While the economic dynamic of art is highly adversarial—creators are engaged in a zero-sum competition for finite human attention—society does not view this escalation in the same way it views the “deadweight loss” of legal paperwork. The resulting explosion of media, storytelling, and design may be born from competitive escalation, but it yields cultural artifacts that society inherently values and consumes differently than pure corporate friction.

Crucially, art is often valued not simply for its outputs, but for its process. In a sense, you could attribute the same to any other field where the participants care about their own work, but society has always given a special place to art in this way.

For those reasons, it’s reasonable to not expect the domain of art to follow all the same dynamics of other adversarial domains. That said, it is clearly an early adopter, like other adversarial domains.

About Real-World Tests

The world is looking at real-world jobs data trying to find confirmations of early AI labor and other predictions. It is not wrong to look, but the expectation of finding confirmation here is unrealistic. Jobs data itself is messy, takes time to become accurate, and has some other recent large scale influences.

Anthropic goes so far as to compare a prediction to another prediction, in search of such confirmation. In their defense, they are clearly aware of the risks there and don’t tout their results heavily.

I look forward to putting my predictions to the test, and seeing the results of other tests. But I would also continue to suggest that we should expect null results and should not force data that only supports a null result into a definitive conclusion about AI’s ultimate labor impact.

Conclusion

When macroeconomic studies aggregate these three top sextants into a single “Highly Exposed” bucket, the stable employment driven by the Arms Race and the Infinite Frontier completely masks the real, acute job losses occurring within the Efficiency Transition. By evaluating occupations through the primary lens of Bounded vs. Non-Bounded (Unbounded/Adversarial) demand, we can isolate the exact sectors where AI will cause job displacement versus where it will merely fuel outcome generation or task escalation.

However, the underlying mechanics of both fields can feel frustratingly inaccessible. By design, cybersecurity is meant to be an invisible shield. Unless you are deeply involved in computing, you usually only notice it when it fails, or when it creates daily friction—like remembering a complex password. The inner workings of how your data stays safe remain mostly opaque, exactly as the engineers intended.

A similar dynamic applies to Artificial Intelligence. Today, it’s easy to experience AI through chatbots. You can ask questions, spiral into deep conversations, or generate images in seconds. But as impressive as that is, talking to a chatbot is just the tip of the iceberg. Behind the scenes, by some estimates, over half of all AI usage today is dedicated to a single task: writing computer code. It is an invisible shift of significant scale.

To understand why applying AI to cybersecurity is so critical right now, we first have to confront a widespread misunderstanding about what software actually is, and why it breaks.

The Myth of Perfect Software

If you don’t have programming experience, it is natural to assume that building software is like publishing a newspaper: you plan the layout, write the articles, print the edition, and the final product is permanently finished. In reality, writing software is much more like writing and maintaining Wikipedia.

When a printed newspaper hits the stands, it cannot be changed; tomorrow brings an entirely new edition, sharing little other than a layout, typeface, and name. But Wikipedia is an ongoing, living document. A single event sparks the first version of an article, but editors will argue over, revise, and correct it for years. Software engineers do the same thing. They write code, users report that something doesn’t work the way they expected, and the engineers go back and revise it.

Because fixing one piece of software often accidentally breaks another, they don’t stop there. They write entirely separate scripts—automated tests—whose only job is to constantly check the original code and ensure that older features keep working as the software evolves.

Testing exists because programmers are human. We misunderstand what users want. We misunderstand the limits of our computer hardware. We mistakenly rely on flawed code written by someone else. Testing protects programmers from their own fallibility.

Historically, programmers wanted their software to be deterministic. That means for every specific action, there is one specific, predictable reaction. If you move $100 from your savings to your checking account, savings goes down exactly $100, and checking goes up exactly $100. It sounds simple. Simple rules like this allow simple tests.

But users are highly unpredictable. They click buttons in the wrong order, type words into boxes meant for numbers, and combine features in ways the engineers never imagined. Add to this the physical realities of computing—hardware inevitably degrades, and surges in user traffic can consume all available memory—and the environment becomes chaotic.

Resilience

This dynamic chaos is difficult enough to manage when users are innocently fumbling around. To manage it, engineers must add another layer of complexity to their work: resilience. They don’t just program what happens when things go right; they have to spend countless hours programming exactly what happens when things go wrong, trying to ensure small failures don’t add up to large failures. This relentless pursuit of perfection makes building software exponentially harder.

Enter the Attacker

Attackers live in the gaps of a programmer’s incomplete plan. They look for the scenarios the engineer forgot to test. Sometimes, this looks like extreme user behavior: What happens if I type 10,000 characters into a password field meant for 20? What if I send thousands of requests at the exact same millisecond?

This doesn’t stop by “acting like a user”. Software has internal communication channels, invisible to users, and attackers will do their best to access and utilize these too.

An attacker is entirely happy with chaos as an outcome. They only need to find one weak spot, one forgotten variable, to force the software to do something it shouldn’t.

The AI Magnifying Glass

How does AI interact with this cat-and-mouse game? Fundamentally, AI is a magnifying glass. For attackers, it is a tool to scan for weak spots faster and more comprehensively than manual reviews allow.

The most obvious response is for defenders to use similar tools. If an attacker is using AI to find the cracks in your walls, you need AI to find—and patch—those cracks first. In the long run, the ability of AI to rapidly spot human errors in code will be a substantial advantage to defenders. But where this was useful before, it’s critical today. As the cat-and-mouse game accelerates, staying ahead is more critical than ever.

But this brings us to another major misconception about cybersecurity: finding the vulnerability isn’t actually the hardest part. Neither is fixing the vulnerability.

To an outside observer, fixing a security flaw sounds highly complex. Often, it isn’t. The majority of security vulnerabilities are born from tiny, simple mistakes: a list that is one item too short, a user granted one permission too many, or a line of code that says “and” when it should have said “or.” In a vacuum, a programmer could fix these errors in five minutes.

There is a worry that a little change might have a bigger impact. Other code may have tried to compensate for the mistake and now breaks after the fix. This is always a worry, and automated testing was a tool to minimize that worry. So fixes aren’t always easy, but they still aren’t the core challenge.

The real challenge is deploying that fix. Modern software is woven into complex corporate environments. A simple five-minute fix might have to pass through multiple testing environments, bureaucratic approvals, and compliance checks before it ever reaches the user. The quality of different companies’ deployment processes varies greatly. The best companies can deploy thousands of small fixes a day. Many other businesses struggle to deploy one update a month.

There are over 40,000 known vulnerabilities, with over 100 more discovered each day. And those numbers only cover known software and libraries. Code a company develops for itself can introduce unique vulnerabilities that aren’t part of vulnerability databases. While these won’t all apply to any particular company’s environments, enough will that one update a month, or even one per day, is not sufficient.

Attackers act as relentless inspectors who will punish a company for any delay. If AI helps an attacker find a flaw today, but your company’s approval process takes three weeks to deploy the fix, you are at a serious disadvantage.

The Economics of Cyber Warfare

This might sound like a losing battle, but the defenders actually have a distinct advantage: economics.

Cyber attackers generally fall into two categories: people who just want to cause random destruction (who are thankfully rare and usually lack the focus to execute complex plans), and people who want to make money.

That second group is large, but they are doing math. If the payoff is too low, or the effort required to break in is too high, they will give up and look for an easier target. You don’t need a perfectly impenetrable wall to stay safe. You just need a wall that is sufficiently expensive for a hacker to penetrate.

This is where AI will shift the landscape. High-value targets (like major banks or tech giants) are already rapidly adopting AI to patch their weak points faster than ever. Attackers will likely find these targets too expensive to hack, assuming they avoid the deployment trap. The security organizations meant to protect are sometimes the impediment, creating the delays that bring risk.

This varies significantly between organizations. All organizations realize the importance of security, but only some have been able to turn that knowledge into reality and bring about the changes that allow for rapid deployment.

Efficient deployment is part of the design of many organizations. Newer organizations with a tech focus usually started out this way, as the template has been demonstrated many times. Older organizations have sometimes moved up, but many older or non-tech-focused organizations sit in an uncomfortable gap here.

For those that have failed to keep up, their fallback is often more layers of security—which carries high costs, but remains effective in raising the barrier to entry for attackers.

The real danger zone will be moderate-value targets—companies that have something worth stealing but may operate with slow, outdated security practices, and lack the justification for the most expensive layered capabilities. AI will turn a harsh lens on organizations that have managed to scrape by unnoticed in the past. These companies will face a strict ultimatum: modernize their security, or risk severe breaches.

Again, there is significant variability. Those with efficient deployment will stay ahead. Those that don’t are at risk, unable to match the expensive high-value protections, but also behind their peers.

Ironically, the lowest-value targets—everyday individuals and small businesses—might actually see an immediate benefit. Because their core reliance is on outsourced platforms (like cloud email providers), they will instantly inherit the new AI-driven spam and scam detection tools built by the tech giants, without having to lift a finger. While outsourcing has its weaknesses, when it comes to core functionality with a broad user base, it’s hard to beat.

A Brief Aside: Adversarial Revenue

This mandatory modernization creates an interesting, somewhat circular side-effect in the tech industry: a concept known as “adversarial revenue.“

Because attackers are rapidly adopting AI, every potential target is forced to buy AI-driven defensive tools just to keep pace. Who sells those tools? Often, it is the broader tech industry that is developing these AI capabilities in the first place. For the companies providing AI security platforms, the rising tide of empowered hackers guarantees a sustained, highly motivated market. The threat itself creates the demand for the cure, making AI defense a uniquely lucrative sector of the economy.

Security revenue is a bit different than other adversarial roles. Here there’s a clear bad guy. In fields like laws and finance, two sides exist, but neither is clearly creating the inefficiency. It’s theoretically possible we might improve the ratio between productive and adversarial revenue here by self-policing or regulatory efforts, though that requires convincing those industries to give up some potential revenue.

Setting aside the financial balance sheets and returning to the mechanics of the conflict, a much simpler question often arises about these adversarial dynamics: why not just prevent attackers from accessing AI to begin with?

Why Not Just Ban the Bad Guys?

Unfortunately, it’s a deeply complex challenge. The best success leverages lesser amounts of privacy, but it would be naive to think a loss of privacy can provide a total solution.

There are two main ways people access AI. One approach is through “open-source” models, which are freely available for anyone to download and use privately on their own computers. Protections here are limited. Creators train them to refuse malicious requests, but determined attackers consistently figure out how to bypass those guardrails (“jailbreak” them). The best protection here is that, so far, open-source models are less capable, and degrade a bit more after being jailbroken.

A more common method is through “controlled hosting”—the major platforms where you must log in to use the AI. Here, the AI companies actually do fight back every day. This isn’t just a theoretical threat; companies like Anthropic and OpenAI routinely detect and disrupt coordinated attackers attempting to use their networks.

But their expectation isn’t to build a flawless barrier. Instead, they use the mechanics of bureaucracy to drive up the attacker’s costs. They require an email to create an account. They monitor activity for suspicious patterns. When they see something shady, they issue a “soft-block,” refusing the prompt. When an attacker repeatedly tries to bypass that block, the company bans the account entirely, forcing the hacker to create a new account. And then they block account creation patterns that look shady.

Even with controlled hosting, stopping malicious actors entirely is difficult. “Shady” is a judgment call, and the other side has the option of changing tactics. They’ll try to look like regular users. They can’t hide forever in this way, but the provider risks harming regular users if they react too quickly by blocking patterns that describe regular users.

The AI Apprentice

AI companies seem highly capable, so why can’t they stop this, even though it’s difficult? You might ask, if they are motivated enough. If you doubt the AI companies are sufficiently motivated, consider the story of “distillation” attacks, which demonstrates the limits of their control.

To understand distillation, imagine a hacker who knows they will eventually get caught on the major, guarded platforms. Instead of using the heavily guarded AI to find vulnerabilities directly, they use it as a master tutor. They feed the secure AI complex coding problems, record its brilliant answers, and use that data to train their own private, open-source AI models.

Think of it like sneaking a camera into a master locksmith’s workshop. You don’t need to steal the locksmith’s tools; you just record how they work, go home, and teach your own apprentice. Once the attacker’s private AI learns enough, they no longer need the major platforms. They have their own unrestricted hacking assistant, operating entirely under the radar.

Distillation attacks also come from rival attempts to improve their own AI model, using outputs from a better model. It should be obvious that the leading AI companies want to retain their lead, and stopping distillation attacks would help. That said, all have reported activity of this type, and while they’ve had partial success in detecting it, it’s only partial. Anthropic reported millions of requests it believes were distillation attacks. Google reports hundreds of thousands of requests too.

Major AI providers are highly motivated to prevent this. Distillation isn’t just a security threat; it’s the outright theft of their multi-billion-dollar intellectual property. The fact that tech giants actively try—and often struggle—to stop distillation proves that preventing misuse isn’t a matter of lacking the desire or financial motivation to build a flawless barrier. They desperately want to build that barrier, but the technical reality makes absolute control nearly impossible.

Privacy and Security

Placing the best models in controlled environments provides some improvements. It does then place some of our privacy in the trust of those controlling those environments. Such environments are designed to preserve privacy in a balanced way. Your requests do go through automated review, but there are internal guardrails on how those are maintained, and who and when someone sees violations. But we have to place some trust elsewhere that’s significantly different from the type of validation we’d need regarding the privacy of a locally run model.

What’s Your Role?

Knowing that this massive contest is occurring behind the scenes, what can you, as an everyday user, actually do?

While the tech giants fight over deploying complex code fixes, attackers will still try to go after the easiest target: you. AI allows hackers to create highly personalized, perfectly spelled scam emails and incredibly realistic fake websites. To stay safe, a few standard pieces of advice are more important than ever:

1. Enable Multi-Factor Authentication (MFA)

With just a username and password, your security depends entirely on no one ever guessing or stealing your password. If you reuse a password, or accidentally type it into a fake “phishing” site created by AI, it’s compromised. MFA, while occasionally annoying, ties your access to something physical that you have—like a phone that receives a prompt or an authentication app. Even if an attacker steals your password, they can’t get in without your phone.

2. Learn to Read a Web Address (and Spot a Fake Browser)

Attackers frequently build fake login pages designed to steal passwords. Because AI makes it easy to perfectly clone the look of a legitimate site, attackers have escalated to a new trick: the “browser within a browser.”

A web browser displays content from the sites it loads. As a side effect, malicious sites can draw a fake window inside the webpage that looks exactly like your browser’s top bar, complete with a perfectly secure-looking—but entirely fake—web address. To protect yourself, you must be familiar with the normal layout of your browser. The real address bar is part of the secure surface of your browser at the very top of your screen, not nested down inside the web page’s content.

Once you are certain you are looking at the real address bar, the URL can look like a long string of gibberish, but there is a simple rule of thumb: find the very first single slash (/) after the https://. Then, look at the word immediately to the left of the .com, .gov, or .org.

• If the address is consumer.ftc.gov/articles/..., the controlling word is ftc. You are on a government site.

• If an attacker tries to trick you with ftc.security-update.com/login, the controlling word is security-update. You are not on a government site; you are on an attacker’s site.

3. When in Doubt, Search

If reading the URL feels confusing, use a search engine instead of clicking a link in an email. Type the company name into Google. It is incredibly difficult for an attacker to manipulate search algorithms enough to place their fake website higher than the real company’s official site. Just be sure to skip past the first few results if they are explicitly labeled as “Sponsored” or “Ad,” as attackers sometimes buy ad space.

4. Be Wary of Voice Calls and Texts

You should never give out your MFA codes or passwords by email or by phone call. However, verifying who is actually on the other end of the line has become much harder. AI makes it trivially easy for scammers to clone voices or generate convincing, conversational text messages. If you get a call from your bank—or even a panicked loved one—asking for money or a security code, hang up. Look up their official phone number yourself, and call them back.

5. Keep Things Updated

You should get to know your computer’s operating system and web browser. Both have built-in mechanisms to install updates automatically. Don’t delay or avoid these updates. As we discussed earlier, deploying fixes is the hardest part of cybersecurity. When you see an update ready to install on your phone or computer, you are often receiving the exact “five-minute fixes” software engineers just wrote to patch a vulnerability. Install them.

What are AI Companies Doing to Protect You?

While your personal vigilance is the last line of defense, the tech industry isn’t sitting idle. AI companies are actively deploying countermeasures:

• Limiting Access by Attackers: When an attacker is identified, their accounts are deactivated. AI companies use complex pattern recognition—analyzing the content and origin of requests—to hunt down malicious users. It is a constant cat and mouse game. While it doesn’t stop all access, it raises the cost significantly. Every moment an attacker spends trying to defeat these protections is a moment they can’t spend conducting destructive attacks.

• Utilizing Guardrails and Training: AI models are trained with guardrails that inspect incoming and outgoing traffic, automatically refusing or modifying prompts that appear intended to facilitate harmful activity. Again, these techniques are not foolproof, but they disrupt access and diminish the utility of the AI for hackers.

• Scanning for Vulnerabilities and Orchestrating Remediation: Vulnerability scanning isn’t new, but AI enables broader and deeper results. AI companies (Google CodeMender, Claude Code Security, OpenAI Aardvark) are working directly with the cybersecurity industry to execute massive scans, instantly generate remediations, and orchestrate campaigns to deploy those fixes before attackers can act.

Securing AI Itself

It is worth noting that protecting traditional software from AI-empowered attackers is only one slice of the overall security story. A complete view of AI security must also engage with other massive topics: how to deploy AI safely within an organization, how to manage how your private data is used by an AI model, and how to manage “agentic” systems (AI that can take actions on its own).

There are also vital, high-level theoretical debates about preventing AI from being used for massively destructive weapons, authoritarian surveillance, or sci-fi “AI overlord” scenarios.

But every conversation needs a focus, and right now, the most immediate, practical threat to the average user and business is the invisible arms race occurring in everyday software.

A Reason for Optimism

Ultimately, the integration of AI into cybersecurity is a narrative of optimism.

Yes, the equilibrium will shift. There will be chaotic periods as attackers test new AI tools. But relying only on defense means attackers get to choose the time and place of the next battle. By using AI to significantly speed up how we write, test, and fix software, we take the initiative away from the attackers. We make the cost of doing bad business too high.

As Dario Amodei, CEO of Anthropic, has noted, the balance between offense and defense is actually tractable in cybersecurity. There is real hope that defense can outpace attacks—but only if we actively invest in it. The tools are here. Someone must do the hard work of putting them to use for good, lest they only be put to use for harm.

Related Articles

In my last post, I mapped the architecture of the AI industry. Today, we are diving into the top of that stack to answer the industry’s most critical question: how do we separate the technological hype from real, sustainable economic value?

While all layers are important, I’d argue that at this point in time, the application layer is the most critical of all. The first reason is this is where theoretical value becomes real value. If social value isn’t created, why should anyone care about AI?

A second and related reason is this is where financial value is realized. Social value and financial value can be unaligned. When they are unaligned, society can miss opportunities and experience waste. This cuts both directions.

A third and also related reason is that as individuals we care about our own financial positions, and this layer’s financial performance will set limits on the financials of every other layer. If revenues can’t be generated here, they cannot pay off the investments already made and planned for model training, nor investments made and planned for data centers filled with AI compute capable chips.

The goal of this post is to understand those aspects. Where does (or could) revenue come from? How large are these flows, and what motivates or justifies them? In the process, we’ll learn more about how AI is changing the world today and in the future.

Who are the providers in the Application Layer?

The way I define it, the application layer will seem like it includes almost the entire industry. I’ll explain a model to differentiate the application layers of these organizations from their contributions to other layers.

At the forefront, we have the foundational model builders like OpenAI (with ChatGPT), Anthropic (with Claude), and Google (with Gemini). While they provide the underlying “Intelligence Layer,” they also act as direct-to-user application providers. Everyone wants to—and needs to—be in the application layer. All the core providers want to enable diverse users within the application layer, but cannot entrust their future solely to a developing ecosystem. They are thus both enablers and active participants.

Alongside them are the massive cloud platforms—most notably Microsoft and AWS. While it’s tempting to look at their packaged applications (like Microsoft Copilot), their true gravity in the enterprise space lies in platforms like Azure AI Foundry and Amazon Bedrock. These platforms provide the crucial API infrastructure that allows other businesses to build their own custom AI applications. Because of their sheer scale and their role in hosting these APIs, their movements dictate much of the financial reality of this layer.

This reliance on API infrastructure is perhaps the most common blind spot in the current AI discourse. Media pundits, the general public, and basically anyone not actively involved in building AI-enabled applications consistently overlook it. It’s easy to fixate on what is visible—the chat interfaces and packaged consumer tools. The enterprise API layer, however, operates quietly behind the scenes, routing data and powering internal corporate workflows. This infrastructure isn’t a secret; the documentation for Bedrock and Azure AI is entirely public. But because the average person has no reason to read it or interact with it, this critical financial and operational engine remains largely hidden from view, and will likely remain so.

Finally, beyond these hyperscalers and hidden API layers, there is a rapidly expanding ecosystem of AI applications. While pure-play startups (like Cursor, Midjourney, or Harvey) provide early examples of entirely new workflows built from the ground up, they represent just a fraction of what is possible. Ultimately, this layer is about embedding AI into existing processes and tools, making it generally pervasive over time, with many of the most transformative use cases yet to be realized.

The Consumer vs. Enterprise Split

A key starting point to making sense of the application layer—and where its revenue comes from—is to categorize usage into two segments: consumer and enterprise.

The consumer segment encompasses personal usage, while the enterprise segment encompasses usage contracted for by a business. The lines can be blurry with self-employed businesses, gig work, and shadow enterprise usage. Imperfect as the distinction is, it still serves a purpose. The consumer segment shares dynamics regarding free usage and subscriptions that are important to understand potential growth. The enterprise segment is very diverse; while it has leading workload types, those workloads share pressures to demonstrate return on investment (ROI) and are challenged by implementation and integration.

Revenues for consumer usage are generally subscription-based. Conversion from free to paid is not straightforward; while limits on free usage create pressure, they can also alienate users. In this segment, usage isn’t measured by ROI but by “vibes”—does the user feel the tool is worth the monthly fee?

Conversely, the enterprise segment defaults to metered API usage (pay-per-token). This is exactly why the API platforms mentioned above (Bedrock, Azure AI) are so critical—they form the backbone of this enterprise business model. This metered structure creates a more predictable future by correlating revenue and costs directly. However, forecasting enterprise revenue depends on how capable organizations are at closing implementation and integration gaps. In a fully rational market, metered usage would create significant profit, but if the market deploys more capacity than the demand can satisfy, we may see usage sold at a loss just to recoup sunk costs.

The Activity Value Matrix

To truly answer the core questions of this article regarding social value, financial value, and financial performance, we need another framing tool alongside the consumer/enterprise split. This is where the Activity Value Matrix comes in.

The Activity Value Matrix is crucial because it establishes a reasoning model for what type of AI activity actually brings net-positive value back to society, versus activity that merely burns compute to propel the industry forward without creating real benefit. Furthermore, it helps us identify precarious gaps: cases where AI could create immense social value, but because we lack a sustainable revenue model to support it, that value will fail to materialize sustainably. By analyzing the relationship between Value (economic/social utility), Revenue (monetization), and Usage (active consumption), we can map the true health of the application layer:

From a social perspective, we have a few rows we prefer, but the most important is productive revenue. Free goods are enjoyable, but unsustainable, and so at a society level, we should expect to eventually pay. Public goods are likewise excellent, but the arrangements are complex and apply to fewer situations. While we might wish for an endless supply of free or public goods, it is via activities with productive revenue that we’ve moved forward.

Likewise, there are some rows we should fear. Waste is clear. The exploitative patent rent is clear. General patent rent is less clear. From a simple preference, we should dislike it. But patent rent is often a useful tool to promise as a privilege. When managed well, that promise encourages investment that subsidizes activity that creates growth that could not occur from productive revenue alone.

Moving Bottlenecks and Remnants

With these tools—the consumer/enterprise split and the Activity Value Matrix—we can start to examine individual use cases. Whether driven by consumer subscriptions or the pressure to recoup enterprise sunk costs, the operational reality is that actual AI adoption is heavily challenged by implementation and integration. As this adoption progresses, we will repeatedly encounter new bottlenecks.

Many bottlenecks are simply standard operational hurdles—like migrating legacy data, reorganizing teams, or passing compliance reviews—that get addressed one by one. While solvable, they still create significant diffusion delays that push the realization of productivity gains further into the future. Another type “remnants”, on the other hand, are the most resistant bottlenecks. They are the stubborn areas left behind after the easy problems are solved.

One of the most stubborn types of remnants is adversarial. As the matrix highlights under “Compelled / Adversarial Revenue,” these actions create a continuous loop of demand for intermediate outputs just to maintain existing outcomes—like AI generating better spam, which requires better AI to filter that spam. This burns energy and resources, generating revenue and usage, without creating net-new social or financial value.

To see how these bottlenecks and remnants shape an industry, consider Coding—arguably the leading edge of the application layer. Software developers are culturally accustomed to disruption, which reduces the organizational friction and diffusion delays seen elsewhere. Yet, their use cases perfectly map to our frameworks:

• Responding to the Adversarial First: A wave of security related investment is underway, and I expect will accelerate. This early adoption is driven by compelled security needs—using AI to find vulnerabilities and harden systems before attackers can use AI to exploit them.

• Paying Down Debt: AI is being deployed to clear existing bottlenecks by paying down technical debt—modernizing deprecated platforms and freeing up locked capital.

• An Engine for Change: Because software underlies modern business, increasing developer productivity acts as an engine for change across all other industries, eventually translating to new productive revenues.

However, if development capacity isn’t solved first, consider how a bottleneck might manifest as an increasing “Proposal-to-Product” ratio. If AI halved the time needed to create software feature proposals of twice the quality, we might describe it as a 400% efficiency improvement. We might also see twice as many proposals. However, if the capacity to actually write the software hasn’t increased proportionally, that 400% internal productivity boost might only translate to a 10% increase in final value.

If we’re not careful, we might incorrectly diagnose the proposal process as the problem. We might even blame AI and suggest all of those new proposals were “AI slop”. While it’s possible for tools to be misused, it would be ironically sloppy to jump to that conclusion. The overuse of AI slop is already a problem, so we should not add to it. If users decrease their effort by 10x, and get half the quality, then yes, we can call their output slop. But that would have been true if they decreased their effort by half without tools.

The real story here is of the bottleneck. The unused proposals aren’t necessarily low-quality slop; they are simply piling up behind a bottleneck further down the line.

Conclusion: The Reality Check

The Application Layer is the industry’s “Reality Check.” While the lower layers (Compute and Intelligence) have seen massive investment and technical breakthroughs, the Application Layer is where those breakthroughs are forced to justify their existence.

The distinction between the Consumer and Enterprise segments is vital to this justification, as it dictates the very survival of business models. In the consumer world, the value is personal and often ephemeral, driven by individual preferences and “vibes.” Here, adoption can be viral and instantaneous, but loyalty is fickle and valuations are tied to user engagement. In the enterprise world, value is rigorous and must survive the gauntlet of organizational change and ROI calculations. While adoption is slower due to integration hurdles, the resulting business models are often more durable and command higher valuations based on proven efficiency gains. While many use cases—like coding or creative generation—will exist in both segments, they will take on different “shapes” and adoption velocities based on these divergent preferences.

As we look forward, the specific tasks—the “Use Cases”—will define which segment wins and how much of that massive investment in lower layers can actually be recouped. In my next post, we will dive deeper into the Activity Value Matrix to see exactly where this “productive revenue” is hiding, how the shapes of use cases shift between consumer and enterprise, and where energy is simply being burned as “waste.”

Introduction

If you look solely at the headlines in 2025, the artificial intelligence industry appears like a single coordinated effort with unprecedented wealth. We see trillions of dollars in valuation, feverish construction of data centers, and a frantic race for silicon. But to view AI as a single, unified industry is to miss the mechanics of how capital flows, and how that creates stability in some places, and precariousness elsewhere.

The reality is a stack—four distinct layers of value, each operating under different laws of physics and economics. At the bottom is the concrete reality of energy and silicon. At the top is the promise of productivity through software. Between them lies a complex web of subsidies, speculative bets, and “soft dependencies” that are challenging the laws of traditional business.

At the base, capital is being aggressively spent on lower layers of the stack, to both meet current and expected future demands from higher layers. Higher layers are operating speculatively, bringing in capital through speculative investments and running at a loss, in expectation of unprecedented future revenue growth.

This creates a commercial stability gradient that decreases as you move up the stack, where revenue covers less of investment and depends more on future growth.

• The Compute Supply Chain (Layer 1) is paid first. While future demand, and thus revenues cannot avoid a dependence on the higher layers, they are also selling hard product today, and being paid well for it. As businesses, they are more stable, though you should be careful about translating that commercial stability into assumptions of stock price stability, which assume both high demand and high margins, neither of which is certain.

• The Operational Infrastructure (Layer 2) is mixing revenue from higher layers and cash flows from existing business to fund purchases from Layer 1. This layer is competitive even before AI, and AI’s influence may make it more commodity-like and more competitive. Investment is speculative, but structured for stability.

• The Intelligence (Layer 3) is precarious. Only a fraction of spending is covered by revenue; the remainder is funded by investment capital and debt, driven by high optimism. While capital costs are not as high as Layer 2, they are still high, and so are operational costs. Almost all this goes to Layer 2 as current revenues. Cash flow is negative and dependent on ongoing investment, while also being highly competitive.

• The Application (Layer 4) is diverse, and where value is realized. Other layers depend upon this to bring the revenues that pay for capital and operational costs. The risk here is an Attribution Gap. Real value is created, but measurement it is difficult. This leaves uncertainty if users will pay enough to justify the massive investments at lower layers.

This structure creates an unstable relationship. If spending in the Application layer doesn’t expand rapidly, the investment optimism funding the Intelligence layer could dry up, leaving a critical gap in the revenues needed to pay back investors and meet the ROI expectations of the infrastructure below.

An interesting exception to this structure is application layer companies can be more commercially stable than Layer 3 companies. If they demonstrate their own value to customers, and decouple via negotiation with Layer 3, they could reach profitability before Layer 3.

As a diverse layer, many different outcomes should be expected, with some companies successfully navigating their own path and others failing. Also, while this shifts more everyday, a reasonably significant part of this layer (25%) is direct OpenAI/Gemini/Anthropic user subscriptions, coupling that revenue to Layer 3.

Layer 1: The Compute Supply Chain

The foundation of the AI stack is not code, but the “hard” reality of atoms: the physical and logical inputs required to create intelligence. This layer is dominated by the high capital cost and expansion timelines of chip manufacturing, which feeds into the economics of the entire industry.

Foundries, such as TSMC, represent the hardest constraint and the largest capital commitment. TSMC alone is projecting $40-42 billion in CapEx for 2025. This creates a “pervasive price” because chip manufacturing costs factor into most other costs. Training needs chips, and inference needs chips.

While foundries deal with the brute force of fabrication, chip designers like Nvidia and AMD operate the Efficiency Lever. Their economic role is to maximize the usage of scarce foundry capacity. However, the lines here are blurring. Hyperscalers like Amazon and Google are increasingly designing their own silicon—Trainium and TPUs—to optimize their own costs, effectively internalizing the supply chain.

Surrounding this is the often-overlooked constraint of utility inputs: energy and real estate. All compute requires reliable power, making grid access a key determinant of land value.

One of the advantages for NVidia and TSMC’s is that chips are being sold today. TSMC may have high CapEx, but for 2025 has $120 billion in revenues, and $40 billion net income. NVidia has an even more appealing 2025 forecast of $115 billion net income and revenue of $200 billion, with only $6 billion in capital expense.

$115 billion in income doesn’t justify a $5 trillion market capitalization, it has to grow to support that, but as a business there’s no lack of commercial stability. It’s also a mistake to look at NVidia’s favorable position, and imply stability for the entire AI industry. That just means the rest of the industry is paying for NVidia, it doesn’t mean the rest of the industry can afford to.

If demand continues to grow this layer’s hard constraints—manufacturing capacity and power availability—will give it pricing power. If it flatlines, that’s lost, but investments will still be repaid. Demand would have to pull back significantly to translate into internalized losses. Stock prices could be volatile, and bad choices there could cause individual investors to experience losses, but that’s partly the nature of stock markets in general.

Layer 2: The Operational Infrastructure

Disclaimer

Now is a good time to remind you that while I work at AWS, this is a personal Substack. Opinions are my own.

Sitting directly above the supply chain is the Operational Infrastructure, defined by the delivered compute. This layer efficiently deploys chips using existing cash flows, and keeps them operational. Very CapEx dependent, but so far, self-financing. The primary examples are the hyperscalers: AWS, GCP and Azure. These companies are absorbing the massive AI infrastructure costs using cash flows from mature, non-AI businesses like search and retail. A full accounting must consider smaller providers and private deployments.

We are witnessing a classic Installation Period: a rapid infrastructure overbuilding driven by FOMO and strategic necessity. Similar to the fiber-optic boom of 1999-2001, physical assets are being deployed ahead of proven demand. While this typically leads to a capacity glut, such a glut is eventually beneficial for the economy, as excess capacity drives costs down and subsidizes the next wave of innovation.

In my follow-up, I’ll dive into

1. The CapEx numbers from each entity

2. The CapEx breakdown by type (chips, land, buildings, etc.)

3. The differences between projected CapEx, announced CapEx, commitments, and actual spending.

4. The revenues from supporting training, inference.

Layer 3: The Intelligence

The third layer is the speculative core of model providers. The current core are those with frontier model programs, such as OpenAI, Anthropic, Google, Amazon, Meta. While this layer captures the public imagination, its economic footing is far more slippery than the infrastructure below it.

These companies face a brutal paradox: they provide the cognitive engine for the entire ecosystem, yet “intelligence” itself is trending toward a commodity.

In this high-velocity environment, legacy advantages are fleeting. A leader can be dethroned by a single release cycle from a competitor. Unlike the infrastructure layer, where assets have a useful life of years to decades, a frontier model can depreciate in months.

Changes in position happen often. At the moment of writing, December 2025, OpenAI is not the leader, and it’s open to debate between Google Gemini and Anthropic Claude Opus/Sonnet as the momentary leader. You’re only as good as your latest model on the frontier. Niches are developing, on a cost basis and use case basis. But even here, a niche can be swallowed by the new frontier release, or lost to a new release targeting the niche.

A long term advantage here would have to be founded on execution: creating effective models cost-efficiently. The most optimistic view is that a recursive feedback loop emerges. If AI improves model architecture via an algorithm loop, or AI improves chip design, these create novel impacts.

A chip design loop would be slow and shared, as a new chip design would have to go next to manufacturing, and dependent on chip designer intellectual property. An algorithm loop could be faster, and self-contained. While this area is fairly speculative, it’s a foundational one to how many model providers think, at least at the fringes.

Financially, this layer is precarious. Revenues are not large enough to pay for past training costs, so companies are still unprofitable. Internally they face a revenue gap where growth projections often presume exponential conversion of free users to paid users, without a proven reason for that assumption. They also are dependent on the realization of many other revenue sources from the application layer.

In my follow-up, I’ll dive into the revenues and expenses at this layer.

Layer 4: The Application

The most critical disconnect in the modern AI economy sits at the very top: The Application Layer (Layer 4). This is where the rubber meets the road—where enterprises and consumers actually use AI to solve problems.

In a healthy market, the revenue from this top layer would cascade down, paying for the models, the servers, and the chips. Today, however, that flow is a trickle compared to the flood of investment rising from the bottom. We are facing an attribution gap. While AI creates real value—writing code, summarizing documents, speeding up research—measuring that value is notoriously difficult.

Besides the attribution gap, there are other frictions, an implementation gap, an integration gap, and an adoption gap.

Before users can make use of the potential capability of a model, for a specific purpose, they often need an implementation. Consider coding and coding tools as an example. Before coding tools, like Kiro, supported spec driven development, the ability to use models for this purpose was challenging. The same goes for earlier advancements, such as generating tests. Sure, you could in theory have done this with a chatbot and a lot of copy and paste, but that’s not good enough to make it practical. The implementation takes theory to practical. The number of implementation gaps, between what the best models could in theory do, and what users have good tools to do, is growing faster than it’s shrinking.

Even with an implementation, another type of gap is an integration gap. Intelligent decisions depend upon access to relevant and current data. Data does not automatically become integrated. For good reason, data has to go through the process of integration, where security, quality and structure have to be resolved. And it’s not just data, but also actions, which quite reasonably have a yet higher bar for proper integration.

Finally, there’s the adoption gap. Even when a tool or process is available, and integrated, skepticism, awareness, and habits often mean it’s not used. There’s a relation back to the attribution gap here too, as the inability to attribute value helps fuel skepticism.

The attribution gap, and the other frictions slow down both the actual implementation, and the creation of revenue. This results in a lag effect. The industry is currently in a race to see if revenue can expand fast enough to justify the valuations before the Installation Period speculation cools.

In my first follow-up (The AI Application Layer), I dive into:

1. Two types of revenue, consumer and enterprise, and their different frictions. A preview here; this is changing rapidly and the enterprise is becoming more important.

2. Different types of value-revenue relationships, and their importance to delivering social value from usage.

A future follow-up, will dive into tasks that can create revenue, and their relationship to value.

Macro-Economic Effects

I’ve written about this a bit in the past…

In my follow-up I’ll dive into:

1. How the current state of the AI industry feeds into the overall economy

2. How different types of revenue have different effects on the economy separate from the effects on the AI industry.

Conclusion

The bottom of the stack is being paid for by upper layers. The largest contribution comes from the internal investment by the infrastructure layer. But this is reaching its limits. Some additional capital is being supplied by the intelligence layer, in the form of external investment spent on training and inference in excess of their own revenues. Finally, the application layer is bringing in revenues. Those revenues wouldn’t be less than enough to pay back CapEx to date, and would be trivial in comparison what would be needed to pay back projected CapEx.

Thus, the current internal and external investment is absorbing the costs of infrastructure in the belief that the application layer will eventually catch up. If projected CapEx is to occur, new sources of investment will be necessary, likely through debt and larger amounts of equity financing. A drop in optimism could cut these plans short and create market volatility.

This is not necessarily a disaster; it is a historical pattern. We are deep in what economists call an Installation Period. Much like the fiber-optic boom of the late 1990s, we are overbuilding infrastructure ahead of proven demand. This is a feature, not a bug, of technological revolutions. This overbuilding is inflationary and chaotic, driven by Fear of Missing Out (FOMO) and strategic necessity.

The eventual result is almost always a capacity glut. It’s not a foregone conclusion, and pinpointing the timing is a much harder prediction than the simple occurrence. If you forecast on potential value only, and ignore the attribution, implementation, integration and adoption gaps, it’s almost certain to occur.

In theory, good forecasting can forestall or avoid a glut. In practice, there’s usually at least one market participant that’s constitutionally inclined to pursue the most optimistic interpretation, and FOMO pulls others behind this wave. Lagging behind too much when optimism prevails risks being left out with a struggle to regain position. In other words, playing it safe isn’t always safe.

When optimism finds its failing point stock valuations correct violently. But for the economy at large, this is the “turning point.” A crash in the cost of intelligence would reduce the barriers of an attribution gap, spawning a burst in adoption that needed to unlock a Deployment Phase. With past revolutions this is where the technology becomes cheap and reliable enough to be woven into the fabric of everyday life.

With AI, we might wonder if the model of the railroads, which experienced serial corrections, could fit better as one correction is not enough to close all of the potential gaps.

For now, the AI economy is a structure supported by massive financial scaffolding. The heavy lifting is being done by legacy profits and speculative investment, all betting on a future where the friction of adoption disappears. The question is not whether AI adds value, but whether the application layer can expand quickly enough to catch the weight of the massive infrastructure being built to support it.

Next in this Series

In the coming posts, we will peel back the layers of this stack one by one, examining the specific data and market dynamics driving the Compute Supply Chain, the precarious economics of Model Providers, and the reality of Enterprise Adoption.

Currently, if I want to read just one specific essay from a writer I don’t subscribe to, I usually can’t. I hit the paywall and leave. That’s a loss for me, and it’s money left on the table for the writer.

I think there is a middle ground that would help readers explore more independent work while putting more money in writers’ pockets: The Reader Pass.

How It Could Work

The idea is simple: a way to support writers individually without committing to a marriage. It acts like a pre-paid tab for the whole platform with volume discounts built in.

1. Set a Budget: I set a monthly limit I’m comfortable with—say, $10.

2. Pay-Per-Post: When I want to read a paywalled post from a writer I don’t subscribe to, I can “unlock” it. This deducts from my budget.

3. Bonus Tiers: Once I hit my limit (e.g. 10 articles for $10), I don’t just get cut off. I unlock a “bonus” set of free articles.

4. Scaling Up: If I exhaust my bonus articles, I can approve the next tier. Each subsequent tier offers significantly better value (more articles for the same price), plus its own set of bonus free reads.

Here is how the value could scale as a reader spends more:

This structure incentivizes heavier reading. The deeper you go into the ecosystem, the cheaper it becomes to explore, removing the hesitation of “is this one article worth it?”

Why This Helps Writers

The biggest worry writers might have is, “Will my subscribers downgrade to this?”

I honestly don’t think so. True fans want the community, the archives, and the direct connection of a full subscription. The Reader Pass is for everyone else—the casual readers who are currently paying $0.

• Earn from Casuals: Instead of bouncing off the paywall, a casual reader pays $1. That adds up.

• Find New Subscribers: This is a great way for readers to “date” a writer before marrying them. If I find myself spending $5 a month unlocking a specific writer’s posts, it becomes an easy decision to just switch to a full subscription.

Why It’s Good for the Community

Right now, Substack can feel a bit siloed—we stick to the writers we already know. A Reader Pass turns the platform into a library where we can wander the stacks. It encourages us to take a chance on a new voice or a niche topic we wouldn’t normally pay for.

Respecting Independence

Of course, independence is the whole point of Substack. This shouldn’t be forced on anyone. If a writer wants to keep their work exclusive to full subscribers, they should absolutely be able to opt out.

Conclusion

The subscription model is amazing for supporting our favorite writers, but it creates a wall for everyone else. A Reader Pass would smooth that out, helping curious readers support more independent voices without the pressure of a full annual commitment.

Let’s Make It Happen

If you agree that a Reader Pass would make Substack better for everyone, let’s make some noise. Share this idea with your favorite writers, or tag Substack to let them know we’re ready for a middle ground between “all or nothing.” We want to support more writers—give us the tools to do it.

Democratic Leadership has been taking flak for the outcome of the shutdown standoff. While that’s not too surprising, I think it should find a different conclusion. To frame that context here, I’ve been unsure about engaging in the shutdown from the start. I’ve been a bit more supportive this time around, as there was a clear outcome the Democrats were seeking here, and it did seem worth a stand. All that said, as recently as Sunday I was saying, if I was in congress, I’m not sure what I’d do. My reason for that is that the shutdown was causing real impacts to people, and it was getting worse.

It’s one thing to make a stand, and put your political reputation on the line, it’s another to do it when people you’re trying to protect are being hurt. I’d want to be sure it was going to succeed, and ideally, I’d want to know those that were at risk, were willing to take the risk themselves. If I had been a senator and broken ranks, that would have been the reason, and would have been how I explained it.

When you look at the explanations of those that broke ranks, their explanations aren’t very dissimilar.

“We have airport controllers. And we were seeing lines to our food banks in northern Nevada. These were lines that I hadn’t seen since the pandemic.”

I’m going to suggest that Democrats shouldn’t be spending their time attacking each other right now. Yes, there is a reason to want the group to act as one, and so I understand the impression that this is a failure. But on the other hand, giving in because you care, that’s something you should consider forgiving. More than that, the real story here should be, the Republicans were willing to throw America under the bus to take away healthcare. The separation between Democrats that were willing to take a stand to protect people’s healthcare and those that were willing to take a drubbing from their own party to protect people harmed by the shutdown, shouldn’t be a division you can’t get past. It’s two sets of good intentions, with a hard choice.

Besides being true, that statement is also one every member of the Democratic party should want to reinforce amongst the public, who were largely on their side throughout the shutdown, so reasonably could be expected to understand how terrible the Republican position is. Additionally, if you value unity as a party, I think this is just the moment to take the initiative. Punitive statements and in-party fighting aren’t necessarily more useful for developing party discipline than recognizing the common cause and acting accordingly.

The bursting of bubbles based on sentiment is not very predictable. For one, when optimism gets into high gear, it’s usually tied to underlying dynamics that carry a lot of uncertainty. Who really knows what work AI will and won’t be able to do, and more importantly, when?

Second, since bubbles are generally based on predictions of future performance, there’s little immediate forcing function. If the current quarter’s numbers come in low, but your target is 5 years away, it’s not that hard to maintain your 5-year optimism. Shrugging off short-term evidence as non-relevant to the long-term outcome has some sensibility.

Third, even someone who knows enough to be certain that optimism is too high will be tempted to hang on and try to time their exit to take best advantage of others’ optimism. This will delay the bursting of a bubble. This dynamic can’t last forever, but it complicates timing significantly.

With this in mind, too much weight may be placed on the idea that a correction would start within the realm of tech and AI investments. This type of correction isn’t the only one we should be worried about, though. A different, though highly complex, scenario is one that shows a smooth connection from point A (now) to point B (the 5-year optimistic view) isn’t possible, because the “main street” economy that supports it begins to buckle.

The Stagflationary Shock

This presents a concerning story: Tariffs create extra costs, and tariff policies create uncertainty that reduces “main street” business investment. Job growth slows as a result. At the same time, crackdowns on immigrants cause business closures and slower business formation in areas that drew workers from immigrant populations, such as farming, restaurants, and residential or informal construction (like renovations and single-family homes).

This scenario creates two powerful, opposing economic forces simultaneously:

An Inflationary Supply Shock: Tariffs (a tax on goods) and labor shortages (raising wage costs) reduce the economy’s productive capacity, pushing prices up.

A Recessionary Demand Shock: Business closures and slowing investment reduce aggregate demand, putting downward pressure on prices.

This aggregate balance is further complicated because these forces may not be uniform. The inflationary pressures from tariffs are broad, while the recessionary effects of workforce loss and business closures are localized to specific industries and regions. It’s possible for the economy to experience both shocks simultaneously without them canceling each other out on a national scale.

Individuals could absorb the pains from those effects by using savings or by taking on more debt, but they’ll also probably stop spending as freely.

When spending slows, it affects retailers, which indirectly slows advertising revenue, or directly slows online sales.

Risk One: The Disinflationary Bust

The first risk is that these forces don’t simply cancel. The inflation from tariffs itself acts as a tax, slashing consumer purchasing power and causing a recession. In this feedback loop, an initial inflationary spike is followed by a recession that becomes so deep that it overwhelms the inflationary pressure from tariffs. Demand is so thoroughly destroyed that businesses cannot pass on their higher costs and instead reduce production and employment. This would not lead to stagflation, but to a deep disinflationary bust—a severe recession where inflation rapidly slows, or even turns negative.

This specific risk—a disinflationary bust originating from “main street”—is one the U.S. economy has arguably been facing. It has, so far, been largely avoided or masked by a powerful counter-force: a massive wave of investment from tech companies and rising stock market optimism fueled by the promises of AI, which has provided a floor for the wider economy. This reliance, however, creates a fragile co-dependence that leads directly to the other major risks.

Risk Two: The Credit Competition Crisis

This fragile balance, propped up by AI, is still dependent on the “main street” consumer. While they have been absorbing the pain from inflation and job slowdowns by using savings or taking on more debt, this cannot last forever.

The second major risk begins when this consumer resilience is exhausted. As “main street” spending contracts significantly, it directly hits the revenue of the tech companies themselves—slowing advertising, e-commerce, and enterprise sales.

This creates a new, dangerous dynamic. Tech companies, which had been funding their massive AI investments from free cash flow, now see those cash flows dry up. They are forced to turn to debt markets to continue investing. Suddenly, these tech giants are in direct competition for a limited pool of credit with the very consumers and “main street” businesses that are also desperate to borrow to survive. This competition for credit would cause interest rates to spike across the board—even without government action—threatening to choke off both the AI boom and any hope of a “main street” recovery.

This is the point where the crisis becomes systemic and demands a government policy response, leading to the next set of risks.

Risk Three: The Crisis Apex and Policy Dilemma

Faced with the systemic credit crisis from Risk Two, the government is forced to intervene. At this apex, the economy is balanced on a knife-edge, with two new, opposing market dynamics coming into play, even as the government weighs its options.

On one hand, there is a slender thread of hope: AI itself might finally start returning tangible, widespread value to “main street,” allowing new business formation and employment to grow again. If AI-driven productivity gains were to suddenly make “main street” businesses profitable and credit-worthy, it could provide a market-based escape. While the dynamic is simple, the reality of this is unclear, as it’s not known what this value would look like or if it could arrive fast enough.

On the other hand, this is the exact moment that AI optimism itself might end. Faced with cratering “main street” demand and a real credit crunch, investors might finally decide the “5-year optimistic view” is no longer plausible. This wouldn’t be a solution, but a different, compounding dynamic: a full-blown tech correction on top of the “main street” recession, leading to an immediate crash (similar to Risk One, but far more severe).

Barring the slender thread of hope, and assuming AI optimism holds just long enough to influence policy, the government must still act. Its options are severely limited by the original stagflationary shock: inflation (from tariffs) is still high, even as all sectors are desperate for cheaper credit.

In this scenario, the government might still try to counteract the recession and credit crunch by lowering base interest rates. This choice, however, creates a new, highly unstable set of risks.

This policy creates a bifurcated economy. It doesn’t mean credit only benefits AI, but rather that in this environment, banks engage in risk-based credit rationing.

This policy is highly unstable. It fails to help “main street” (which is too risky to lend to) while simultaneously fueling an asset bubble and adding inflationary pressure (by pumping cheap money into the “hot” AI sector).

Risk Four: Path A - The Politically-Driven Hard Landing

Faced with the unstable, inflationary, and politically toxic situation created by Risk Three, policymakers arrive at a critical fork in the road. The first, and most economically orthodox path, is the hard landing.

The policy in Risk Three is politically toxic. A government facing an electorate where everyone is moderately affected by inflation—a universal and highly visible political poison—is far less likely to tolerate it than unemployment, which affects a smaller group severely.

The political pressure would be immense for the Federal Reserve to do the opposite of Risk Three: keep interest rates high (or raise them further) to crush inflation, even at the cost of deepening the recession. This “Volcker-style” hard landing, driven by political necessity, would itself trigger the tech correction by making financing prohibitively expensive, leading to a bust in both tech and “main street.”

Risk Five: Path B - The Idiosyncratic Low-Rate Gamble

There is, however, a second, less orthodox path. An idiosyncratic political administration, one more concerned with short-term stock market performance and headline employment figures than long-term price stability, might attempt to force the Federal Reserve to keep interest rates low, despite the stagflationary pressures.

This path involves rejecting the painful (but historically validated) lesson of the 1970s and 80s, and instead gambling that the economy can “grow its way out” of inflation. In this scenario, the government essentially accepts the unstable dynamic of Risk Three. It continues to fuel the AI asset bubble with cheap credit while hoping for a productivity miracle, all while persistent inflation erodes the value of “main “street” savings and wages.

A Choice Between Two Harmful Paths

Neither fork in the road leads to a happy state. Path A (Risk Four) is the orthodox, “Volcker-style” hard landing: a deliberate and deeply painful recession. It is a harmful path that would cause significant unemployment and business failures, but it is a known strategy designed to crush inflation and prevent an uncontrolled spiral.

Path B (Risk Five) is an unorthodox gamble that likely leads to a worse fate. If President Trump chooses this second path—cutting and holding rates during stagflation—the scenario ends grimly. The policy would fail to stop the main street recession while runaway inflation, now unchecked by the central bank, takes hold.

This onset of runaway inflation would eventually force the government’s hand anyway, compelling them to aggressively increase base interest rates after all. This U-turn would force AI investment to finally slow, but only after significant economic damage has been done. In the end, the economy would be a weaker one, with more inflation, more unemployment, fewer productive businesses, and a complicated unwinding of debts.

How likely is this?

The likelihood of this scenario appears uncomfortably high, as the key premises are not hypothetical but are already visible in late 2025 data.

1. The Stagflationary Shock is Underway. The “Stagflationary Shock” is not a future risk; it is the current reality. Job growth has stalled, averaging just 29,000 per month over the last quarter (near-recessionary levels) and showing “little change since April” according to the Bureau of Labor Statistics. This is happening while inflation remains “sticky” at 3.0%, fueled by tariffs. This combination of weak growth and persistent inflation is the textbook definition of stagflation.

2. Tech Vulnerability is Real. “Risk Two” is also proving plausible. The assumption that Big Tech would fund the AI boom from its own cash reserves is incorrect. Tech giants are already turning to debt markets. Recent reports show AI capital expenditures are approaching 94% of operating cash flow, forcing massive bond sales, including Meta’s recent $30 billion offering. This confirms their new vulnerability to a “main street” downturn. That’s not to say that tech itself is at risk of collapse, but more that they aren’t insulated from the rest of the economy.

3. Fed Independence is Under Attack. “Risk Five,” the “idiosyncratic low-rate gamble,” has become a tangible political risk. The Federal Reserve is under “immense pressure” from the administration. The recent appointment of a political adviser, Stephen Miran, to the Fed’s board—who is already dissenting in favor of the larger rate cuts the White House desires—demonstrates that the Fed’s traditional independence can no longer be taken for granted.

Because the three critical links in the chain—the stagflationary shock, the tech debt reliance, and the erosion of Fed independence—are already in place, this alternative path to a correction is not just a theory, but an unfolding reality.

What should we do?

Confronting the question of what would reduce these risks, the most direct path is to remove the initial stagflationary shocks—the tariff policy and the workforce disruptions from restrictive immigration policy—which are the root cause of the entire risk cascade. Tariff policy is easily undone, especially if it comes with a commitment to not make that mistake again. Immigration policy shifts should be more forward-leaning, creating an environment for orderly, legal immigration that meets workforce needs.

Reducing our risk by these means is the most obvious and sound policy. The question though that arises is, what could convince the Trump administration to follow through with measures that contradict the largely irrational policy it’s pursued so far?

Postscript: Quantifying

The scenario here, it’s plausible, but not quantified. Job numbers, cash flows and attacks on Fed independence, do neutralize the strongest counterarguments. That still doesn’t make this a quantified scenario, and without quantification, the timeline is unknown. Anytime a timeline is unknown, you have to also consider the possibility that the timing is “too far away to matter”.

While there’s a solid story that main street weaknesses could overwhelm optimism, financial tricks, and tech’s contributions, it’s also possible these prop up main street and slow main street erosion. In that time, policies could reverse. Maybe it could even be so long to allow new elections and new policies from that.

It would be nice to have numbers. Numbers are not easy though. Quantifying this scenario presents a significant challenge. I have no intention of attempting a full econometric model of the U.S. economy, which is the minimum that would be required to know the outcome of the interaction of these forces. I’d watch for others taking on that work, but do you really need to wait?

I do have the idea that I can take a deeper look at the tech economy, and think through how the specific companies or layers would be impacted by changes in wider economic behaviors. That’s an interesting question all of its own, but I also think it helps understanding how powerful feedbacks would be. This is still a big topic, so it will be a future post.

The idea that AI has been necessary for slop is a lazy one of its own. The more and more we use the term AI slop, the more I see two mistakes happening. First, we forget that non-AI slop hasn’t gone away; in fact, the same incentives that created it are now simply being supercharged by new tools. Second, there’s a lazy classification of everything AI related as slop. Ultimately, these two mistakes will lead to a failed understanding of the problem, preventing us from developing effective solutions.

There’s protections against slop already, though we can all attest that these haven’t fully protected us from slop, otherwise most of us wouldn’t even know what it was. But we do. Will the addition of AI radically change this? I think it’s not going to be all that dramatic. I know that’s in contrast to the common belief that it’s going to be an unstoppable tidal wave that creates the “end of the Internet”.

I believe the panic is overstated for three reasons. The first is, AI can improve the protections. While it’s also useful to avoid the protections, it’s hard to say which is going to be the more powerful force. The second is those protections, they already set a status quo which is more stable than is generally acknowledged.

The third, and most powerful reason is, slop has intent. You don’t have to protect against slop by directly focusing on the content, but can also focus on the intent. The motivation that goes into the intent isn’t endless either.

Let’s think about the protections we have. It’s gone unacknowledged that multiple platforms have thrived on slop. Facebook, Tiktok, etc. What is the majority of the content there other than slop?

At this point, I have to take a minor tangent. Since we define slop as low-quality, we have to acknowledge that quality is in the eye of the beholder. You can identify factual details and use them in a debate about quality, but only when there’s an agreement on goals. It’s a generally acknowledged goal of most products that they don’t fall apart (though some, like toilet paper, must be designed to fall apart at the right time). But what is entertaining is far less agreed upon. What is informative is somewhat in between.

These platforms use algorithms to prioritize content, using user activity as a key input. However, their goal isn’t to find ‘entertaining’ content; it’s to find ‘engaging’ content—anything that keeps your eyes on the screen longer so they can show you more ads. While the two sometimes overlap, they are not the same thing.

It’s close enough that if you ask a representative of Facebook or TikTok, they’ll probably claim they are trying to prioritize entertaining content. However, their systems receive clearer signals about engagement than entertainment. The fact that this easier approximation is closer to their business objective, is a coincidence. A coincidence that they’re happy to embrace, but a coincidence nonetheless.

The point is not to defend social media platforms. I actually have little interest in them myself, and think their objectives and thus actions, have some problems. The point though is that they’ve somewhat tamed slop to their purposes. This gives a reasonable cause for hope that if you choose a different purpose, you can tame it there as well.

Recognizing slop can be done by looking at the quality, but that’s time consuming compared to some shortcuts. Most slop will carry its intent with it, and can be recognized by looking for tell-tale markers of its intent. Is a page littered with 1,000 ads, such that if there was any value in the base material, it’s rather hard to find? Well, it’s probably slop. Is it repeating some lazy conspiracy theory? Slop.

The interesting thing about the anti-slop opportunities that modern AI tools promise is they can listen to us. This does depend on us retaining control, which we mostly gave up in the last run of algorithm deployment.

When preference algorithms were first deployed, they used likes, dislikes, and correlational data. In Pandora, we indicated music we liked, and Pandora contributed data about music patterns that would help find the patterns in the likes and dislikes. We didn’t have a lot of control here, but at least there wasn’t much in there other than our goals.

Later, as platforms gathered more user data, they began assuming a correlation between your choices and those of similar users. But things got really messy when they started measuring subconscious actions. Suddenly, metrics like how long you spent scrolling or how your cursor lingered over a post became key inputs for the algorithms. After that, the algorithm optimizers started to optimize for these aspects. Initially this helped grow those platforms, but slowly users started to realize that more of their interactions with these platforms weren’t serving their own interests, at least not fully. At some level, it was satisfying something, else, why would they keep coming back? But we’re not immune to doing things we regret, and regrets have started to pile up.

With this history, I can see why someone worries about another iteration of AI assisted content selection. That said, there’s some differences this time. Large language models are pretrained as general purpose tools. Intents as specific as guiding you to particular content aren’t part of the training. This presents the opportunity to take more control. If you wanted to explain to Facebook what you were interested in, you could choose interests from a list, but how those would affect outcomes was in the control of the algorithm designer. In theory, we might have been offered the ability to write our own algorithms, but this was never in the business model for Facebook, and besides most users would have never mastered it (though we might imagine a world in which enough did to share their algorithms with other users).

With GenAI tools, the genie is out of the bottle already. Preferences can be described in natural language. That’s still less trivial than it sounds, but it’s something that average users can use trial and error to gain enough experience to get something useful. Facebook still may never offer you this, so the social world may have to wait for a transition to something more open.

While a solution for the walled gardens of social media remains elusive, there is opportunity in other environments. This is where users are active participants, not passive consumers, and in decentralized spaces with competitive services that prioritize their own discoverability. Examples are the world of news, opinion writing, and academic research. In these cases, the data will be easy to consume, allowing production of a feed that serves your interests. Imagine telling your news aggregator, ‘Show me articles about urban planning, but filter out any that are just clickbait or designed to provoke outrage.’

While this leaves the problem of the passive user unsolved, the problem of matching our interests there is more complex. In one sense, providing entertaining content by algorithm, is the user’s interests. That is not the healthiest choice when overused, but it’s not one we’d typically deny a user a choice in. Providing better alternatives, which a better feed would do, is probably the most attractive, though not the easiest way, to draw users away from that pattern.

While the term AI slop does have a real meaning, and does describe something that will create annoyances, the casual overuse misses the point of describing slop in general. Yes, there will be more slop, and yes, we’ll need to be more active in filtering it out.

My intent here is not to apologize for AI generated slop, or slop in general. Yes, we will give some credit based on intent. If someone tries to create something great, but fails, we don’t want to discourage future efforts. But if it’s commercially motivated, or criminally motivated, we shouldn’t think twice about asserting our interests. The greater intent is, let’s not panic, and let’s be as specific as we can.

But it’s not a hopeless battle, and lazy use of “AI slop” as a term for the purpose of creating antipathy to AI in general misses a real opportunity: to use these new tools to reduce the impact of slop—AI-generated or not—and at the same time reassert our own intentions, rather than a platform’s.

This is not a discussion of an AGI world. I explain why in “Why should we sometimes avoid talking about AGI?” near the end.

I’m also not talking about what people are doing today. I’ll leave that to the journalists and researchers who compile such data. OpenAI has some good data here, and so does Anthropic. A lot is being done, and I’ll mention this when it’s relevant to what I’m writing about, but I don’t intend to present any new data of my own or do a comprehensive review of what’s available today.

The part I’m interested in is between AGI and tracking usage. Using the high level capabilities available today, what are the incremental steps that make the best use of it? I want to constrain myself to talk about practical things, but also be exploratory.

As I see it, this part tracks back to data. AI models, like any type of reasoning, thrive on data. The models in use today, they did utilize a massive repository of data to enable them to perform what they do. There are questions this has raised, like how creators of data (which in this case all includes creative outputs, not just data in a scientific form) are able to ask for and get compensation for the use of their creations. That’s a big topic all of its own, where there are both unresolved questions today, and questions from the past that have taken on another level of relevance.

I’m not trying to focus on this use of data. Training of AI models will continue, but a lot of data isn’t going to be part of that. Sometimes it’s about privacy, but often it’s about recency. There’s also the thought about relevance.

Privacy here has a lot of dimensions when we get past thinking of ourselves alone. It’s one decision regarding how something I know about myself gets used, it’s another question about how something I know about you gets used. Add to that organizations you or I are part of, and that serve us, or that serve organizations we are part of and it gets complex quickly.

If we think about how we’d want to interact with all of this, I think a common baseline might go something like this. We’d like the people and organizations that serve us to do so intelligently, which does require them to be informed. We’d like to not repeat ourselves, or be burdened with the responsibility to inform every interaction. We’d like for people or organizations with our data to only do things that serve us.

That last point does lead to tension. We can’t honestly expect people or organizations to only serve us. In the extreme form, if someone does something bad to someone else, being unable to use a video of a public space to show the presence or actions of the first person, would to understate things, seem harmful. So there’s a clear need to not be absolute about people’s data only being used for them.

There’s some balance here, and that’s a great topic I don’t think I can adequately cover here. But one thing I can pull out of that topic is that the balance often incorporates how personal a set of data is. My appearance in a public space is personal, but it’s as personal as my appearance in a private space.

AI and Private Data

I think I’m saying something non-controversial if I say that there’s many tasks we could not expect to be done, either by AI or by human, without access to data. I cannot respond to a customer’s concerns without access to the email where those concerns were voiced.

One of the most major challenges with AI adoption beyond what it’s adopted so far, is getting data where it needs to be, when it needs to be there, while also ensuring it retains privacy that it should have. Without this data, the tasks dependent upon it are not automatable. But this type of data has long been difficult to handle. Even without AI systems, ensuring that private data is kept private, and used only where required. This is true whether it represents a customer's personal data, or represents a company's internal data.

As a direct user of AI platforms, you can handle some of this yourself. You can attach a document, or copy and paste it in. But for an AI system to be able to perform many operations, that’s impractical. What we generally want is for AI systems to automate the most boring parts of our jobs. Replacing those activities with some nearly mindless copy and paste isn’t going to help. More powerful AI systems do longer actions. Identifying what data is needed for those actions is a progressive multi-step exploration. It’s both impractical and dumb to assume the solution is having a human is the solution here.

What is needed is workflows that put data where they need to be. This isn’t a novel problem. For humans to access the data they need to fulfill the roles their job is composed of, they need practical ways to access data. Controls and safeguards around that data are often needed to prevent misuse. Studies have often shown that one of the greatest risks to information is insiders. That’s not the only motivation in building these systems. Even if it was safe to allow everyone in a company direct access to the database, most of them couldn’t use it, because they didn’t have the skill set.

AI potentially would solve the second part, but not the first. If you allowed it unfettered access to a database, in theory it could learn how that data was organized and use it. But the first problem would remain. In some ways it’d be worse, as more people would know how to use it.

I think it’s very unlikely then that the current paradigm of building systems to manage access becomes less critical. In a sense, AI can help, by helping to build those systems. But building new ones, and updating existing ones to incorporate AI, and new safeguards necessary, it’s been a slow process. If it wasn’t a slow process, we’d be seeing many more and broader benefits already.

The jobs that have seen the most impact from AI adoption—software development and call centers—perfectly illustrate this data access challenge. They represent opposite ends of the spectrum in data trust and system readiness.

The association of software developers with AI developers is a ready explanation for software development’s early adoption, but I think there’s more to the story than that alone. Software development turns out to be a high-trust activity. The average software developer has access to read a lot of code directly. Quite a lot of code is open-source, meaning anyone in the world can read it, and even within organizations, the efforts to silo code visibility is relatively low. This is a little surprising considering how valuable most organizations consider their software, but there’s also a few good explanations here.

The influence of open-source conventions creates a cultural acceptance that pushes most controls to the output side. Software development does have a lot of controls about what goes into a code repository. There’s generally one or two manual reviews, large numbers of automated reviews and a rock-solid history that tracks who did what. The solid focus on the output does take some concerns about the viewing away. It also helps motivate liberal viewing, so that a wide group can be part of the review process.

Another enabling aspect is that software code, unlike other pieces of data a company manages, generally isn’t third party. When third parties are engaged, you get binaries, not original code. The most fine grained access controls in organizations revolve around managing data for third parties, either individuals like you and me, or organizations that are customers. That makes sense, in terms of retaining customer’s trust, but is also further motivated by many compliance activities that legally require those types of controls.

With all this in mind, it might seem odd that call centers which are so directly engaged in interacting with customers and thus customer data, would be another early implementer. I think a lot of people assume the reason here is that call center jobs are easy and so AI doesn’t have to be too smart to do them. I think that’s both an insufficient, and wrong explanation. The real reason is simpler: the call center industry, by necessity, had already invested heavily in the rigorous data workflows required to put exactly the right information in front of an employee at exactly the right time. It’s essentially the opposite side of the pendulum from the high-trust software developers. Call center staff operate in a low-trust environment.

Call center jobs aren’t easy. Partly it’s because of you. When people make a call, it’s often because they have a reason to be angry, or worried, or impatient. Of course, it doesn’t really make sense to take that anger out on call center staff, but we’re all human, and talk to enough people, and one will let that reasoning slip. Dealing with emotionally charged customers isn't the only challenge involved in call center roles though. Those roles are also operating in a low-trust environment, which means lots of extra complications for anything they do.

When you experience bad service, it’s tempting to ascribe it to bad staff. While that’s also possible, it shouldn’t be the first assumption. Many times, bad service means bad systems. Systems are forced to be restrictive because they handle customer data, but also because call center staff often have an employment relationship that encourages low-trust. Access to systems is often very constrained. There may be a complex “runbook” to perform even seemingly simple activities. Permissions are micromanaged.

My point isn’t to suggest there aren’t good reasons for that, but to point out that companies have already invested in making explicit all the questions about data and moving it about. They’ve realized that bad systems equals bad service, and so in many ways the systems were more ready to add an AI layer to them than elsewhere in companies where general purpose systems like email are conduits for information.

The challenges of developing workflows and systems to manage the data is a key part of what AI engineers call context engineering. Many AI pilots don’t include that type of activity. Predictably, they produce fewer results. That’s not entirely saying those pilots were mistakes, they often spread some awareness of what to expect, challenges. But it does limit their ability to offer real productivity improvement. The mistake would be to keep repeating this over and over, and not moving to the real challenges.

Like any technological change, there’s problems of technical implementation and organizational implementation. Some organizations understand this quicker than others. Some may never understand it fully, and find their competitive position eroded. It takes a surprisingly long time for bad organizations to fail. All of this slows actual adoption far beyond what the technical challenges alone do.

If you’re in an organization you’re trying to change, being aware of all of this helps a lot. It gives a consistent direction to head toward, a message to convey, and the ability to craft plans. That said, it doesn’t make it easy.

If you’re studying AI adoption from the outside and trying to understand differential rates of adoption, this is a key component. It’s also not all starting from ground-zero here. The challenges of handling data existed before AI. Getting the right data to the right person at the right time, but never the inverse has been a hard problem. As the tale of software developers and call centers shows, the solutions haven’t always been the same. Sometimes the solutions are implemented as technical systems, like with call centers. Sometimes they depend more on human judgement, culture, or both.

Understanding this data bottleneck is one thing; acting on it is another. The divergence between high-trust and low-trust environments provides a clear roadmap for where investment, technological effort, and strategic planning should be focused.

So, What Now?

For Investors: Bet on the "Plumbers"

One take for an investor should be that AI’s adoption curve is more complex than the supply curve. Consumer workflows have been able to grow on an awareness wave, coupled with capability waves that were more or less mediated by the supply of compute capacity. Future growth won't come from compute power alone; it will be driven by companies that can navigate the messy, real-world challenges—both technical and organizational—of handling private data.

You should also look beyond models and compute capacity, and look to firms with deep capabilities in dealing with those impediments. Firms that have already done this for their own data will have an advantage of being able to adopt AI more deeply, sooner. Firms that can help others do this will be in demand.

Don’t assume that only firms founded with the purpose of addressing AI data challenges are capable of doing so. Many existing firms have been addressing data challenges for some time now. If they use that experience in the new domain that’s an advantage they start from, both when competing against traditional competitors and maintaining relevance during a disruptive wave.

For Technology Professionals: Build the "Pipes"

As a technology professional, you should be thinking about the challenges to getting your private data where it needs to be, and how to reduce that friction. While it is something that could be tackled one by one, that is not the efficient route. There are repeating patterns here that are relevant. If you’re in the type of role that builds such patterns, you should start focusing on identifying what’s missing and building it. If you’re in a role that leverages platforms others build, you should do the research to know who has a platform for this and who has the commitment to keep building that platform because they understand the needs and opportunities here.

For Business Leaders: Evolve and Revolutionize

One thing learned from prior technology revolutions is that while companies can gain by integrating a technology into an existing business, there’s always an even greater gain to be made by a bigger rethinking of business processes. Machine tools and static assembly vs. an assembly line for example.

The same will undoubtedly be true with AI and existing business processes. That insight can encourage companies to attempt to remake their business processes. On net, that’s a good thing, as there are so many forces at work encouraging companies to not do this, often with the result that the business process only changes when a new company introduces it and disrupts existing companies. Sometimes those companies fail to ever change, sometimes they are very slow and lose competitiveness in the process. That said, the change has to be the right change, and in the historical sense looking back at disruptive startups, we only remember the ones that succeeded. Some of those that failed, did so because the disruption they attempted wasn’t well structured. An incumbent attempting to disrupt themselves can make the same mistake.

As such, it makes sense to understand what can be done by evolution, alongside revolution. The insight here is that giving up on the modernization of the existing business is certainly premature. Don’t convince yourself to stop maintaining the “legacy” because a revolution is coming. You're far more likely to make the leap to the other side if you find the narrowest point to cross.

Further Context & Definitions

What about tacit knowledge?

Tacit knowledge is underrated in businesses because it’s so invisible. It’s not written down, it’s hard to measure, and you mostly know it’s there from its effects. Thus it’s correct to point out that the training of general purpose large language models, they don’t consume tacit knowledge.

Where that thinking can go awry though, is equating tacit knowledge to private data, like it’s a subset. This leads to the thought that what’s needed is to extract that tacit knowledge and turn it into data. The reality is that tacit knowledge is more about experience than data. In my experience, public data is generally sufficient for its acquisition.

While private data might include trade secrets, mostly, it’s about current facts that are simply critical to carrying out an action. Tacit knowledge has more to do with the ability to make good judgements about using data of any type, to make decisions. While I wouldn’t suggest that LLMs are excellent at complex judgements, they are a lot better than they are commonly given credit for and the potential that they may become better is one of the great uncertainties.

I don’t think the next frontier for AI is particularly dependent on extracting tacit knowledge. There’s opportunities there for sure, but it’s also a difficult thing to do, and so I’d be sure you have the real experts, both in the domain you’re extracting, from the domain of the process of extraction, and from the domain of training necessary to turn extracted knowledge into real systems. A half-hearted effort will probably fail.

I think progress will be made faster via integration of private data and that general purpose models contain enough tacit knowledge to put that data to practical use in short order.

What’s AGI?

AGI, or artificial general intelligence, refers to the science fiction like possibilities where AI capabilities exceed human capabilities in a general, rather than a local way. Those are capabilities that do not exist today.

Why should we sometimes avoid talking about AGI?

My honest take on AGI is that its future timeline is indeterminate. I think anyone who tells you otherwise either has access to some really interesting data I’d like to see, or they are making overconfident predictions. Now, that said, making predictions here isn't entirely wrong. Someone might say they have an argument for 2040 as a reasonable timeline. Every prediction though must admit to not just being subject to miscalculation by actual numbers input, but also of novel influences we don’t have a structure to reason about yet. This cuts a bit both ways. We could get surprised by something that emerges in 2 years as much as we could be disappointed by predictions for 2040 receding farther and farther as we approach.

It’s not that hard to use your imagination, or connect with someone else’s, who thinks about what could occur for the science fiction possibilities. But it’s less obvious how you put that into present day planning. The investment in it can’t be too high if your timeline is uncertain, and the effects have wide divergences in possibility. A lot of the debate that happens here is closer to mood affiliation with optimistic or pessimistic outcomes than hard reasoning about probabilities, which makes it difficult to do much in terms of future planning other than more research.

It's great that this is occurring, and I’ll talk about it at other times, but I think a great mistake in much AI debate is dismissing the need to think about short term impacts under conventional assumptions. It’s no more reasonable to enter a room talking about short term impacts and change the topic to, “what about AGI?”, then it is to walk into an AGI discussion and say “that’s impossible, who cares?”. Both conversations need enough space between them to be able to progress, add nuance, without becoming a debate between them.

A way to sum this all up, I think there’s two discussions, “If AGI”, and “If less than AGI”, which are both interesting. While there is an obvious connection point between them, that connection point is coming back with answers like “needs more information”. In light of that, putting the connection aside, and thinking about each independently makes a lot more sense than treating it as one big continuum.

In the near term, AI's integration into the economy will manifest through familiar channels, primarily impacting labor markets, financial investment, and industrial competition.

A more novel danger is what I call 'The Great Masking': the risk that intense investor optimism in AI is hiding serious, slow-building threats to the wider economy. Those threats are trying to hide their effects already, and are eager to utilize AI investment as another layer of protection, to mask the effects of tariffs or irresponsible fiscal policy.

Labor Market Disruption: Beyond Simple Job Churn

Historically, technological advancements have caused job disruption, not a net reduction in jobs. Some roles, companies, and even industries shrink or fail, while others are created to meet new or previously unmet needs. We don’t have reasons to think that AI is breaking this pattern. While we have reasons to wonder about this in the long-term, those are the existential concerns we covered in part one. In the present day, we’d need more concrete data. This data is limited and not conclusive at this point.

For instance, Derek Thompson supports the idea that AI is already shifting employment among recent grads, but he is careful to note that overconfidence is unadvisable.

Derek Thompson

The Evidence That AI Is Destroying Jobs For Young People Just Got Stronger

In a moment with many important economic questions and fears, I continue to find this among the more interesting mysteries about the US economy in the long run: Is artificial intelligence already taking jobs from young people…

Read more

8 months ago · 344 likes · 48 comments · Derek Thompson

When looking at recent data, it's crucial to distinguish AI's impact from other confounding factors. For instance, a perceived slowdown in hiring for software developers—a key data point in the discussion around recent grads—may be less a result of AI replacing jobs and more a consequence of the post-pandemic hiring rebalancing in the tech sector, which saw furious hiring in 2020-2021 followed by a correction. This highlights the difficulty in isolating AI's specific effects from broader economic trends.

To make the case that AI is breaking the prior patterns, you’d need to make the case for structural unemployment. This is not merely the temporary friction of workers moving between jobs, but a fundamental, lasting mismatch between the skills held by the workforce and the skills demanded by an AI-driven economy. If AI automates a wide swath of cognitive and manual tasks, a large segment of the population may find their skills devalued, creating a challenge that standard economic churn cannot easily resolve. This practical concern, if it grows large enough, becomes the mechanism for a more existential crisis.

While that conceptually is sound, it’s applicability depends on data. Weak data suggesting shifting employment among recent grads doesn’t suggest structural unemployment, much less attribute it to AI rather than other economic conditions.

Investment, Bubbles, and Financial Contagion

The development of AI is fueled by staggering levels of investment, creating both immense opportunity and significant financial risk. The source of this capital is critical to understanding the potential fallout.

• Investment from Corporate Profits: When tech giants fund AI development from their vast cash reserves, the primary risk is an opportunity cost. If these investments fail to generate expected returns, shareholder value will fall, but the direct impact on the broader financial system is relatively contained.

• Investment from Bank Loans: When investments are funded by bank loans and other debt instruments, the risk of contagion is much higher. If AI companies fail to meet lofty profit expectations, they could default on these loans. A wave of defaults could destabilize the lending institutions, forcing them to tighten credit across the entire economy. This is how widespread unemployment can occur. A person laid off may seek a new job or try to start their own business, but if loans for new ventures aren't available, job creation stalls. It is this failure to create new jobs, not just the disruption of old ones, that leads to prolonged unemployment.

This leads to the risk of an AI investment bubble. This doesn't require investors to be irrational, but rather for individually rational views to create a collectively irrational market. For example, one group of investors may rationally believe Company A will dominate the market, while another group rationally believes Company B will. If both groups invest heavily based on their beliefs, the aggregate market valuation can reflect a future where both companies win, an impossible outcome. The market as a whole may be pricing in a cumulative level of future profit that the industry cannot possibly deliver due to competition. The dot-com bubble of 2000 serves as a pertinent example of a financial collapse sparked by over-enthusiasm for a transformative technology, even one that ultimately delivered immense productivity gains.

Market Structure: Layers and Concentration

The AI industry is developing in distinct layers: hardware providers (e.g., AI chips), compute providers (cloud services), model providers (foundational models), and workflow integration (applying AI to specific industries). Competition within and between these layers will shape the distribution of profits. A key uncertainty is who will capture the value at the workflow layer.

• Startups: New, specialized startups could focus on different niches, leading to a differentiated market with high potential profits for the successful disruptors.

• Cloud Providers: The large cloud computing companies could extend their offerings, creating products for each market. This would likely lead to less differentiation as providers feel pressure to match each other's offerings.

• Existing Companies: Incumbents could develop their own AI solutions internally. In this case, profits that might have gone to startups would remain within these established firms.

While these layers are distinct, there will be pressure to break them down. Cloud providers, for instance, have a strong incentive to expand into the other layers. The economics of AI—with massive upfront R&D costs and powerful network effects—create a tendency toward market concentration.

It’s useful not to overstate this point though. Multiple layers allow for multiple points where concentration, and the effects of concentration can be weakened.

Model providers are only as good as their latest model, and so far current investments haven’t provided a clear moat. Mostly AI companies with early success have stayed in the forefront by ever increasing investment, not by resting on laurels. New entrants have arrived and shown results that would have fully obsoleted early investments, at a fraction of those costs.

Amazon, Azure and GCP may remain the primary compute providers, but three competitors are enough to retain a competitive market, and other avenues for compute will remain. In the extreme scenario that competitive forces were muted, the efficiency factor from that scale is still within a single order of magnitude, limiting the exploitative capacity here.

NVidia has had great success, but stays in that place through continued investments. If they ever did hit a brick wall in terms of ability to improve performance, there’s no reason not to expect competitors to reach that same wall quickly, and thus have complete competitive equality.

But above all, each lower layer can only charge something less than what higher layers can capture in terms of value. A higher layer cannot pay for services it has not gathered enough revenue to pay for. With workflow at the top, and the most diversified of all the layers, this is a strong force toward a more broad distribution of economic surplus.

A Counterforce: Defensive Competition and Economic Surplus

For some areas, like cloud providers, forces toward concentration aren’t demonstrably stronger than existing forces. Investing in AI is not simply an attempt to build a new business, but one to retain a competitive position in an existing one. This dynamic could ultimately lead to more economic surplus being more generally by society through usage, rather than by companies through the various means of capturing revenue. This surplus can be:

• Direct: Users may get free services or receive far more value than the price they are charged for those services.

• Indirect: Many smaller businesses, empowered by more accessible AI tools, become more efficient or capable and pass on those savings or enhanced capabilities to their own customers.

While that is a great deal for consumers generally, it’s a risk to investors. If they assumed a different outcome, or followed others without considering the outcome, their paper wealth will decrease. To the degree that investors are individuals or entities that can survive that risk, it’s not something we need worry about, but that’s not always the case with investors. To some degree we are those investors, but even more, we interact with a system that could be disrupted by turmoil.

The Great Masking: How AI Optimism Obscures Deeper Economic Risks

While the financial risks within the AI sector are significant, a more novel and under-discussed danger is how the AI investment boom may be masking other serious, slow-building threats to the economy. The intense market optimism and capital flows generated by AI can create a "sugar high," temporarily hiding the destructive effects of tariffs, trade disputes, and irresponsible fiscal policy.

Economic Forces

6 reasons why tariffs are a terrible way to raise revenue

Economists are not fans of tariffs. There, the newsletter is coming out the gate with some hot takes! Meanwhile, regular people think tariffs are good for jobs. At the very least, they may ask, “What’s the big deal? It’s just a tax on imports. We need taxes to fix the deficit…

Read more

9 months ago · 116 likes · 77 comments · Brian Albrecht

The mechanism of this interaction is perilous. These external factors, such as tariffs breaking down global cooperation or budget deficits driving inflation, act as a persistent drag on the economy. They increase costs for businesses and reduce purchasing power for consumers. In a normal environment, the negative effects of these policies would be more visible in economic data. However, the powerful forward-looking optimism of the AI boom can overwhelm these signals, keeping investment sentiment high and stock market valuations buoyant.

This phenomenon aligns with observations from economists like Paul Krugman, who notes that markets are often poor at pricing in long-term policy risks, allowing them to build until a crisis becomes undeniable. An AI boom and tariffs both fit this narrative of market myopia, focusing attention on future technological gains while downplaying present-day policy costs.

Paul Krugman

Why Aren’t Markets Freaking Out?

For those of us who follow economic policy in general and the Federal Reserve in particular, the past week has been shocking and terrifying. Donald Trump’s ongoing attempts to bully the Fed into large interest rate cuts have escalated into an attempt to fire Lisa Cook, a member of the Fed’s Board of Governors, over unsubstantiated claims that she commit…

Read more

8 months ago · 2766 likes · 597 comments · Paul Krugman

Matthew Yglesias expands on this, explaining why this effect may be particularly severe now. The logic is that traders believe Trump is so sensitive to the stock market that if a sell-off occurs, Trump will back down. This creates a powerful incentive for investors to ignore the initial disruptive action and "buy the dip," confident that the policy will be reversed. This confidence, however, can prevent the very market crash needed to trigger the policy reversal. This dynamic, where the market's faith in its own influence creates a dangerous tolerance for risk, is amplified by the sheer optimism surrounding American AI companies. Since the major AI players are American, and are perceived as having the support of the political establishment, investors are willing to overlook institutional risks that would cause panic in other circumstances.

Slow Boring

How the stock market learned to love Trump

Whenever Donald Trump does something that amounts to putting the institutional foundations of American prosperity through the wood chipper — whether that’s attacking Fed independence, turning his back on mRNA technology, attacking the integrity of federal economic statistics, or any of a dozen other things that happened this su…

Read more

8 months ago · 330 likes · 577 comments · Matthew Yglesias

The danger is that these problems don't disappear; they fester beneath the surface. This sets the stage for a potential cascade of concurrent failures. An eventual, and likely inevitable, correction in AI valuations would not happen in a vacuum. Instead, it would act as a trigger, suddenly exposing the underlying weaknesses that the boom had papered over. This could lead to a multi-pronged crisis:

1. A Financial Shock: The AI bubble deflates, erasing wealth and crushing investor confidence.

2. A Corporate Shock: Businesses, already weakened by higher input costs from trade barriers, face a sudden drop in demand and tighter credit, leading to widespread insolvencies.

3. A Consumer Shock: Households, whose purchasing power has already been eroded by inflation, face the added threat of mass layoffs.

This "perfect storm" scenario, where a financial correction and a real-economy crisis hit simultaneously, is far more dangerous than either event occurring in isolation.

Too many triggers, and weakened foundations

What’s more, either can create the precipitating event. Bad policy could induce stagflation or a recession, spook AI investors and force them to accept that expected profits are now more distant. AI would still succeed, but the financing costs will still impact companies. Some may fail, but those that survive will still have a weaker financial position that merits a valuation change. The weaker financial position would cause investments to realign, and slowdown, removing the lift it’s been providing to the economy and exacerbating a policy induced recession.

On the other hand, an adjustment to optimism about long term effects can happen at any time. This could be a rational adjustment, or an irrational one. Normally an irrational adjustment would be temporary, but if it triggers a break in an unstable foundation, the normal recovery would not occur. Even a logical, healthy correction in AI stocks could trigger a catastrophe. Investors who thought they had safety nets would discover those protections were eroded by bad policy, causing a much wider economic collapse.

Finally, you have how the Trump administration would react to this. TACO (Trump Always Chickens Out) logic presumes that Trump can undo the damage done. That works far better for a policy announced yesterday that has not been put into effect. It does less for one that’s been slowly eating away at the American economy for months, maybe even years. In that case, which policy do they reverse? All of them? While that would be my choice, it presumes a lot to assume it would be the Trump administration's choice.

It might instead think it can placate markets as it has in the past with partial reversals of the most recent policies. This of course would not work, as something fundamental would have shifted, and so only something more fundamental can correct it. We also have signs that the Trump administration would take a page from China’s playbook and create the numbers it wants, and hope it can fool enough of the public to restore optimism. There’s a chance that might work once or twice, but ultimately if the bad policies remain, it only allows the foundation to erode further, creating a yet more future crisis.

Conclusion: The Risk of a Great Unmasking

While the long-term, existential questions surrounding AI command attention, the most immediate and tangible threats to the economy are rooted in the complex interplay between the current investment boom and other festering economic problems. The path to any future, utopian or otherwise, must first pass through a period of significant short-term risk, where the greatest danger is not a single point of failure, but a cascade of them.

The central, under-analyzed threat is that the AI investment bubble is actively masking the slow-building damage from irresponsible fiscal policy and the breakdown of global trade. Should the bubble burst, it will trigger a "Great Unmasking." The initial financial shock from deflating tech valuations will be immediately compounded by the sudden exposure of a real economy already weakened by inflation and supply chain friction.

This creates the potential for a multi-faceted crisis where a credit crunch, a collapse in business investment, and a surge in unemployment all happen concurrently. This is not just a tech-sector correction; it is the risk of a systemic economic downturn sparked by the tech sector but amplified by pre-existing conditions the boom helped to hide.

Therefore, while navigating the long-term societal transition is crucial, the immediate priority for policymakers must be to recognize and address this dangerous interaction. It requires looking beyond tech-sector regulation to the interplay of fiscal, trade, and financial stability policies. Successfully managing the practical, interconnected turbulence of today is the absolute prerequisite for realizing the profound promise of tomorrow.

Both of these are important topics. It can be tempting to ignore one for the other. Existential concerns have such large impacts, that they can make present day concerns seem trivial. The flaw in that thinking is that we don’t know when, or even if, existential concerns will emerge. Even if we assume they will emerge, our management of the present day has a lot to do with how stable and strong a society we have to manage those concerns. And if they don’t emerge, today's default concerns are more important.

That frame of reference can sometimes be used to ignore existential concerns, which would also be a mistake. While their certainty and timing is less knowable, the adjustments we make for them can be modest in the present, yet still have large impacts, when carried across time. Sometimes this is nothing more than the understanding that would help us recognize early impacts. Sometimes it’s plans we’d enact once recognized.

It’s tempting to demand responses to existential risks that would be as total as the risk. For example, with nuclear weapons, it’s tempting to think, why have any? While that sounds reasonable, we should ask, if that was the only response we considered, would we be better or worse than we are with the far less obvious set of non-proliferation treaties, monitoring, and other responses that have been employed? If we assume we would have gotten what we demanded, we can make a good, though not air-tight case for that. But the biggest risk is that we might not have been able to achieve that demand, and ended up with nothing as a response.

So both are important, and both interact, but they are different enough that it’s unproductive to continually introduce one into the other domain. With that in mind, I’d like to spend some time on the existential concerns to put them aside before moving into the present day concerns.

Two Horizons of AI's Economic Impact

How Artificial Intelligence will affect the economy unfolds across two distinct but interconnected horizons. The first is a practical, near-term landscape of familiar economic forces: jobs will be created and destroyed, investments will be made, and markets will adapt. These are the everyday concerns of disruption and growth that have characterized technological shifts of the past. AI is normal technology, at least for the short-term.

The second horizon raises questions about the nature of work, value, and social organization. If AI is as transformative as predicted, it may force a fundamental restructuring of the mechanisms that distribute wealth and opportunity. Understanding AI's total economic impact requires analyzing both of these horizons—the immediate disruption and the potential long-term transformation—and recognizing how the former may ultimately lead to the latter.

The Existential Horizon: Abundance, Agency, and the Social Contract

When we talk about an existential horizon, we’re assuming a radically changed environment. Not just incremental changes, but something fundamentally different. One fundamentally different environment would where there is no longer an incentive to utilize more human labor. Historically, productivity improvements have led to more labor specialization, more benefits to labor, and similarly complete employment of labor across society. While some jobs were reduced, and some disappeared, others were created.

If the historical pattern no longer holds, the result is more than temporary friction of workers moving between jobs, but a fundamental, lasting mismatch with the demands of an AI-driven economy. If AI automates a wide swath of cognitive and manual tasks, a large segment of the population may find their skills devalued, creating a challenge that standard economic churn cannot easily resolve. This practical concern, if it grows large enough, becomes the mechanism for a fundamental crisis with the structure of an economy.

It’s important to note that the part of the context of that crisis is radical productivity gains. This implies an abundance of production, which if distributed, could fulfill all current day physical needs and desires.

Unstable Dystopia and Inevitable Transformation

The core issue remains the distinction between production and distribution. AI may solve the problem of production, but our primary distribution mechanism—employment income—could be broken by widespread structural unemployment.

An existential thinker considers a potential dystopian state where the owners of AI capture the gains, leaving a majority without income. However, such a system would be profoundly unstable. An economy where the vast majority of the population cannot afford to purchase the goods and services being produced is one that no longer functions for the majority, creating immense pressure for change. It is difficult to envision a scenario where a large, disenfranchised majority would peacefully accept deprivation while a paradise of abundance is technologically possible. The social contract would be broken, and the economic system's legitimacy would evaporate.

Therefore, the more enduring pathway is one of transformation. Faced with systemic collapse, social and political systems would be forced to adapt. Through democratic pressure or mass social movements, new mechanisms for distributing the gains of productivity—such as a Universal Basic Income (UBI) or other forms of social wealth distribution—would likely emerge not as a matter of choice, but of necessity, to ensure social and economic stability.

The Inevitability of Change: Why a Dystopian State Cannot Last

The notion that a wealthy elite could maintain a dystopian system against the will of the majority overlooks the fundamental levers of power in society. A population that has nothing to lose has no reason to respect the existing economic or political order.

• Democratic Power: In nations with functioning democratic processes, a disenfranchised majority would have the votes to enact radical change. The political imperative to ensure the well-being of the populace would eventually overwhelm the influence of a small, wealthy minority.

• The Power of Mass Action: If democratic channels were to fail, the risk of mass civil unrest and rebellion would become acute. A system that creates widespread deprivation alongside visible, immense wealth is inherently unstable and invites revolutionary change.

• The Appeal to Humanity: Beyond coercion and political maneuvering, there remains the appeal to the shared humanity and self-preservation of the powerful. A society in a state of constant, simmering revolt is not a desirable or stable one for anyone, including the elite.

• Alternate Economies: A final aspect is that if a formal economy is not functioning for the majority, alternate economies can be formed. This happens today in exploitative economies around the world. The difference here is that in those economies today, productive capacity is low, so they distribute little. The formal economy would have to successfully suppress or deny alternate economies access to productive capacity of both their own individuals and AI.

These corrective forces make a prolonged, technologically enforced dystopia an unlikely long-term outcome. The friction and conflict during the transition would be immense, but the ultimate direction would be toward a new social contract that aligns with the new economic reality.

The "Brave New World" Scenario: The Dystopia of Contentment

A more insidious, and perhaps more stable, dystopian outcome is not one of overt oppression but of sophisticated pacification. In this scenario, "bread and circuses" would pacify the population. An accurate view upgrades "bread and circuses" to far more than subsistence. It would be a high standard of living for all, with material needs and entertainment amply provided for.

The trade-off would be a loss of genuine agency. The population would be consumers and spectators in a world run by a small elite, not active participants in shaping their society. The core conflict is not freedom versus suffering, but freedom versus comfort. This presents a more philosophical challenge: whether a comfortable, secure, and entertained population would still value the burdens and responsibilities of self-determination.

Navigating the Transition: A Pragmatic Approach

Given the scale of the potential disruption and the inevitability of the technological advance, the central policy debate should focus on how to manage the transition.

The Case Against Preemptive Policy and Prohibition

A key debate is the timing of major social and economic reforms. While some advocate for preemptively implementing policies like Universal Basic Income (UBI) to soften the blow of disruption. A system like UBI is a response to a fundamentally different economic reality where the link between labor and survival has been severed for a large part of the population. Insisting upon it as a condition of AI development is premature.

To the degree that UBI makes sense today, maybe in a more limited form, that can be advocated for. But it only becomes absolutely necessary after a systemic change has occurred, not before. To insist upon it now, with an uncertain timeline, would be to enact a cure for a condition that has not yet manifested.

Similarly, arguments to halt or severely restrict AI development due to these risks are both overly cautious and impractical. The potential for AI to solve humanity's most pressing problems creates an enormous opportunity cost. Furthermore, AI development is a global geopolitical race. Any nation that unilaterally pauses its efforts risks falling catastrophically behind, making a global moratorium unenforceable. Progress is, for all practical purposes, inevitable.

The most viable path forward is not to stop progress or to preemptively re-engineer society, but to focus on managing the risks and guiding the technology's development.

Conclusion: The Risk of a Great Unmasking

While the long-term, existential questions surrounding AI command attention, the most immediate and tangible threats to the economy are rooted in the complex interplay between the current investment boom and other festering economic problems. The path to any future, utopian or otherwise, must first pass through a period of significant short-term risk, where the greatest danger is not a single point of failure, but a cascade of them.

Those near-term threats will be the topic of part two.

The common thread of trust here makes me want to talk about another concept, that while fairly pervasive, has escaped labelling and deep discussion. I’m choosing to describe this concept as the “Technologies of Trust,” a set of formal, abstract systems that provide the foundation for large-scale cooperation. This bucks some commonplace uses of “technology” but is true to the defined term, and creates a space for something that has no such space today.

The term "technology" is often associated with the physical sciences, but a broader definition is "the application of scientific knowledge for practical purposes." This is a useful lens. Just as categorizing physical tools under "technology" gives us a shared language to discuss them, we need similar categories for our abstract tools. We can think of a hierarchy of dependencies. Physical technologies allow us to manipulate the world, and organizational technologies allow us to coordinate groups to apply those physical tools. But for organizations to function at a large scale, beyond the limits of personal relationships, a separate category is needed: the technologies of trust. These are primarily applications of economic and social science, creating the formal frameworks that make broad, impersonal collaboration possible.

What are these technologies of trust? The list is long, but some of the most fundamental examples include:

• Money: The primary form of depersonalized trust. It is a purely abstract system whose power comes from our collective confidence that it can be exchanged for goods and services in the future, allowing transactions between strangers who have no personal reason to trust each other.
  

  norabble

  Money: More Than Just Stuff, It's Trust

  We all understand trust and how essential it is for people to live and work together. Civilization itself is built on it. Money, while newer in human history, is just as woven into the fabric of our lives…

  Read more

  a year ago · 3 likes · Ryan Baker

• Loans: An instrument built on personal trust. Unlike money, a loan requires a direct assessment of a borrower's reliability, including their intent and capability to repay. This personal nature limits its scale compared to a depersonalized system.
  

  norabble

  Money, Trust, and Loans

  In a previous discussion, we established a framework for understanding money: it's a form of trust that has been depersonalized and made exchangeable. This transformation allows society to operate with a greater total amount of trust than would be possible through personal relationships alone, thereby fostering greater potential for prosperity…

  Read more

  a year ago · 2 likes · Ryan Baker

• Contracts & Law: Formal agreements and rules that create predictable, enforceable expectations between parties.

• Corporations: Legal structures that enable groups of people to act as a single entity, limiting individual liability and enabling large-scale investment and enterprise.
  

  norabble

  Trust, Money, and Companies

  While companies are ubiquitous, and most people have a general idea of what they are, a fully-formed understanding of the concept of a company is less prevalent. At the base level, it’s just an entity doing business. The extra nuance comes from all the structure needed to make that practical. They have to be mostly flexible, but in other ways predictabl…

  Read more

  a year ago · 1 like · Ryan Baker

• Finance: A complex system that bridges personal and depersonalized trust. Through banking, for example, personal trust (a loan agreement with an individual) is converted into depersonalized trust (newly created bank deposits that function as money in the broader economy). It uses tools like interest and collateral to bolster personal trust where it might be weak.

• Professional & Academic Standards: Credentials, licenses, and peer review that create a baseline of trust in specialized knowledge and skills.

• Government: The overarching set of institutions that create, administer, and enforce the laws of a society, providing a framework of stability.

• Bureaucracy: The operational systems of rules, procedures, and hierarchies used to manage large organizations—public or private—in a consistent and predictable manner.

• Digital Trust Systems: Modern extensions of these frameworks, including blockchains, digital identity systems, and encryption, which allow trust to be distributed without central intermediaries. These highlight both the promise and peril of new, untested technologies of trust.

Formal vs. Informal Trust

A common thread among these examples is that they are formal systems. Unlike the informal, intuitive trust we place in friends and family, these technologies are designed to be evaluated through reason. They create a framework of predictability that allows us to engage in complex collaborations with people we have never met. A formal system, however, is only as strong as the informal trust that individuals place in it. When confidence in these institutions erodes, so does their ability to facilitate cooperation.

Contemporary life shows how fragile this balance can be. For example, social media both erodes and reinforces institutional trust: on the one hand, disinformation campaigns weaken confidence in governments or media; on the other, platforms also serve as vehicles for new forms of verification and accountability.

Understanding these technologies is not merely an academic exercise; it is essential for a functioning society. When individuals lack a basic understanding of these systems, they are less likely to trust them, leading to reduced participation and a loss of potential collaboration. This can lead to unproductive modes of reasoning and the embrace of radical ideologies that fail to appreciate the delicate balance required.

Another aspect to consider is how these technologies complement and balance against each other. This balance is one reason it’s useful to think of them as a group, as many flawed ideologies can be described as a failure to balance amongst these techniques.

When Trust Systems Are Rejected

Some ideologies are defined by their skepticism of certain technologies of trust. While this is sometimes expressed in stark, theoretical terms, the reality has usually been more nuanced.

• Communism: In theory, many communist traditions reject markets, corporations, and money itself, relying instead on centralized planning and bureaucracy. In practice, however, even communist regimes such as the USSR or China never fully abolished money or markets—both sanctioned and unsanctioned markets continued to exist. This shows how difficult it is to eliminate these technologies of trust completely. What communism demonstrates most clearly is an overemphasis on government and bureaucracy at the expense of other balancing systems.

• Libertarianism: Many libertarian strands distrust government and bureaucracy, instead placing heavy faith in contracts, markets, and voluntary associations. But this perspective often underplays the fact that governments do far more than simply “enforce contracts.” They create the very conditions under which markets exist—defining property rights, establishing money, and ensuring stability. Beyond that, governments must intervene to correct for failures (like pollution or monopolies) and to unwind outdated rules that distort rather than enable economic life. Libertarian experiments such as charter cities or cryptocurrency-based communities illustrate both the appeal of market-centric trust and the difficulties of maintaining order, fairness, and adaptability without strong public institutions.

• Anarchism: Anarchist thought often rejects nearly all formal trust systems, seeking instead to build cooperation through informal trust, mutual aid, and federated local assemblies. Examples like the Rojava region of Syria show attempts to create alternative, decentralized institutions. These systems reveal both the strengths of informal trust in tight-knit groups and the difficulty of scaling them to larger societies.

Recent populist movements also show how selective rejection of trust systems—such as undermining courts, electoral processes, or expert institutions—can destabilize societies. Even when not fully ideological, these attacks erode confidence in the shared frameworks that make cooperation possible.

The lesson across these cases is not that certain ideologies are simply wrong, but that any attempt to radically unbalance the ecosystem of trust technologies—by over-relying on one and dismissing others—creates fragility. Durable societies require a balance among these different tools of trust.

The Price of Complexity

The challenge is finding the right balance. While trust expands our potential for good, it is not without risk. On one level, trust can be exploited, if we are not vigilant enough. On another level, systems dependent on trust can be forced to shrink when trust is lost. As a combination, our capability for successful vigilance puts a limit on the amount of trust that both logical and intuitive systems support.

Technologies like finance enable a level of prosperity analogous to building an economic skyscraper, reaching heights unimaginable from the ground. A financial crisis, in this view, is a catastrophic failure, but it is a failure that could only happen from such a height. The alternative is not a skyscraper that cannot collapse, but a simple hut with a much lower ceiling for growth.

In this metaphor, regulation serves as the skyscraper’s safety systems: fire codes, structural reinforcements, and emergency exits. They cannot eliminate risk altogether, but they can reduce the likelihood of collapse and increase resilience when shocks occur.

This is why, even after a major crisis, trust is rarely abandoned entirely. It is damaged, but it proves resilient. The response is not to tear down the system, but to repair and renegotiate trust through new regulations and reforms. The alternative—a complete return to simpler, less powerful systems—is often unthinkable, demonstrating our fundamental dependence on these complex technologies, for all their flaws.

Educating for Trust

Ultimately, the technologies of trust are not merely abstract systems; they are the essential tools we have engineered to overcome the natural limits of personal relationships, enabling the vast collaborations that define our world. Small-scale, personal trust is innate, a concept that barely needs to be taught, as it is coded into our function from evolving in small communities. Understanding the abstract technologies that allow for this large-scale cooperation, however, does not come so naturally.

This is why, just as we teach our children about the physical world, we must also educate ourselves and future generations about these abstract systems. Schools, universities, and public institutions should include civic education on money, law, and emerging trust systems. The goal is to build critical literacy so citizens can weigh their potential and their risks. Policymakers can help by ensuring transparency, accountability, and accessible regulation. And as individuals, we can cultivate trust by participating in institutions thoughtfully and critically, rather than retreating into cynicism or ideological views.

Understanding the value in each system helps reinforce the need for balance. At its simplest, this is immunization to ideology.

Balance itself is a bit harder to obtain, as it's not enough to understand one's own position, or even those close to us. Balance requires listening to others as a first step. But in large communities we need to trust others who aggregate views distant from us, either physically or along other dimensions.

Understanding in this context also allows acknowledging flaws, and working to improve implementation. Putting this all together, we can foster a more collaborative and prosperous society for all.

While developing those capabilities is valuable, AI research has often stopped short of addressing what comes after: deciding how we should use them. Humanity has rarely agreed on such questions. It’s possible someone has found a complete answer, but more likely, we have only partial and competing ideas. Even when we agree, translating agreement into action proves difficult.

AI enters a world already full of unresolved issues. To use it wisely, we must acknowledge the societal patterns we’ve long ignored.

Example: AI Companions

Researchers ask if AI companions could reduce loneliness. This is promising for those who feel isolated, yet reactions vary.

Some dismiss it as “unnatural” or “creepy,” ignoring how our social structures actively contribute to loneliness. While some causes are personal, many are external—rooted in social biases and behaviors. Knee-jerk objections often defend the status quo, offering platitudes (“we should all be nice”) that history shows we rarely fulfill.

Others worry about exploitation—people becoming dependent on AI and vulnerable to manipulation. This is a valid concern, but banning AI for companionship ignores the underlying problem: many people are already left lonely by societal design.

The Social Ecosystem Problem

The deeper risk lies not in AI itself, but in the social environment it enters. Human biases determine who we befriend and who we exclude. These biases are difficult to overcome, because personal happiness is tied to them. Progress in changing this has been slow for centuries, with only incremental gains.

In this sense, we live in a manipulative ecosystem. Human social life often revolves around implicit negotiations for status, influence, and belonging. We signal our own value, interpret others' signals, and use those social cues to navigate our standing. This process can subtly or overtly marginalize those who do this less, or less effectively, reinforcing their exclusion. This can leave people feeling rejected or excluded, reinforcing cycles of isolation. Those without sufficient supportive relationships may turn to artificial companionship—whether AI chatbots or other digital agents—that promise consistent validation without the risks of human judgment. While such companionship can meet real needs, it can also increase dependence on entities whose motivations may not align with the user’s well-being.

If this ecosystem persists—as history suggests—it will shape how AI is used and abused. Delaying AI in hopes that society will “be ready” may simply leave us unprepared when AI inevitably becomes widespread.

Instead, we should ask: what can we do, with old or new capabilities, to make this ecosystem safer before AI becomes deeply embedded?

Advertising as a Case Study

Advertising is one of the most organized forms of social influence, and it provides a clear case study for the risks of AI integration. Consider the AI companions discussed earlier. If they are provided by people who genuinely aim to reduce loneliness, we can be optimistic. But if they come from companies whose main goal is ad revenue, we should be cautious. This same dynamic applies to any system—from search engines to social media feeds—that could be funded through advertising.

Society has rarely addressed the broader effects of advertising. Rules generally only forbid outright falsehoods, leaving manipulative but technically true messages untouched. In an AI-powered world, this gap becomes more dangerous.

We often avoid regulating advertising because of free speech concerns. While this principle is important, advertising is distinct from the personal, expressive speech we most value. There will be blurry cases, but clear ones exist—and addressing them could protect vulnerable people without dismantling free expression.

Balancing Free Speech and Advertising Limits in an AI Age

When addressing the risks of AI-powered advertising—especially in companionship contexts—we face a classic dilemma: how to curb manipulation without undermining free speech. The tension lies in the fact that too much personal judgment in setting limits risks turning regulation into a tool for enforcing one person’s biases over another’s speech. This would strike at the very root of free speech protections.

One path forward is to prioritize rules that require minimal subjective interpretation. These rules should be inherently more equal in application, less reliant on individual moral standards, and thus less prone to abuse.

Examples of Lower-Judgment Rules:

• Limiting the Total Volume of Advertising: Capping the overall amount of advertising exposure—whether in minutes per hour, or ad impressions per day—reduces manipulation opportunities without privileging one viewpoint over another.

• Restricting Advertising from Certain Spaces:

• Physical spaces: We already prohibit most advertising in schools, recognizing that young people deserve protection from certain commercial pressures.

• Digital spaces: Online environments used primarily for education, mental health support, or community-building could adopt similar restrictions. Currently, digital spaces often import norms from the open web, where advertising is pervasive and largely unregulated.

• Context-Based Exclusions: Banning advertising in contexts where people are unusually vulnerable—such as grief counseling forums, addiction recovery platforms, or AI companionship apps—could help safeguard well-being without making content-based judgments about the ads themselves.

A Further Step: Regulating Manipulative Methods

Beyond rules about the context of advertising, a more ambitious, though complex, path involves regulating the specific manipulative methods used. This approach rightfully raises concerns about potential overreach, so it must be handled with care. The focus would not be on the content of a message, but on its structure, targeting verifiably manipulative techniques that often exploit known psychological shortcuts.

Many of these techniques are applications of well-documented principles of persuasion, such as those identified by Robert Cialdini in his foundational book, Influence. An automated system could, in theory, be trained to detect and block advertising that employs these methods:

• Exploiting Scarcity: Creating artificial pressure with misleading countdown timers or "limited supply" claims that are demonstrably false.

• Inflating Social Proof: Using fabricated testimonials, fake reviews, or inflated user counts to create a false sense of popularity.

• Deceptive Interface Design: This includes a wide range of techniques often referred to as "Dark Patterns," a term coined by UX researcher Harry Brignull. Examples include using confusing navigation, hidden opt-outs, or "confirmshaming" language ("No thanks, I hate saving money") to trick users into making unintended choices.

The key advantage of exploring this path is that an AI could be a more consistent and less biased arbiter for these kinds of structural rules than a human reviewer. While the risk of bias in the AI's training data would still exist, it avoids the motivated reasoning a human might use to permit a borderline case, making enforcement more uniform. This remains a highly contingent idea, but one worth exploring as AI capabilities mature.

The Funding Problem: Alternatives and Competitive Realities

Restricting advertising in vulnerable digital spaces immediately raises a critical question: how will these services be funded? Blocking a primary source of revenue is not enough; viable alternatives must exist. This requires considering the funding ecosystem alongside the regulatory one. Potential models include:

• Direct Funding: Services could be supported by government grants or non-profit organizations, treating them as public goods akin to libraries or mental health services.

• User-Supported Ecosystems: Deliberately creating infrastructure for subscriptions or micropayments could allow users to directly fund the services they value, aligning the provider’s incentives with the user’s well-being.

However, it is not an either/or situation. In a competitive market, the existence of an alternative funding model may not be enough to drive out ad-based systems entirely. A slight competitive edge from having access to even a small amount of advertising revenue can compound over time. While some users may value an ad-free experience, this appreciation is often insufficient to overcome the advantages—in scale, features, or price—that an ad-supported competitor can offer. Any serious attempt to create ad-free zones must also account for these powerful market dynamics.

Why This Matters in the AI Era

If AI companions, search algorithms, or social media feeds are driven by ad-based revenue, they inherit the same risks that have long plagued advertising—but with greater personalization and subtlety. As one of the adversarial industries, advertising will respond to scaling differently than other industries, so some extra attention is merited.

norabble

AI and the Zero-Sum Game

AI is advancing quickly, and if there’s any one consensus about it, it is that it will have broad impacts on jobs. What impact, is an area of more debate, but it’s uncommon to view it as non-impactful. Some believe that jobs will disappear, and there would be large amounts of unemployment. Some draw on past periods of technological change, such as the I…

Read more

a year ago · 1 like · 2 comments · Ryan Baker

We already use simple guardrails in some areas, like limiting ads in schools, but we have not consistently re-evaluated limits to quantity and placement as trade-offs have shifted. Revisiting those trade-offs, can limit harm, and does not need to entangle us in debates over which messages are “acceptable.”

The ability to systematically implement more advanced rules would be more transformative, but rightfully needs more caution. Caution though, advocates of more study and discussion, rather than treatment as a settled topic.

Such solutions won’t remove all risks, but they can help ensure that free speech protections remain intact while still shielding people from the most pervasive forms of manipulation.

Conclusion

The central challenge is not just what AI will do, but how it will operate within human systems already shaped by bias, neglect, and power imbalances. The examples offered here are far from comprehensive and are ultimately exploratory. Reducing the influence of advertising won’t create a utopia; the scope is only so broad, and the effort will not be easy. It’s a concern that hasn’t entirely escaped our awareness, but one where we’ve been limited by the trade-offs. As those shift, we must re-examine.

We should remain aware of our own individual and group social interactions. Best efforts there are consistent with our humanity, and we should applaud anyone who tries to be a better person. But we must also recognize our limits and how systems encourage patterns that individual efforts either ride or resist. AI could reinforce these patterns—or help change them. A willingness to re-evaluate and adjust trade-offs can find wins where we’d otherwise see losses. But we must be clear-eyed: even positive changes come with some set of losses. The goal, then, is not to seek a perfect, cost-free solution, but to consciously decide which changes we are willing to bear. That choice—what we are prepared to change in order to gain something better—is the work that separates shaping our future from letting it happen to us.

If your main mechanism for judging the different claims is by looking at your own personal situation, you’d likely be uncertain that anything bad was going to happen. Unfortunately, that’s not a good way to judge how an economy of 340 million will behave.

There is an effect of tariffs, and the business uncertainty they have created. While it may be somewhat masked, the masking has limits.

How tariffs affect businesses that produce goods

In a simple world, where a tariff was established without all the reversals, delays and uncertainty, the effect on business largely depends on what the businesses do, and what options they have to make changes.

For a business involved in manufacturing, they would be affected by inputs and outputs. In terms of outputs, if they have a foreign competitor that has the same output taxed, it gives that business an opportunity to raise their prices or get more market share, at least in the long term.

In the short term, local business has to react to what foreign competitors do. If those competitors keep the same pre-import price, the post-import price increases. Local business then can increase their prices, and gain more profit. Alternatively, if local businesses have the capacity to produce more, they can maintain their current prices and find willing customers who choose their product over the foreign competitors due to the relative price change.

It’s also possible their competitors choose to reduce their pre-import price. This inherently means less profit, unless they can force their suppliers or workforce to take less for their inputs or labor. If foreign competitors take less profit, this either will lead to less ability to invest long term which will reduce their market share eventually, or to the existing investment becoming negative value.

In the short term, there would be limits to expanding production, but with some predictability about the future, a business that expects to have a competitive advantage would invest in expanding. If a local manufacturer knows they’ll make more profit per item at the same price to the consumer, they can price lower to attract more business and be sure that extra production capacity won’t go to waste.

The timing of this does matter. It doesn’t make sense for a local producer to undercut their foreign competitor until they can produce enough to take additional market share. Until that point, the local producer would be better off maintaining a higher price, accumulating more profit, and using that profit (and the additional access to capital it enables) to invest in more production. Only after that investment adds more production capacity would they undercut their foreign competitors' prices.

Foreign producers will want to continue to use their existing capacity. They’ll have less incentive to invest in more capacity, but in the immediate sense, there’s no reason for them to reduce prices. If they assume the tariffs will remain in place, they would want to capture what profit they could until the local producer undercuts them in a way that makes the existing capacity unprofitable to operate, at which point they would idle that capacity, and possibly take more final measures, such as selling equipment or parts, declaring bankruptcy, etc.

Again timing matters. One set of decisions, continued operation or idling, depends on the realities of today. Another set of decisions affords a more complex set of options:

• New Investment

• Maintenance with no investment

• No maintenance, and no investment

• Dismantling operational capacity

These decisions depend on future expectations. If the competitive advantage looks favorable for the future, new investment makes sense, either to capture any growth in demand, or to force a competitor to choose option (3) or (4). If competitive advantage looks even, option (2) makes sense. If the reality is a competitive disadvantage, you’ll choose between (3) and (4) based upon how fast your competitor expands and the prices they set.

If a competitor can't meet all the market demand, a company might continue selling with its existing equipment. However, the most profitable move could be to let a shortage happen and dismantle operations (option 4), especially if the costs of maintaining idle equipment are high. In other words, as a company produces less, its competitive disadvantage can worsen, forcing it to dismantle its operations even faster than its rivals can grow to fill the gap. If not, the company would likely continue to produce and sell until its competitors capture the entire market through lower prices and greater supply.

Effect of tariffs on inputs

In addition to all of these considerations about the effect of tariffs on outputs, there’s also the consideration that an input may have a tariff applied. Assuming there is a local producer for that input, that full story about the balance between foreign and local producers will play out. During that time, the input’s price would initially increase. The simplest logic assumes the initial price increase would be equal to the tariff rate, but it could be less. As expansions and contractions take place, the price would reduce from this increased price. The most likely settling point is still higher than the initial price.

While it is possible to craft a story that ends up with the final local price being lower, it would depend on specific equilibriums. Then there is the counter point that you could end up with shortages that cause prices to exceed the original price plus tariff. Anyone who tells you such situations are probable without a very detailed set of data is making overconfident predictions.

If you’re a local producer that both has inputs increase in price and has their foreign competitors affected by tariffs on the output product, timing your investments becomes more complicated. For one, you probably don’t have enough information about the business of your potential suppliers to know what their optimal pricing model would be. For two, they may not either. For three, even if they have the information to choose an optimal pricing model, they may be run by less than fully rational individuals, or hope that their competitors have less than full information.

Most businesses must have a lot of confidence in their own competitive advantage or the growth potential of their industry before they commit to investments. In general, they’d prefer to risk their customer’s experiencing a shortage, and thus being able to raise prices, then to risk having unproductive investments.

We often make the mistake of only thinking about high growth industries' business models. In those industries, early investments are somewhat defensive. Beyond creating customer loyalty, there are also scale advantages. The producer that reaches a certain scale first captures that part of a competitive advantage. Dislodging them from this position at the apex of growth is more expensive than an early lead, as exponential compounding of those costs applies. As such, in a high growth industry everyone wants an early lead.

But industries in which growth is minimal, flat or contracting, this preference for investment doesn’t exist. Even if a competitor invests first, the scale advantage is trivial or non-existent, and the amount it costs to invest early and later are equal or roughly equal.

It’s in this environment that a country imposing a tariff should worry about the possibility of shortages. The foreign producers may no longer see the local market as worth exporting to, and yet the local producers may have favored being conservative about their investments to the degree that their capacity expansion isn’t available until after a shortage is experienced. In a liquid global market, a local shortage would be resolved by bidding up the pre-import price. However, markets aren’t fully liquid, and so it could be left unresolved, due to contracts and agreements and the simple nature of shipping delays.

How tariffs policies create business uncertainty

In a long term, static economy, reasoning about the effect of tariffs is simpler. In a more dynamic model, or during periods or transition, it becomes more complex, and timing must be factored in.

When Trump announces a policy, reverses the policy, reimposes the policy at half the initial level and then adds another layer on top, this more complex model adds yet another layer of complexity. The effect of uncertainty is a greater degree of delay for investments, increasing the probabilities of high local prices or local shortages, which would lead to even higher local prices, in addition to the actual shortage.

Risks of Oversimplifying

At times it’s appealing to try and apply our own personal experiences with money and finance to economic reasoning. For example, when you think of businesses delaying investments, you might think of it like a personal decision to buy a new car or keep driving the old one. If you don’t spend on a car today, you save that money, which you retain as potential to buy a new car (or something else) later.

For an individual business this is a little bit similar. They do have the complexity of building relationships, organizing operations, to worry about. Still, a company with $100 million in the bank has a lot of potential saved up.

Where this analogy breaks down fully is at the level of a national or global economy. Despite all the talk about imports and exports, most national economies have the majority of their production and consumption locally consumed and produced. For example, about 89% of U.S. consumer spending is on domestically produced goods and services. When you or a business saves money, you can later exchange the money for a good or service, with the most minimal of restrictions. But national economies have limited options to use currency to demand goods or services, and a global economy has no options.

An economy can store value by stockpiling produced goods or raw materials, but the outcome of delayed investment usually doesn’t translate into stockpiling. Rather it tends to mean less circulation of money. In many cases that would cause less employment, and labor can’t be stored. If a person is unemployed in 2025, you can’t get them to do two years worth of work in 2026.

At an economy wide level, if one business doesn’t invest, another may find it easier to get the loan necessary to invest. But when investment is lower across an entire economy, the result is mostly waste that can’t be recovered.

The connections between businesses

The combined effects of uncertainty and actual tariff costs affects US businesses. Despite braggadocio claims that foreigners would pay for tariffs, the evidence so far is that Americans are bearing them. So far a lot of it is landing on American businesses. Some people might consider that a win, a take down of profitable US corporations. But ultimately that’s a shortsighted view, and ignoring the composition of American businesses.

The problems are (a) this is unlikely to continue for long, (b) affects many smaller businesses, and (c) many Americans are invested in those corporations.

One of the reasons this won’t continue? Some of those costs are being buffered by pre-tariff stockpiling. When those stockpiles are gone, businesses will have harder decisions to make. They could have raised prices, earning a small profit for being wise enough to stockpile, but they weren’t forced to. In the future, the choice of raising prices will not represent a decision to forgo a small one-time profit, but may represent a choice to be unprofitable generally. Businesses that aren’t profitable eventually stop being businesses. If that’s the case, the pattern of bearing costs will end and consumer prices will rise.

Even if the change in costs is small enough to only reduce profit, rather than invert it to a loss, these costs aren’t borne just by large corporations, many are smaller businesses. Those business investment plans are altered by a change in cash flow.

Finally, remember that Americans own quite a lot of those corporations, and that changes to their profitability will lower rational valuations. Those corporations also do a lot of investing too. In the end, one thing must decline, either investment or dividends, and in both cases this should translate into a lower valuation.

The risks of delayed effects

I worry that a set of delayed effects that have masked the developing effects of tariffs and inconsistent policy will all land at the same time. The effect here is far more worrisome than the stock market's initial reaction to the tariffs. That reaction was based on expectations, and expectations are easily corrected by a course change. The wave of effects that have been working their way through our economies layers, are not so easily reversed.

Investments that businesses put off to build stockpiles, and due to uncertainty meant real work wasn’t being done. We finally saw the effect of that in a revised jobs report. It’s not simply that the stockpile is gone, it’s that the stockpile itself had costs, and we’ve largely been missing those costs as we take advantage of the stockpile. But when that’s exhausted a new price must be paid, in addition to the loss of investment. Even if all tariffs are permanently reversed next week, those effects will continue to show up.

Businesses, large and small, that use profits or a cash cushion to absorb costs will make that clear in quarterly statements, or in more personal ways for smaller businesses. Even if all tariffs are permanently reversed next week, those costs remain. If they continue, they keep growing.

We’re deep enough into the tariffs that foreign shipments are disincentivized by high tariff rates, and yet businesses have not invested in replacing that supply. This creates a balance sheet problem if they’ve lost their cash cushion while waiting, and now finally see enough certainty to make investment worthwhile. In theory this can be fixed by credit, and investment can start and eventually these local producers would have more market share. But there’s a looming shortage between these points. We can’t really avoid those shortages where they are lurking. Maybe I’m wrong and such shortages aren’t lurking, but I’m not sure who could give you a reason to be confident in that.

Even if all tariffs are permanently reversed next week, arranging new supply will come with many complications. Supply contracts can take time to restart.

And then there’s the businesses that have had their production costs increased by higher input prices, or will have higher input prices. Those businesses have a hard set of choices. They can raise prices. If they do that alone they lose customers. But if all businesses in an industry come to the same conclusion, they may just shrink the market. If the market shrinks then the industry has to scale back costs or lose money.

In a competitive market, in the long run, a business only has one option, to raise prices. If they operate at a loss, they’ll go bankrupt. If they operate at a lower margin, they’ll lose access to credit and be unable to make investments. That will have different effects depending on the market's potential for growth.

In a growth market, not making investments will cause market share to drop. At an individual business level, that would be destructive, making raising prices a better option than accepting lower margins (which are sometimes already negative in growth industries), and lower investment. If, counter to that, it does make sense to accept lower margins, the result is the growth industry transitions to a period of no growth, which brings effects to the wider economy.

In a stable market, investments are less frequent, but can still be necessary to perform maintenance. Loss of access to credit can then force such businesses to shrink, albeit slowly as deferred maintenance slowly reduces efficiency and capacity. Capacity reductions can cause a loss of scale advantages, raising per unit costs and thus forcing the increased price vs. lower margin tradeoff to begin again.

In the end, the only real option for businesses that were at an equilibrium before, is to raise prices. It’s tempting to imagine that most businesses are out of equilibrium, and that profits are high enough that none of these effects are triggered. It’s a hard story to disprove in any given moment because finding the equilibrium for individual businesses and industries is quite difficult. For the most part, we don’t have a better mechanism for discovering this than the actual conduct of markets. If you knew a stable equilibrium other than what markets discover for themselves, that information would enable you to make a lot of money by outsmarting the markets. While it’s not impossible, history tells us this is far more difficult and rare than overconfident predictors make it look.

Conclusion: Cascading Risks and Lasting Costs

The true risk of tariff policies lies not just in their immediate costs, but in their potential to create a cascade of lasting economic damage. The uncertainty generated by shifting trade rules discourages business investment, a loss of productivity that cannot be easily recovered. As this article has detailed, these policies create a ripple effect:

• Delayed Investment: Businesses postpone crucial investments in new equipment and capacity, leading to long-term economic waste and a higher risk of future shortages.

• Supply Chain Fragility: As foreign supply is disincentivized and domestic replacement fails to materialize, supply chains become brittle and prone to disruption.

• Absorbed Costs and Rising Prices: While businesses may initially absorb the costs of tariffs through lower profits or cash reserves, this is not sustainable. Eventually, these costs are passed on to consumers in the form of higher prices.

These factors create a fragile economic environment. Unlike a temporary market fluctuation, the structural changes caused by prolonged trade conflicts, such as lost investment, atrophied supply chains, and bankrupt businesses, are not quickly reversible, even if the tariffs themselves are lifted.

These changes can develop with less visibility, below the level of awareness that drives market prices. But when that lack of awareness is shattered, a course change is insufficient to undo the change in expectations that awareness brings. The economic consequences, therefore, will persist long after the policies that created them have changed.