When AI Becomes Infrastructure, Understanding Becomes A Risk

Infrastructure has a strange habit of becoming invisible, even though it’s the backbone of…well…everything. Electricity is infrastructure, but we don’t think about that when we’re making our morning coffee. Networks are infrastructure, but we don’t think about them when we’re messaging our colleagues. 

Identity systems are infrastructure, but we don’t think about them when we’re logging into our workspace. Cloud platforms are infrastructure, but we rarely think about them while jumping between the dozen or so SaaS tools we use every day. The moment something becomes infrastructure, people stop noticing it. 

AI is starting to enter the same category.

For a while, it was easy to treat AI as something separate from the real machinery of the business. It lived in pilots, dashboards, copilots, chatbots, and slightly breathless strategy decks. It was useful. Sometimes impressive. Sometimes deeply irritating in the way only a mistakenly confident machine can be irritating.

But it felt contained.

Now AI is moving into software development, customer service, analytics, cybersecurity, business automation, workflow management, and decision support. It’s not only helping people work faster. It’s starting to shape how work happens in the first place.

Which raises a more interesting question than whether AI can perform a task. Which is: What happens when organisations become dependent on systems they can’t fully understand? That’s where AI governance starts to become much more than a compliance exercise. It becomes a way of keeping organisations connected to the systems they increasingly rely on.

When AI Becomes Infrastructure

AI rarely becomes infrastructure through one big decision. It happens in smaller, less noticeable ways. 

A developer starts using an AI coding assistant. A customer service team adds an automated response tool. A security team uses AI to sort alerts. A marketing team uses synthetic content in campaign workflows. A finance team uses AI-supported forecasting. An operations team adds automation to move work between systems.

None of these choices look dramatic on their own. Most of them look sensible. But that’s how dependency usually starts. Not with a grand transformation moment, but with a series of reasonable decisions made by teams trying to move faster, reduce manual work, or cope with complexity they already had.

Over time, those tools stop feeling optional.

People stop asking whether they should use them and start building work around them. The AI assistant becomes part of the development workflow. The automated summary becomes part of the reporting process. The recommendation engine becomes part of the decision. The chatbot becomes part of customer experience.

At that point, AI isn’t just a tool anymore. It’s part of the operating environment.

This is what makes AI infrastructure such an important idea. It doesn’t always look like infrastructure in the traditional sense. It may not sit neatly inside an architecture diagram. It may not be owned by one central team. It may arrive inside a vendor product, a workflow platform, a development tool, or a customer-facing application.

But if people rely on it to get work done, it’s becoming infrastructure. And infrastructure brings responsibilities. You need to know how it works. You need to know how it fails. You need to know who owns it, who monitors it, who can fix it, and what happens when its outputs become unreliable.

That’s where many organisations are still catching up.

IBM’s 2026 CEO study found that surveyed CEOs expect 48 per cent of codifiable operational decisions to be made by AI without human intervention by 2030, compared with 25 per cent today. That’s not just “AI helping people work faster”. That’s AI moving closer to the decision layer of the business.

The 2026 Stanford AI Index also points to the same direction of travel. AI adoption has accelerated sharply, with generative AI reaching 53 per cent population-level adoption within three years.

None of this means AI adoption is bad. That’s too simple. It means the enterprise conversation needs to mature. Because once AI becomes part of how an organisation operates, the question is no longer only whether the system can perform. It’s whether the organisation understands enough about that system to rely on it safely.

The Rise Of Comprehension Risk

Every major technology shift creates a new risk language.

Cloud gave us shared responsibility. Remote work forced a rethink of identity and access. Software as a Service created new concerns around visibility, data control, and vendor dependency. Cybersecurity moved from perimeter protection to continuous verification because the old model stopped matching the way work actually happened.

AI is creating its own risk language now. One of the most useful terms for this is comprehension risk. 

Comprehension risk is what happens when an organisation becomes dependent on a system it can’t meaningfully inspect, explain, validate, or reason about at the level required for the work that system is doing. Put simply, it’s the risk of relying on something you don’t understand well enough.

Not in a vague, philosophical way. In a very practical way.

• If an AI system recommends who should receive a loan, who can explain the logic clearly enough to defend it?
• If an AI tool generates code that ends up in production, who understands what’s now running inside the application?
• If an AI agent updates records, sends messages, triggers workflows, or makes operational choices, who knows exactly what happened if something goes wrong?

This is where AI is different from a lot of traditional enterprise technology.

Most older systems are deterministic. That means the same input should produce the same output if the conditions stay the same. You can test that. You can map it. You can document it. You can usually trace the logic from beginning to end, even if doing so makes everyone involved question their career choices.

AI systems are more probabilistic. They work through patterns, context, and likelihood. They can produce useful outputs without following a simple fixed path that a person can easily read back. That doesn’t make them unusable. It does mean trust becomes more complicated though.

Because trust and understanding aren’t the same thing.

A team may trust an AI system because it usually works. But if they can’t explain why it produced a specific result, spot when it’s starting to drift, or understand where its inputs came from, that trust has limits.

McKinsey’s 2026 AI Trust Maturity Survey found progress in AI trust maturity, but also persistent gaps in strategy, governance, and risk management, especially as organisations move toward agentic AI. 

That gap is the interesting part. AI can be good enough to use before the organisation is ready to understand it properly. And once that happens, successful adoption can create its own kind of fragility. Not because the system is useless. But because it becomes useful enough to depend on before the organisation has built the right habits around it.

Why Enterprise AI Is Becoming Harder To Understand

Enterprise technology was already complicated before AI arrived and started confidently volunteering to help. Most large organisations aren’t running one clean, elegant technology stack. They’re running layers.

There are legacy systems. Cloud platforms. Data warehouses. SaaS tools. Security controls. Identity systems. APIs. Automation workflows. Custom applications. Reporting tools. And, somewhere in the background, at least one spreadsheet that absolutely should not be business-critical but somehow is.

AI doesn’t replace all of that complexity. It connects to it. And that’s where the problem starts. Modern enterprise AI is rarely one model doing one clearly defined task in isolation. It may involve a foundation model, a retrieval system, a prompt layer, a workflow engine, a data pipeline, a vendor platform, and a set of integrations into existing business tools.

Each part may be understood by a different team.

• The data team knows the source systems. 
• The engineering team knows the application. 
• The security team knows the controls. 
• The compliance team knows the policy. 
• The business team knows how the output is being used. 
• The vendor knows the model behaviour, up to a point.

But the actual system is all of those pieces working together. And that’s much harder to understand. This is how organisational blind spots form. Not because people are careless. Not because teams aren’t doing their jobs. But because nobody owns the full chain from input to output to business consequence.

Take a simple customer-facing AI assistant. From the outside, it looks like a chat window. Nice and harmless. Possibly too cheerful. Behind the scenes it may be pulling in product documentation, customer records, policy files, support tickets, pricing data and past conversations. 

And it may integrate with workflow tools, ticketing systems, identity controls and escalation paths. If it gives the wrong answer, where did it go wrong?

• Was the source data wrong? 
• Was the policy outdated? 
• Was the retrieval layer weak? 
• Was the prompt unclear? 
• Did the model infer something it shouldn’t have? 
• Did the workflow send the answer somewhere it shouldn’t go?

There may not be one clean answer. That’s why this is no longer just a technical problem. It’s becoming an enterprise architecture problem, a governance problem, and a leadership problem.

The risk sits in the gaps between teams. And because AI systems can appear simple at the user level, those gaps are easy to underestimate. A clean interface can hide a messy chain of dependency. Which is fine, until the organisation needs to explain what happened and why.

What AI-Generated Code Reveals About Dependency

AI-generated code gives us one of the clearest examples of this shift. For years, software development has been built around a simple assumption. People write code, people review code, people maintain code, and people are held responsible when the code does something unfortunate.

Obviously, that’s the tidy version. Real development also includes rushed deadlines, strange dependencies, old documentation, half-remembered decisions, and comments in the code that say “temporary fix” from 2019. Still, the basic accountability model was clear enough. 

AI coding tools are changing that. Developers are now using AI assistants to generate, complete, refactor, explain, and test code. In many cases, that’s useful. It can speed up routine work. It can help developers move through repetitive tasks. It can make complex problems feel less stuck.

But it also changes the work. The developer isn’t always writing every line anymore. Increasingly, they’re reviewing, correcting, validating, and securing code the machine helped produce.

That’s a meaningful shift. SonarSource’s 2026 State of Code Developer Survey found that 82 per cent of developers agree AI helps them code faster. But 96 per cent don’t fully trust AI-generated code to be functionally correct.

That tension says a lot. AI is useful enough to keep using, but not trusted enough to leave alone. And that’s exactly where AI-generated code becomes an early warning sign for the rest of the enterprise. The issue is not simply that AI might write bad code. Humans write bad code too. We’ve made a whole industry out of proving that daily.

The deeper issue is that organisations may start inheriting systems made from code that was partly written by humans, partly generated by AI, partly reviewed under time pressure, and later maintained by people who weren’t involved in the original decision.

That makes understanding thinner. Over time, teams may know that the system works without fully knowing why it works, where the risks sit, or how fragile certain parts have become. This has real implications for software governance and application security.

If AI-assisted code enters production, organisations need more than a general feeling that someone checked it. They need review standards. They need security testing. They need traceability. They need documentation that shows what was generated, what was changed, and what was approved.

Because the risk isn’t only faulty code. It’s unexplained code becoming operational code. Once that happens, software teams don’t just manage applications. They manage inherited uncertainty.

The Invisible Labour Behind AI

There’s a popular story about automation that refuses to die. It goes something like this: The system does the work. The people get their time back. The business becomes faster, leaner, more efficient, and somehow less dependent on manual effort. Lovely idea.

Not always how it plays out though.

Automation often removes one kind of work and creates another. AI is no different. The work may move away from direct execution, but it doesn’t disappear. It becomes monitoring, testing, reviewing, validating, auditing, exception handling, and explaining. Which is less glamorous than an AI demo, but far more important once the system is live.

This is the invisible labour behind AI.

Someone still needs to check whether outputs are accurate. Someone needs to investigate weird behaviour. Someone needs to maintain permissions. Someone needs to update source material. Someone needs to define escalation paths. Someone needs to check whether the system is drifting.

Model drift is when an AI system’s behaviour or performance changes over time because the data, environment, users, or business context has changed. It’s a simple idea, but it can be easy to miss if nobody is actively watching for it. The more autonomous systems become, the more important human oversight becomes.

Not because humans need to approve every single thing. That would defeat half the point of automation and probably make everyone miserable by Thursday. But humans still need to decide where judgement belongs. 

They need to know which outputs can move automatically, which need review, and which should never be handed to a machine without a person in the loop. This affects more teams than people often realise.

• Security teams have to understand whether AI is reducing alert noise or hiding signals they should see. 
• Operations teams have to manage automated workflows. 
• Data teams have to maintain the quality of the information feeding the system. 
• Governance teams have to translate policy into usable controls. 
• System administrators have to keep increasingly abstracted environments stable, even when nobody fully sees how all the pieces connect anymore.

System Administrator Appreciation Day sits at the end of July, which feels almost too fitting. Because if there’s one group that understands invisible operational labour, it’s system administrators. They know what happens when a system that everyone forgot about suddenly stops doing the thing everyone quietly depended on.

That’s the lesson AI leaders should take seriously. The work that keeps intelligent systems reliable may not always be visible. But if it’s underfunded, understaffed, or poorly owned, the weakness will show up eventually. Usually at the least convenient time. Because systems do have a sense of drama.

The Rise Of AI Observability

At some point, organisations may have to make peace with an uncomfortable truth. They may not be able to fully explain every AI output in every system, every time. That doesn’t mean they should stop trying to understand their systems. It means they need a more practical way to manage them.

This is where AI observability comes in.

Observability is already familiar in cloud, infrastructure, and software environments. It’s the practice of understanding what a system is doing by watching its signals. Logs, metrics, traces, errors, performance patterns, and behaviour over time all help teams work out whether something is healthy, slow, broken, or about to ruin someone’s afternoon.

AI observability applies that same thinking to intelligent systems. But instead of only asking whether the system is online, organisations also need to ask whether it’s behaving as expected.

• Are the outputs still accurate? 
• Is performance getting worse? 
• Are certain users getting weaker results? 
• Are biased patterns appearing? 
• Are prompts producing strange responses? 
• Are AI agents taking actions they shouldn’t take? 
• Is the system relying on outdated information?

Gartner predicts that 40 per cent of organisations deploying AI will use dedicated AI observability tools by 2028 to monitor model performance, bias, and outputs. That prediction points to a practical shift. Traditional monitoring isn’t enough for AI.

When a normal application fails, it’s usually pretty obvious. It throws an error. It crashes. It slows down. It refuses to load, usually when you’re presenting something to people with job titles that make the room tense.

AI can fail a whole lot more quietly. It can give a wrong answer that sounds reasonable. It can summarise outdated content. It can recommend an action based on incomplete data. It can behave well in testing and then degrade after users start interacting with it in ways nobody predicted.

So organisations need visibility into behaviour, not just uptime. AI observability won’t solve every governance problem. It won’t magically explain every decision. It won’t turn a complex model into a neat flowchart. But it gives teams a way to see whether trust is still justified. That’s the practical centre of the issue.

The goal isn’t perfect certainty. It’s enough visibility to catch problems before they become business problems.

From Execution To Validation

AI is changing enterprise skills, but not always in the way the public conversation suggests. The loudest debate is still about replacement. 

• Which jobs will go away? 
• Which tasks will be automated? 
• Which teams will shrink? 
• Which roles will be safe?

Those questions are understandable, but they can flatten the real shift. In many organisations, AI isn’t simply replacing work. It’s changing what valuable work looks like. For a long time, a lot of enterprise roles were built around production. 

Write the report. Build the dashboard. Draft the code. Analyse the data. Answer the query. Create the document. Complete the process. AI can now help with many of those first-pass outputs. But someone still has to decide whether the output is correct. Someone still has to know whether it fits the business context. 

Whether it’s safe to use. Whether it’s compliant. Whether it reflects the right data. Whether it has missed something obvious to a human but invisible to the system. That makes validation a core skill. Validation isn’t passive checking. It’s not glancing at a machine-generated output and thinking, “Looks fine enough, off we go.” 

It requires judgement. It requires domain knowledge. It requires enough AI literacy to understand where systems can go wrong. It also requires enough business understanding to know what a wrong output could affect. This is where workforce transformation becomes more practical than dramatic.

• Developers will need stronger code review and security validation skills. 
• Analysts will need to test AI-supported insights against business reality. 
• Managers will need to know when automation is improving a process and when it’s covering up a weak one. 
• Administrators will need to manage increasingly abstracted systems. 
• Security teams will need to separate useful signals from AI-shaped noise.

The next generation of enterprise skills won’t only be about using AI. It’ll be about supervising it. That means AI skills need to move beyond prompt writing. Prompting is useful, yes. But it’s not enough.

People need to understand how AI systems behave. Where they’re strong. Where they’re brittle. Why confidence is not the same as correctness. Why a polished answer can still be wrong. Why automation still needs ownership.

The valuable employee in an AI-enabled organisation won’t always be the person who can produce the most. It may be the person who can tell when the machine-produced answer shouldn’t be trusted. 

Governing Systems We Can’t Fully Understand

AI governance is often treated like a policy problem. Create the guidelines. Define approved tools. Add risk categories. Train employees. Set review processes. Keep records. Make compliance happy enough to stop sending follow-up emails.

All of that has a place. But it’s not enough if AI is becoming part of the operating environment. Traditional governance often assumes systems can be clearly inspected, explained, tested, and owned. 

AI complicates that assumption because the system is often spread across models, data sources, prompts, workflows, vendors, applications, and user behaviour. So the governance question can’t only be: Was this AI use case approved?

It has to become: Do we understand what this system is doing inside the business?

That’s a much bigger question.

• It includes accountability. Who owns the output?
• It includes visibility. Where is the AI being used?
• It includes data. What information is the system relying on?
• It includes control. When does a human step in?
• It includes resilience. What happens when the output is wrong?
• It includes auditability. Can the organisation explain the role AI played in a decision after the fact?

NIST’s AI Risk Management Framework was developed to help organisations manage AI risks to individuals, organisations, and society. Its generative AI profile also frames risk management around the need to govern, map, measure, and manage risks across common business processes.

ISO/IEC 42001 takes a similar direction by setting requirements for establishing, maintaining, and continually improving an Artificial Intelligence Management System for organisations that provide or use AI-based products and services. 

The EU AI Act adds another layer. It entered into force on 1 August 2024 and is set to become fully applicable on 2 August 2026, with some exceptions. The direction is clear enough. AI governance is moving closer to operational accountability. And that means organisations need to stop thinking about governance as something that sits outside the system. 

Governance has to sit inside the way AI is selected, deployed, monitored, reviewed, and maintained. This doesn’t mean leaders need to understand every model parameter. That’s not realistic, and pretending it is just creates a new theatre of control. Lovely paperwork. Very little safety.

What leaders do need is organisational understanding. They need to know where AI is being used, which decisions it influences, which teams own it, which risks have been accepted, and how problems will be caught. That’s the real governance challenge.

Not controlling AI from a distance. Understanding what the organisation has allowed itself to depend on.

Final Thoughts: Dependency Without Understanding Creates Fragility

AI is becoming part of the machinery of the enterprise. Not all at once. Not always visibly. Not always through systems labelled as “AI strategy”. Sometimes it arrives through a coding assistant, a workflow feature, a chatbot, a reporting tool, a security platform, or a vendor update someone approved because it looked useful.

That’s how infrastructure often grows. Without anyone really noticing. Then suddenly everyone relies on it. This is why the next stage of enterprise AI won’t only be about adoption. It’ll be about comprehension. The organisations that move fastest won’t automatically be the strongest.

Speed can help, but only if it’s paired with understanding. Otherwise, the business may simply create a faster route to decisions it can’t explain, systems it can’t inspect, and workflows it can’t properly govern.

The stronger organisation will be the one that knows where dependency is forming. It’ll understand which AI systems are shaping decisions, where human oversight sits, and how accountability is maintained.

Because the real risk is not that AI becomes useful. It already is. The risk is that it becomes useful enough to disappear into the background before organisations understand what they now depend on.

As AI moves deeper into enterprise operations, the conversation is shifting from what these systems can do to how organisations can govern, validate, and remain accountable for them. EM360Tech will keep following the technologies, leadership decisions, and operational challenges shaping that transition.