Securing AI at Scale: What Enterprises Get Wrong

Rolling out artificial intelligence across a business sounds pretty straightforward until security enters the conversation. On a recent episode of the Security Strategist Podcast, host Richard Stiennon sat down with Omar Khawaja, Field Chief Information Security Officer at Databricks, and Danny Healy, the company's Lead Data & AI Strategist, to explore why so many organisations fail when moving AI from pilot to production. Everything always looks good in theory, but it's harder to execute in reality.  This discussion offers a practical take on AI governance, risk and the cultural shifts security teams need to make to protect their business at scale.

Why Shadow AI Grows on Indecision

Khawaja opens with a warning that should resonate with any CISO watching AI adoption outpace their controls. The instinct to "wait and figure it out" is, in his view, the single biggest misconception in AI security. Every month spent deliberating is a month in which employees quietly adopt unsanctioned tools themselves, and that delay creates a wider window for shadow AI to spread unchecked.

The bigger issue, he explains, is that security teams keep reaching for playbooks built for deterministic systems, basically software that behaves predictably every time. However, AI doesn't work in this manner. It's probabilistic, which means outcomes are very different, and organisations that expect old-world controls to transfer seamlessly are setting themselves up for failure. This mismatch tends to push companies toward one of two extremes, which is either drowning use cases in exhaustive control lists or quietly ignoring the problem until it becomes unavoidable.

Khawaja’s solution is straightforward, despite the complexity of the problem. Before writing a single policy, he asks leadership teams a question: can your architects actually draw what your AI system looks like? Without a shared view of the main components, which range from data pipelines and models to agents and permissions, different governance teams can end up solving different problems, all while thinking they are on the same page when they are not.

He also takes a pragmatic view of AI adoption. Drawing on Thomas Aquinas, if the job of a ship's captain was to keep it from sinking, he would never leave harbour. He suggests that while avoiding all risk may seem safe, it also limits what can be achieved. The goal for security, he argues, is to help organisations find a safe and defensible path to using AI effectively.

Why Trust Comes Before Speed

Meanwhile, Healy brings the conversation back to a more fundamental issue: trust. Databricks learned this the hard way inside its own operations. An early attempt at agentic threat triage used a single generalist model across multiple data sources and produced far too many false negatives. Swapping in smaller, specialist models for each source improved accuracy, a reminder that AI security maturity is built through iteration, not theory.

Healy breaks trust down into three main areas: auditability, so teams can understand how an agent reached a decision; limited data access, so agents only access what they need; and resilience against manipulation, particularly as agents combine multiple actions that could create greater risks.

That last point echoes something Khawaja raises later: the concept of contextual policies. Rather than static, role-based permissions that struggle to scale, contextual policies assess the actual risk of a sequence of actions in real time. Instead of users robotically approving every access request until they switch on "auto mode" out of fatigue, the system flags genuinely risky actions and lets routine ones pass - a smarter, more human-centred approach to access management.

From 97 Risks to a Focused Shortlist

The most useful takeaway from the episode is how Databricks approaches AI governance frameworks. Rather than starting with controls, the company's open, vendor-agnostic AI Security Framework, now in its third version, starts by mapping the AI system itself, then cataloguing the risks that could affect it. The list currently runs to 97 risks, nearly double what it was three years ago, largely due to the rise of agentic AI.

In practice, organisations are not expected to address all 97. Khawaja notes that a well-governed, modern data platform already neutralises a large chunk of them, leaving perhaps five to fifteen genuine concerns per use case. In this regard, each is mapped to specific, actionable controls rather than vague objectives. It's a philosophy borrowed from Khawaja's OODA loop framework (observe, orient, decide, act). This means most organisations are competent at observing problems and acting on them, but weak at the orientation and decision-making in between- the stage where risk triage happens.

Healy makes a clear case that a well-governed data layer is about more than compliance. It's a competitive advantage. Organisations with clean, contextual, well-governed data give their AI agents an edge that attackers who lack that internal context simply don't have.

The message from both guests is consistent throughout, and that is securing AI isn't about building the longest possible list of controls. It's about shrinking an intimidating problem down to something teams can actually solve, deliberately, iteratively, and without grinding the business to a halt. If you would like to find out more, please visit databricks.com or follow Omar Khawaja and Danny Healy on LinkedIn.

Takeaways

• Challenges of deploying AI securely at enterprise scale.
• Evolving security strategies for probabilistic AI systems.
• Importance of AI governance and risk management.
• Drawing a system picture for AI security.
• Implementing controls and permissions for AI agents.
• AI governance frameworks and standards.

Chapters

00:00 Introduction to AI security challenges in the enterprise

01:00 Misconceptions about securing AI and shadow AI risks

02:12 Learning from organisations that have secured AI

03:25 How AI changes cybersecurity strategies

04:52 The importance of understanding AI system architecture

06:50 The risks of banning AI versus managing it responsibly

08:37 Obstacles in operationalising AI securely

10:00 Building trust through model auditability and control

12:00 The role of external consultants and frameworks

13:03 Databricks' approach to AI security and governance

15:11 AI governance complexities and the UDA loop

17:27 Using risk-based controls instead of exhaustive controls

18:46 Designing permission controls for AI agents

21:52 Content and intent analysis for agent security

24:35 Developing effective AI security frameworks

27:47 Key takeaways for AI security and governance

29:28 Final thoughts on making AI security manageable