The Real Number is 0%: Ohalo CEO on Unstructured Data in Enterprise AI

The person who calls the shots picks up his regular cup of drip coffee, listens to Fukui’s Scenery album and goes through X, Apple’s newsfeed and finally Slack. 

Before tackling fast data rotation in enterprise tech, the CEO reads the latest business insights on unstructured data from a new BARC study conducted by Kevin Petrie, VP of Research at BARC, and Merv Adrian, Senior Analyst, Data & AI, BARC Fellow. 

This new BARC research revealed both exciting and surprising information about how enterprises manage their unstructured data. 

The study found that the majority of enterprises based in the US and Europe believe they can retrieve value from unstructured data without compromising governance. 

However, there’s a conundrum: Just 29 per cent of enterprises fully know where their relevant files are located, and 70 per cent of enterprises reported that less than half of their unstructured data is discoverable and usable for AI.

Ultimately, the report concludes that enterprises are often overconfident in their readiness, and that "you cannot govern what you cannot find.”

This is why EM360Tech’s Shubhangi Dua, Podcast Host, Producer and Tech Journalist, sat down with Kyle DuPont, the CEO of Ohalo, for an interview on what the research really means and how it is going to impact unstructured data in enterprise tech.

Ohalo sponsored the BARC research titled: Harnessing Unstructured Data for AI Innovation. The firm is also the technology provider addressing the specific gaps highlighted by the study.

Dua also requested Kevin Petrie for comments on the report. The author of the research said that unstructured data holds the key to proprietary context that feeds AI models and agents. 

“Tables have valuable context, but if you're looking for context that really builds competitive advantage in terms of how your agents understand your distinct business processes, a lot of that shows up in unstructured data.”

He added that the unstructured data shows up in emails, documents, contracts, business records, and images. Unless an enterprise captures the context in its unstructured data, you're not gonna feed your AI agents the necessary insights to build sustainable competitive advantage.

In this interview article with DuPont, Dua dives into why the research was necessary, what Ohalo’s role is, key Agentic AI-related issues enterprises need to face right now and the BARC report’s shocking statistics. Additionally, we address why enterprises are slow to adopt AI, the political atmosphere that creates a ripple effect on how large enterprises manage their unstructured data, and key takeaways for C-suites. 

Dua: Why research unstructured data for AI innovation in the first place?

DuPont: There are a lot of enterprises that have been "doing AI," or deploying AI. The fact is that the gap between what enterprises are actually doing and what they believe they ought to do is really wide. 

Enterprises are 18 to 24 months behind the state-of-the-art of what's happening in Silicon Valley. When I speak to people in Silicon Valley, I often tell them that they are pursuing something that was meant to be adopted in 2024. 

However, the BARC research sponsored by Ohalo helped identify the real gap in the market.

Dua: Where does Ohalo come into play?

DuPont: We sell a product called Data X-Ray. Data X-Ray provides a map of your unstructured data, your files, and tells you where they are. The big problem a lot of companies face is that while AI tools work miraculously for one person working alone, enterprises aren't one person — they're teams of people collaborating across multiple functions. Current AI doesn't naturally support that.

As an example, one of Ohalo’s clients is a Department of Defence contractor, and they're writing a proposal for a line item budget for the Department of Defence. 

Now they’re faced with an issue with writing a proposal because it's not just about one person dumping their brain out onto a piece of paper and then submitting it. Your bid team is required to understand the finances around the bid. Then the technical team is necessary to understand whether or not the technical thing that you're delivering meets the spec of the RFP. You have to have an understanding of historical work in this area, what the company does. All these factors that are pulled together.

Dua: What’s the key issue with AI agents and how should enterprises tackle it?

DuPont: If you want to make that an agentic workflow, the enterprise needs to know about what files are where and what files are relevant, and what files are, from a temporal standpoint, relevant to a certain time period. The agent needs to know about what your files are and where they are.

The issue is the Agent can’t be trusted with a trillion files.

This is where Ohalo comes in. We're working at billion-files scale in large enterprises. 

You can't just send a billion files to an agent, and you don't have the capacity to just retrain a model every day as your files change all the time. This is why we have to compress the files down into an index similar to Google. 

Agents need a contextual metadata layer to be able to respond to user requests, or other agents' requests, to build that out. That's where the world's going. It’s moving away from just chatbots that are helping somebody singularly be more productive, to actually helping teams be productive within large tasks within enterprises.

Dua: The BARC report says 79–80% of enterprise data is unstructured and unsorted, but even before you can optimise it, doesn't there still need to be a filtration process?

DuPont: My co-founder Alistair often says there really shouldn't be any unstructured data. 

The problem is that there is unstructured data in the first place, and nobody can query it. If everything was structured, you could query it like you query a database, and the problem would be solved. 

However, when the personal computer came out in the late '70s, you started getting unstructured data. The user has the ability to put random natural language into a digital format; everything becomes unstructured data. That's kind of 50 years of legacy that we're dealing with now. 

This is why 80 per cent of enterprise data is unsorted, and most of that 80 per cent is actually junk. An enterprise doesn’t want to spend AI tokens on junk.

If I'm a defence contractor, or I'm a top-four accounting firm, or I'm a government entity, I have a lot of data out there, and recipes for chicken soup are probably not relevant for a lot of the use cases that I'm trying to deal with. This is why you have to compress all that into a searchable thing.

AI needs to have an index that tells you exactly where that particular data is located, so that enterprises don’t burn tokens with random data that's irrelevant to a user's job or query or task that they're trying to get done.

Dua: What is the biggest number that shocked you from the research

DuPont: The first was that 29 per cent of people said that they knew where all their unstructured data was, and they could use it. However, 70 per cent of people don't know where that is. That's too few.

In fact, it's strange that 29 per cent even know about it.

The other data point is the vectorisation story, and how enterprises are shifting from standing up their own vector databases to pivoting to agentic frameworks, and its implications. More research is required to really understand the details, but my own hypothesis is that enterprises used vector databases and got them stood up as RAG systems.

Enterprises are struggling to scale RAG systems. This is because vector databases might handle hundreds or thousands of files, but at a million documents they only return the top 50 results, and the best answer is unlikely to be in there. That's the enterprise reality.

Pure RAG systems are going to fade away. Spending six to twelve months building something that just queries HR policies isn't going to boost the stock price. 

The real value is everyone having agents that can build proposals, win new business, actually move the needle. That's where enterprise AI is heading.

Also Watch: Why Enterprise RAG Systems Still Produce AI Hallucinations

Dua: Large enterprises are slower to adapt because of security and governance concerns — what does the research say about navigating that?

DuPont: One of the more interesting findings was a clear gap in AI adoption confidence between US and non-US companies. US companies were significantly more confident. 

The research didn't go into why, but I have two hypotheses: one is political, where non-US companies know Anthropic and OpenAI have the best models, but they're wary of becoming dependent on them. 

The second is that non-English languages are less supported, though the models are improving so fast that'll likely fade. It's a finding that warrants a follow-up study.

Also Watch: Chief Data Officer Strategies for the AI Era

Dua: The political and regulatory atmosphere is more unpredictable than ever. Does that create a ripple effect on how large enterprises manage their unstructured data?

DuPont: It might be a provocative take. To put it into perspective, imagine an AI-purist world, where the future of the world is AI, robots, AI undertaking most tasks, and an enterprise makes profit from a fiduciary of their shareholders' finances. The most logical move as an enterprise is just to purchase Anthropic and OpenAI stock. And that's not a great future.

At an enterprise level, you have to question what truly is the best thing for the organisation; what’s your moat? It's the specialist knowledge locked in their systems like a big-four firm's 25 years of M&A history, a DoD contractor's manufacturing know-how. That's data no model has access to, and that's the competitive edge.

Where does that specialist knowledge live? It lives in a plethora of emails, PowerPoint presentations, Excel documents, Word documents, PDFs, CAD documents, CAD files. It lives in all this information that is in the world. 

The ripple effect is that things that are commodities, that are in public training data, for instance- accounting standards, or how to sell software, or CRM- could be built with AI in the future. This will vanish. 

However, firms will get much more specialised in the things that they do well. That's the ripple effect of understanding your unstructured data. It means to ultimately protect the moat that you have, because it's the thing that you know how to do the best. 

The information already in the public training data, that can be vibe-coded easily, such as a CRM, will go away eventually. That’s going to be hard. 

Dua: What is the main key takeaway for C-suites from CFOs, CTOs, CEOs, to CISOs?

DuPont: The key takeaway is the confidence gap. Although about 70 per cent of people say that they can use AI and that it's making them more productive, in reality just about 30 per cent believe that they actually have the data to use it. 

I hypothesise that most people think they're more productive with chatbots, and they probably are. However, it's really hard to measure productivity. As you build real AI systems, you need to control everything; you need the index of what's where. That's the main takeaway for me — the confidence gap.

The real question is: if only 30 per cent, and honestly I think it's closer to 0 per cent, actually understand their data, how do we get that to 70%? Because if AI is going to build your future, you have to start there.

It's about understanding, firstly, standard data governance practices, but with data quality, with data cataloguing. Most importantly, records management, which is considered a boring part of the enterprise, but it's actually part of what will drive structuring unstructured data so that you can understand it. 

Records managers are usually people without budget and without much power in the enterprise, but they're fundamental and foundational to this AI world. The record managers are the ones that should organise data for people in the AI space. It's about the basics of data governance, in addition to discovering, classifying, and managing an enterprise’s unstructured data.

Read the report here: https://barc.com/research/harnessing-unstructured-data-for-ai-innovation/