Last week, I attended a Snowflake session on data products, and one comment in particular stood out. The Data Architect presenting said that, in his experience, data analysts at most companies know the actual data better than anyone else. That stuck with me, not because I’m a former data analyst, but because few organizations actually acknowledge it amid today’s automation and AI obsession.
Later that week, during our analyst “watercooler” call, we lamented a problem that seemed remarkably similar: over-reliance on AI to solve problems that are fundamentally human. We discussed the frustrating experience of modern customer service and interacting with bots that seem “almost human” but consistently miss the mark.
This lack of context made me realize that we risk steering data quality into that same “uncanny valley” where data volumes are exploding, but trust doesn’t keep up. Despite widespread adoption, EMA research has shown that fewer than half of enterprises feel their observability strategy is actually successful. In this framework, the data analyst role should be viewed through a new lens, less technical and more as a practitioner of human judgment.
Quality Is About Judgment
We talk about data quality in technical terms and measurable properties, but what makes the data actually useful is whether a human can trust it to make a decision that ultimately impacts other humans. That's a judgment call, and the best and most critical judgment calls move beyond what can be clearly quantified or encoded.
It’s a relationship between the data and the person trying to use it. Maintaining that relationship requires human judgment at virtually every layer. Organizations need to leverage their data analysts as the institutional bearers of that judgment.
The unit of value in data quality isn’t fewer incidents, it’s fewer arguments. If a tool produces fewer results but more confusion, it’s failing.
Empathy as an Analytical Skill
Empathy is implicit in everything good data analysts do; they know which metrics will be misinterpreted, and importantly, they anticipate confusion. Knowing what you know is as important as knowing what you don’t know.
By empathy here, I don’t mean kindness or sentiment; I mean the ability to anticipate how other humans will interpret, misuse, overtrust, or misunderstand data.
Data represents people, behavior, and outcomes that affect humans. Percentages are abstractions, but a conversion rate is a decision someone made, and churn is a customer relationship at risk. When analysts do their job well, they're translating raw information into something another human can act on with confidence, aware of both its power and limits.
This becomes especially critical when data feeds automated systems and AI-driven decisions. If there’s a gap or a bias in the data used for training, the consequences scale rapidly. The judgment required is way beyond technical. It's interpretive, ethical, and relational. It requires understanding not just what the data says, but where it comes from, how it will be used, and who it will affect.
What Happens When Humans Are Removed
In our watercooler discussion, customer service came up as the cautionary example. Sure, AI can handle volume, but the most important cases still require humans. They need to, because trust erodes when automated systems feel "almost human" but miss the nuance. Humans prefer information when it comes from other humans, even when AI can produce it faster.
This is the uncanny valley of automation. Systems that work 95% of the time but act “weird,” or worse, fail catastrophically in edge cases, create more friction than the systems they replace. Users tolerate limitations when they're predictable. They don't tolerate systems that seem intelligent but fail in ways that feel arbitrary.
The same dynamic applies to data quality. You can automate anomaly detection, schema validation, and lineage tracking. But you can't automate the judgment call about whether an anomaly matters, whether a schema change will break downstream logic, or whether lineage documentation reflects the actual trust boundaries in the organization. Those require context, experience, and empathy.
Human-in-the-Loop as a Design Principle
The question isn't whether AI is capable of managing data quality. In narrow, well-defined contexts, it already does. The question is where judgment must remain—and how systems should be designed to support it rather than bypass it. Detection is cheap. The hard part is the decision.
Human-in-the-loop is an acknowledgment of where value actually sits, and the data analyst is the natural control point. They know the data best, apply institutional knowledge, context that can’t be programmed, interpret ambiguity, and make trade-offs that no rule-based or probabilistic system can encode.
This matters for tool design. The best tools amplify judgment rather than replace it. They surface the right information at the right time. They make exploration fast, explanation clear, and intervention seamless. They assume the analyst knows things the system doesn't, and they make that knowledge actionable.
What the Best Tools Get Right
The most effective data quality tools focus on usability, transparency, and feedback loops. They empower the analyst to investigate, override, or contextualize what the system flags and prioritize collaboration over control. The tools that work are the ones that treat the analyst as a partner, not a user who can be abstracted away.
Automation handles scale, and humans handle judgment. The best systems make the most of that distinction.
Data Is Always About Humans
We measure systems, processes, and behaviors, but the impact is always human. Someone will make a decision based on that data, and someone will be affected by that model's output. More bluntly, trust in an organization often hinges on whether its data holds up under scrutiny.
The data analyst is the one best equipped to conduct that scrutiny with the necessary judgment and empathy. Automation can’t replace that role, and as it scales, it becomes even more important. The data analyst is the one who knows the data best and can interpret what it says. The biggest mistake would be to stop listening.
Comments ( 0 )