5 Data Engineering Lessons We've Learned From Real Client Work

Quick answer

Across client work, the same patterns keep showing up: bad KPIs almost always trace back to a modeling problem, not a dashboard problem. A hundred tables can usually be modeled down to ten. Systems that don't talk to each other hide the full picture of the business. Spreadsheet exports are a bigger risk than most companies realize. And employees are already putting company data into AI tools whether or not there's a system built for it.

Key takeaways

• A KPI that looks wrong is almost always a modeling problem. Fix the data before you build another dashboard.
• Good data modeling can compress a hundred tables into ten without losing any insight.
• Disconnected systems, like two CRMs that don't talk to each other, hide the full picture of the business until someone integrates them.
• Spreadsheet exports, not system breaches, are one of the most common ways sensitive data actually leaks.
• Employees are already using AI with your company's data. The question is whether that's happening deliberately or by accident.

When a KPI looks wrong, check the data before building another dashboard

When a company suspects its KPIs are inaccurate, the instinct is almost always to build another dashboard, or add another view, or try a different analysis angle. Most of the time, that's not the issue. The underlying data usually isn't modeled properly, or the metric being tracked doesn't actually answer the question being asked. If a number looks off, look at how it's assembled before you build anything new around it. This is the same root cause behind why your Monday reports never match.

A hundred tables can become ten

We worked with a client who had over a hundred tables. Once we actually understood their business and what they needed to track, we compressed all of it down to ten tables that covered every insight they cared about. That's not a small difference. Learning ten tables instead of a hundred changes how fast people can actually use the data, and how much easier metric tracking becomes across the board.

Disconnected systems hide the real picture

Every company ends up with a handful of tools that don't talk to each other, usually because different teams picked different systems for good reasons at the time. We had a client running two CRMs for two different purposes, which made sense for how they operated, except the two never connected. We pulled both into one place for analysis, and later found two more systems other teams were using that told other pieces of the story.

Once everything was together, they could see the whole business instead of one narrow slice of it, whether that was the customer experience side or the lead side. That full picture is what actually drives better business decisions. A narrow view of one system can't do that on its own.

Your biggest data risk might just be a spreadsheet export

If you're looking for the most common way sensitive data actually gets exposed, it's rarely a system breach. It's someone exporting a table or a spreadsheet and sharing it with colleagues, without much thought about where that file ends up next. Most of the time there's no bad intent behind it. People just aren't always as careful with the details as you'd want them to be, especially with something as ordinary-feeling as a spreadsheet. When that file contains confidential or personally identifiable information, that ordinary habit becomes a real liability. We go deeper on the compliance side of this in what GDPR and HIPAA require from your data pipeline.

Your team is probably already feeding data into AI, whether you've planned for it or not

If you're a business owner, it's worth asking yourself whether you actually know what your employees are putting into AI tools. Most people are already using AI for their own productivity with whatever tools they have access to, whether or not the company has thought it through. What's much rarer is a company deliberately leveraging its own data for AI-driven process improvements across the whole business, rather than leaving it to individual employees to figure out on their own. For what that actually looks like once it's done deliberately, see what AI needs from your data and why plugging your data straight into AI is riskier than it looks.

Turning lessons into infrastructure

None of these are one-off problems. They're patterns that show up across almost every company we work with, at different stages. The fix in every case is the same: get the data centralized, modeled, and governed before it turns into a bigger issue, whether that's an inaccurate KPI, a compliance risk, or an AI tool giving confident wrong answers.

This is what we build

Every one of these five lessons is something we check for on day one with a new client: how the tables are modeled, which systems aren't talking to each other, where exports are leaking, and whether the data is actually ready for the AI tools the team is already using. See how we build your data foundation.