Data warehouse: what it is and when you need one

Half the companies that ask about a data warehouse have, all told, about 40,000 rows of data. That fits in a spreadsheet. It fits in Google Sheets, with room to spare.

The other half has information scattered across seven systems, closes the month with three different revenue figures, and believes the problem is the report. That half genuinely needs one, and almost never understands what it is buying.

The two conversations deserve to be separated, because the mistake runs in both directions. Build too early and you burn six months on infrastructure nobody queries. Wait too long and you make decisions on wrong numbers for years, noticing only once the damage has become routine.

What is a data warehouse?

A data warehouse is a central database, organized for analysis, that brings information from several systems, ERP, CRM, ad platforms, spreadsheets, into a single model with history preserved. It does not run the business: it exists so people can query, compare periods and build reports without slowing down the source systems.

That last part usually gets skipped. A data warehouse is not "a bigger database." It is a database with a different purpose, and the purpose changes everything: the modeling, how often data is written, the kind of query it can absorb.

Why won't your current database do the job?

The system that issues invoices was designed to write one transaction at a time, fast, losing nothing. That design is called OLTP. It is excellent at answering "what is customer 4,812's balance right now" and terrible at answering "what was margin by product line, by region, month over month, across three years."

When someone runs that second question straight against production, two things happen. The query drags, because the structure was never meant to scan millions of records. And the system slows down for everyone working in it, because the heavy query competes with the operation. So IT eventually forbids it, and the business team goes back to exporting CSVs.

There is a quieter problem too: operational systems overwrite the past. If a customer changed tier in March, the database shows today's tier. You lose the ability to answer "how did things look back then." A data warehouse keeps those snapshots. It is one of the real reasons it exists, and it rarely shows up on benefit lists.

When does a company actually need one?

There is no magic row count. What usually decides is the combination of three things: how many sources need to talk to each other, how far back you need to look, and how many people query without asking someone for help.

Concrete signs the moment has arrived:

• Two departments present different numbers for the same question, and nobody can say which one is right.
• The monthly report depends on one specific person stitching five exports together by hand.
• You cannot tie ad spend to closed revenue because each number lives somewhere else.
• You have hit the spreadsheet's ceiling: it freezes, corrupts, or takes minutes to recalculate.
• You need a year-over-year comparison and the source system already overwrote that state.

Signs it is not time yet: you have two sources, the whole operation fits in a weekly export, and the real pain is sloppy data entry rather than missing technology. A warehouse does not fix bad records; it just distributes the error faster, to more people, with more authority. Before investing, deal with data quality at the source.

Data warehouse, data lake or lakehouse: what is the difference?

All three store data, but they start from different assumptions about when you organize it.

Trait	Data warehouse	Data lake	Lakehouse
Format	Structured, modeled before loading	Raw, exactly as it arrived	Raw with a structured layer on top
Daily users	Analysts and business teams	Engineering and data science	Both profiles
Storage cost	Higher per GB	Lower per GB	In between
Typical risk	Rigidity: changing the model is work	Becoming an uncatalogued dump	Operational complexity
Fits	Most mid-sized companies	High volume and unstructured data	Teams with dedicated engineering

An opinion from having watched this go wrong: if your company has no one dedicated to data engineering, a data lake becomes a shared folder with a nicer name. Start with the warehouse, which forces organization, and migrate only when volume or data type demands it.

What does a data warehouse actually cost?

The infrastructure bill is far less frightening than most people expect. In BigQuery, Google charges US$6.25 per tebibyte scanned, with the first 1 TiB of queries and the first 10 GiB of storage free every month, according to the official pricing page.

A worked example. A distributor with 3 million sales rows, 800,000 CRM records and two years of ad data takes up roughly 8 GB, inside the free storage allowance. If the team runs queries scanning 200 GB a month, it still sits under the free 1 TiB. Infrastructure cost: zero, or close to it.

The real cost sits elsewhere: who builds it and who keeps it alive. The ETL routine needs an owner. The ad platform's API changes, the category field gains a new value, the ERP gets updated, and the load breaks on a Friday. Budgeting the warehouse without budgeting maintenance is the most common way to end up with a dead project in twelve months, with the tables still there, stale, and everyone back on CSVs.

Worth remembering the size of the damage on the other side. Gartner estimates that poor data quality costs organizations at least US$12.9 million a year on average, as published on the firm's own site. That is not your number, it reflects large organizations, but the proportion tends to hold: bad decisions cost more than the infrastructure.

How to leave the spreadsheet without stalling the project

Sequence matters more than tooling. A path that works:

1. Pick a business question, not a set of tables. "What is CAC by channel, by month" is a scope. "Centralize the company's data" is not.
2. Bring in only the sources that answer that question. Two, three at most. Resist the urge to plug in everything.
3. Model with the names the business already uses. If sales says "closed won," the table should not be called stage_id_7.
4. Automate the load before building the pretty dashboard. A dashboard on stale data is worse than a spreadsheet, because it looks trustworthy.
5. Only then open it to other teams. Every new team brings new business rules, and it is better to find that out once the structure is stable.

Scoped this way, you solve the most expensive symptom of data silos (numbers that never reconcile between departments) without needing a twelve-month project before anything delivers value.

The mistakes that kill the project halfway

The first is starting with the tool. The meeting turns into Snowflake versus BigQuery versus Redshift before anyone has defined which decision the warehouse is supposed to support. For most mid-sized companies all three work; the choice barely changes the outcome and badly delays the start.

The second is replicating the mess. Dump the ERP tables into the warehouse without modeling and you have bought a pricier copy of the same confusion. The translation work, from system structure to business structure, is what creates the value.

The third is underestimating the routine. Anaconda's survey of data professionals found that, on average, 45% of their time goes to preparing data (loading and cleaning) before any analysis begins, according to the State of Data Science 2020. A warehouse lowers that share but shifts part of the effort into keeping the loads healthy. Skip that in planning and you will believe the job ended the day the first dashboard shipped.

Frequently asked questions

Is a data warehouse a database?

Technically yes, with a different purpose. An operational database is tuned to write transactions quickly; a warehouse is tuned to read large volumes and cross dimensions. The underlying technology may look similar, but the modeling and usage patterns are not.

What is the difference between a data warehouse and a data lake?

The warehouse organizes data before storing it; the lake stores first and organizes later. In practice the warehouse gives business users answers faster, while the lake offers engineers more flexibility. Mid-sized companies usually do better starting with the warehouse.

Does a small business need a data warehouse?

In most cases, not right away. While the operation fits in two or three sources and one person can consolidate it in a day, a spreadsheet or a simple database with BI will do. The turning point is usually when consolidation time becomes recurring and versions start to diverge.

How long does it take to implement a data warehouse?

With a tight scope (two sources and one clear business question), four to eight weeks to the first usable result. Projects promising to centralize everything at once tend to run past six months and lose their sponsor before delivering.

Do I need a data professional to maintain it?

Someone has to own the load, even part time. Managed tools remove much of the infrastructure work, but source changes still break routines. Without a named owner, the data ages and the team stops trusting it.

The decision that matters is not technical

The question worth asking is not "which data warehouse should we pick," but "which decision have we been making blind, and what is that costing us." If the answer is a monthly meeting where every department brings a different number, the structure pays for itself quickly. If it is dashboard curiosity, it is expensive for the return.

Centralizing data only pays off when people can actually ask it things. That is the gap Sherlok covers: connecting the sources you already have (ads, CRM, spreadsheets, SQL databases) and letting anyone on the team ask a question in plain language and get the analysis back, instead of joining a queue for the technical team. It works as a first step, including for finding out which questions justify a warehouse later.