Your Lakehouse Doesn’t Need Another Query Engine

The industry’s obsession with benchmark performance is solving the wrong problem entirely.

When I joined Flamelake’s platform team in late 2021, we were processing four petabytes of analytical queries daily across a distributed lakehouse architecture. Our latency target was 95th-percentile under 30 seconds for ad-hoc queries, and we hit it consistently. Nobody complained about query speed. Yet our data platform was quietly falling apart around us in ways that no benchmark would ever surface.

The Metadata Layer Is Where It Breaks

The problem was never performance — it was coherence. Every morning, engineers asked why a pipeline had failed overnight. The answer was almost always a silent schema change upstream breaking a dependency three hops away. We had no lineage graph, no contract enforcement between producers and consumers, and no way to predict whether renaming a single column would break a revenue dashboard used by the CFO every Monday.

We could scan a terabyte in twelve seconds but could not tell you which team owned a given table, or who had queried it last.