It is tempting to ask ChatGPT or Claude a simple question: Which is cheaper for my workload, Snowflake, BigQuery or Redshift? The answer will usually look convincing. It may contain current-looking prices, a neat table, some arithmetic and, at the end, a recommendation.
The arithmetic is rarely the difficult part. The difficult part is establishing what should go into it.
Software vendors often charge for fundamentally different things. Snowflake charges for credits consumed by warehouses, BigQuery can charge for data scanned, and Redshift Serverless charges for RPU-hours. Even storage is not necessarily comparable: one platform may charge for compressed physical data while another charges for logical bytes. A comparison therefore requires more than finding three prices and putting them into the same table; it requires translating one workload into three different billing models.
A plausible number is not necessarily a reliable number
LLMs are very good at producing plausible answers, which is precisely what makes pricing comparisons tricky. A rate may be outdated, apply only with an annual commitment, or refer to a different region, and an allowance that materially changes the bill may be left out. None of these mistakes necessarily makes the resulting answer look wrong.
The same problem exists with assumptions. Suppose an analysis concludes that one warehouse will cost $400 per month and another $250. Before the numbers are useful, we need to know what configurations produced them: how much data is stored, how much is queried, how long compute runs, which warehouse size or capacity was assumed, and whether the two configurations are even expected to provide similar performance. Without that information, the apparent precision of $250 versus $400 can be misleading.
This is why a good cost comparison should expose its assumptions rather than hide them. In many cases the useful conclusion is not “BigQuery is cheaper than Snowflake,” but something closer to “BigQuery is cheaper for the current workload, while Snowflake becomes cheaper if query volume grows under these assumptions.” The second statement is less dramatic, but considerably more useful.
Pricing also has a time dimension
There is another complication: vendor pricing changes. Knowing that a product charges per credit, token or gigabyte is not enough; the exact rate, region, plan and date matter as well.
An LLM can search for these facts (for example, on modern-datatools.com 😅), and increasingly it can do this very well. But for a decision involving meaningful spend, it is worth asking a more prosaic question: Where exactly did this number come from? Ideally, every important rate in the calculation should point to a current first-party source and have a verification date. If the source cannot be identified, the apparent sophistication of the rest of the analysis does not help much.
The same applies when the analysis is repeated later. If the workload and pricing snapshot have not changed, the result should not depend on how the question happened to be phrased that day. A useful cost model should behave more like a spreadsheet than a conversation: the inputs, pricing rules and assumptions determine the answer, and changing one of them changes the result in a predictable way.
This does not make LLMs less useful
In fact, LLMs are excellent companions to this kind of analysis. They can explain unfamiliar billing models, identify the questions worth asking, interpret a sensitivity analysis, and help decide which scenarios deserve attention. What they should not have to do from scratch every time is reconstruct the underlying pricing database and calculation model.
The distinction is between explaining a pricing model and being the pricing model. For casual research, the difference hardly matters. For a decision that may affect tens of thousands of dollars of annual spend, it does.
A useful comparison should therefore be easy to audit. The prices should come from identifiable sources; units, regions and plans should be explicit; workload assumptions should be visible; and the same calculation should be reproducible later. Once those foundations are in place, an LLM is an excellent way to interrogate and explain the result.
This is also the principle behind Cost Comparison at Modern DataTools. Check our sample pricing report: we use verified public pricing and an explicit workload model to calculate the comparison first, then summarize what the calculation means. The writing is the easy part; getting the numbers and assumptions into a form that can be checked is the real work.