I have been changing how I build Modern DataTools with AI. This post is about the setup rather than the product: which models do which job, what that costs me, and what happened when I stopped running one workflow for everything.
Modern DataTools is a platform for deciding what data and AI technology to use. What matters here is its shape: a few thousand pages that quote each other, full of prices, metrics, facts, competitors and rankings. Change one fact and you have changed a dozen pages, or you should have.
The two problems that wouldn't go away
Both issues that I've been solving are about stopping the platform from saying things that are not true.
The first is identity. Snowflake and dbt sit in the same category and are not competitors, and for a while my code did not know that, so it offered readers alternatives that do a completely different job.
The second is consistency. A price shows up on the product page, the pricing page, two comparisons and an alternatives table. When the vendor changes it, all of them have to change too. Some of what I found:
- "last verified on September 4" meant only that something had written to the page that day
- a CloudWatch price came from the wrong part of the vendor's site, and then spread to fifteen pages
- a fix I had checked myself was undone later by a pass that trusted an older page
My previous set up - codex only
As I pay for the development myself, I care a lot about the set up. For about four weeks I ran gpt5.6-sol for planning and gpt5.6-terra with high reasoning for execution. It was a good setup, and I would use it again on one bounded feature.
