I talk to a lot of martech product teams. Part of my advisory work is getting an early look at features before they go GA, sometimes months ahead… well, weeks, more often, and giving feedback while it can still change something. Most of those previews have little in common. Lately one pattern keeps showing up, across companies that don't know they're building the same thing.
A marketer types a question in plain language. An LLM queries the warehouse and comes back with insights, an audience, and sometimes a full activation plan. No SQL, no BI ticket, no waiting on a data engineer.
To be honest, it's impressive. As someone who started out as an analyst, I'd have loved it. My problem is with what it removes.
What goes missing when marketers skip the BI ticket
Everyone hates the BI ticket, and for good cause. It's slow, and it sits in a queue. It's also, in a lot of companies, the only regular moment where a marketer and a data engineer talk about what a field actually means. That's where a definition like "active customer" gets challenged, or where a marketer finds out the loyalty table stopped updating after a migration. The data engineer usually has a question back, too:
What's this audience actually for?
The interfaces I've seen leave a few questions unanswered.
Does a marketer need to understand the underlying data? I don't think they need the schema, but they do need to know which questions the data can answer at all.
Trust is the next problem, because an LLM presents a wrong answer with the same confidence as a right one.
And then there's what's missing.
An audience of 847 profiles looks complete whether it drew on every relevant source or skipped one.
Why text-to-SQL accuracy depends on the data team
These tools are only as accurate as the modeling work the data team has done.
dbt Labs benchmarked this earlier in the year. With a well-modeled semantic layer underneath, current models answered close to every covered question correctly.
Working from raw tables, accuracy dropped to about two in three.
On BEAVER, a benchmark built from real enterprise warehouse query logs, the best agentic setups got around 11% right.
So your product's quality depends on the people your interface is designed to route around. When the audience is wrong, the marketer won't blame the schema. They'll blame your tool, probably somewhere near renewal.
There's a sales angle too.
In warehouse-native deals the data team usually sits on the buying committee, or at least has a veto. Telling them your product removes the need for them is an odd pitch. I mean, would you buy that?
What I'd build into a natural-language audience builder
Very little of this slows the marketer down. Most of it is about what the interface shows next to the answer.
Start by making answers inspectable. Show which tables, definitions and joins produced the output, in language a marketer can follow, so someone can stop and say "hang on, that isn't how we count active." Alongside that, show coverage: which sources were included, which fields were sparse, and what the system couldn't map. A count without that context is how people get into trouble.
The second ask is to let it decline. "I need more detail" or "I can't answer that reliably with this data" should be a normal outcome, designed in from the start. I've seen at least one proprietary CDP build its natural-language segment assistant this way, returning a valid segment, a clarifying question, or a rejection. It's a small design choice that does a lot for trust.
The last one is about people. If a definition has an owner, name them. When the system is uncertain, offer the marketer a handoff to that person with the question, the generated query, and the gap already attached. Make it opt-in on both sides, because plenty of data engineers would prefer fewer messages, and that's their call. Then keep a log of the questions the system couldn't answer. That log is a ready-made backlog for the semantic layer, and each item cleared makes your product more accurate.
Look, it's rare for a feature to help the customer and your own roadmap at once.
About that copy
One small request. Please stop putting "no dependency on data teams" on your website. It pitches the wrong thing to the people who often sign off on the purchase, and it promises marketers an independence the accuracy figures don't support. 😉
The human side of the same problem
I've been working on this from the user end too. My Talking to Data Engineering series, a sponsored set of guides, helps marketers have these conversations without a tool in between. This piece covers the product side of the same problem.
Read the latest guide in the series here 👇🏻
Matthew Niederberger
Will this idea really make a difference?
I genuinely don't know whether most data engineers want more contact with marketing. Some would welcome it. Others have spent years on the receiving end of vague requests and are perfectly happy with a queue between them. That's why the handoff has to be opt-in.
If you're building one of these interfaces and want an outside view before it ships, consider this an open invitation to get in touch with me. I'd much rather see it in preview than help untangle it after go-live.
Do you have any questions after reading this article?
Or need support with your Martech projects?
Discussion