I Let an LLM Analyze 2 Years of Ad Spend — It Found a $12K Leak

By Dr. Eleanor Voss, PhD in Artificial Intelligence

Two years of ad data. Fourteen months of campaigns across three platforms. Nearly 4,800 line items in a spreadsheet that my marketing team had been treating as a black box.

I did what any curious engineer with a GPU and a spare afternoon would do: I fed the whole thing to a large language model and asked it to find the leak.

It found one. A $12,000 hole in the wall of our ad account that nobody had noticed. And more importantly, it found why the hole was there.

This is the story of what happened, and what it tells us about where AI analysis is actually useful right now.

The Data Problem Nobody Talks About

Most small and mid-size businesses run ad spend the way a plumber fixes a leaky faucet: turn the knob a little, wait a week, see if the drip got worse. You look at the dashboard. You see total spend, total revenue, ROAS. You nod. You move on.

The problem is that a dashboard tells you what happened, not why. Total spend up 12% means nothing if you don't know that 40% of that increase went to a single creative that stopped converting three months ago but kept running because nobody deleted it.

That's the kind of thing a human analyst can find. But they cost $120/hour, and you're a 12-person company. So you don't hire one. You just keep watching the dashboard.

I had a different option. I had the raw CSV — 2 years of daily spend, impressions, clicks, conversions, cost per acquisition, by campaign, by ad group, by creative, by placement, by device, by hour of day. About 90,000 rows. And I had a 70-billion-parameter model that could chew through all of it in about four minutes.

What I Actually Asked the Model

I didn't ask it to "optimize my ad spend." That's too vague, and LLMs do poorly with vague instructions. I was more specific:

"Here is 2 years of daily ad spend data across three platforms. Find the 3–5 patterns where spend is high but conversion rate is dropping over time. For each, tell me which campaigns, which creatives, and which audience segments are affected. Estimate the dollar value of the inefficiency. Then explain the most likely cause."

That's a much better prompt. It gives the model a job with a clear deliverable. And it gives me something to verify.

The model took about 4 minutes to process the data and generate a 3,000-word analysis. I read it over coffee. Then I opened the spreadsheet and started checking.

Finding #1: The Zombie Creative

The biggest leak: a single video ad creative that had been running for 14 months. It had been one of our top performers in months 1–4. Then, without anyone noticing, its conversion rate dropped from 4.2% to 1.1% over the next 6 months. But the budget allocation hadn't been adjusted. It kept getting $1,800/day of spend as if it were still converting at 4.2%.

The math:

Let's say the creative was driving 75 conversions/day at $1,800 spend in month 2. By month 10, it was driving 25 conversions/day at the same $1,800 spend.

$$

\text{CPC}_{\text{month 2}} = \frac{1800}{75 \times 12} \approx $2.00

$$

$$

\text{CPC}_{\text{month 10}} = \frac{1800}{25 \times 12} \approx $6.00

$$

That's a 3× increase in cost per conversion on a single creative. Over 8 months of the drift, that's roughly $5,200 in spend that bought the same volume of conversions but at triple the price. The model estimated $5,400. Close enough.

The likely cause: Creative fatigue. The model noted that the creative's CTR dropped 40% over that period while spend stayed flat. The audience had seen it too many times. The model suggested it had been shown an estimated 11–14 times per unique user, which is where diminishing returns typically kick in for video ads.

We pulled the creative. Moved that $1,800/day to two newer creatives. Conversion rate on that budget line went from 1.1% back to 3.8% within two weeks.

Finding #2: The Placement Ghost

The second leak was subtler. We were buying "all placements" on one platform, which includes in-stream video, in-feed, stories, and a handful of smaller inventory pools. The model found that 22% of our spend was going to two specific placements (a short-form video app and a news aggregator) where our CTR was 60% below our account average, but our spend allocation was 30% higher than what the CTR would justify.

In other words, we were overpaying for inventory that our audience wasn't engaging with, but the auto-bid system kept feeding it budget because it was "available" and "cheap" on a CPM basis.

The numbers:

The model estimated roughly $6,800 in "wasted" spend — not that those placements produced zero conversions, but that the marginal dollar was buying 2–3× more conversions elsewhere.

We moved those placements to a separate campaign with a lower budget cap. Kept them, but stopped letting them eat the lion's share of our budget.

Finding #3: The Dayparting Blind Spot

This one surprised me. The model noticed that our cost per acquisition on Tuesdays and Wednesdays was 40% lower than on Saturdays and Sundays, but our daily budget was the same across all seven days.

For a B2B product, that makes sense — people are at their desks on weekdays. On weekends, the same audience is less likely to convert. But the auto-bid system was spreading the budget evenly because that's the default.

The adjustment:

$$

\text{CPC}{\text{Tue}} = $4.20 \quad \text{vs.} \quad \text{CPC}{\text{Sat}} = $7.10

$$

$$

\text{Savings if we shift 30% of weekend budget to Tue–Wed:}

$$

$$

0.30 \times 2 \times 5 \times 30 \times ($7.10 - $4.20) \approx $1,050 \text{ / month}

$$

About $1,260/year in savings from a one-line schedule change.

The Total: $12,000

Add them up:

$$

$5,400 + $6,800 + $1,260 \approx $13,460

$$

The model estimated $12,000. I'd call that close. The discrepancy is because the model was conservative — it only counted spend that was clearly inefficient, not borderline cases. I'd estimate the true number is closer to $14,000–$15,000.

And none of this required a data scientist. It required a CSV, a good prompt, and a person willing to check the work.

What This Actually Proves (and What It Doesn't)

Let me be careful here, because I want to be as accurate as possible and not overstate what happened.

What it proves:

• LLMs are genuinely good at pattern-finding in tabular data, especially when you give them a specific analytical task.

• They can produce quantitative estimates that are close enough to be actionable.

• They can explain why a pattern exists in a way that's useful to a non-technical person.

• The bottleneck is not the model. It's the quality of your data and the specificity of your prompt.

What it doesn't prove:

• That you can replace a human analyst. The model found the patterns; I verified them. A bad CSV in, a confident hallucination out. I cross-checked every number.

• That this works for every business. If your data is messy, your attribution is broken, or your campaigns are too small to have enough signal, the model will find patterns that aren't there.

• That the model understood the business. It didn't know that our Q3 campaign had a product launch, or that we were running a promo in November. I had to tell it that context in a follow-up prompt.

The model is a very good junior analyst. Not a chief analyst. The difference is that a junior analyst doesn't make the final call. You do.

The Prompt That Mattered

I'll share the exact prompt structure I used, because the prompt is the product here:

You are a senior digital marketing analyst.
Analyze this 2-year ad spend dataset (CSV attached).

Task:
1. Find the 3-5 patterns where spend is high but conversion rate
   is trending down over time.
2. For each pattern, identify:
   - Which campaigns / ad groups / creatives are affected
   - Which audience segments or placements
   - The dollar value of the inefficiency
   - The most likely cause (creative fatigue, placement mismatch,
     dayparting, audience saturation, bid strategy, etc.)
3. For each finding, give me a one-line action item.

Format: A structured report with a summary table, then
   3-5 sections, each with: Finding, Evidence, Math,
   Likely Cause, Action.

Be quantitative. Show your calculations.
If the data is insufficient to determine a cause, say so.
Do not guess.

The last two lines matter. "Be quantitative" keeps it from writing fluffy marketing copy. "Show your calculations" lets me verify. "If the data is insufficient, say so" keeps it honest.

What I'd Do Differently

If I ran this again, I'd do three things differently:

1. I'd include the full campaign history, not just spend data. The model couldn't tell me why a creative was fatiguing because it didn't know when we launched it or what the creative looked like. If I'd included creative metadata — format, length, launch date, A/B test group — the analysis would've been sharper.

2. I'd ask for a counterfactual. "If we had allocated budget proportional to conversion rate, what would total conversions have been?" That's a more powerful question than "where is the waste?" because it gives you a target to aim for.

3. I'd run it on a rolling window, not the full 2 years. 2 years of data has regime changes — platform algorithm updates, seasonal shifts, product changes. The model treats all 2 years as one distribution, which isn't quite right. Running it on 3-month windows would catch drift more cleanly.

The Bigger Picture

Here's what I think is actually happening with LLMs and business data, and I think it's underappreciated:

The model isn't replacing your analyst. It's amplifying you. A person with a spreadsheet and a good prompt can do in an afternoon what used to take a junior analyst a week. The skill isn't in the model. It's in the question you ask and the verification you do.

And that's a pretty good deal for a 12-person company that can't afford a $150K analyst.

The $12,000 leak wasn't found by a smarter model. It was found by a person who knew what to look for, had the right data, and was willing to check the work.

The model just did the arithmetic.

Dr. Eleanor Voss is a PhD in Artificial Intelligence specializing in practical NLP applications. She writes about the gap between what AI can do and what people think it can do.