I Built an AI Anomaly Detector for Marketing in a Weekend — Here's the Result 📊

By Dr. Elias Thornwood, PhD in Artificial Intelligence

There's a particular kind of frustration that every marketing analyst knows well: you open your dashboard on a Monday morning, and something looks off. A campaign's click-through rate has dipped. A channel that's been stable for three months suddenly spikes by 40%. A specific product category's conversion rate has quietly halved, and no one on the team has noticed. You spend the rest of the week digging through spreadsheets, cross-referencing cohorts, and arguing with stakeholders about whether the change is "meaningful." By the time you've built the case, the anomaly is old news.

I got tired of that. So one Saturday morning, with a cold coffee and a half-empty notebook full of half-ideas, I set out to build an anomaly detector for marketing metrics that could do the watching for me. This is the story of what I built, how it works, and what it actually caught when I pointed it at real campaign data.

The Problem: Marketing Data Is Noisy, and Humans Are Slow

Marketing metrics are inherently noisy. A/B tests create artificial variance. Seasonal effects, holiday traffic, and platform algorithm changes all add background static. On top of that, most dashboards are built to show you what happened, not what's unusual. A line chart that dips by 8% can be noise. A dip of 12% on a metric that's been flat for six months might be a broken pixel tracker. Without a model, you're pattern-matching with your eyes, which is slow and inconsistent.

The goal wasn't to build another dashboard. The goal was to build a watchdog — a small, fast model that watches a set of marketing metrics continuously and flags deviations that are statistically unlikely given recent history. Not every deviation. Just the ones worth a human looking at.

The Design: A Simple, Interpretable Architecture

I deliberately kept the architecture small. For a weekend project, you want something you can explain to a PM in three sentences. Here's what I ended up with:

Input layer. A sliding window of the last 28 days of daily metric values for each KPI we want to monitor — impressions, clicks, conversions, revenue, cost-per-acquisition, email open rates, and so on. Each KPI is tracked independently, which keeps the model simple and makes debugging easy.

Baseline estimation. For each KPI, I compute a rolling baseline using a 14-day exponential weighted moving average (EWMA), with a decay factor of $\lambda = 0.8$. This gives us a smooth estimate of "what the metric is supposed to be doing right now," which adapts to slow trends (seasonality, growth) without being thrown around by single-day noise.

Variance estimation. I compute a rolling standard deviation over the same window, also EWMA-smoothed. This captures how much the metric normally fluctuates. A KPI that's stable will have a small $\sigma$; a KPI that's naturally volatile will have a large one. This is important — we want the alert threshold to be relative to the metric's own behavior, not absolute.

Anomaly score. For each new daily value $x_t$, I compute a z-score:

$$z _t = \frac{x_t - \mu_t}{\sigma_t}$$

where $\mu_t$ is the EWMA mean and $\sigma_t$ is the EWMA standard deviation. Then I apply a soft threshold: a day is flagged as a candidate anomaly if $|z_t| > 2.0$. I use 2.0 rather than the classic 3.0 because in marketing, you want to be a bit more sensitive — you'd rather get a few extra alerts and filter them than miss a real shift.

Trend confirmation. A single-day z-score spike can be a fluke. So I add a confirmation step: an anomaly is confirmed if 2 out of the last 3 days also have $|z| > 1.5$, or if the current $|z_t|$ exceeds 3.0 on its own. This cuts down on single-day noise while still catching genuine regime shifts quickly.

Output. A simple JSON event: KPI name, date, observed value, expected value, z-score, and a direction flag (up or down). I pipe these into a small internal dashboard and a daily digest email.

The whole thing runs in under 200 lines of Python. No GPU. No training. No feature engineering. It's a statistical model wearing a lab coat, but it works.

What It Actually Caught

I pointed it at about 18 months of campaign data across five channels — paid social, paid search, email, organic, and affiliate — tracking roughly 12 KPIs. Here's a sample of what it flagged in the last 90 days:

Out of roughly 120 KPI-days that crossed the soft threshold, I reviewed 34 and confirmed 28 as genuine signals. That's about an 82% precision rate, which for a weekend project with zero tuning is genuinely encouraging. The 6 that weren't real were mostly edge cases — a budget change that hadn't been communicated, a data pipeline delay, and a couple of days where the 14-day window was still recovering from a prior anomaly.

A few patterns stood out:

Downward anomalies are more actionable than upward ones. A spike in revenue or open rate is usually good news and rarely requires intervention. A dip in CTR, CPA, or conversions is almost always a signal that something broke or shifted. My team started treating confirmed downward anomalies as "investigate within 24 hours" and upward ones as "log and move on."

The confirmation step mattered more than the threshold. Without the 2-of-3-day confirmation, false positives roughly doubled. Marketing data is too lumpy for single-day z-scores to be trustworthy on their own.

Variance-aware thresholds beat fixed thresholds. A fixed "alert if CTR drops below 1.5%" approach would have generated a flood of alerts during our naturally volatile holiday season and missed a real dip during a stable period. The EWMA-variance approach handled both gracefully.

The Math, Plainly Stated

The whole system reduces to one idea: compare today's value to a smoothed estimate of what it "should" be, weighted by how much that metric normally wobbles. In equation form:

$$\ mu_t = \lambda \mu_{t-1} + (1-\lambda) x_t$$

$$\ sigma_t^2 = \lambda \sigma_{t-1}^2 + (1-\lambda)(x_t - \mu_t)^2$$

$$\ text{flag if } \left| \frac{x_t - \mu_t}{\sigma_t} \right| > 2.0 \text{ and confirmed over 3 days}$$

That's it. No neural networks, no gradient descent, no GPU cluster. Just exponential smoothing and z-scores. The elegance is in the deployment — running it daily, wiring it into a digest, and letting the team triage a short list instead of a wall of charts.

What I'd Do Differently

A few honest notes for anyone replicating this:

Add a holiday calendar. Seasonality is the biggest source of false positives. A simple lookup of known campaign dates, sales events, and platform changes would let you suppress alerts on those days or at least annotate them.

Track KPIs by cohort. Right now, each KPI is a single time series. If you segment by audience, device, or geography, the signal gets stronger but the maintenance grows. For a weekend project, aggregate KPIs are the right call.

Build a feedback loop. When a human confirms or dismisses an alert, log the outcome. Over a few months, you can tune $\lambda$, the threshold, and the confirmation window based on actual precision and recall.

Consider a simple regression baseline. If a KPI has a clear weekly pattern, fitting a day-of-week effect on top of the EWMA would sharpen the baseline. It's a small change with a meaningful accuracy bump.

The Takeaway

The biggest lesson isn't technical. It's cultural. A team that gets a short, curated list of "these 4 things look unusual, here's the math" makes faster decisions than a team that stares at 40 charts. The model doesn't need to be perfect. It needs to be consistent, explainable, and fast enough to matter before the anomaly becomes a trend.

I built this in a weekend because the architecture was simple. I kept it in production for months because the output was useful. That's the whole game: a small, interpretable model that does the repetitive watching so your team can do the thinking.

And if your dashboard is mostly red, or mostly green, or mostly "it depends," maybe it's time to build a watchdog. ☕

Dr. Elias Thornwood is a fictional author. The model architecture described here is a simplified, production-adjacent design and is shared for educational purposes.