From Raw Clicks to Revenue Signals: How Our LTV Pipeline Actually Works

By Dr. Julie Williams

Predicting customer lifetime value isn't about throwing a neural network at a spreadsheet and hoping for the best. It's about building a data pipeline so clean, structured, and defensible that when your model outputs "this customer is worth $4,200 over 18 months," you can explain exactly which transactions, time windows, and feature transformations produced that number.

This article walks through our exact pipeline — the one we run in production for a mid-market e-commerce client with roughly 2.3 million active customers and an average order value of $67. No fluff. Just the stages, the choices, and the reasoning behind each decision.

Why "LTV" Isn't One Number

Before touching code, we define what LTV means operationally:

$$

\text{LTV}i = \sum{t=1}^{T} \frac{\mathbb{E}[R_i(t)]}{(1+r)^t}

$$

Where $R_i(t)$ is the expected revenue from customer $i$ in month $t$, and $r$ is a discount rate (we use 8% annualized, or ~0.67% monthly). This matters because two customers can have identical 12-month spend but very different LTV if one spent it in months 1-3 and the other spread it evenly. The time-weighted version rewards early monetization and penalizes leakage.

Our pipeline outputs three artifacts per customer:

• Point estimate (single dollar figure)

• 95% prediction interval (lower/upper bound)

• Component decomposition (how much comes from purchase frequency vs. AOV vs. retention probability)

The decomposition is what makes the output actionable for marketing teams. They can say "we need to lift frequency" rather than staring at a black-box score.

Stage 1: Ingestion and Normalization

Raw transactions arrive from three sources:

• POS system (batch CSV, daily)

• E-commerce platform API (webhook per order)

• Wholesale/enterprise CRM (SFTP, weekly)

Each has slightly different schemas. The pipeline normalizes every record to a canonical event:

(customer_id, timestamp, gross_amount, discount_applied, tax_amount, net_revenue, channel, payment_method, return_status, product_category_l1, product_category_l2)

Key decisions:

• net_revenue = gross − discounts + tax (we include tax because the business cares about cash-in-hand)

• Returns are not deleted, they're marked return_status ∈ {pending, confirmed, reversed} and the pipeline waits 14 days before finalizing a transaction as "confirmed" (our return window). This prevents LTV inflation from products that get sent back.

• Deduplication uses (customer_id, order_id) not just timestamps — two customers can buy in the same second.

Data volume: ~4.1 million events/month. We use Apache Kafka for streaming and Postgres as the normalized store (partitioned by month).

Stage 2: Customer-Level Aggregation

This is where most pipelines get sloppy. We compute a rolling feature vector per customer at each "decision time" $t$:

A few notes:

• Recency is computed at the exact decision timestamp, not "today." This avoids look-ahead bias. If the model scores customers as of January 15, we don't include February's purchases in their features.

• Category breadth captures whether a customer is exploring (high breadth = likely growing) or narrowing down (low breadth = possibly churning).

• Channel mix entropy ($H = -\sum p_k \log p_k$) tells us if a customer uses one channel or many. High entropy correlates with higher retention in our data — these customers are "omnichannel" and less likely to defect to a competitor's app.

We compute these features for every 30-day snapshot over the trailing 24 months, giving each customer ~8 observation points. The final feature vector is concatenated: $x_i = [f_1(t), f_2(t), \dots, f_{36}(t)]$ — we use 36 features total after including lagged versions of the core metrics.

Stage 3: Label Construction (The Part Everyone Gets Wrong)

We need a "ground truth" LTV for training. The challenge: customers who joined last month only have 1 month of data, but we're predicting 18 months ahead.

Our approach:

• Training set: Only include customers with at least 20 months of history (so their 18-month forward window is fully observed).

• Exclusion: Newer customers are scored by the model using their shorter feature windows, but they're not used to train the label mapping. This keeps training labels clean.

• Noise floor: For the interval output, we run a quantile regression forest on top of our main gradient-boosted model (XGBoost, 400 trees, depth 6). The forest gives us $Q_{0.05}$ and $Q_{0.95}$.

Label: $\text{LTV}{\text{observed}} = \sum{t=1}^{18} R(t) / (1+0.0067)^t$

Stage 4: Model Architecture

We use a stacked ensemble:

• Base models (3): XGBoost, LightGBM, and a 2-layer GRU (hidden=128) that ingests the raw monthly revenue sequence directly. The GRU captures temporal shape that tabular features miss — e.g., a customer whose spend is decelerating looks different from one who's accelerating, even if their trailing-90-day totals are identical.

• Meta-learner: Ridge regression over the 3 base predictions, trained on a held-out 15% split.

Hyperparameters were tuned via Bayesian optimization (Optuna, 200 trials). No hand-tuning.

Performance (5-fold CV):

We optimize for rank correlation rather than absolute RMSE because the marketing team uses LTV scores to segment, not to predict exact dollar amounts. Getting the ordering right matters more than getting magnitudes exactly right.

Stage 5: Serving and Caching

• Features are recomputed nightly (batch) + updated intra-day for new orders via a lightweight Python worker.

• Model inference runs on a 4-core CPU node; full re-score of 2.3M customers takes ~11 minutes. We cache the last score per customer in Redis with a 6-hour TTL.

• Output: a daily CSV + Parquet file that loads into the marketing platform (Segment → BigQuery).

Stage 6: Monitoring and Drift Detection

A model you don't monitor is a model you don't trust. We track:

Last quarter, a schema change in the POS system silently nulled out discount_applied for two weeks. Our PSI monitor caught it on day 3. Without that check, our LTV scores would have been quietly inflated by ~$40/customer.

What We'd Change If Starting Over

1. Incorporate behavioral telemetry (session duration, cart abandonment) as auxiliary features. Currently we only use transactional data; adding engagement signals improved a prototype's rank IC from 0.58 to 0.67.

2. Use a survival-analysis layer for the retention probability component. Our current approach treats "will this customer still be here in month $t$?" as a simple logistic regression. A Cox model or deep-survival network would better capture heterogeneity in churn risk.

3. Publish feature importance per segment. The marketing team benefits from knowing why a specific cohort scores high — is it frequency, AOV, or channel diversity?

Closing Thought

A good LTV pipeline is 70% data hygiene and 30% modeling. The model can only be as smart as the transactions you feed it. If your returns aren't reconciled, if your discount fields are inconsistent across channels, if you're scoring customers with next month's purchases leaking into their features — no amount of architecture will save you.

Build the pipeline first. Then let the numbers speak.