Six standard models agree on who gets the credit. Tested against a known answer, only one holds up.
Splitting marketing budget across channels means knowing which touchpoints actually drive conversions — and real conversion data carries no ground truth to check that against. So this project builds one: a simulator generates 8,000 journeys with fixed, known per-channel effects, every method is scored on whether it recovers that known ranking, and only then is the surviving method pointed at 267,084 real journeys from the GA4 public e-commerce dataset. Six standard methods — five touch-position heuristics and a Markov chain — all fail the test; a data-driven, Shapley-value model is the only one that passes.
A conversion never tells you which touchpoints caused it, so no attribution method can be scored for correctness on real data. The workaround is a simulator: 8,000 journeys generated from a fixed rule where each channel's true effect on conversion is set by construction — email strongest, display almost inert, the rest in between. Every method is then ranked on one question: does its channel ordering match the known truth? Spearman ρ = 1.0 is perfect recovery, 0 is chance, negative is backwards.
None of the five touch-position heuristics beats chance (ρ between −0.04 and 0.07). Markov removal effect does worse still at −0.18 — actively backwards: it ranks the rare-but-powerful email channel last, because removal effect is confounded by how often a channel appears, not just how much it moves conversion. Only the data-driven Shapley model recovers the true ordering (ρ = 0.79) — strong, not perfect.
One thing the Markov chain gets exactly right: its predicted overall conversion rate matches the empirical rate to four decimal places, on both simulated and real data. The chain arithmetic is sound; it's the per-channel credit split that fails — and it fails consistently, holding the same backwards ranking across bootstrap resamples. Stable and correct are different things.
That's the license for everything below: the real GA4 data is read mainly through the one model that passed — with the caveat that clearing a simulation is necessary, not sufficient, and the real dataset carries problems of its own (section 06).
First-touch, last-touch, linear, time-decay, position-based, and Markov removal-effect all land in roughly the same place: organic search leads, direct is a clear second, paid search is reliably last. That agreement looks reassuring — until you notice it tracks touchpoint frequency almost exactly, not incremental conversion lift. It's the same exposure bias the simulator exposed in section 01.
Swap in the data-driven model and the ranking doesn't just shift — parts of it invert. Organic search drops from ~31% average heuristic credit to 12.8% (0.42×), direct drops from ~23% to 10.3% (0.45×), while “other” jumps from ~10% to 43% (4.1×) and referral rises from ~20% to 25.5% (1.29×). Paid search is the one channel every model agrees on — weak everywhere, 2–5% of credit regardless of method.
Worth the caveat: this is the model that passed the simulator test, but it passed at ρ = 0.79, not 1.0, and its AUC on the real data is only 0.712 — decent, not excellent. Treat the reshuffling as directional evidence that heuristic credit is inflated by exposure, not as a precise reallocation of budget.
| Channel | First-touch | Last-touch | Linear | Time-decay | Position-based | Markov removal | Data-driven (Shapley) |
|---|---|---|---|---|---|---|---|
| Paid search | 0.050 | 0.033 | 0.039 | 0.037 | 0.041 | 0.036 | 0.023 |
| Organic search | 0.353 | 0.280 | 0.307 | 0.300 | 0.313 | 0.290 | 0.128 |
| Referral | 0.166 | 0.216 | 0.198 | 0.203 | 0.194 | 0.207 | 0.255 |
| Direct | 0.230 | 0.225 | 0.226 | 0.224 | 0.227 | 0.226 | 0.103 |
| Display | 0.156 | 0.107 | 0.125 | 0.119 | 0.129 | 0.116 | 0.060 |
| Other | 0.045 | 0.140 | 0.104 | 0.117 | 0.097 | 0.124 | 0.430 |
Line this up against the heuristic attribution chart above and the match is almost exact: organic search is both the highest-credited channel and the most frequent touchpoint. That's the signature of models rewarding exposure rather than causal contribution.
Organic search alone (87,082 journeys) and direct alone (54,267) each dwarf the best multi-hop combination — organic search → referral, at 3,262. The five single-channel paths in navy are each an order of magnitude bigger than any two-step sequence in red.
The attribution debate above is entirely about splitting credit across touchpoints in a journey — but for 83% of the 267,084 journeys, there's only one touchpoint. Multi-touch attribution modeling only meaningfully applies to the ~17% of journeys with two or more touches, and the long tail past 5 touches is just over 1% of total volume.
Looking at what immediately follows a visit to each channel — convert, continue to another channel, or drop — drop-off dominates everywhere — for every channel except “other” it's the outcome roughly four times in five. But the conversion rates that do differ are telling. “Other” converts at 3.3% per touch and referral at 1.7%, against roughly 1% for organic search, display and paid search. Those are the same two channels the data-driven model rewards and the heuristics ignore.
These rates come from the full first-order transition matrix — every channel→outcome edge, including the small conversion edges the flow diagram above drops for readability. One caveat remains: “other” is a catch-all bucket for traffic the channel-grouping logic couldn't label, so its 3.3% rate blends together sources that may behave nothing alike. In the obfuscated GA4 sample that bucket is large and disputed — read the number as a reason to break the bucket apart, not as a channel to fund.
On simulated data with a known answer, five touch-position heuristics scored no better than chance and Markov removal effect scored worse (ρ = −0.18). Only the data-driven Shapley model recovered the true order (ρ = 0.79). Last-touch dashboards are measuring exposure, not lift.
MethodBoth collapse under the one model that passed the test (0.42× and 0.45× their heuristic credit). Treat the drop as directional, not exact — that model cleared the simulator at ρ = 0.79 and scores only 0.712 AUC on the real data.
CautionBoth punch above their heuristic weight in the causal model, and both convert faster per touch (1.7% and 3.3% vs. ~1% elsewhere). But “other” is an unlabelled catch-all — its jump to 43% credit is more likely a channel-grouping artifact than a channel. Break that bucket apart before acting on it.
Investigate83% of journeys are single-touch and the blended conversion rate is 1.63%. Most of this model debate concerns a minority of traffic — why single-touch conversion is so low is arguably the larger question.
Conversion