预测分析：有购买可能的人，未必是能被说服的人
Predictive analytics uses historical and current data to estimate unknown outcomes with statistical models, including machine learning. It produces probabilities, forecasts, or expected values to support decisions. It sits inside the broader field of data analytics, but its job is narrower. Put simply, it answers "what is likely?" and stays silent on "what would change?"
Google's documentation shows the gap. In the Google Analytics predictive metrics documentation, purchase probability, churn probability, and predicted revenue are all defined as expected outcomes for a user inside a fixed window. None of those definitions describes what a campaign would change. (VERIFIED, product documentation with no geography, undated page, accessed Oct 28, 2026.) That gap runs through the rest of this entry.
Prediction, Explanation, and Why They Differ
Statistics distinguishes two goals that are easy to blur. Explanatory modeling tests a theory about why something happens. Predictive modeling estimates what will happen to new cases. Galit Shmueli set out the distinction in "To Explain or to Predict?", Statistical Science 25(3), 289-310, 2010, and you can read the author's arXiv copy in Shmueli's paper on explaining versus predicting. (VERIFIED; the arXiv copy differs from the journal version only in pagination and typographic detail.)
The paper's warning applies to marketing. Models with strong explanatory power are often assumed to predict well too. They don't have to be. Shmueli argues the two should be assessed separately, and that predictive power should be assessed on holdout data instead of being inferred from in-sample fit such as R².
How Predictive Analytics Works
Define the Outcome, Window, and Data Available
Every model starts with a question that has three parts. Which outcome, for which people, over what period? Add a fourth: which data exists at the moment you score someone.
GA4 makes those parts visible. Purchase probability covers a user who was active in the last 28 days and the chance of a purchase key event in the next 7. Churn probability covers a user active in the last 7 days and the chance they won't be active in the next 7. Predicted revenue covers purchase key events over the next 28 days. The page doesn't say what counts as "active." (VERIFIED, Google Analytics Help, undated page, accessed Oct 28, 2026; metrics are probabilities and expected revenue, no geography.)
Why this matters: GA4's "churn" is a seven-day inactivity forecast. It isn't a canceled contract, a lapsed subscription, or a customer who left for good. Your business may define churn differently, and a score built on someone else's definition answers someone else's question. Check the event and the window before you act on any score.
Inputs can come from first-party data you own, such as transactions, CRM records, and timestamped events.
Train, Validate, and Score
Training means fitting a model to past cases where the outcome is already known. Validation means checking it on cases it hasn't seen. Skip that second step and you learn nothing about how the model behaves on tomorrow's customers.
The scikit-learn cross-validation documentation calls testing a model on its own training data a methodological mistake, because a model can memorize training labels, score perfectly, and still fail on new data. The standard fix is a held-out test set. Time changes the rules. Ordinary cross-validation assumes cases are independent, so on time-ordered data it can train on the future and test on the past, which gives misleading estimates, and the documentation points to time series splits instead. Repeated customers need care as well. Group-aware splitters keep every record from one customer on the same side of the split, so the model isn't tested on a person it already learned from. (VERIFIED, scikit-learn 1.9.1 documentation.)
Where Marketers Use It
A score does nothing on its own. It matters when it changes a decision, such as who sees a search engine marketing campaign, who gets a retention email, or who's shown social media ads. Whether any of those decisions pays off is a separate question.
Purchase Probability, Churn Risk, and Revenue Prediction
GA4 is the one documented example here with published rules. To qualify, a property needs at least 1,000 returning users who triggered the predictive condition and at least 1,000 who didn't, within a seven-day period inside the last 28 days. Model quality also has to hold up over a sustained period, and the purchase event must carry value and currency parameters. Predictions are generated once a day for each eligible model. If quality drops below the minimum, Analytics stops updating them and they may become unavailable. The page doesn't publish the quality threshold or a retraining schedule. (VERIFIED, Google Analytics Help, undated page, accessed Oct 28, 2026; the thresholds are user counts per window, product rules with no geography.)
Analysis, not a quote from the page: a small store that cannot reach 1,000 returning purchasers within a seven-day period may never see the purchase metrics. That's a limit of the data, not a flaw in your marketing.
Lead Scoring and Customer Value
The same logic applies to prospects. In rules-based lead scoring, a person decides how many points each action earns. A predictive version learns those weights from past outcomes instead. Customer value works similarly, and a predicted value remains an estimate built on assumptions, not a target. Both need the checks in the next section.
How to Judge a Model Before Trusting It
Use this as a review list with an analyst, an agency, or a platform vendor. Each question can be answered with evidence, not a promise.
| Check | What to look for | Question to ask the team or vendor | Source |
|---|---|---|---|
| 1. Outcome, window, and available data defined | The event, the population, the prediction window, and the fields available at scoring time are written down, and the platform definition matches your own business definition. | What exactly is the outcome, over what window, and does it match how we define purchase or churn? | Google Analytics Help (VERIFIED) plus the entry analysis on churn definitions |
| 2. Validation free of leakage | Data was split before preprocessing, every field exists at scoring time, and time-ordered or repeated-customer data used time or group splits. | Was the data split before any preprocessing, and was each field available at the moment we would score someone? | scikit-learn 1.9.1, Common pitfalls and Cross-validation (VERIFIED) |
| 3. Ranking checked separately from calibration | A calibration curve exists, not only a ranking score such as ROC-AUC. | Do the probabilities match observed frequencies, and who checked that besides the ranking score? | scikit-learn 1.9.1, Probability calibration (VERIFIED) |
| 4. Threshold chosen from costs | The cut-off reflects the cost of a false positive and a false negative, and it was tuned on data separate from training data. | What does each kind of mistake cost us, and how was the cut-off chosen? | scikit-learn 1.9.1, Tuning the decision threshold (VERIFIED) |
| 5. Drift monitored after deployment | Someone owns monitoring, and a rule says what drop in quality means the score is no longer used. | Who watches quality after launch, and what drop makes us stop using the score? | NIST AI 100-1 and the Google Analytics Help quality note (VERIFIED) |
Each row is a source used in this entry. VERIFIED means the primary page or text was opened and supports the statement. REPORTED means only an abstract or a secondary release was read.
| Source | What it establishes | Label |
|---|---|---|
| Google Analytics Help, predictive metrics (support.google.com, undated page, accessed Oct 28, 2026) | Defines purchase probability (user active in the last 28 days, key event in the next 7), churn probability (user active in the last 7 days, not active in the next 7), and predicted revenue (purchase key events over the next 28 days). The page does not define "active." | VERIFIED |
| Google Analytics Help, eligibility requirements (same page, accessed Oct 28, 2026) | Needs at least 1,000 returning users who met the condition and 1,000 who did not, within a seven-day period inside the last 28 days. Predictions are generated once a day. Updates stop if model quality drops below the minimum. The quality threshold and retraining schedule are not published. | VERIFIED |
| scikit-learn 1.9.1, Common pitfalls | Defines data leakage as using information that will not be available at prediction time. It inflates performance estimates and the model does worse on new data. Split first, learn preprocessing on training data only, use a pipeline. | VERIFIED |
| scikit-learn 1.9.1, Cross-validation | Testing on training data is a methodological mistake. Use a held-out test set, time series splits for time-ordered data, and group-aware splits when one customer has many records. | VERIFIED |
| scikit-learn 1.9.1, Probability calibration | A calibrated model scored near 0.8 is positive about 80% of the time. Calibration generally does not change ranking metrics such as ROC-AUC. Brier score and log loss blend calibration with discriminative power. | VERIFIED |
| scikit-learn 1.9.1, Tuning the decision threshold | Classification has a statistical part (probabilities) and a decision part (action). The default 0.5 cut-off is rarely ideal. Tune the threshold on data separate from training data. | VERIFIED |
| NIST AI 100-1, AI Risk Management Framework 1.0 (year not readable on the page opened) | AI systems can need corrective maintenance because of data, model, or concept drift. Training data can go stale. Controlled-condition measurements may differ from operation. Ongoing testing and monitoring is recommended. | VERIFIED |
| Shmueli, Statistical Science 25(3), 289-310, 2010 (arXiv copy read) | Explaining and predicting are different goals. High explanatory power is often wrongly assumed to imply predictive power. Predictive power should be measured on holdout data. | VERIFIED |
| Ascarza, Journal of Marketing Research 55(1), 80-98, February 2018 (Crossref abstract) | Two field experiments combined with machine learning found that the highest churn-risk customers are not necessarily the best retention targets. Targeting by sensitivity to the intervention is recommended. | REPORTED |
| AMA award release about Ascarza (ama.org) | Reports churn reductions of 4.1 to 8.7 percentage points from sensitivity-based targeting. Two settings: North American membership organization and Middle Eastern wireless firm; sample sizes not stated. | REPORTED |
| Stanford GSB, Susan Athey, May 23, 2018 | Says deep neural nets are good at finding patterns in a stable environment. Gives a hotel pricing example showing that a correlation does not show what changing the price would cause. | VERIFIED |
A model that looks accurate can still be useless. Three checks are worth running before a score touches a budget.
Leakage: When a Great Score Is a Warning
The scikit-learn common pitfalls guide defines leakage as building a model with information that won't be available when you actually need a prediction. It inflates performance estimates, including cross-validation results, and the model then does worse on new data. The guide's advice is to split the data first, learn preprocessing steps from the training subset only, and wrap the steps in a pipeline so nothing touches test data. (VERIFIED, scikit-learn 1.9.1 documentation.)
Picture a churn model trained with a field called "cancellation reason." That field only gets filled in after someone cancels. The model looks brilliant in testing and is worthless the day you score a live customer. (Hypothetical example, not real data.) A very high score isn't proof of skill. Worth checking how the data was split and when every field was recorded.
Ranking Is Not the Same as Calibration
A model can rank people well and still give probabilities you shouldn't trust. The scikit-learn calibration documentation describes a well-calibrated model as one where, among cases scored near 0.8, about 80% truly belong to the positive class. Calibration generally doesn't change ranking metrics such as ROC-AUC, so a strong AUC says little about whether 0.8 really means 80%. The same page cautions that Brier score and log loss blend calibration with discriminative power, so a model can improve its Brier score without becoming better calibrated. (VERIFIED, scikit-learn 1.9.1 documentation.)
Why this matters: if you multiply a probability by order value to estimate expected revenue, the probability has to mean what it says.
Choosing a Threshold From Costs, Not Defaults
The scikit-learn decision threshold documentation splits classification into two jobs. One is a statistical problem, predicting probabilities. The other is a decision problem, turning those probabilities into an action. scikit-learn's default cut-off of 0.5 is, in the documentation's words, "most certainly not ideal for most use cases." It also warns against tuning the threshold on the same data used to train the model. (VERIFIED, scikit-learn 1.9.1 documentation.)
In marketing terms, the cut-off sets which mistake you'd rather make. A false positive might be a discount handed to someone who'd have bought anyway. A false negative might be a loyal customer you never contacted. Which one costs more depends on margin, capacity, and the offer. (Hypothetical framing, no data.) That's also why no accuracy figure is "good" on its own. The task, the baseline, how rare the outcome is, and the price of each error all change what a number means. That conclusion follows from the three documentation pages above, not from a single sentence in any of them.
Predicted Likely Is Not the Same as Persuadable
A purchase-probability score answers one question: who is likely to buy? It doesn't answer another: who will buy because of this offer? Those two groups can differ sharply, and the difference decides where money goes.
Take two customers. (Hypothetical example, not real data.) Anna scores 0.92 for purchase probability and would have bought without any message. A discount just costs margin. Ben scores 0.30, but a timely reminder tips him over. A model built to predict purchases ranks Anna first. A model built to estimate the effect of the message would put Ben first. The same logic applies to churn, where the customer most likely to leave may also be the one an offer can't reach.
Eva Ascarza tested that idea in "Retention Futility: Targeting High-Risk Customers Might Be Ineffective," Journal of Marketing Research 55(1), 80-98, February 2018. The abstract reports that she combined two field experiments with machine learning and found that customers at the highest risk of churning aren't necessarily the best targets for proactive retention programs. Programs can fail because of the targeting rule, not the incentive, and she recommends targeting by sensitivity to the intervention. The Crossref record for Ascarza's paper carries the abstract. (REPORTED, abstract only; the full text wasn't accessible.)
One further number comes from an award release. According to an AMA award release, targeting by sensitivity to the offer instead of churn risk reduced churn rates by 4.1 to 8.7 percentage points. The release describes a subscription membership organization in North America and a wireless telecommunications firm in the Middle East. It does not give sample sizes or map each endpoint of the range to a firm. (REPORTED, award-release summary; the number is not in the Crossref abstract.)
Stanford's Susan Athey draws the broader line. In a Stanford GSB article featuring Susan Athey, she makes a point about the limits of pattern-finding:
"Deep neural nets are very good at finding patterns in a stable environment."
Susan Athey, Stanford GSB, May 23, 2018 (VERIFIED)
She also used a hotel example: higher prices correlate with a full hotel, yet raising your price will not fill more rooms. (Paraphrased, VERIFIED: Stanford GSB.) A pattern describes what happened. It doesn't promise what an action will cause.
Two terms help here. In marketing, a propensity score often means the estimated probability that someone takes an action, such as buying. In causal inference, the term instead means the probability of receiving a treatment. Uplift modeling estimates how much a specific action changes the chance of that outcome for a person.
Testing the difference doesn't need a fancy model. Hold out a random group in each score band, treat the rest, and compare outcomes band by band. Incrementality testing methods covers the method, and a plain A/B test works for simple offers.
Keep the limits in view. Ascarza's evidence comes from two field experiments, and your customers may behave differently. Nothing here shows that targeting by sensitivity always wins, and this entry doesn't claim teams commonly get it wrong. The claim is narrower: the gap between "likely" and "persuadable" is real, and you can test it.
Where Predictive Analytics Stops Working
Models age. The NIST AI Risk Management Framework says AI systems can need more frequent corrective maintenance because of data, model, or concept drift, and that training data can go stale relative to the setting where a system runs. It also notes that measurements taken in controlled conditions may differ from what happens in operation, and it calls for ongoing testing and monitoring of deployed systems. (VERIFIED, NIST AI 100-1, AI RMF 1.0, summarized rather than quoted; see Appendix B, Section 1.2.1, and Section 3.1.)
GA4 shows a platform-level version of the same problem. When model quality falls below its minimum, updates stop and predictions may disappear. That's the safe behavior. Your own models need a similar rule: decide in advance what drop in quality means you stop trusting the scores.
Two more limits apply. A prediction is a likelihood, not a promise, and a platform's definition of an outcome might not match yours, so a score can be accurate and still answer the wrong question.
Predictive Analytics FAQs
Why might a platform's predictive metric be unavailable?
In GA4, the property needs purchase or in-app purchase events, at least 1,000 returning users who met the condition and 1,000 who didn't within a seven-day period in the last 28 days, and model quality that holds over time. If quality drops below the minimum, updates stop. (VERIFIED, Google Analytics Help, accessed Oct 28, 2026.)
Does an accurate prediction prove ROI?
No. Accuracy tells you the score matches outcomes. ROI depends on what a campaign changed, which needs a comparison against a group that didn't receive it. Ascarza's work (REPORTED) and Athey's pricing example (VERIFIED) both point to the gap between pattern and effect. This entry offers no ROI figure for predictive analytics because none could be verified.
Can I trust the probability a model gives?
Only if it's calibrated. A calibrated model's 0.8 scores come true about 80% of the time. Ranking quality and calibration are different properties, so check a calibration curve before you multiply probabilities by revenue. (VERIFIED, scikit-learn 1.9.1 documentation.)
What's the difference between churn probability and my business's churn definition?
GA4's churn probability is the chance that a user active in the last 7 days won't be active in the next 7. Your business may define churn as a canceled plan, a missed renewal, or 90 days of silence. Match the event and window to your own definition before you act. (VERIFIED for the GA4 definition; the comparison is analysis.)
What is data leakage?
Leakage means building a model with information that won't exist when you need the prediction. It inflates test results, and the model performs worse on new data. Split the data before preprocessing and use a pipeline. (VERIFIED, scikit-learn 1.9.1 documentation.)
Related Terms
- Customer Retention and churn risk: the goal many churn scores serve.
- Customer Lifetime Value estimates: a predicted value that's still an estimate.
- Uplift modeling: estimates how much an action changes someone's chance of an outcome.
- Propensity score: the estimated probability that someone takes an action.
Sources
- Galit Shmueli, "To Explain or to Predict?", Statistical Science 25(3), 289-310, 2010 (arXiv copy). https://arxiv.org/pdf/1101.0891. VERIFIED.
- Google Analytics Help, predictive metrics. https://support.google.com/analytics/answer/9846734. Undated, accessed Oct 28, 2026. VERIFIED.
- scikit-learn 1.9.1, Common pitfalls (data leakage). https://scikit-learn.org/stable/common_pitfalls.html. VERIFIED.
- scikit-learn 1.9.1, Cross-validation. https://scikit-learn.org/stable/modules/cross_validation.html. VERIFIED.
- scikit-learn 1.9.1, Probability calibration. https://scikit-learn.org/stable/modules/calibration.html. VERIFIED.
- scikit-learn 1.9.1, Tuning the decision threshold. https://scikit-learn.org/stable/modules/classification_threshold.html. VERIFIED.
- NIST, Artificial Intelligence Risk Management Framework (AI RMF 1.0), NIST AI 100-1. https://nvlpubs.nist.gov/nistpubs/ai/NIST.AI.100-1.pdf. VERIFIED.
- Eva Ascarza, "Retention Futility," Journal of Marketing Research 55(1), 80-98, February 2018, Crossref record. https://api.crossref.org/works/10.1509/jmr.16.0163. REPORTED (abstract).
- American Marketing Association, Eva Ascarza receives the 2023 Weitz-Winer-O'Dell Award. https://ama.org/press-releases/eva-ascarza-receives-2023-weitz-winer-odell-award-for-retention-futility. REPORTED.
- Stanford GSB, Susan Athey on business leaders and AI, May 23, 2018. https://www.gsb.stanford.edu/insights/susan-athey-why-business-leaders-shouldnt-have-blind-faith-ai. VERIFIED.
