A churn model I reviewed a few years ago reported 73.77 percent accuracy on its holdout set. That number sat in a slide deck for six weeks before anyone printed the confusion matrix, at which point one column turned out to be entirely zeros. Every customer had been scored as staying. Accuracy was simply reciting the base rate back at us.

Where the churn project stands after Part 9

Last part we fitted a linear model and spent most of our effort learning to read coefficients without lying to ourselves. That model predicted a continuous number. Our actual business question is not continuous at all. Retention wants to know which customers will leave, which is a yes or no outcome, and a straight line fitted to a column of zeros and ones will happily predict a churn probability of 1.4 or minus 0.2. Neither of those exists.

This part swaps the target for the binary Churn column and produces our first real classifier. We keep the same feature set built in Part 6, add the encoding work needed for a categorical target, and finish with a scored customer list and a threshold we can defend in a meeting. Data throughout is the public Telco Customer Churn file, 7043 rows of a fictional California telco, published by IBM and reachable at the raw GitHub link in the references. To keep the examples quick to rerun I worked from a 486 row extract of that file, so every number printed below comes from that extract rather than the full set. Directionally the two agree closely, but if you run the full 7043 rows your third decimal place will move.

Log odds, and why a straight line cannot output a probability

Probability is trapped between 0 and 1. Linear functions are not, so fitting one directly to a probability guarantees nonsense at the extremes. Logistic regression solves this by moving the linear part somewhere unbounded. Odds are the probability of the event divided by the probability of no event, so a 20 percent churn probability is odds of 0.25, or one churner for every four stayers. Odds run from 0 to infinity. Take their natural logarithm and you get log odds, which run from minus infinity to plus infinity. That is a scale a straight line can live on safely.

So we fit a linear combination of features to the log odds, then invert it back into a probability with the sigmoid, also called the logistic function. Its shape does all the work: far from the middle it flattens, so extreme inputs are pushed towards 0 or 1 without ever arriving. A useful anchor is that log odds of 0 corresponds to probability 0.5 exactly, and log odds of 2 corresponds to roughly 0.88.

Look carefully at that curve, because it explains the single most common misreading of a logistic model. Near the middle the line is steep, so one unit of log odds moves probability by roughly 24 percentage points. Out at the edges the same unit moves it by less than one point. A coefficient therefore never means a fixed change in probability. It means a fixed multiplication of the odds, which is a different sentence entirely.

Coefficient recovery on data with a known answer

Before pointing anything at real data, I always fit it to data whose answer I already know. Generate features, pick coefficients yourself, produce the probabilities, draw the outcomes, and then check whether the fitting procedure hands your coefficients back. If it cannot recover a truth you planted, no result on real data means anything. Below is a logistic fit written out in full using Newton iterations, which is the same objective scikit-learn optimises with its default solver.

# tested on Python 3.10 with numpy 2.2.6, pandas 2.3.3, scikit-learn 1.7.2
import numpy as np

rng = np.random.default_rng(0)
n = 4000
tenure  = rng.normal(0, 1, n)
monthly = rng.normal(0, 1, n)

# a truth we choose, so we can check it comes back
z = -1.0 + (-1.2 * tenure) + (0.8 * monthly)
p = 1 / (1 + np.exp(-z))
y = rng.binomial(1, p)
X = np.column_stack([tenure, monthly])

def fit(X, y, C=1.0, iters=100):
    n, k = X.shape
    Xb = np.hstack([np.ones((n, 1)), X])
    w = np.zeros(k + 1)
    lam = 1.0 / C                      # L2 strength, intercept left unpenalised
    for _ in range(iters):
        pr = 1 / (1 + np.exp(-(Xb @ w)))
        g = Xb.T @ (pr - y)
        g[1:] += lam * w[1:]
        W = pr * (1 - pr)
        H = (Xb.T * W) @ Xb
        H[1:, 1:] += lam * np.eye(k)
        step = np.linalg.solve(H, g)
        w -= step
        if np.max(np.abs(step)) < 1e-10:
            break
    return w

w = fit(X, y, C=1.0)
print('positive rate %.4f' % y.mean())
print('intercept %.4f  (true -1.0000)' % w[0])
print('tenure    %.4f  (true -1.2000)' % w[1])
print('monthly   %.4f  (true  0.8000)' % w[2])
print('odds ratio for tenure %.4f' % np.exp(w[1]))

positive rate 0.3240
intercept -1.0223  (true -1.0000)
tenure    -1.1515  (true -1.2000)
monthly   0.7116  (true  0.8000)
odds ratio for tenure 0.3162

Notice that monthly came back as 0.71 against a planted 0.80. On first sight that gap looks like a bug. It is not. With 4000 rows and a binary outcome, each row carries very little information, so the standard error on these estimates sits around 0.05 and an estimate roughly two standard errors from truth is unremarkable. Pushing the same script to 400,000 rows returns an intercept of minus 0.9973, tenure of minus 1.1920 and monthly of 0.7948, which is the behaviour you want to confirm before trusting anything. Binary outcomes are expensive in sample size, and this is precisely why a churn model built on 300 rows of a pilot cohort should never be reported to three decimal places.

First churn classifier end to end

With the mechanics understood, the library version is short. One detail deserves attention before any of it runs. Scaling is not optional here in the way it is optional for plain least squares. Regularisation is switched on by default, and a penalty applied to raw coefficients punishes whichever feature happens to be measured in small units. TotalCharges runs into the thousands while tenure runs to 72, so without standardising, the penalty falls almost entirely on tenure for reasons that have nothing to do with churn.

import pandas as pd
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import train_test_split
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler

df = pd.read_csv('Telco-Customer-Churn.csv')
df['TotalCharges'] = pd.to_numeric(df['TotalCharges'], errors='coerce')
df = df.dropna(subset=['TotalCharges'])

y = (df['Churn'] == 'Yes').astype(int)
num = ['tenure', 'MonthlyCharges', 'TotalCharges']
cat = ['Contract', 'InternetService', 'PaymentMethod',
       'PaperlessBilling', 'TechSupport', 'OnlineSecurity']
X = pd.get_dummies(df[num + cat], columns=cat, drop_first=True).astype(float)

X_tr, X_te, y_tr, y_te = train_test_split(
    X, y, test_size=0.25, random_state=42, stratify=y)

# StandardScaler first, because the L2 penalty is scale sensitive
model = make_pipeline(StandardScaler(), LogisticRegression(max_iter=1000))
model.fit(X_tr, y_tr)

proba = model.predict_proba(X_te)[:, 1]   # column 1 is the positive class
print('scored customers:', proba.shape[0])

I raised max_iter to 1000 deliberately. Its documented default is 100, and lbfgs frequently fails to converge inside that budget on a widened one hot encoded matrix, which produces a ConvergenceWarning rather than an error. Warnings scroll past in a notebook and the model still returns predictions, so an under fitted model quietly reaches your evaluation cell. Raising the ceiling costs a fraction of a second and removes a whole class of confusion.

Two lines above are load bearing and both relate to a genuine defect in this dataset. Eleven of the 7043 rows carry a single space character in TotalCharges rather than a number, all of them new customers with a tenure of zero. Because one non numeric value poisons the whole column, pandas types it as object. Reach for astype directly and Python stops you:

>>> df['TotalCharges'].astype(float)
Traceback (most recent call last):
  File "pandas/core/dtypes/astype.py", line 133, in _astype_nansafe
    return arr.astype(dtype, copy=True)
ValueError: could not convert string to float: ' '

Using pd.to_numeric with errors set to coerce turns those spaces into NaN so they can be handled deliberately. Dropping eleven rows out of 7043 is defensible; silently letting them through as zeros is not, because a zero total charge for a zero tenure customer is a real value and a missing one is not, and the model cannot tell the difference. Cleaning decisions of exactly this shape are covered at length in cleaning messy data from the Data Analyst Series, so I will not repeat that ground here.

Accuracy as a misleading single number

Roughly 26 percent of these customers churn, so a model that predicts nobody ever leaves scores about 74 percent accuracy. That is the trap from the opening paragraph and it is not a rare pathology. It appears whenever features are weak, regularisation is heavy, or the encoding has quietly destroyed the signal. Accuracy will not tell you, because accuracy cannot distinguish a model that discriminates from a model that has memorised the base rate.

That crosstab is now my first move on any classification problem, and on genuine Telco data it looks nothing like the broken export. Month to month customers churn at 42 percent while two year contract customers churn at under 3 percent, a fifteen fold difference visible before a single model is fitted. Any feature with that much separation should show up as a large coefficient, and if it does not, something between the raw file and the design matrix is broken.

Fitted properly, the coefficients recover exactly that structure. Below are the eight largest by absolute value, standardised so they are comparable, with each one exponentiated into an odds ratio. Values under 1.0 reduce the odds of churn and values above 1.0 raise them.

Fiber optic deserves a comment because it is the one result here that is not obvious in advance. Faster internet raising the odds of leaving by 43 percent sounds backwards until you remember that fiber customers pay considerably more and therefore shop around more. That is a hypothesis rather than a finding, and confusing the two is the classic error described in correlation and causation. A coefficient tells you what moves together with churn in this sample. Establishing why requires an experiment.

Threshold choice as a business decision

Calling predict gives you labels at a threshold of 0.5. Nothing about 0.5 is special. It is a convention, and treating it as a default rather than a decision is how good models produce bad retention campaigns. What predict_proba returns is a score, and where you cut that score determines how many customers you contact and how many leavers you miss.

Read that table as a menu of business positions rather than a search for a best row. Dropping to 0.2 catches 93.5 percent of leavers but flags 62 customers to find them, and slightly more than half of those flags are wasted. Holding at 0.6 wastes far less budget yet misses more than half the churners. Accuracy peaks at 0.5, which is exactly why optimising accuracy is the wrong instinct: the accuracy maximising threshold is simply the one that agrees with the base rate most often.

Arithmetic settles this faster than argument. Where a retention call costs about 15 dollars and a saved customer is worth roughly 500 dollars of margin, a wasted call is cheap relative to a missed churner and the low threshold wins comfortably. Flip the numbers, with an expensive hardware giveaway against a low margin plan, and 0.6 becomes correct. Ask for those two figures before choosing, and if nobody can supply them, that conversation is more valuable than any tuning you could do.

Area under this curve was 0.853, and its interpretation is pleasantly concrete: pick one churner and one stayer at random, and the model gives the churner a higher score 85.3 percent of the time. Because that statement holds at every threshold at once, AUC is the number I quote when comparing models and precision at a stated threshold is the number I quote when discussing a campaign. Those are different audiences and they need different metrics. Part 11 takes this apart properly, including cross validation and the ways leakage manufactures an AUC you should not believe.

Regularisation defaults that changed under you

One surprise catches almost everyone arriving from a statistics background: scikit-learn regularises by default. C carries a default of 1.0 and is the inverse of regularisation strength, so smaller values penalise harder. Unpenalised maximum likelihood, which is what a statistics course teaches and what statsmodels gives you, requires setting C to infinity explicitly. Comparing a scikit-learn fit against a statsmodels fit and finding the coefficients disagree is nearly always this, not a bug.

A more current trap concerns how that penalty is specified. Documentation for version 1.9 records that the penalty argument was deprecated in 1.8 and will be removed in 1.10, replaced by l1_ratio, whose own default changed from None to 0.0 in the same release. Code written as penalty set to l1 becomes l1_ratio set to 1.0, penalty set to l2 becomes l1_ratio of 0.0, and penalty set to None becomes C of infinity. Anyone pinning scikit-learn loosely in a requirements file has roughly one minor release before that breaks, and it will break as a deprecation warning first and an error later, which is the worst possible failure shape for a scheduled retraining job that nobody watches.

On solver choice, lbfgs is the documented default and I would keep it for a first classifier. It handles L2 penalties and multiclass problems, and it converges quickly on a matrix of this size. Reach for saga only when you actually need L1 or elastic net, since it is slower and genuinely fussy about feature scaling. Steer clear of liblinear on anything with three or more classes, because it cannot minimise the full multinomial loss and raises an error rather than doing something reasonable. Newton cholesky suits the case where rows vastly outnumber features, though its memory use grows with the square of features times classes, so it is a poor fit for wide one hot matrices.

Setup I would use for a first classifier

My recommendation for any first pass at a binary problem is narrow and deliberately boring. Wrap StandardScaler and LogisticRegression in a pipeline so scaling cannot leak across the split. Set max_iter to 1000 and leave C at 1.0 until you have a reason. Stratify the train test split, because with a 26 percent positive class an unstratified split can hand you noticeably different base rates in train and test. Crosstab your strongest categorical feature against the target before fitting anything. Then report AUC alongside a confusion matrix at a threshold you can justify in one sentence.

Keep this model even after gradient boosting beats it in Part 12, and it will. Logistic regression stays useful as the baseline everything else must clear, and a tree ensemble that fails to beat it is telling you something important about your features. Its coefficients also remain explainable to a regulator in a way that a boosted ensemble is not, which matters more than accuracy in several industries. Our churn project now has a scored customer list and a defended threshold. What it does not yet have is any honest estimate of how those numbers will hold up on customers we have never seen, and a single train test split of 122 rows is a thin basis for a retention budget. Part 11 fixes that with proper evaluation, cross validation, and the leakage patterns that make a model look far better than it is.

Run the code in this part against the full 7043 row file before moving on, and print your own confusion matrix at three different thresholds. Seeing the flagged customer count move while accuracy barely twitches is the lesson here, and it lands far harder when the numbers are yours.

References

• scikit-learn, LogisticRegression API reference, for solver, C, max_iter and class_weight defaults and the penalty deprecation in 1.8.
• scikit-learn user guide, Linear Models, for the regularised logistic loss and the solver comparison table.
• Telco Customer Churn dataset, IBM sample data on GitHub, 7043 rows, the file used throughout this series.
• pandas, to_numeric reference, for the errors argument used to handle the blank TotalCharges values.

About The Author

Dr. Pranay Jha

Dr. Pranay Jha is a Cloud and AI Consultant with 18+ years of experience in hybrid cloud, virtualization, and enterprise infrastructure transformation. He specializes in VMware technologies, multi-cloud strategy, and Generative AI solutions. He holds a PhD in Computer Applications with research focused on Cloud and AI, has published multiple research papers, and has been a VMware vExpert since 2016 and a VMUG Community Leader.

See author's posts