Scaling and transforms

Many models care about the size of their inputs. A linear model treats a value of 10,000 very differently from a value of 10, and its weights shrink or grow to compensate. Scaling and transforming put numeric features on comparable footing before the model sees them. Trees are much less sensitive, because their split thresholds absorb the units.

Scaling

• Standardization computes (x − μ) / σ. It is the default for linear and logistic models.
• Min-max scaling maps values to [0, 1]. It suits inputs with known bounds.
• Robust scaling uses the median and the interquartile range, so outliers have less effect.

Fit the scaler on the training data only, then apply the fitted scaler to validation and test data. Fitting it on everything is a quiet form of leakage.

Log transforms for skewed values

Revenue, counts and bytes are often skewed, with a few very large values. log1p(x) = log(1 + x) compresses the tail while keeping zero valid, because log(0) is undefined.

Ratios

Raw counts depend on size. Raw error counts are not comparable between a large tenant and a small one, but errors / requests is. Decide in advance what a zero denominator means: missing, zero, or a separate flag.

Outliers: signal or noise?

Decide whether an extreme value is signal or noise before you act on it. A fraudulent $50,000 charge is a real event, and in fraud data it may be exactly the positive you want. A sensor reading of -999 is almost certainly noise. Winsorizing, which clips values at chosen percentiles, is a good default for glitches. Dropping outliers blindly is not.

Look before you transform

Plot histograms and count values before you choose a transform:

• Zero inflation. Many exact zeros can mean "none", or "missing logged as zero".
• Heavy tails. A few large values dominate sums. Prefer rates, logs or clipping.
• Strong label correlation. A category that is 99% positive is either a strong signal or leakage.

Common mistakes

• Scaling before the split. The validation data then shapes the scaler, and the score is optimistic.
• Taking log of zero. Use log1p.
• Dropping every outlier. In fraud, abuse or medical data, the outliers may be the positives.

Check yourself

1. Why use log1p instead of log for counts? log1p handles zeros, while log(0) is undefined.
2. Which common model type cares least about scaling? Trees, because their split thresholds absorb the units.

Key takeaways

• Standardize for linear and logistic models, and fit the scaler on training data only.
• Use log1p for skewed positive values, and check the tail before choosing a transform.
• Prefer ratios when the size of the group varies, and define the zero-denominator case.
• Decide whether each outlier is signal or noise before you remove it.

Further learning

Numeric features are only half the story. The next page covers categories and text: encoding categories and text.