Statistics for CMT Candidates: Describe Data Before Testing a Signal

Why a technician needs statistics

Charts summarize observations; statistics help describe variation and test whether a proposed pattern survives beyond a few memorable examples. The 2026 CMT Program Guide includes descriptive statistics, correlation, regression, probability, and distribution concepts in Level I objectives. Level II adds inferential statistics. Its public objectives set scope; they do not tell you which calculation will occur in a future exam.

The first discipline is to distinguish description from inference. A sample average describes the observed sample. Claiming that a signal has an enduring edge in future data requires a defensible test, adequate data, and explicit assumptions.

Mean, median, and dispersion

Consider five simple daily returns in percent: −2, 0, 1, 1, and 5. Their arithmetic mean is (−2 + 0 + 1 + 1 + 5)/5 = 1%. The median is 1%, and the mode is 1%. In this sample the three coincide, but the 5% return still stretches the distribution. The squared deviations from 1 are 9, 1, 0, 0, and 16, summing to 26 percentage-point-squared units. Treating these five observations as a population gives variance 26/5 = 5.2 and standard deviation √5.2 ≈ 2.28 percentage points. Treating them as a sample gives variance 26/4 = 6.5 and standard deviation √6.5 ≈ 2.55 percentage points. Always specify which convention you use.

A z-score expresses distance from a mean in standard-deviation units: z = (observation − mean)/standard deviation. A 5% return using the population calculation above has z = (5 − 1)/2.28 ≈ 1.75. That does not mean it has a known probability unless the distributional model is appropriate. Financial returns can be skewed and have fat tails. A normal-model “empirical rule” is a model property, not a promise about market losses.

For compounded wealth, arithmetic average return can mislead. A +50% period followed by −50% leaves 1.5 × 0.5 = 0.75 of starting wealth, a 25% cumulative loss. The arithmetic average is 0%, while the two-period geometric average is √0.75 − 1 ≈ −13.4% per period. The compounding calculation answers a different question.

Correlation and regression

Pearson correlation r measures linear co-movement from −1 to +1. If r = 0.8 for two return series, they have strong positive linear association in that sample. It does not establish that one causes the other, that future correlation stays 0.8, or that both have similar volatility. The coefficient of determination in a simple one-predictor regression is r², so 0.8² = 0.64. In that narrow regression setting, 64% of sample variation in the dependent variable is associated with the fitted linear relationship. It is not a 64% probability that a trade succeeds.

A linear regression has the form predicted y = intercept + slope × x. Suppose a fitted relation for monthly percentage returns is predicted asset return = 0.4 + 1.2 × benchmark return. For a benchmark return of 2%, the fitted asset return is 0.4 + 1.2×2 = 2.8%. If the actual asset return is 1.5%, the residual is 1.5 − 2.8 = −1.3 percentage points. The fitted line describes the sample relationship; forecast use requires checking stability, assumptions, and out-of-sample performance.

Scatterplots should precede a coefficient. One outlier can dominate r, while a curved relationship can have a small linear correlation despite a visible pattern. Plot both axes with units and inspect clustering, outliers, and regime shifts.

From a chart idea to a testable hypothesis

Suppose the hypothesis is “a close above the upper Bollinger Band is followed by above-average next-month returns.” Define the universe, band settings, decision timestamp, holding period, comparison group, costs, and whether overlapping signals count separately. A null hypothesis could be no difference in mean next-month return between signal and non-signal observations under the defined sampling plan. A p-value, if calculated, is the probability of data at least as extreme under the specified null model; it is not the probability that the trading idea is true.

There are two broad error types. A false positive rejects a true null; a false negative fails to reject a false null. Testing 100 parameter combinations and reporting only the smallest p-value increases false-positive risk. Split development and evaluation periods, or use an appropriate walk-forward design. Even an out-of-sample result can fail after market structure changes.

Price series are not always independent and identically distributed. Volatility clustering, overlapping returns, and shared market factors violate simple textbook assumptions. Use appropriate methods for the data and describe uncertainty. Do not convert a small historical sample into a precise forecast.

A decision rule with invalidation

A signal should be proposed before examining its validation sample. For example: use a fixed 20-bar indicator, enter on the next open after a qualifying close, risk no more than a specified fraction of capital, and compare net return with a declared benchmark. If the out-of-sample estimate is economically trivial after costs or depends on one month, reject the claim of a useful edge. This is a research rule, not a promise of performance.

Statistical significance and practical value differ. A tiny average advantage might be statistically detectable in a huge sample yet too small after spread, fees, and slippage. Conversely, an apparently large mean from ten observations can be too uncertain to trust.

Common mistakes and self-check

Do not use percentage-point and decimal units interchangeably. Do not call r² a success probability. Do not assume normality because a histogram looks roughly symmetric. Do not drop failed signals or delisted securities from a historical test. See Backtesting Trading Systems for full implementation pitfalls.

Self-check: A strategy produces +20%, then −10%. What are the cumulative and geometric average returns? Answer: Wealth multiplies by 1.2 × 0.9 = 1.08, so cumulative return is +8%. The two-period geometric mean is √1.08 − 1 ≈ 3.92% per period. The arithmetic mean of 20% and −10% is 5%, a different descriptive statistic.

Next, hand-calculate mean and sample standard deviation for five returns from a practice chart, then describe one assumption you would need before treating that sample as evidence about future returns.