Quantmodel · Strategy research
Carlson's Defense First
Carlson's Defense First receives a research grade of F (composite 47 / 100) on Paper-to-Profit robustness tests run against published monthly ours versus SPY. On these tests the published path does not currently support a constructive allocation case.
Factor grades
Published CAGR 11.59% / Sharpe 1.09 / max DD -16.4% over 647 months. Those headline stats are context only; the grade is the robustness tests above.
Assessment
Statistical edge
The path makes money vs zero (p=0.000) but is not a significant edge versus SPY (p=0.512, excess -0.13% per month). Same failure mode as a strategy that only rides beta.
Bootstrap versus SPY
Block bootstrap of consecutive months (400 paths): P(beat SPY) over 6.0 months is 33%; over 24.0 months is 34%. This is the Paper-to-Profit 'realistic path' test, not a reshuffled iid sample.
Alpha decay
19% of 12-month OLS windows have a significantly positive slope and 46% are significantly negative. Terminal strategy/SPY wealth ratio is 0.76. A steadily rising log-relative-wealth line is the decay test from the tutorial; a late collapse is a warning.
Overfit / stability
70/30 OOS/IS Sharpe ratio is 1.15; second/first-half Sharpe ratio is 0.92. Walk-forward parameter search is not estimable on a frozen catalog snapshot; this split is the overfit proxy. Return-space noise did not halve Sharpe on the tested grid.
Transaction costs
Average daily turnover is 1.025%. Additional one-way costs through 24 bps did not cut Sharpe in half. That is the tutorial's cost-robust case.
Regimes and crises
The path beats SPY in 19% of classified months (4.0/7.0 regimes). In bear/strong-bear/vol-down months it beats SPY 100% of the time. Crisis windows: Dot-com bust: strategy 4.5% vs SPY -38.4%; Global Financial Crisis: strategy 12.4% vs SPY -46.0%; COVID crash: strategy 4.1% vs SPY -19.4%; 2022 inflation shock: strategy 4.8% vs SPY -23.9%.
Limitations
Tests use published monthly ours, already lagged one bar in the engine, versus SPY. t-tests assume roughly comparable monthly means; Wilcoxon is shown as a non-normal check. Bootstrap paths are overlapping consecutive months (not iid reshuffles). Noise is added to monthly returns, not to OHLC, because this producer does not rerun backtests. Walk-forward parameter optimization, SPA, and PBO are not estimable without a trial ledger. Fee drag is extra one-way bps on catalog average daily turnover, on top of the published costed path.
Edge tests
| Test | Statistic | p-value | Significant? |
|---|---|---|---|
| t-test vs zero | 7.98 | 0.0000 | yes |
| t-test vs SPY (paired) | -0.66 | 0.5120 | no |
| Wilcoxon vs zero | 140400.0 | 0.0000 | p<0.05 |
Part 1: a strategy that only beats zero but not SPY has no statistically significant edge versus the market.
Monthly return distributions
Strategy versus SPY on the shared window. Input to the t-tests above.
Block bootstrap — 6 months
400 overlapping consecutive-month paths. P(beat SPY) = 33% . Fan is p10 / p50 / p90 of terminal wealth from $1.
Block bootstrap — 24 months
Same construction at a two-year horizon. P(beat SPY) = 34% .
Bootstrap excess CDF
Sorted 24-month strategy minus SPY terminal wealth. Fraction above zero is P(beat SPY).
Relative wealth versus SPY
Strategy equity divided by SPY equity. A rising line is persistent outperformance; a late roll-over is decay.
Rolling OLS slope of log relative wealth
12-month window on ln(strategy/SPY). Positive significant slopes are the tutorial's "edge is still there" test.
70/30 chronological split
Train through 2010-05. OOS/IS Sharpe ratio 1.15 . Walk-forward parameter search is not estimable here; this is the overfit proxy.
Transaction-cost sensitivity
Extra one-way bps applied to average daily turnover, on top of the published (~10 bps) path. Half-Sharpe at beyond 24 bps.
Return-space noise
Gaussian noise scaled by 12-month rolling vol. Not an OHLC re-backtest. Critical multiplier (Sharpe halves): not reached.
SPY regimes
Seven trend/vol states on a 12-month SPY window. Mean monthly return of strategy versus SPY in each state.
| Regime | Months | Strategy mean | SPY mean | Beats SPY? |
|---|---|---|---|---|
| Strong bull | 161.0 | 1.09% | 1.74% | no |
| Bull | 96.0 | 0.64% | 0.83% | no |
| Sideways | 15.0 | 0.02% | -2.32% | yes |
| Bear | 11.0 | -0.33% | -1.84% | yes |
| Strong bear | 3.0 | -2.55% | -7.47% | yes |
| Volatility up | 63.0 | 1.15% | 1.98% | no |
| Volatility down | 48.0 | 0.48% | -0.64% | yes |
Named crisis windows
| Episode | Status | Strategy | SPY |
|---|---|---|---|
| Dot-com bust | ok | 4.5% | -38.4% |
| Global Financial Crisis | ok | 12.4% | -46.0% |
| COVID crash | ok | 4.1% | -19.4% |
| 2022 inflation shock | ok | 4.8% | -23.9% |
Growth of $1
Published path versus SPY, 60/40 when present, and a leave-one-out catalog mix. Context only — not a scoring factor.
Drawdown
Related tools
- Strategy report — open (ok) Equity, allocations, and replication metrics.
- QuantStats tearsheet — open (ok) Full QuantStats HTML from the same monthly ours path.
- Paper to Profit robustness — open (ok) t-test vs zero and SPY, overlapping-block bootstrap, rolling OLS alpha decay, 70/30 stability, extra-bps cost grid, return-space noise, and seven SPY regimes. Walk-forward parameter search is not estimable on this catalog snapshot.