Kalshi EV ML

Minimal project for training and evaluating +EV Kalshi models.

Debrief: approach and what we measure

This project models Kalshi binary market outcomes (YES vs NO) and then converts model probabilities into an expected-value trading policy.

The pipeline has two stages:

  1. Data pipeline (src/process_data.py)
    • Load raw market snapshots.
    • Clean/normalize columns and parse timestamps.
    • Build derived features (spreads, momentum, liquidity/volume ratios, time-to-event features).
    • Create a chronological train/validation/test split.
  2. Model + strategy pipeline (src/train.py)
    • Train three models (logistic regression, gradient boosting, and ExtraTrees).
    • Calibrate probabilities with isotonic regression.
    • Build a fixed weighted soft-voting ensemble from the calibrated model outputs.
    • Run an edge-threshold sweep based on an EV trading rule.
    • Save both classification and strategy metrics.
    • Compare standalone models against the ensemble under the same evaluation flow.

We evaluate two kinds of performance:

  • Classification metrics (probability quality):
    • roc_auc: ranking quality for positives vs negatives.
    • log_loss: penalizes overconfident wrong probabilities.
    • brier: mean squared probability error.
    • precision, recall, f1: thresholded prediction quality (at 0.5).
  • Trading/strategy metrics (economic utility):
    • total_pnl: aggregate realized expected-value payoff.
    • mean_pnl_per_row: average contribution per market row.
    • mean_pnl_per_trade: average payoff conditional on taking a trade.
    • trade_rate: fraction of rows where policy trades.
    • win_rate: fraction of executed trades with positive realized payoff.

The key objective is not only predictive accuracy, but robust +EV decisions under realistic thresholding.

Quick start

  1. Create a virtual environment and install dependencies:
    • python -m venv .venv
    • source .venv/bin/activate
    • pip install -r requirements.txt
  2. Add your raw Kalshi export at the path expected by src/process_data.py (see RAW_CSV in that file).
  3. Run data processing:
    • python src/process_data.py
  4. Run model training/evaluation:
    • python src/train.py

Directory and file reference

Paths below are relative to this repository root (kalshi_ev_ml/). Only paths currently tracked in git are listed; running the pipeline creates additional local files (see src/*.py).

/: Project root: documentation, dependencies, source, notebooks, and committed report/plot outputs.

/README.md: Overview, approach, metrics, quick start, and directory map.

/requirements.txt: Pinned Python dependencies for the pipeline and notebook.

/.gitignore: Git ignore rules for envs, caches, and large/generated assets.

/src/: Python entrypoints for data processing and training.

/src/process_data.py: Load raw CSV, clean and engineer features, and apply a chronological train/validation/test split; output locations are set in this module.

/src/train.py: Train logistic + GBM + ExtraTrees, calibrate each model, blend them with a fixed weighted soft-voting ensemble, and run the shared EV strategy + threshold sweep; metrics path and model save locations are set in this module.

/reports/: Committed evaluation snapshot for the tracked run.

/reports/metrics.json: Val/test classification metrics, default strategy stats, threshold sweep, and best threshold per model.

/notebooks/: Exploratory and reproducible walkthroughs (not required for CLI runs).

/notebooks/first_run_pipeline_walkthrough.ipynb: End-to-end mirror of process_data + train with tables and saved plots.

/results/: Committed figures from the notebook visualization cells.

Notes

  • Pipeline uses time-based split by close_time.
  • Primary target is result == "yes".
  • Leakage columns (result, settlement_*, etc.) are excluded from training features.