About this project

# Volatility Nowcasting from Market Data + News This project predicts short-horizon S&P 500 realized volatility (H=5 trading days) by combining market features (returns, realized vols, price ranges) with daily-pooled news signals (FinBERT sentiment + embeddings). ## Problem Statement Near-term volatility forecasting is central to risk management (VaR, capital buffers), trading strategies (volatility arbitrage, hedging), and derivatives pricing (options, structured products). The task is to estimate realized variance (RV) over the next H trading days using both market data and textual news sentiment. The model targets log(1 + RV) for stability and transforms back into volatility for interpretation. ## Data Science Approach - **Data**: S&P 500 daily returns, realized volatility, high-low ranges, plus daily-pooled news sentiment and embeddings from FinBERT - **Target**: Realized variance over H=5 trading days, modeled as log(1+RV_H) - **Validation**: Expanding-time cross-validation with embargo to avoid look-ahead leakage from overlapping horizons - **Metrics**: R² (fit to magnitude), IC (Spearman rank correlation, ordering skill), QLIKE (robust volatility loss) - **Interpretation**: Converts back to daily and annualized volatility (σ_daily = √(RV_H / H), σ_annual ≈ σ_daily × √252) ## Key Features - Hybrid feature set combining market data and news sentiment - Robust embargoed expanding-time cross-validation - Interactive Streamlit dashboard showing out-of-fold truth vs prediction, derived daily/annualized volatility, and rolling 63-day IC & R² - Full pipeline from data processing to model training and evaluation - Lightweight Dockerfile for containerized runs - CI checks and automated tests ## Live Demo - **Streamlit App**: Available at the live link in the repository - **Colab Notebook**: Runnable notebook for full pipeline exploration ## Preview of Results Example latest forecast: annualized σ (H=5) of 17.9%, RV_H ≈ 0.000637, σ_daily ≈ 1.13%. ## How to Run Locally Clone the repo, install dependencies, and run the app: ```bash git clone https://github.com/ingo-stallknecht/volatility-nowcasting.git cd volatility-nowcasting pip install -r requirements.txt streamlit run app.py ```