Markovian

Markovian is an experimental Haskell package focused on finite stochastic kernels, Markov reward processes (MRP), Markov decision processes (MDP), policies, and bounded interpreters. The package provides exact finite-horizon expected-return evaluation with explicit model, policy, horizon, and discount parameters. It includes exact discounted value iteration with residual, value-error, and greedy-policy bounds, as well as exact deterministic policy iteration. The package supports Bayesian inference with canonical exact finite beliefs, post-transition prediction, and conditioning. It implements circuit representations with purity-indexed stochastic-circuit syntax, explicit sharing, and exact convex choice. For system modeling, it offers finite typed hypergraphs, structured cospans, and commuting open-system cells. Markovian includes implementations for finite open games with exact pure contextual equilibrium enumeration. The package is organized into separate modules: the root library depends only on 'base' and exposes finite exact semantics, while optional packages handle numerical operations, sampling, learning, dense exact representations, and benchmarks. Additional optional packages provide capabilities for continuous processes, autodiff, tensor operations, GPU acceleration, and neural network integration, all with explicitly bounded scopes. The package includes inventory control benchmarks and evidence for specific mathematical properties and implementations.