novafabric

NovaFabric is a self-hosted command-line tool for making AI and HPC runs reproducible, comparable and auditable. Its unit of value is a capsule: a ULID-named directory written under ~/.novafabric/capsules/ holding the observable facts of one execution, rather than a trace row in a hosted database. Install with `pip install novafabric` (Python 3.12+); optional extras add dashboard, query, server, compliance and Sigstore support, and `nova doctor --check-extras` reports what is missing. Capture: `nova capture python my_agent.py --dataset data.csv` works with any command and requires no application changes. It injects a sitecustomize.py loader through PYTHONPATH and installs monkey-patches for OpenAI, Anthropic and MCP SDKs, plus wire-level hooks on httpx, requests, aiohttp and urllib3, with a layering guard so nested calls yield a single record. OpenTelemetry GenAI semantic-convention fields (temperature, top_p, max_tokens, seed, finish reasons and more) are extracted. For non-Python clients such as Claude Code, Cursor or Node/Go agents, two transparent proxies — `nova api-proxy` and `nova mcp-proxy` — capture LLM and MCP traffic without touching the client. Capsules are written on both success and failure, so a crash is recorded evidence with status, exit code and an error block. Capsule contents: capsule.yaml manifest, trace.jsonl spans, model-calls.jsonl, tool-calls.jsonl, env.lock, replay.yaml, inputs and outputs (stdout/stderr), and redaction-proof.json. Secret scanning runs against every artifact before finalization; a capsule without its redaction proof is invalid to `nova validate` and cannot be exported. Replay and diff: `nova replay` offers four modes — forensic (read-only inspection, no network or subprocess), mocked (re-runs the command with model and tool calls served from the capsule cache), semantic (judges meaning rather than tokens, 0.0–1.0 similarity) and exact (byte-exact eligibility requiring a deterministic environment and per-call seed). The README explicitly does not claim exact replay of remote LLM calls. `nova diff` compares two runs, and `nova diff --assert-no-regressions` exits non-zero on behavioural change, positioned as a CI gate for agents that worked yesterday and fail today. Governance and lineage: a local SQLite Asset Registry stores versioned assets of seven types (model, agent, prompt, tool, dataset, evaluation, deployment), addressed as name@version with a lifecycle running from development to archived; promotion is governance metadata only and deploys nothing. The lineage layer emits mechanical consumed, produced_by and replayed_from edges with observed versus inferred confidence, supports provenance, blast-radius and replay-chain queries, and can emit OpenLineage events plus W3C PROV-N export. Experimental optional backends exist for larger graphs. `nova export-evidence` builds a signed ZIP containing the capsule, a lineage subgraph, in-toto DSSE attestations and ed25519 signatures, verifiable with sha256sum and an ed25519 verifier without the NovaFabric runtime. Fit: the project states it is not an orchestrator, model trainer, inference server, vector DB, LLM gateway or CI runner, and that a managed SaaS observability platform will be less work when real-time dashboards, alerting and multi-user analytics are the goal. Server mode and the live dashboard are marked experimental, capsules are not frozen until a planned v1.0 schema freeze, and the tool produces evidence that supports compliance workflows rather than certifying them. Intended audiences named in the README include ML/platform engineers, HPC and research teams, compliance officers, incident responders and agent-framework authors.