video-analyzer

Video Analyzer is a command-line tool for generating descriptions of video content from both visual and audio information. Its pipeline has three main stages: key-frame extraction and audio processing, frame-by-frame vision analysis, and final video reconstruction. The analyzer uses OpenCV to select key frames and OpenAI Whisper to transcribe available audio, including checks for poor-quality audio. Each extracted frame is sent to a vision-capable language model, with context from earlier frames used to preserve chronological progression. The frame analyses and transcript are then combined into a comprehensive description. By default, local inference uses Ollama with the Llama 3.2 Vision model; users can alternatively configure any OpenAI-compatible endpoint, such as OpenAI or OpenRouter, through CLI options or a JSON configuration file. Reported features include fully local operation without cloud services or API keys, optional cloud APIs, configurable Whisper and vision models, custom prompts, detailed JSON output containing metadata, transcript, frame analyses, and the final description, and a cascading configuration system where command-line arguments override user configuration and defaults. Requirements include Python 3.11 or newer and FFmpeg. The README recommends substantial RAM and GPU/Apple Silicon hardware for local LLM execution, while API-based use avoids those local model requirements. Installation is performed by cloning the repository and installing it with pip. An optional video-analyzer-tune package can optimize prompt templates from representative edited outputs using DSPy MIPROv2, writing tuned prompts to separate files. The project is released under the Apache License and includes usage, design, and contribution documentation.