dlstreamer

DL Streamer (Deep Learning Streamer) is an open-source streaming media analytics framework built on the GStreamer multimedia framework. It is designed for constructing video and audio intelligence pipelines, from a single command-line detection example to multi-stream, multi-sensor deployments at the edge or in the cloud. Inference is powered by OpenVINO, targeting Intel CPU, GPU and NPU hardware. Pipelines are described as simple strings (or via Python/C++ code) and executed with hardware acceleration. The project ships with 30+ ready-to-run samples covering detection, classification, tracking, vision-language models, LiDAR and radar processing. It is part of the Intel Open Edge Platform. Key capabilities highlighted in the README: - One-line pipelines: a working detection pipeline can be built with a single gst-launch-1.0 command. - Hardware acceleration: the same codebase targets CPU, GPU and NPU on Intel platforms; switching device is a parameter change. - VLM and GenAI support: Vision-Language Models such as MiniCPM-V, CLIP and Whisper can run inside a GStreamer pipeline. - GstAnalytics compliance: supports the GStreamer metadata standard for interoperability with other GStreamer-based tools. - Messaging integration: built-in elements publish inference results to MQTT or Kafka without extra code. - Python-first extensibility: custom logic can be added as Python callbacks or full Python GStreamer elements. - Multi-stream, multi-sensor: mux/demux of many RTSP streams, LiDAR frames and radar point clouds in one process. - Model support: deploy models from Geti Studio, Ultralytics, Hugging Face, or any ONNX/OpenVINO IR model. Installation options include a Docker image (intel/dlstreamer:latest) and native packages for Ubuntu 22.04/24.04, plus Windows 11 support. A prerequisite script installs Intel GPU/NPU drivers. The README provides quick-start commands for model download, running a YOLO detection pipeline, outputting JSON metadata, and using the Python API via GStreamer bindings. Element categories include inference (gvadetect, gvaclassify, gvainference, gvagenai, gvaaudiotranscribe), analytics (gvatrack, gvaanalytics, gvastreammux/demux, gvamotiondetect), output (gvawatermark, gvametaconvert, gvametapublish, gvafpscounter) and 3D/sensors (g3dlidarparse, g3dinference, g3dradarprocess). Samples span detection, segmentation and pose, tracking, VLM/GenAI, multi-stream, 3D sensors, integration (ONVIF discovery, Geti deployment, MQTT/Kafka) and Python extensibility such as a smart NVR. Supported hardware listed includes Intel Core Ultra series 1-3, Intel Arc discrete GPUs, and 11th-13th Gen Intel Core. The project is MIT licensed and welcomes contributions.