mediapipe

MediaPipe is an open-source project from Google AI Edge that provides on-device machine learning capabilities for live and streaming media. According to the README, the primary developer documentation has moved to developers.google.com/mediapipe as of April 3, 2023, and the repository README now serves mainly as a pointer to that site. The project is positioned as "on-device machine learning for everyone," offering tools to customize and deploy ML features to mobile (Android, iOS), web, desktop, edge devices, and IoT. The README describes MediaPipe Solutions as a suite of libraries and tools for applying AI and ML techniques in applications, which can be plugged in immediately, customized, and used across multiple development platforms. Core components described include: - MediaPipe Tasks: cross-platform APIs and libraries for deploying solutions, with developer guides for vision (e.g., object detection), text (e.g., text classification), and audio (e.g., audio classification) tasks, plus setup guides for Android, web apps, and Python. - MediaPipe models: pre-trained, ready-to-run models for each solution. - MediaPipe Model Maker: for customizing models with your own data. - MediaPipe Studio: for visualizing, evaluating, and benchmarking solutions in the browser. The README notes that MediaPipe Solutions Preview is an early release. It also states that support for MediaPipe Legacy Solutions ended as of March 1, 2023, with remaining legacy solutions to be upgraded to new MediaPipe Solutions; the legacy code repository and prebuilt binaries continue to be provided on an as-is basis. For lower-level work, MediaPipe Framework is described as the low-level component used to build efficient on-device machine learning pipelines, similar to the premade solutions. Key framework concepts listed are Packets, Graphs, and Calculators, with installation and example applications in C++, Android, and iOS. On privacy, the README states that when using MediaPipe Tasks, processing of input data such as images, video, and text takes place on device, and MediaPipe does not send that input data to Google servers. It also notes that MediaPipe Tasks APIs send metrics about performance and utilization to Google, and that developers are responsible for obtaining informed consent from app users as required by applicable law. Community resources mentioned include a Slack community, a Google discussion group, and an "Awesome MediaPipe" curated list. Contributions are welcomed via the project's contributing guidelines, and GitHub issues are used for tracking requests and bugs. The README also lists numerous publications and a YouTube channel as further resources.