higress

Higress is a cloud-native API gateway built on Istio and Envoy, designed as an AI-native gateway that unifies LLM API management, MCP (Model Context Protocol) server hosting, and traditional API gateway functions. It originated at Alibaba to address long-connection disruption during gateway reloads and improve gRPC/Dubbo load balancing, and is now developed as a vendor-neutral CNCF Sandbox project. Core capabilities include: (1) AI Gateway — connects to all mainstream LLM model providers via a unified protocol with AI observability, multi-model load balancing, token rate limiting, and caching; (2) MCP Server Hosting — hosts MCP Servers through its plugin mechanism, enabling AI Agents to call tools and services with unified authentication, fine-grained rate limiting, audit logs, and dynamic updates without connection drops; (3) Kubernetes Ingress Controller — listed in official Kubernetes documentation, compatible with NGINX Ingress annotations, supports Gateway API standard, and reduces resource overhead compared to ingress-nginx; (4) Microservice Gateway — discovers services from Nacos, ZooKeeper, Consul, Eureka, and integrates with Dubbo, Nacos, and Sentinel; (5) Security Gateway — supports WAF, key-auth, hmac-auth, jwt-auth, basic-auth, and OIDC. Key technical advantages: production-grade with over 2 years of validation at Alibaba supporting hundreds of thousands of requests per second; true streaming processing of request/response bodies including SSE for AI workloads; Wasm plugin extensibility in Go/Rust/JS with sandbox isolation and traffic-lossless hot updates; out-of-the-box UI console; Let's Encrypt integration for automatic certificate management; and single-Docker-command deployment for individual developers. Installation is straightforward via Docker (ports 8001 for UI, 8080 for HTTP, 8443 for HTTPS) or Helm on Kubernetes with configurable regional registry mirrors. The project provides a rich official plugin library covering AI, traffic management, and security, meeting over 90% of business scenario requirements. Related repositories include Higress Console, Higress Standalone, Plugin Server, and Wasm Plugin Golang SDK.