llm-voucher-gateway

llm-voucher-gateway is a specialized application that acts as a reverse proxy for vLLM (Very Large Language Model) servers. Its primary function is to add a layer of security and access control by implementing token-based authentication before requests reach the underlying model serving infrastructure. This tool is particularly useful for organizations deploying large language models where direct exposure of the vLLM endpoint is not desirable. By sitting in front of the vLLM server, the gateway validates incoming requests against a set of tokens or vouchers. Only authenticated requests are forwarded to the model server, while unauthorized attempts are rejected. Key features include: - **Reverse Proxy Functionality**: Transparently forwards valid API requests to the vLLM backend. - **Token-Based Authentication**: Ensures that only clients possessing valid tokens can interact with the LLM service. - **Security Layer**: Protects the vLLM server from direct, unauthenticated access, reducing the attack surface. This solution fits well into infrastructure setups requiring controlled access to generative AI models, allowing administrators to manage usage via token issuance rather than network-level restrictions alone.