ai-relay Serverless AI Gateway

Project Summary

MoyuFamily published

ai-relayMoyuFamily

on GitHub. The repository contains a serverless AI API gateway built on Vercel Edge Runtime. It handles multi-provider routing across OpenAI, Claude, DeepSeek and custom endpoints, performs key rotation with 429 backoff, and supports chained failover. Deployment requires only three environment variables and an Upstash Redis connection. The project targets users who need an OpenAI-compatible proxy without managing servers or Docker containers.

Architecture Details

The gateway runs entirely on Vercel Edge functions and stores state in Upstash Redis via REST API calls. Environment variables KV_REST_API_URL and KV_REST_API_TOKEN are injected automatically after the database is linked in the Vercel dashboard. Routing logic selects providers in round-robin order and falls back on failure. Each request can carry a temporary key signed with HMAC-SHA256 using RELAY_SIGNING_SECRET; keys expire without server-side storage. Virtual model names map to real provider endpoints, and streaming responses pass through as Server-Sent Events without buffering. An admin interface protected by RELAY_ADMIN_KEY exposes quota settings, usage metrics and model testing endpoints. Webhooks can be configured for daily reports or threshold alerts to Slack, DingTalk or Lark.

Deployment Steps

Fork or deploy directly from the repository using the Vercel button. Set RELAY_API_KEY for client authentication, RELAY_ADMIN_KEY for the dashboard, and RELAY_SIGNING_SECRET for temporary keys. After the first deployment, navigate to the project’s Storage tab, create an Upstash Redis instance on the free tier, and connect it to the project. The build injects the required KV variables. Update the OpenAI client by changing only the base_url to the deployed endpoint while keeping the original SDK calls to /v1/chat/completions. For local testing, copy .env.local.example and supply the same three relay keys plus the Redis credentials.

Practical Considerations

The free Vercel tier plus free Upstash tier covers low to moderate traffic for individuals or small teams, but sustained high request volumes will require paid plans on both platforms. All traffic passes through Vercel’s edge network, introducing an additional hop compared with direct provider calls. Because the project relies on Redis for state, any outage in the Upstash instance blocks key rotation and quota enforcement. The admin panel and webhook features are useful for production monitoring yet add configuration surface that must be secured with strong keys. No custom rate-limit algorithms beyond round-robin and simple backoff are provided; teams needing fine-grained control must extend the source.

---

📖 Related articles

Need a consultation?

I help companies and startups build software, automate workflows, and integrate AI. Let's talk.

Get in touch