cc-relay

CC-Relay is a fast LLM API gateway written in Go that solves the single-provider limitation of Claude Code by routing requests across multiple Anthropic-compatible providers. It enables operations teams to pool rate limits across multiple API keys, automatically route simpler tasks to lighter and cheaper models, and implement failover to prevent service interruption. The gateway supports Anthropic, AWS Bedrock, Azure, Google Vertex, and Anthropic-compatible services like MiniMax, Z.AI, and Ollama. Teams use CC-Relay to reduce LLM API costs, improve reliability through automatic provider failover, and integrate both enterprise and personal API credentials in a single unified gateway.

[{"step":"Install and configure cc-relay","description":"Download cc-relay from GitHub and set up provider credentials in the config file (e.g., `~/.cc-relay/config.yaml`). Add your Anthropic, OpenAI, and Mistral API keys with appropriate rate limits.","tips":"Use environment variables for sensitive keys. Test with `--dry-run` flag first to verify provider connectivity."},{"step":"Define your routing criteria","description":"Create a routing profile in cc-relay for your specific use case (e.g., `marketing-content`, `code-review`, or `customer-support`). Specify quality thresholds, cost limits, and latency tolerances.","tips":"Start with conservative thresholds (e.g., max latency 3s, min quality 8/10) and adjust based on initial results."},{"step":"Integrate with your workflow","description":"Replace direct Anthropic API calls in your scripts/tools with cc-relay endpoints (e.g., `http://localhost:8080/v1/messages`). Use the `--provider` flag to force a specific provider for testing.","tips":"Use cc-relay's `--stats` endpoint to monitor performance in real-time. Set up Grafana dashboards for long-term tracking."},{"step":"Optimize and scale","description":"Analyze cc-relay logs and provider metrics weekly. Adjust routing rules based on actual performance data. Consider adding new providers (e.g., Google Vertex AI) for redundancy.","tips":"Use cc-relay's `--benchmark` mode to compare providers under load. Implement circuit breakers for providers with >5% failure rates."}]