mirror of
https://github.com/ghndrx/starlane-router.git
synced 2026-02-10 06:45:01 +00:00
79 lines
2.4 KiB
Markdown
79 lines
2.4 KiB
Markdown
# a2a-router (FastAPI + DigitalOcean Gradient)
|
|
|
|
Super-basic app that routes messages either to a local echo or to a DigitalOcean Gradient inference endpoint.
|
|
|
|
## Features
|
|
- FastAPI with `/route` and `/healthz`
|
|
- Simple router: keyword/length heuristic or explicit `route_hint`
|
|
- Config via env/ConfigMap/Secret
|
|
- Docker + Kubernetes manifests (Deployment, Service, Ingress)
|
|
- Script to create a tiny DO Kubernetes cluster via `doctl`
|
|
|
|
## Configure
|
|
Environment variables used by the app:
|
|
- `GRADIENT_ENDPOINT_URL`: Full URL to your Gradient inference endpoint
|
|
- `GRADIENT_API_KEY`: API key/token for Gradient
|
|
- `GRADIENT_AUTH_SCHEME`: `authorization_bearer` (default) or `x_api_key`
|
|
- `ROUTE_KEYWORDS`: comma-separated keywords to force Gradient routing
|
|
|
|
## Local Run
|
|
```bash
|
|
python -m venv .venv && source .venv/bin/activate
|
|
pip install -r requirements.txt
|
|
uvicorn app.main:app --host 0.0.0.0 --port 8080
|
|
```
|
|
|
|
Test:
|
|
```bash
|
|
curl -s http://localhost:8080/healthz
|
|
curl -s -X POST http://localhost:8080/route -H 'Content-Type: application/json' \
|
|
-d '{"message":"call the gradient model"}' | jq
|
|
```
|
|
|
|
## Build & Push Image
|
|
Replace `YOUR_REGISTRY/YOUR_IMAGE:tag` appropriately.
|
|
```bash
|
|
docker build -t YOUR_REGISTRY/YOUR_IMAGE:latest .
|
|
docker push YOUR_REGISTRY/YOUR_IMAGE:latest
|
|
```
|
|
|
|
## Create a Small DO K8s Cluster
|
|
Requires `doctl` and `kubectl`.
|
|
```bash
|
|
chmod +x scripts/doctl_create_cluster.sh
|
|
./scripts/doctl_create_cluster.sh a2a-cluster nyc1 s-1vcpu-2gb 1.29.1-do.0 1
|
|
```
|
|
|
|
Install nginx ingress controller if needed:
|
|
```bash
|
|
kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v1.10.1/deploy/static/provider/cloud/deploy.yaml
|
|
```
|
|
|
|
## Deploy to Kubernetes
|
|
1) Create/update secrets and config:
|
|
```bash
|
|
kubectl apply -f k8s/config-secrets.yaml
|
|
```
|
|
|
|
2) Set the image and domain in manifests, then apply:
|
|
```bash
|
|
# edit k8s/deployment.yaml -> set image
|
|
# edit k8s/ingress.yaml -> set host domain
|
|
kubectl apply -f k8s/deployment.yaml
|
|
kubectl apply -f k8s/ingress.yaml
|
|
```
|
|
|
|
3) Get the ingress address and point your DNS (A/AAAA record) to it manually.
|
|
```bash
|
|
kubectl get ingress a2a-router -o wide
|
|
```
|
|
|
|
## API
|
|
- `POST /route`
|
|
- body: `{ message: string, route_hint?: 'local'|'gradient', metadata?: object }`
|
|
- response: `{ route: 'local'|'gradient', output: object }`
|
|
- `GET /healthz`
|
|
|
|
## Notes
|
|
- To force Gradient usage, pass `route_hint: "gradient"` or include a keyword in `ROUTE_KEYWORDS`.
|
|
- If Gradient call fails, endpoint returns 502 with message. |