Caching the right things keeps you under rate limits and reduces latency. The rule: cache static references, never cache another company's generated content.
Safe to cache
- The model list (
GET /v1/models) — changes rarely; cache for minutes. - API metadata and capability descriptors — cache for minutes.
- Your own stored completions — only if you persist them server-side.
These responses include Cache-Control and ETag headers:
Cache-Control: public, max-age=300
ETag: "a1b2c3"Use conditional requests with If-None-Match to revalidate cheaply.
Do not cache
- Generated chat completions and tool results — they are company-specific and may contain private business data.
- Anything keyed only by request body without company isolation.
Client-side recommendations
1. Cache /v1/models for ~5 minutes.
2. Send If-None-Match on revalidation.
3. Honour X-RateLimit-Remaining and back off before 429.
4. Never share a cached completion across companies.Generative responses are computed per request and are not served from a shared edge cache. This keeps tenant data isolated while still letting you cache the small, stable reference data.
Was this article helpful?