Model Distribution in Kubernetes: Practical Approaches and Pitfalls
We use a mix of OCI registries, HTTP servers.
We use a mix of OCI registries, HTTP servers, and shared storage with workflow automation to distribute LLM models efficiently in production Kubernetes clusters.
