NetflixDifficulty: Advanced
In-House LLM Serving at Netflix
Netflix built infrastructure to deploy and serve large language models efficiently within their own data centers rather than relying on external APIs.
Interview angle
How would you design a system to serve large language models at scale with low latency and cost efficiency?
- Failure / topic
External LLM API dependency and latency constraints for real-time personalization
- Source
- netflixtechblog.com