← Questions
LLM provider pool with failover
0
I was asked this
No reports yet. Sign in to say where you were asked it.
Write a client that sends completion requests to a pool of external LLM providers. It should be able to switch between providers, and when one goes down, move traffic to another without the caller noticing. Decide how a provider is marked unhealthy and brought back, how retries and timeouts work, and how you would test the failover.
No official answer yet. Write yours on the right and run it.
Nothing published yet.
Sign in to submit your code; every attempt is kept here.
Write and test your solution herePython in your browser, the question's tests level by level, every submission kept for you.Sign in to start coding