Handle and retry after rate limits from OpenAI

[dcadenas]

No description provided.

[dcadenas]

There's Retry after exponential backoff delay already enabled in the topic so any rate limit should be retried correctly.

I'm also adding:

• Dead Letter topic. So we don't lose any message that still may not be processed and ease monitoring and alerting
• Random jitter inside the function so when the exponential backoff is elapsed we don't get all retries done at the same time.
• Not sure if we have slack alert integrations in case we detect rate limit issues of topics increasing.

If this is not enough, in the future we could:

• Send some kind of token summary of the content to avoid hitting the token limit. It would be naive to just grab a random chunk of the content because the offensive part could be in another place but there may be other options we could investigate
• Fill the Increase Request Limit Form but we'd need to probe we really need that

[dcadenas]

Related #15