Getting OpenAI API error : 429 in azure databricks while using OpenAI API key

Question

Getting OpenAI API error : 429 in azure databricks while using OpenAI API key

Sooriya E 35

Hi Team,

In Azure Databricks, our pipeline could successfully process datasets with up to 18,000 records via the LLM endpoint without issue.

Recently, with the same application code and logic, the API is frequently returning "OpenAI API error: 429 Too Many Requests" for much smaller batches.
Now, even with batches as small as 500 records, ~300 records are failing due to this error.
As suggested online, I regenerated our Azure OpenAI API key and updated our secret. This also did not resolve the issue.
Even after this, when processing 400 records, about 40 fail due to the same 429 error.
There have been no significant changes to our code or request pattern. Could you please investigate if there have been any changes to our Azure subscription, OpenAI service quotas, network throttling, or policy restrictions that could explain these errors? This issue is impacting our deliverables and client commitments — your urgent attention would be greatly appreciated. ERROR:dqapp.llm.llm_handler:OpenAI API error: 429 ERROR:dqapp.llm.llm_handler:OpenAI API error: 429 ERROR:dqapp.llm.llm_handler:OpenAI API error: 429 ERROR:dqapp.llm.llm_handler:OpenAI API error: 429 ERROR:dqapp.llm.llm_handler:OpenAI API error: 429 ERROR:dqapp.llm.llm_handler:OpenAI API error: 429 ERROR:dqapp.llm.llm_handler:OpenAI API error: 429 ERROR:dqapp.llm.llm_handler:OpenAI API error: 429 ERROR:dqapp.llm.llm_handler:OpenAI API error: 429 ERROR:dqapp.llm.llm_handler:OpenAI API error: 429 ERROR:dqapp.llm.llm_handler:OpenAI API error: 429 ERROR:dqapp.llm.llm_handler:OpenAI API error: 429 ERROR:dqapp.llm.llm_handler:OpenAI API error: 429 ERROR:dqapp.llm.llm_handler:OpenAI API error: 429 ERROR:dqapp.llm.llm_handler:OpenAI API error: 429

Thanks,

Sooriya E

Sooriya E 35 Reputation points

2025-08-22T08:17:31.0166667+00:00

Any updates??
Deleted

This comment has been deleted due to a violation of our Code of Conduct. The comment was manually reported or identified through automated detection before action was taken. Please refer to our Code of Conduct for more information.
Manas Mohanty 9,655 Reputation points Microsoft External Staff Moderator

2025-08-25T04:32:43.2033333+00:00

Hi Sooriya E

To deal with 429 rate limits,

You can reduce max_tokens on Deployment configuration, increase TPM on Model setting, load balance between multiple Azure OpenAI, Enable retry with exponential backoff.

Reference thread

https://learn.microsoft.com/en-us/answers/questions/2263211/429-rate-limit-errors-on-gpt-4-1

Hope it helps address the issue.

Thank you
Sooriya E 35 Reputation points

2025-08-28T03:56:41.9366667+00:00

Hi @Manas Mohanty

We tried the methods you suggested, but we are still encountering the same issue (OpenAI API error code: 429). It would be better to schedule a call with the engineering team, as this is causing delays and preventing us from moving forward. This is very urgent for us, so we kindly request your immediate assistance in resolving this issue.

Thanks,
Sooriya E
Manas Mohanty 9,655 Reputation points Microsoft External Staff Moderator

2025-08-28T04:51:30.5133333+00:00

Hi Sooriya E

I have requested details in private message to create support ticket and connect in a team's call.

Thank you
Manas Mohanty 9,655 Reputation points Microsoft External Staff Moderator

2025-08-29T02:05:46.28+00:00

Hi Sooriya E

Thank you for sharing the details on call.

Summary of the call

Calls are not being processed through Batch API but async chat completion.

Each prompt being around 2.2 to 4.4 k Token being bundled in batch of 10 and passed as dictionary to Endpoint.

Suggested to reduce max_tokens and batch size and adopt back off retry logic

I have already escalated the case to product group and waiting for their inputs for any backend degradation in East US side.

Thank you

Your answer

Sooriya E 35 Reputation points

2025-08-22T08:17:31.0166667+00:00

Any updates??
Deleted

This comment has been deleted due to a violation of our Code of Conduct. The comment was manually reported or identified through automated detection before action was taken. Please refer to our Code of Conduct for more information.
Manas Mohanty 9,655 Reputation points Microsoft External Staff Moderator

2025-08-25T04:32:43.2033333+00:00

Hi Sooriya E

To deal with 429 rate limits,

You can reduce max_tokens on Deployment configuration, increase TPM on Model setting, load balance between multiple Azure OpenAI, Enable retry with exponential backoff.

Reference thread

https://learn.microsoft.com/en-us/answers/questions/2263211/429-rate-limit-errors-on-gpt-4-1

Hope it helps address the issue.

Thank you
Sooriya E 35 Reputation points

2025-08-28T03:56:41.9366667+00:00

Hi @Manas Mohanty

We tried the methods you suggested, but we are still encountering the same issue (OpenAI API error code: 429). It would be better to schedule a call with the engineering team, as this is causing delays and preventing us from moving forward. This is very urgent for us, so we kindly request your immediate assistance in resolving this issue.

Thanks,
Sooriya E
Manas Mohanty 9,655 Reputation points Microsoft External Staff Moderator

2025-08-28T04:51:30.5133333+00:00

Hi Sooriya E

I have requested details in private message to create support ticket and connect in a team's call.

Thank you
Manas Mohanty 9,655 Reputation points Microsoft External Staff Moderator

2025-08-29T02:05:46.28+00:00

Hi Sooriya E

Thank you for sharing the details on call.

Summary of the call

Calls are not being processed through Batch API but async chat completion.

Each prompt being around 2.2 to 4.4 k Token being bundled in batch of 10 and passed as dictionary to Endpoint.

Suggested to reduce max_tokens and batch size and adopt back off retry logic

I have already escalated the case to product group and waiting for their inputs for any backend degradation in East US side.

Thank you

Share via

Getting OpenAI API error : 429 in azure databricks while using OpenAI API key

Your answer