Intermittent CPU spikes and metric gaps

Hi Olof Montin,

Thanks for reaching out Microsoft Q&A portal

It looks like your Function App is having issues with high CPU usage and interruptions in custom metrics.

On Premium (EP) plans, the Function runtime runs continuously, like a regular web app. If your code or the runtime (for example, telemetry exporters, leftover threads, or open connections) gradually uses more CPU or gets stuck, it can build up over time, leading to long CPU spikes and pauses in the custom metrics pipeline.

On Consumption (Y1), the runtime is automatically restarted often these leaks/issues get cleaned up before they become a problem.

So, the issue isn’t necessarily your code alone it’s the difference in how plans handle runtime lifecycle.

You’ve already done thorough troubleshooting reducing concurrency and I/O, disabling OpenTelemetry, and more which indicates the issue isn’t solely in your code. The fact that it runs fine on Y1 but has problems on EP1/EP2 points to the way Premium plans keep the runtime process running continuously

Follow the few recommended steps to resolve the issue:

1.Add proactive restarts on EP plan

• Because EP instances don’t restart automatically, you might need to set up a ‘self-healing’ approach: use Azure Functions Health Check with automatic app restarts if the process hangs, or schedule regular restarts (daily or weekly) to clear any accumulated issues

Refer document: https://learn.microsoft.com/en-us/azure/app-service/monitor-instances-health-check?tabs=dotnet

https://learn.microsoft.com/en-us/powershell/module/az.functions/restart-azfunctionapp?view=azps-14.3.0

2.Tune EP Plan scaling / isolate workloads

• Separate the 24h heavy workload and the 5-min frequent workload into different Function Apps. This reduces the chance one long-running job locks up resources for everything.
• Scale out with a minimum instance count: Setting a minimum instance count on your EP2 plan forces Azure to keep a certain number of instances warm at all times. This can improve reliability and distribute the load.

Refer document: https://learn.microsoft.com/en-us/azure/azure-functions/functions-premium-plan?tabs=portal

3.Diagnostic Tools

• Use Application Insights to further investigate what's consuming CPU during those spikes. Check for any particular function executions or external calls that take up more resources.

4.Implement ‘process recycling’ similar to IIS. On Premium plans, you can enable Always On and configure scheduled restarts through Azure App Service settings.

• Go to App Service → Configuration → General settings → Auto-Heal, or set a restart on a time

Refer document: https://learn.microsoft.com/en-us/azure/app-service/web-sites-monitor

5.Check for .NET or OpenTelemetry-related leaks. OpenTelemetry exporters can sometimes cause threads to get stuck, and even if disabled, the runtime may behave differently on EP versus Y1. Use Application Insights Profiler to monitor memory and CPU usage over long-running executions.

refer document: https://learn.microsoft.com/en-us/azure/azure-monitor/optimization-insights/code-optimizations-profiler-overview

6.Ensure that OpenTelemetry is set up correctly. Double-check the telemetry channel configuration to see if it aligns with recommended practices for Azure Functions

• Look into telemetry validation errors in Application Insights. Sometimes, data can be dropped due to validation failures linked to incorrect configuration.
• Review and upgrade your OpenTelemetry SDK to avoid telemetry stalls

Refer document:https://learn.microsoft.com/en-us/azure/azure-functions/opentelemetry-howto?tabs=app-insights%2Cihostapplicationbuilder%2Cmaven&pivots=programming-language-csharp

7.Check the logs for both Azure Functions and OpenTelemetry to look for any errors or potential performance issues or bottlenecks.

if you need more info on reliable Azure Functions refer https://learn.microsoft.com/en-us/azure/azure-functions/functions-best-practices?tabs=csharp#scaling-and-performance

I hope this addresses your concern. Feel free to ask if you have any more questions. I’m happy to assist.

Thank you for your answer, Sandhya! And thanks for the clarification regarding the Y1 and EP plans and their respective lifecycles.

1. Add proactive restarts on EP plan
   1. This should not be required. Although, there can be multiple reasons to terminate instances, leakages and CPU spikes should not be the case. We'd rather understand what's going on than just terminate.
2. Tune EP Plan scaling / isolate workloads
   1. Yes, good point and we've considered batching workloads with messaging on a cheaper plan
3. Diagnostic Tools
   1. Sorry that I didn't mention this, but we've tried it. But when the CPU hogs, the diagnostic tooling doesn't have enough resources to complete, making it useless.
4. Implement ‘process recycling’ similar to IIS. On Premium plans, you can enable Always On and configure scheduled restarts through Azure App Service settings.
   1. This is an App Service setting and not a Function App setting, and thereby not usable.
5. Check for .NET or OpenTelemetry-related leaks. OpenTelemetry exporters can sometimes cause threads to get stuck, and even if disabled, the runtime may behave differently on EP versus Y1. Use Application Insights Profiler to monitor memory and CPU usage over long-running executions.
   1. Can you please provide any reference for this? Like a GitHub issue or similar.
6. Ensure that OpenTelemetry is set up correctly. Double-check the telemetry channel configuration to see if it aligns with recommended practices for Azure Functions
   1. Yes, good point. This we've done.
   2. Another aspect of this is the language dependency injection logic. This is my lack of experience, but I can assume that references to OpenTelemetry class instances can get lost in the dependency injection logic.
7. Check the logs for both Azure Functions and OpenTelemetry to look for any errors or potential performance issues or bottlenecks.
   1. Good point! This we've done but haven't found anything yet.

Hi Olof Montin, Thanks for your response

There’s a confirmed memory leak in the OpenTelemetry .NET SDK tied to how metrics are handled:

• Issue #5922 shows that MetricPoint[] instances continuously accumulate unless you explicitly dispose Meter objects—otherwise, metrics consume more memory over time. This matches the symptoms of your metrics stalling until a function restart
• Even if your function doesn’t send telemetry very often, the OpenTelemetry library may leave old metric objects behind each time it runs. Over time, these pile up in memory and can cause the app to slow down or stop sending metrics until you restart it.

https://github.com/open-telemetry/opentelemetry-dotnet/issues/5922?utm_source

• Make sure to call .Dispose() on your Meter or MeterProvider at the end of each function execution.
• Instead of creating the provider once at app startup, initialize it inside each function or ensure it’s properly cleaned up after use.
• Keep your OpenTelemetry setup simple, stateless, and lightweight to avoid leftover objects accumulating over time.

https://github.com/open-telemetry/opentelemetry-collector-contrib/issues/29762

let me know if this helps