Troubleshooting Queue-Triggered Azure Function in .NET 8.0 Facing Intermittent Internal CLR Errors.

@Anca Corbos .

Thanks for reaching out. The error you're seeing, 0x80131506, usually points to a problem within the .NET runtime, specifically related to memory allocation or thread management. This can sometimes happen when there’s a conflict or corruption in the environment.

Here are a few things to try:

Reinstall .NET SDK (Locally): If you're testing locally, I recommend reinstalling the .NET SDK to rule out any corruption in the installation. You can do this by uninstalling the current version and installing the latest one:

• Run dotnet --list-sdks to list your installed SDKs.
• Then, uninstall the version with dotnet sdk uninstall <version>.
• Finally, install the latest version with dotnet sdk install <latest-version>.

Check Resource Limits: This might be due to insufficient resources. Try increasing the memory allocation in your function app settings. If you're using the Consumption Plan, keep in mind that resources are limited, and sometimes increasing memory can help with this type of error.

Function App Scaling: Make sure your function app is properly scaled, especially under heavy load. If you're on the Consumption Plan, you might run into resource limits that can cause these errors.

Restart/Deploy Again: Sometimes a simple restart or redeploy of your function app can fix temporary issues or corrupted environments. You can restart the app directly from the Azure Portal or use the CLI:

az functionapp restart --name <function-app-name> --resource-group <resource-group-name>

Check Application Insights: If you have Application Insights set up, it could give you more details about what might be causing the issue. Look for any memory spikes or timeouts that could be triggering the error.

Sorry, I forgot to say that we are on tier Elastic Premium EP2 and there is plenty of resources. It seems it happens when the function is starting, because it does not get to log any of our traces.

All we find in AppInsights is the exception I put in the first post. There, in the custom properties we have

TriggerReason="New queue message detected on 'ams'."

Succeeded=false

Category=Host.Results

I included an invocation ID for Microsoft employees to be able to look into our logs.

Hello @Anca Corbos,

Could you please check private message and provide the requested details?

As an update from my side: we updated all our NuGet references to the last version they had for .Net 8.0 and deployed to Prod. No improvement. 20% failures in the last 24 hours.

My impression is that the failures occur when the 'always on' instance is busy and a new instance should be started. Maybe I should also say that (on Microsoft recommendation) we don't use dynamic concurrency anymore. We changed to these settings:

 "queues": {
      "maxPollingInterval": "00:00:02",
      "batchSize": 2,
      "maxDequeueCount": 2
}

We also started the "Crash monitoring" 2 days ago, but no dumps are generated: