Azure Log Stream Logs missing - Java

Question

In my Azure Function, I am using SLF4J for logging (backed by logback). Application Insights supports auto-instrumented logging. Where the logs are automatically collected. Details here.

But I find that randomly a lot of logs get missed. They are present during a run and missing some other times when viewed in LogStream. What could be going wrong?

Answer 1

This answer does not fully answer the question. I still find the issue present. And don't really know the cause for missing logs in LogStream. And it is frustrating.

---

One thing that you may want to further take care is: Sampling. Pasting from the docs:

Sampling is helpful if you need to reduce cost. Sampling is performed as a function on the operation ID (also known as trace ID), so that the same operation ID will always result in the same sampling decision. This ensures that you won't get parts of a distributed transaction sampled in while other parts of it are sampled out.

For example, if you set sampling to 10%, you will only see 10% of your transactions, but each one of those 10% will have full end-to-end transaction details.

To avoid filtering due to sampling, add the function configuration:

APPLICATIONINSIGHTS_SAMPLING_PERCENTAGE: 100

This does not omit anything. For more details, visit: https://docs.microsoft.com/en-us/azure/azure-monitor/app/java-standalone-config#sampling

---

In my case, I wanted to include all the logs. But perform sampling for other metrics (like dependencies which includes calls to REST API, database or a file system).

An alternative way as well. Edit your host.json as:

{
  "version": "2.0",
  "extensionBundle": {
    "id": "Microsoft.Azure.Functions.ExtensionBundle",
    "version": "[2.*, 3.0.0)"
  },
  "logging": {
    "applicationInsights": {
      "samplingSettings": {
        "excludedTypes": "Exception;Trace;Request"
      }
    }
  }
}

The log statements as such go under Trace. So we exclude sampling for it. The rest are sampled (Dependency, Event, PageView, Request)

To validate if the above config is working as expected, run the below kusto query:

union requests,dependencies,pageViews,browserTimings,exceptions,traces
| where timestamp > ago(1d)
| summarize RetainedPercentage = 100/avg(itemCount) by bin(timestamp, 1h), itemType

Load test the function. In the output, you should see 100 across the Exception;Trace;Request

To view full set of config: https://docs.microsoft.com/en-us/azure/azure-functions/functions-host-json#sample-hostjson-file

To selectively override (different behavior for different logs): https://docs.microsoft.com/en-us/azure/azure-monitor/app/java-standalone-config#sampling-overrides-preview

---