Analytic Rules not triggering but data is on log analytics

Question

Analytic Rules not triggering but data is on log analytics

Arthur Moreth 20

Hi Guys,
I'm try to resume the question.
I created an analytics rule for a specific situation. If I take the KQL and put it in the Analytics Log, it returns the records I want. But it doesn't trigger the incident at all. I have several incidents with the same structure (just changing the KQL) and they all trigger normally. Only this one doesn't.

I've tried to check the time within KQL, but I didn't get the same result. I've tried to make NRT rules and I also didn't get any results. Each one generates a different number of incidents, but everything is in the analitycs log.

Edit: I put more details in the anwser for the first comment

Accepted answer

0 additional answers

Your answer

Answer 1

Hi, this is really hard to triage without seeing some / most of the code as an example, which maybe in another thread?

Typical issues (I suspect you have discounted 1 & 2):

certain commands like union * are not supported in a detection
detections that look back more than 14days
Supporting data from some sources may have arrived later due to latency, so when the detection ran at say 9:00 it failed, however at 9:02 (or whatever time) the data arrived so it would be picked up in the next run (assuming your sliding window looks back far enough), and would be there when you look later in the day manually!

e.g. add this at line one and test in your logs blade

set query_now = datetime(2025-04-24T15:15:00.0000000Z);
< add your query >

source: https://learn.microsoft.com/en-us/kusto/api/rest/request-properties?view=microsoft-fabric

and

https://techcommunity.microsoft.com/blog/microsoftsentinelblog/handling-sliding-windows-in-azure-sentinel-rules/1505394

You can see query_now in action by a quick test, run these two example queries:

// Run this to simulate a fixed point in time 
set query_now = datetime(2025-04-24T15:15:00.0000000Z);
print now()


// run this to show the "real" current time 
print now()

As an Example sometimes there are delays from Defender products (look at the ProcessingTime)

SecurityIncident
| extend Alerts = extract("\\[(.*?)\\]", 1, tostring(AlertIds))
| mv-expand AlertIds to typeof(string), Labels to typeof(string), Comments to typeof(string), AdditionalData to typeof(string)
| join kind=leftouter
(
    SecurityAlert
     | extend ProductProcessingMin = datetime_diff('minute',  ProcessingEndTime, EndTime), ingest_ = ingestion_time()
     | extend sentinelIngestionDelayinMinutes_ = datetime_diff('minute', ingestion_time(), ProcessingEndTime)
     | extend StarttoEndDelayinMinutes_ = datetime_diff('minute', EndTime, StartTime)
) on $right.SystemAlertId == $left.AlertIds
| summarize AlertCount=dcount(AlertIds),arg_max(TimeGenerated, *) by IncidentNumber
| extend sentinelIngestiontoCreated_ = datetime_diff('minute', ingest_, CreatedTime)
| extend InvestigationElapsedTime_   = datetime_diff('minute', LastModifiedTime, CreatedTime)
| summarize arg_max(TimeGenerated,*) by IncidentNumber
| project IncidentNumber, Title, StartTime, EndTime, ProcessingEndTime, TimeGenerated, ingest_, StarttoEndDelayinMinutes_, ProductProcessingMin,
        sentinelIngestionDelayinMinutes_, AlertName, FirstActivityTime, LastModifiedTime, 
        CreatedTime
        , InvestigationElapsedTime_ 
        , sentinelIngestiontoCreated_
        , Severity, Status, Comments, ProductName, ProviderName

example
User's image

Arthur Moreth 20 Reputation points

2025-04-25T01:06:03.19+00:00

I'm sorry Clive, the question was so bizarre that I forgot to post more information. First of all, the query is super simple:

ImpervaWAFCloud_CL

| where ingestion_time() >= ago(5m)

| where ImpervaAAPercentBlocked_s != "100" and Attack_Severity_s != "MINOR"

| extend Attack_Name_s, Attack_Severity_s, _CRITICAL_msg_s, _MAJOR_msg_s, src_s, dhost_s, request_s, requestClientApplication_s, ImpervaAANumberOfEvents_s,ImpervaAAPercentBlocked_s, ImpervaAACountry_s, ImpervaAAAttackType_s, TimeGenerated

I've tried to make it cleaner by removing the extend part, but it didn't make any difference.

The grouping is to trigger for for each event.

There is no automated response. And Suppression is disabled.

For exemple, in log analytcs , last 24 hours i have :

And in the Incidents tab i just had:
Clive Watson 7,636 Reputation points MVP

2025-04-25T06:48:19.1633333+00:00

Hi, in a NRT detection you normally don't put a time filter at all, as its done as part of the Rule configuration (1min / 1min is hard coded). I wonder if this is creating a conflict?

https://learn.microsoft.com/en-us/azure/sentinel/near-real-time-rules#how-nrt-rules-work

Because of the nature and limitations of NRT rules, however, the following features of scheduled analytics rules will not be available in the wizard:

Query scheduling is not configurable, since queries are automatically scheduled to run once per minute with a one-minute lookback period.
Clive Watson 7,636 Reputation points MVP

2025-04-25T08:49:15.6433333+00:00

You use of "extend" may also be causing a little overhead, I think you should replace with "Project". That way you are only projecting those named columns, with extend you are projecting all columns and then also extending them (for no reason as you are not changing the column name or vale

e.g. | extend a="the new a", b="the new b"

Also if the row count for this data source is large, you want to summarize() the results
Arthur Moreth 20 Reputation points

2025-04-26T01:54:45.44+00:00

@Clive Watson Thanks for the tips, i did so many things trying to solve these, that i forgot about the time in que query. I'm trying without a time filter and with project instead of extent. i will get the results tomorrow.

Meanwhile i use the "test with current data" and get no results

And i'm sure that i have at least 3 or 4 alerts per day.

Share via

Analytic Rules not triggering but data is on log analytics

0 additional answers

Your answer