Skip to main content

4. Run Your First Search

The fourth step of the Quickstart: find your own data in the data lake and read one record end to end. This page is for both administrators and analysts: it is the first step of the path everyone repeats daily, and the point where the data stops being a bar on a chart and becomes records you can read.

Before You Begin​

You confirmed data arriving in step 3, and you know which index it landed in. For this quickstart's source that is one of Office365, MSEntra, or AzureSigninLogs (on an older deployment, AzureAudit, the legacy name for MSEntra).

Searching an index with no data in the selected window returns nothing and looks identical to a broken search. That is exactly why step 3 comes first; if you skipped it, go back, because you will otherwise spend this step debugging a search that is working correctly.

Step 1: Open the Search Page​

In the navigation rail, open Datalake Search.

Datalake Search page before a search: the index selector, search box and time-range buttons across the top, the event histogram, the Filters panel with the Default facet's empty field groups, and the result table prompting to choose a time range and press Search

The page is headed Datalake Search, with the subtitle Faceted search across all collected events in the data lake. It is the platform's primary search surface and the page you will spend the most time on. From top to bottom it holds:

  • Search bar: the Data lake index selector, the search box, the time-range buttons, and Search.
  • Toolbar: the record count on the left; Save Facet, the sort order, JSON line, Columns, and Export JSON on the right. Active filters appear as chips below it.
  • Histogram: the event count over time for the selected index.
  • Filters panel (left): the saved search facet in use, chosen from the FACET drop-down, and its field groups, each listing the values found in the results.
  • Result table (right): one row per event, 25 to a page, with ‹ Prev and Next › below it.

Before any search has run, the result table reads Choose a time range and filters, then press Search. and the record count is 0 of 0 records. When a search matches nothing, the table reads No events match this search.

Step 2: Select the Index​

  1. Open the Data lake index drop-down at the left of the search bar. The page opens on managed-default.
  2. Choose the index your application writes to. Each index is named after its data lake and index, so platform-managed indexes carry a managed- prefix here: this quickstart's Office365 index appears as managed-Office365. The example below lists managed-behavior, managed-default, and managed-MSEntra.

Data lake index drop-down open, listing the managed-behavior, managed-default and managed-MSEntra indexes with managed-default selected

Choosing another index redraws the histogram for that index straight away; the results follow when you press Search.

Why index selection comes first​

The index decides three things at once: which records are in scope, which schema describes them, and therefore which fields the facet panel can break down. That is why the facets change when you change the index: they are not a fixed sidebar, they are computed from what the selected index actually contains.

It also decides cost. A search scans the indexes it is pointed at, so searching more than you need scans more data than you need. Start with one index and widen deliberately.

Step 3: Set the Time Range​

Set a window that certainly contains your data, and start wide: the last hour or the last day for a source connected this morning.

The time-range buttons sit between the search box and Search: 15m, 30m, 1h, 6h, 24h, and Custom. The page opens on 1h; 24h is the widest preset. Custom adds two date-and-time fields to the bar, Start time and an end time after the →, for any window you like.

The timestamp used is the event's time, not the time it arrived. For a polled cloud source those differ: Microsoft aggregates unified audit log events across datacenters into content blobs and does not guarantee ordering, so events can arrive hours after they happened and a later blob can carry earlier events. Judge freshness over hours, not minutes.

Starting wide and narrowing​

Confirm data exists in a broad window first, then narrow. Narrowing first is the single most common reason a new user concludes they have no data: the search is correct and the window is simply on the wrong side of the ingest lag.

Step 4: Run a Query​

  1. Leave the search box empty, so the search returns every event in the selected index and window.
  2. Click Search. The page's URL then records the search's time range and index (/v2/events?from=…&to=…&index=…), so the address of a results page opens the same search again: any search you can see, you can bookmark and share.
  3. Read the record count at the left of the toolbar (for example 1,000 of 5,106 records · first 1,000 loaded) and the Events in Selected Range figure above the histogram. A search loads at most the first 1,000 matching events; the count tells you how many matched in all.
  4. Add one condition and watch both change.

Datalake Search after a search of the managed-MSEntra index over the last hour: the toolbar reads 1,000 of 5,106 records, first 1,000 loaded; the Filters panel lists event types, a source, a sender and a parser with their counts; and the result table shows each event's row followed by its raw JSON line

Query input basics​

The query, the index, and the time range together define what comes back; the query field is the only one of the three you type. For the syntax itself (field terms, boolean operators, wildcards, and ranges), see Queries and Lucene syntax rather than experimenting here.

The search box's placeholder, Search events – an IP, a user, @eventType:fortigate…, shows the two forms it accepts: a bare value, matched anywhere in the event, and a @field:value term that matches one field.

Sorting results​

The sort drop-down in the toolbar orders the results by time: Newest first, the default, or Oldest first. Newest-first is the sensible default for a first look, because the most recent records are the ones you can still corroborate from memory.

Step 5: Use the Facets​

  1. Read the Filters panel: one group per field in the selected facet, each headed by its name and, in brackets, the number of distinct values found, Event Types (3), with each value listed beside a checkbox and its number of matching events.
  2. Use the groups to see what is actually in your data: which workloads are represented, which users appear most, where sign-ins come from.
  3. Switch facets from the FACET drop-down at the top of the panel when you want a different breakdown of the same results.

With the Default facet the groups are Behaviors, Event Types, Sources, Senders, Parsers, and Tags, a general-purpose view. The other facets are named after the data they suit, such as Default-AWS_CloudWatch, Default-Fortigate, or GSuite, and bring their own groups; GSuite, for example, adds groups such as City, Country, ISP, Org, Email, and IP Address. The facets offered are the ones on Configuration → Search Facets that fit the selected index, and choosing one asks before it replaces your query and filters with the facet's own.

Facet drop-down open in the Filters panel, listing Default followed by facets such as Default-AWS_CloudWatch, Default-BoxCom_Events, Default-Cisco_ASA and Default-Fortigate

Clicking a value filters on it: the first click includes it (its checkbox is ticked and a chip such as Event Types: ServicePrincipalSignInLogs appears above the histogram), the next excludes it, striking it through in red with a red NOT chip, and a third clears it. Press Search to run the search with the filters that are set; Clear all beside the chips removes them.

Why facets are the fastest way to learn a new data source​

A per-source facet's groups are, in effect, an inventory of what a Microsoft 365 audit record contains and what real values those fields take in your tenant: which workloads are busy, which operations dominate, which countries your sign-ins come from. On day one that is worth more than any schema document, because it is your own data rather than an example, and it takes one click rather than a read.

Managing and grouping facets​

Save Facet in the toolbar saves a search facet from the page (a set of field groups, with a query and indexes), and the saved facets are managed on Configuration → Search Facets. + Add field at the bottom of the Filters panel adds a field group of your own, from a Display name and the event field to count (Field value, such as @field.path); the × on a group's heading removes it. The « button in the panel's header folds the panel into a narrow strip to give the result table the full width. Datalake Search documents each control, and Facets covers the concept.

Step 6: Read One Record in Full​

Facets tell you what fields exist; at some point you need to see a whole record, with every field, in the order the platform stored it.

Click any row in the result table, or its JSON line, to open that event in a detail panel that slides in from the right:

Event detail panel for a ServicePrincipalSignInLogs event from AzureEventHub, with the Overview fields Time, Event Type, Source, Sender and Parser, and the Other Fields table listing each field with its value and a filter button

  • OVERVIEW: the event's Time, Event Type, Source, Sender, and Parser.
  • OTHER FIELDS: every other field, one row each, with nested fields written as a dotted path. The filter button at the end of a row adds that field and value to the search's filters.
  • RAW JSON: the event's complete JSON, folded until you click Show, with a Copy JSON button.

The × in the panel's header closes it. With JSON line on in the toolbar, as it is when the page opens, each row of the result table is also followed by the event's complete JSON on one line.

To query records as a table instead, use KQL Search:

  1. In the navigation rail, open KQL Search.

  2. In the Indexes list on the left, click your index to expand its columns in place. Note that index names here have no managed- prefix: the same index that is managed-Office365 in Datalake Search is Office365 here. Indexes with typed columns of their own, such as AzureSigninLogs, are listed only here, not in Datalake Search.

  3. Read the columns it lists, each with its storage type on the right: INT64, INT32, BYTE_ARRAY, and so on. An index that uses the standard event layout shows Default event schema instead.

  4. Put a query in the editor that returns a handful of recent records, then click Run Query:

    Office365 | where timestamp > ago(24h) | take 100

    timestamp is the time column of an index with the standard event layout. An index with typed columns has a time column of its own, which its expanded entry lists; for AzureSigninLogs it is TimeGenerated, and a filter on timestamp fails with an error naming that column:

    AzureSigninLogs | where TimeGenerated > ago(24h) | take 100
  5. The page switches to the Results tab, which reports the number of records searched, the size of the data, a time in milliseconds, and the time window searched above a result table headed by its name and row count, with an Export CSV button. With nothing in range the body reads No results found.

Results tab after running the starter query: a summary line reading 104 searched, 46.8 KB, 0 ms and the time window searched; the PrimaryResult table of 99 rows with an Export CSV button; and the timestamp column with its datetime type

Every query you run is added to your Query History tab with its time, range, status, and execution time, and each row's ▷ (Rerun query) runs it again, so a query that returns something useful stays one click away. Search History in the page header opens the account-wide record of KQL searches, where a completed search's stored results can be reopened without running it again.

As you read a record, look for the normalized field names: the ones behavior rules and reports depend on, as opposed to the vendor's own field names carried through from the source. Normalization explains the distinction, and it is what makes the same rule work across two vendors' logs.

How to Tell It Worked​

  • The Data lake index drop-down lists your index, prefixed managed-.
  • A broad time range with an empty search returns a non-zero record count in the toolbar.
  • The histogram shows events in the period since your source was connected.
  • The Filters panel lists groups with non-zero counts, and the values in them are recognisably yours: your own domain, your own users, your own workloads.
  • KQL Search returns records for the same index and window.

If It Did Not Work​

SymptomLikely causeWhat to do
No events match this search. and a record count of 0 of 0 recordsThe window is on the wrong side of the ingest lag, or the wrong index is selectedWiden the range to 24h or a Custom window, confirm the index, and re-check step 3
Results, but the fields are not the ones you expectedThe facet applied is not the one for that sourceSwitch it from the FACET drop-down
Results, but fields look unparsed or rawThe processor for that source is not matching the formatParser cookbooks
The record count reads first 1,000 loaded and the events you want are not among themA search loads at most the first 1,000 matching eventsNarrow the time range or add filters
The search is slow, or refuses to runToo wide a window, or more indexes in scope than you needNarrow both. Search volume is charted on the Usage Report under Reports, as Search
Export JSON is greyed outNo search has returned records yetNothing to fix: run a search that matches first

Next Step​

5. See Your First Detection: from raw events to something worth acting on.