Skip to main content

KQL

KQL is the pipe-oriented query language the platform uses against data lake indexes. It is what the KQL Query Builder on Data Lake → KQL Query Builder executes, and it is what the platform's programmatic search and validation calls accept.

Deliberately partial

This page documents what has been confirmed against the product: where KQL runs, the data model it queries, the shape of a query, and how runs are costed and recorded. The operator and function surface is not documented here. This dialect has not been tested against another KQL implementation, so familiarity with one elsewhere is not a safe guide — check a query with the validation call, or run it over a short range in the KQL Query Builder, before relying on it.

What KQL is used for here​

KQL is the query language of the data lake. It answers tabular questions — filter these records, project these columns, aggregate them this way — over indexes that may hold months of collected events.

Where KQL is available​

  • Data Lake → KQL Query Builder (/dataLake/schemas/builder) — the primary interactive editor. A left INDEXES panel lists the queryable indexes, each expanding in place to show its columns and types; a toolbar carries the Run query button and a Set in query control marked with a calendar icon; the dark editor below holds the query with line numbers and syntax highlighting; and a Query History / Results tab pair sits underneath, opening on Query History.
  • Programmatic search — the platform exposes KQL search over its automation surface, so an external client or AI agent can run the same queries. See MCP Server.
  • Query validation — a separate call that parses and checks a query without executing it, for editors and agents that want to fail fast.

KQL Query Builder page with the index list, KQL editor, and Query History table

KQL vs. Lucene vs. FPL​

Three query languages coexist and are easy to mix up. Lucene drives the free-text Events Search on Data Lake → Search — the platform's primary event search — over the event model. KQL answers tabular questions over data lake indexes. FPL is the language of the programs that produce and process the data in the first place. See Lucene Queries and FPL.

The tabular data model​

Every KQL query runs against a table.

Indexes​

An index is the named table a query starts from. The KQL Query Builder's INDEXES panel lists the queryable ones.

Base columns​

The base columns present on the demo indexes are:

ColumnTypeNotes
timestampINT64The record's time; the column time filters operate on.
sizeINT32The record's size.
docBYTE_ARRAYThe record body.
labelsMAPKey/value labels attached to the record.

The types shown are the physical types the INDEXES panel reports. Whether these four columns are guaranteed on every index or merely common on these ones has not been established.

Schemas and typed columns​

Beyond the base columns, an index's shape comes from its schema. Schema definitions carry a name, a description, a dynamic flag, aliases, virtualTables, and a fields list where each field has a name, a description, and type information (fpltype, type, convertedtype). The definitions are on Data Lake → Schemas; a row's eye button opens the Editing Schema dialog with that schema's JSON in the Content editor, which is where those property names can be read directly:

Editing Schema dialog with the schema's name, description, and JSON definition loaded in the Content editor

Virtual tables​

A virtual table is a projection of an index defined by a name, a root field, and a label selector, letting a subset of a wide index be queried as its own table. See Data Lake → Management.

Query structure​

A KQL query names a source table and then applies a sequence of operators, each separated by |, each consuming the table produced by the one before it. The in-product sample query is:

default | where timestamp > ago(24h) | take 100

This reads as: start from the default index, keep only records from the last 24 hours, then return at most 100 of them. Every stage takes a table in and passes a table out, which is the mental model the rest of the language rests on.

Time​

ago() produces a time relative to now, and is the idiomatic way to write a rolling window — where timestamp > ago(24h) is the in-product default.

The KQL Query Builder also has a Set in query control beside the editor, and query-history rows whose window comes from the query text itself show Set in query in their Search Range column. What takes precedence when a query carries its own time filter and a range is set outside it has not been established — set the range in one place, not both. See KQL Query Builder.

Query cost​

Data lake searches are metered, so the cost of a query matters before it runs.

How searches are costed​

The Investigation Search dialog previews a run's scope and shows an estimated Search Cost in dollars before the search starts, and the Investigations page records the cost of each completed run. See Data Lake → Search and Investigations.

What the Results view reports​

A completed run in the KQL Query Builder reports the search range, the total records searched, and the total bytes searched — the figures to read when predicting the cost of a wider run.

Writing cheaper queries​

Filter on timestamp first, restrict to one index, project early, prefer aggregation over returning raw rows, and use take while iterating.

Query history​

Every run is recorded in the KQL Query Builder's Query History tab with its Time, Execution Time, Search Range, Status, and KQL text. The list is searchable and refreshable. Completed runs are also kept, with their stored results, on Data Lake → KQL Search History (/dataLake/search/history/kql).

The Rerun Query button on a history row re-executes that entry's KQL and switches to the Results tab. Note that a query using a relative expression such as ago(24h) covers a different window on each rerun.