Skip to main content

Search

Describes an older version of the product

Parts of this page reference pages that do not exist in the current web interface (see Page Functionality): the Network Flows page and the Sources Devices page (under the old "Data Ingress" menu section) are not part of the current interface, and the flow/fusion pipeline described under "Message Flow" reflects the older product architecture. In the current interface, stored events are searched from Data Lake → Search, and the data pipeline is shown on Platform → Configuration.

Ad Hoc Searching: Global Pages​

Events and Flows​

There are two main databases that messages (logs) are stored in:

  • Event Database: Contains the incoming message and how it got there (its metadata)
  • Flow Database: Contains merged messages that shares a common tuple along with enhanced data

Both databases are useful and serve different purposes. Events are searched from the Search page under the Data Lake menu (/dataLake/search), headed Events Search on screen with the tagline "Database Search for all collected Events." (The Network Flows page from the older interface is no longer present.)

Events Search page showing the search bar, the facet panel, the event count timeline, and an empty results list

Pick an index, enter a query and time range, then review the matching events with a count timeline and a configurable facet breakdown. The page is laid out top to bottom as a header (Events Search title, Export Data and Run Investigation buttons), a search bar (index selector, query input, Search button, Sort By menu), the Facet Search Configurations panel down the left column, the Event Count Timeline in the main column, and the results list with its paging footer. See Events Search for the page in detail.

Message Flow​

Everything from here to the end of this section describes a build of the portal that is no longer shipping — the "Legacy pages" and the flow architecture behind them. The current pipeline view is Platform → Configuration.

Sign in to your Fluency portal at https://<companyname>.app.ingext.io — see Fluency Web Interface for the URL formats.

Open the Main Menu from the upper left-hand corner and choose the Sources Devices option under the Data Ingress section.

Sources Devices page from the older interface, on the Diagram tab

Choose the "Diagram" tab, and it shows an active sankey diagram of information flowing through the system.

Sample SanKey Diagram

On the left side of the diagram is information entering the system.

Metaflow Engine data is the Fluency protocol analyzer adding flow data to fill in the gaps that are missed by the security tools. The boxes below that are data sources provide messages. The information is forwarded to the stream INPUT.

The message collector (stream INPUT) examines the message and sends it to the correct parser.

The parsers normalize the data into key-value fields producing a JSON document and define fields that are sensitive and have the value replaced with a Pseudonym.

The record and how the data is collected is placed in the Event Database and sent to the Fusion engine. The fusion engine enhances the record with table and third party lookups and merges the record with other records that share the same network tuple.

Before the record is stored, the RiskScore processor provides a confidence score and saved that data in the Flow Database.

Fusion Process

In this section​

  • Events Search — the search page in detail: indexes, the query bar, facets and the event count timeline.
  • Defining the Query — building a query, and the search and where command reference.
  • Lucene Queries — the free-text syntax the search bar accepts.
  • Facets — configuring the breakdown panel down the left of the results.
  • KQL — the tabular query language used by the KQL Query Builder.
  • Data Storage — the indexes and schemas a search runs against.
  • Investigate — searching as part of working a ticket.