Skip to main content

Data Routers

Processors allow you to change the shape and content of data. While pipes ensure the movement and continuous flow, processing adds functionality into the flow. In Processors you learned how to create a process. In this section you will assign the process to a pipe by creating a Router.

Routers in the pipeline diagram​

Routers are the middle of the three columns on the Platform page: Sources (left), Routers (middle), and Sinks (right), read left to right in the direction data flows.

Platform page showing the pipeline diagram with a column of sources on the left, routers with their pipes in the middle, and sinks on the right, joined by connection lines, with Add Source, Add Router, and Add Sink buttons at the foot of the columns

Each router card lists the pipes it runs. A pipe row shows the pipe name and its processor count, for example (1 procs). The outline reports the router's state: green while it is actively moving data, gray while it is idle, and red, with an exclamation mark (!) above the card, when it is in error. A green router can also carry a small green throughput graph above the card's upper-right corner; an idle one has none.

Small circular ports sit on the edges of each card. Colored lines run from a source's output port to the router's input port, and from each router pipe's output port to a sink, tracing the exact path events take through the platform. Line colors distinguish the individual flows where several cross.

Click Add Router + at the bottom of the Routers column to add a new router.

Creating the Pipe​

Click Add Router + to open the Adding Router panel. It starts from the processor: in this example, you are using the process created previously, called LinuxServer.

  • Processor Name: the processor the router's pipe runs, chosen from the account's processors.
  • Name: the router's name, filled in from the processor, so the overall router takes the processor's name.
  • Use Custom Name, under ADVANCED SETTINGS: tick it to give the router a name of its own; doing that allows you to edit Name.

Adding Router panel with the Processor Name list, the Name field, and the Use Custom Name checkbox under Advanced Settings

Click Save at the foot of the panel. The new router appears in the Routers column, with a row for its pipe. The pipe is named after the processor, whatever name you gave the router.

Click the router's title to open its panel: under PIPES it lists the pipe, with Processor 1: and the processor's name beneath it. Clicking the processor opens its panel, with the processor's details and its FPL code under PROCESSOR CODE; its Events tab charts the processor's traffic. See Routers.

As you can see, the processor is there but there is no data or histograms yet.

Connecting Data​

In this example you are going to use a Data Source called "Syslog Endpoint." Syslog is a common data source. In this case you want to move an existing source that is turned on.

Changing the flow is as simple as deleting the current flow and then reassigning the new one. The full procedure is: disable the source, delete the old flow, draw the new flow, then re-enable the source.

Disable the source​

If you are moving the flow from a source, you first need to turn the source off.

Click the source's card to open its panel. On the Details tab, under STATUS, switch Active off. The Deactivate Source confirmation asks Are you sure you want to deactivate followed by the source's name; confirm it.

Source panel of the SyslogEndpoint cloud syslog source, showing its Format json, the Active and Batch Mode switches, Path /var/log/remote under Configurations, Properties JSON, Receiver Configuration, and tags application CloudSyslog and appInstance default

Delete the existing flow​

With the source turned off, click Disconnect Source From Router at the foot of its panel. The confirmation asks Are you sure you want to disconnect followed by the source and the router it feeds. Confirm it and the connection will be removed. While the source is still active, the platform refuses instead, with Failed To Delete Connection: the source needs to be disabled first.

Create the new flow​

Now you have a data source with no Router. This allows you to route the data somewhere else.

Start by clicking the source's output port, the circle on the right edge of its card. Then click the input port (the circle on the left edge) of the router you want to connect. The Adding Source Connection confirmation asks Are you sure you want to connect followed by the source and the router. Confirming will establish the connection. (Connecting does not need the source turned off; only removing a connection does.)

If the router's pipe does not deliver anywhere yet, connect it the same way: click the output port at the right end of the pipe's row, then the input port of the sink. The Adding Pipe Connection confirmation names the sink and the pipe. See Making a Connection.

Re-enable the source​

There is one more step you need to do to get the data flowing into the process: turn on the data source. Remember you turned it off to change the connection.

Open the source's panel again and switch Active back on. The Activate Source confirmation asks Are you sure you want to activate followed by the source's name; confirm it. The source's outline turns green once it is moving data again.

Last updated on