Explode

The Explode function transforms a single event containing an array into multiple events, each containing one element of the array. This is useful for breaking down complex nested data into simpler, individual events.

Purpose

Use the Explode function when you need to flatten arrays within your events. Common use cases include:

• Processing nested JSON structures.

• Breaking down multi-value fields (e.g., tags, categories) into individual events.

• Preparing data for further transformations or analysis.

Usage

Select Explode transform. Add Name (required) and Description (optional).

General Configuration:

• Bypass Transform: Defaults to disabled. When enabled, this transform will be bypassed entirely, allowing the event to pass through without any modifications.

• Add Filter Conditions: Defaults to disabled. When enabled, it allows events to filter through conditions. Only events that meet the true condition will be processed; all others will bypass this transform. Based on AND/OR conditions, "+Rule" or "+Group" buttons.

Explode:

• Enabled: Defaults to enabled, meaning it does evaluate all events. Toggle Enabled off to prevent event processing to feed data to the downstream Transforms.

• Explode Conditions: Defaults to empty. When set, allows events to explode through conditions. Specify the criteria for exploding events. Explode only occurs if the specified criteria are met. Leave this field blank to explode all events. Based on AND/OR conditions, "+Rule" or "+Group" buttons.

• Field Name: The field to be exploded must contain a valid array. The array will be exploded into multiple events, each event retaining the original field name. If you wish to rename the field, utilize the Rename transform.

Examples

Examples require that Enabled is toggled on.

Expand Single Entry Array into Multiple Events

Examples require that the these criteria are meet:

• Explode Condition: n/a

• Field Name: root.array

Scenario: Transforms a single event containing an array into multiple events.

Input:

{
  "timestamp": "2023-06-15T15:00:00Z",
  "source": "application_logs",
  "root": {
    "array": [
      {
        "id": 1,
        "status": "success",
        "message": "Operation completed"
      },
      {
        "id": 2,
        "status": "warning",
        "message": "Partial success"
      },
      {
        "id": 3,
        "status": "error",
        "message": "Operation failed"
      }
    ]
  }
}

Output:

{
  "timestamp": "2023-06-15T15:00:00Z",
  "source": "application_logs",
  "root": {
    "array": {
      "id": 1,
      "status": "success",
      "message": "Operation completed"
    }
  }
}
{
  "timestamp": "2023-06-15T15:00:00Z",
  "source": "application_logs",
  "root": {
    "array": {
      "id": 2,
      "status": "warning",
      "message": "Partial success"
    }
  }
}
{
  "timestamp": "2023-06-15T15:00:00Z",
  "source": "application_logs",
  "root": {
    "array": {
      "id": 3,
      "status": "error",
      "message": "Operation failed"
    }
  }
}

Results: A single entry array comprising id1, id2 and id3 are expanded into three (3) separate log entries.

Behavior

• Field Name: If the specified array field does not exist or is not an array, the event remains unchanged.

• Output: The output will contain one element of the array per event. The original array field is removed from the resulting events.

• Count: The number of output events will match the number of elements in the array.

Limitations

• Exploding large arrays can significantly increase the number of events, impacting pipeline performance.

• Nested arrays or complex structures may require additional transformations.

Related Functions

• Filter Event: Apply conditions to filter data before or after removing fields.

• Aggregate Metrics: Aggregate multiple metrics into a single metric based on a set of conditions.