Lua Script

The Lau Script Function Processor allows you to write a custom transform for your pipeline using Lua script.

Purpose

The Lua Script Processor is a versatile data transformation component designed to execute custom Lua scripts on incoming data within a pipeline. This processor is particularly useful for applying bespoke data transformations, enriching data, or filtering events based on custom logic defined in Lua. The Lua scripts are executed in a sandboxed environment to ensure restricted access to system resources, maintaining a secure processing environment.

Usage

Select Lua Script transform. Add Name (required) and Description (optional).

General Configuration:

• Bypass Transform: Defaults to disabled. When enabled, this transform will be bypassed entirely, allowing the event to pass through without any modifications.

• Add Filter Conditions: Defaults to disabled. When enabled, it allows events to filter through conditions. Only events that meet the true condition will be processed; all others will bypass this transform. Based on AND/OR conditions, "+Rule" or "+Group" buttons.

Lua:

Enabled: Defaults to enabled, meaning it does evaluate all events. Toggle Enabled off to prevent event processing to feed data to the downstream Transforms.

Lua Script: This field specifies the Lua script to be executed for each incoming event. The script must define the processEvent(event) function, which is invoked for every event received. This function should return the event (potentially modified) to allow it to proceed to the next stage of processing. If the function returns nil, the event will be discarded. Additional helper functions can also be defined and called from within processEvent(event).

Metric Event: Indicates whether the script should process log data or metric data. Set this to true if the script should handle metric data; otherwise, it will process log data by default.

Script Function Details:

• processEvent(event): This is the main entry point for processing an event. The event parameter contains the data to be processed. The function name and parameter names must remain unchanged. The function should return the event object to continue processing or null to discard it.

• Fields in the JSON data are accessed using event.field. For nested fields, use event.field.nested.

Examples

Examples require that Enabled is toggled on.

Clean Fields

Scenario: Remove fields where field value is empty.

function cleanTable(t)
    local keysToRemove = {}

    -- Iterate over the table
    for key, value in pairs(t) do
        if type(value) == "table" then
            -- Recursively clean nested tables
            cleanTable(value)

            -- If the nested table is empty after cleaning, remove it
            if next(value) == nil then
                table.insert(keysToRemove, key)
            end
        elseif value == "" or value == nil then
            -- Mark empty values for removal
            table.insert(keysToRemove, key)
        end
    end

    -- Remove marked keys
    for _, key in ipairs(keysToRemove) do
        t[key] = nil
    end
end

function processEvent(event)
    cleanTable(event)
    return event
end

Results: All fields with empty field values are removed.

Pass Through Without Modification

function processEvent(event)
    return event
end

Modify a Field

function processEvent(event)
    -- Add root level field
    event.field = "new value"
    -- Add nested field
    event.nested.field = "nested value"
    -- Rename field
    event.renamed_field = event.log.field_to_rename
    event.field_to_rename = nil
    -- Remove fields
    event.field_to_remove = nil
    return event
end

Discard Event

function processEvent(event)
   return nil
end

Multiple Functions

-- Main function called for each incoming event
function processEvent(event)
    -- Call the helper function to process the event
    processHelper(event)
    
    -- Additional processing can be done here
    if event.level == "error" then
        return nil
    end
    
    -- Return the modified event
    return event
end

-- Define a helper function that processes a field in the event
function processHelper(event)
    if event.message then
        event.message = event.message .. " - Processed by Helper"
    end

Related Functions

• Filter Event: Apply conditions to filter data before or after removing fields.

• Aggregate Metrics: Aggregate multiple metrics into a single metric based on a set of conditions.

Additional Resources

• Lua Reference Guide