Windows Event Cleaner

The Windows Event Cleaner (Optimizer) in Observo AI allows users to remove events that contain such entities such as null and zero values.

Purpose

The Windows Event Cleaner (Optimizer) is designed to automatically filter out redundant, low-value, and noisy entries from Windows event logs, ensuring that only actionable and relevant data is retained. It normalizes and cleanses the log data so that downstream security and observability tools can process it more efficiently. By reducing log volume and enhancing data quality, it helps organizations lower storage costs, improve query performance, and accelerate incident detection and response.

Usage

Select Windows Event Cleaner (Optimizer) transform. Add Name (required) and Description (optional).

General Configuration:

• Bypass Transform: Defaults to disable. When enabled, this transform will be bypassed entirely, allowing the event to pass through without any modifications.

• Add Filter Conditions: Defaults to disable. When enabled, it allows events to filter through conditions. Only events that meet the true condition will be processed; all others will bypass this transform. Based on AND/OR conditions, "+Rule" or "+Group" buttons.

Windows Event Cleaner: Enabled: Defaults to enabled, meaning it does evaluate all events. Toggle Enabled off to prevent event processing to feed data to the downstream Transforms.

Cleanup System Event Content: When enabled, it will remove the fields that have 0 value in System Event.

Example

Scenario: A Windows Security event (ID 4740), which logs user account lockouts due to repeated failed login attempts, is streamlined to include only essential, non-empty fields. Empty fields—such as guid, task, and user ID—are excluded, while key details like channel, message, and hostname are retained for clarity.

Windows Event Cleaner

Enabled	Cleanup System Event Content
Toggled to enabled	Toggled on

Input

{
  "appname": "ServiceControlManager",
  "event@4740": {
    "binary": "7300700070007300760063002F0034000000",
    "channel": "System",
    "event_source": "",
    "guid": "",
    "keywords": "",
    "level": "Information",
    "opcode": "",
    "param1": "Software Protection",
    "param2": "running",
    "qualifiers": "",
    "record_id": "17953",
    "task": "",
    "thread_id": "",
    "user_id": ""
  },
  "facility": "user",
  "hostname": "EC2AMAZ-C8O17BP",
  "message": "The Software Protection service entered the running state.",
  "msgid": "7036",
  "procid": 720,
  "severity": "info",
  "timestamp": "2025-07-09T10:09:48.678946700Z",
  "version": 1
}

Output

{
  "appname": "ServiceControlManager",
  "event@4740": {
    "binary": "7300700070007300760063002F0034000000",
    "channel": "System",
    "level": "Information",
    "param1": "Software Protection",
    "param2": "running",
    "record_id": "17953"
  },
  "facility": "user",
  "hostname": "EC2AMAZ-C8O17BP",
  "message": "The Software Protection service entered the running state.",
  "msgid": "7036",
  "procid": 720,
  "severity": "info",
  "timestamp": "2025-07-09T10:09:48.678946700Z",
  "version": 1
}

Results: Empty fields in the Windows event log entry are filtered out, preserving only actionable and relevant data such as channel, message, and hostname for effective analysis.

Best Practices for Windows Event Cleaner

To maximize the effectiveness of the Windows Event Cleaner (Optimizer) transform in Observo AI pipelines, follow these best practices to leverage its capabilities for filtering out redundant, low-value, and noisy entries, ensuring only actionable and relevant data is retained for efficient downstream processing.

Field Cleanup

• Enable Cleanup System Event Content: Activate the Cleanup System Event Content option to automatically remove fields with 0 or empty values such as guid, event_source, qualifiers, task, opcode, thread_id, user_id, keywords from System event logs. This reduces log volume and enhances data quality by retaining only meaningful fields like channel, level, record_id, and event-specific data such as param1, param2.

• Target High-Noise Fields: Focus on eliminating fields that frequently contain null or zero values, such as guid, task, or user_id, which are often empty in system-level events like service state changes or security audits. This ensures downstream tools process only relevant data, improving query performance.

• Preserve Critical Fields: Ensure essential fields like record_id, channel, level, and event-specific data such as binary, param1 are retained, as they provide context for analysis and correlation in SIEM systems or observability platforms.

Event Filtering

• Utilize Filter Conditions: Enable Add Filter Conditions to selectively process events based on specific criteria such as event ID, channel, or severity using AND/OR logic with +Rule or +Group buttons. This targets high-value events (e.g., Security events like ID 4740 for account lockouts) and bypasses irrelevant ones, reducing noise before cleanup.

• Bypass Low-Value Events: Use the Bypass Transform option to skip cleaning for events that should remain unprocessed such as events requiring all fields for compliance. This preserves raw data for specific use cases while applying cleanup to others or when testing warrants bypass the transform.

Transform Configuration

• Toggle Enabled for Selective Processing: Keep the Enabled setting on (default) to process all events, or toggle it off to pass events to downstream transforms without modification. Use this to control which pipeline stages apply cleanup, optimizing resource usage.

• Assign Descriptive Names and Descriptions: Provide a clear Name (required) for the transform to identify its purpose in the pipeline such as "SecurityEventCleaner". Add an optional Description such as "Removes empty fields from Windows Security logs" to document its role, improving pipeline maintainability.

Output Optimization

• Validate Output for Relevance: After cleanup, verify that the output retains only non-empty, actionable fields such as channel="System", message, record_id to ensure compatibility with downstream tools like SIEM systems or analytics platforms. For example, a cleaned Event ID 7036 should retain fields like param1="Software Protection" and param2="running" while omitting guid="" or task="".

• Minimize Storage Costs: By removing empty fields, the transform reduces log volume, lowering storage requirements. Regularly review cleaned logs to confirm that only essential data is stored, optimizing costs without sacrificing critical information.

Example Application

• Scenario Example: For a Windows Security event like ID 4740 (user account lockout), enable Cleanup System Event Content to remove empty fields such as guid, task, opcode, and user_id. Apply filter conditions to process only Security channel events, retaining fields like account_name, account_domain, and caller_computer_name. The cleaned output ensures streamlined data for efficient SIEM ingestion and faster incident response.

By adhering to these best practices, the Windows Event Cleaner (Optimizer) transform enhances data quality, reduces log volume, and accelerates analysis by focusing on actionable fields, making it ideal for security and observability workflows in Observo AI pipelines.

Related Functions

• Cloudtrail Optimizer: Transform group to process AWS Cloudtrail events.

• GCP Flow Logs: Optimize GCP flow logs using this transform.