CEF Serializer

The CEF Serializer in Observo AI allows you to format log data into the Common Event Format (CEF) for compatibility with SIEM systems such as Splunk, ArcSight, Qradar and Sumo Logic.

Purpose

The CEF (Common Event Format) Serializer converts structured security event logs into the standardized CEF format, ensuring compatibility with SIEM systems such as Splunk, ArcSight, Qradar and Sumo Logic. It normalizes log fields, preserving key details such as timestamps, source/destination IPs, protocols, and user information while ensuring efficient log ingestion and correlation. This transformation enhances log interoperability, making it easier for security teams to analyze and respond to threats.

Usage

Select CEF Serializer transform. Add Name (required) and Description (optional).

General Configuration:

  • Bypass Transform: Defaults to disable. When enabled, this transform will be bypassed entirely, allowing the event to pass through without any modifications.

  • Add Filter Conditions: Defaults to disable. When enabled, it allows events to filter through conditions. Only events that meet the true condition will be processed; all others will bypass this transform. Based on AND/OR conditions, "+Rule" or "+Group" buttons.

CEF Serializer: Enabled: Defaults to enabled, meaning it does evaluate all events. Toggle Enabled off to prevent event processing to feed data to the downstream Transforms.

CEF Field Name: Field name which contains CEF data. Leave empty for using the root of the event as the CEF data.

Syslog Fragment Field: To recreate the CEF format we want to prepend the Syslog part which gets preserved as part of parsing the CEF format. The value here should refer to the one which was used while adding the CEF parser. Default: _ob.cef.syslog_part.

Examples

Convert PAN Traffic Logs to CEF

Scenario: Convert PAN Threat Log to CEF format.

  • CEF Field Name: Message

  • Syslog Fragment Field: _ob.cef.syslog_part.

Input

Feb 26 12:34:56 PA-FW1 1,2024/02/26 12:34:56,THREAT,virus,1,2024/02/26 12:34:56,10.1.1.100,192.168.1.200,0.0.0.0,0.0.0.0,app-unknown,vsys1,Trust,Untrust,ethernet1/1,ethernet1/2,Log-Forwarding,1001,1,54321,80,TCP,block,1,example.com,1,2,0,50,10.1.1.100,192.168.1.200,example-user,Threat-Id: 12345
Feb 26 12:35:01 PA-FW1 1,2024/02/26 12:35:01,THREAT,malware,1,2024/02/26 12:35:01,10.1.1.101,192.168.1.201,0.0.0.0,0.0.0.0,app-web-browsing,vsys1,Trust,Untrust,ethernet1/1,ethernet1/2,Log-Forwarding,1002,1,34567,443,TCP,block,1,malicious.com,1,2,0,30,10.1.1.101,192.168.1.201,unknown-user,Threat-Id: 67890
Feb 26 12:35:05 PA-FW1 1,2024/02/26 12:35:05,THREAT,exploit,1,2024/02/26 12:35:05,10.1.1.102,192.168.1.202,0.0.0.0,0.0.0.0,app-dns,vsys1,Trust,Untrust,ethernet1/1,ethernet1/2,Log-Forwarding,1003,1,23456,53,UDP,block,1,exploit.com,1,2,0,80,10.1.1.102,192.168.1.202,example-user,Threat-Id: 54321
Feb 26 12:35:10 PA-FW1 1,2024/02/26 12:35:10,THREAT,command-and-control,1,2024/02/26 12:35:10,10.1.1.103,192.168.1.203,0.0.0.0,0.0.0.0,app-http,vsys1,Trust,Untrust,ethernet1/1,ethernet1/2,Log-Forwarding,1004,1,13579,80,TCP,block,1,c2-server.com,1,2,0,5,10.1.1.103,192.168.1.203,example-user,Threat-Id: 98765
Feb 26 12:35:15 PA-FW1 1,2024/02/26 12:35:15,THREAT,phishing,1,2024/02/26 12:35:15,10.1.1.104,192.168.1.204,0.0.0.0,0.0.0.0,app-ftp,vsys1,Trust,Untrust,ethernet1/1,ethernet1/2,Log-Forwarding,1005,1,98765,21,TCP,block,1,phishing-site.com,1,2,0,60,10.1.1.104,192.168.1.204,unknown-user,Threat-Id: 11223

Output

  {
    "_ob": {
      "log_path": "message",
      "source": "100000000000000472",
      "ts": "2025-06-16T17:04:00Z"
    },
    "message": "Feb 26 12:34:56 PA-FW1 1,2024/02/26 12:34:56,THREAT,virus,1,2024/02/26 12:34:56,10.1.1.100,192.168.1.200,0.0.0.0,0.0.0.0,app-unknown,vsys1,Trust,Untrust,ethernet1/1,ethernet1/2,Log-Forwarding,1001,1,54321,80,TCP,block,1,example.com,1,2,0,50,10.1.1.100,192.168.1.200,example-user,Threat-Id: 12345",
    "timestamp": "2025-06-16T17:04:00Z"
  }
  {
    "_ob": {
      "log_path": "message",
      "source": "100000000000000472",
      "ts": "2025-06-16T17:04:00Z"
    },
    "message": "Feb 26 12:35:01 PA-FW1 1,2024/02/26 12:35:01,THREAT,malware,1,2024/02/26 12:35:01,10.1.1.101,192.168.1.201,0.0.0.0,0.0.0.0,app-web-browsing,vsys1,Trust,Untrust,ethernet1/1,ethernet1/2,Log-Forwarding,1002,1,34567,443,TCP,block,1,malicious.com,1,2,0,30,10.1.1.101,192.168.1.201,unknown-user,Threat-Id: 67890",
    "timestamp": "2025-06-16T17:04:00Z"
  }
  {
    "_ob": {
      "log_path": "message",
      "source": "100000000000000472",
      "ts": "2025-06-16T17:04:00Z"
    },
    "message": "Feb 26 12:35:05 PA-FW1 1,2024/02/26 12:35:05,THREAT,exploit,1,2024/02/26 12:35:05,10.1.1.102,192.168.1.202,0.0.0.0,0.0.0.0,app-dns,vsys1,Trust,Untrust,ethernet1/1,ethernet1/2,Log-Forwarding,1003,1,23456,53,UDP,block,1,exploit.com,1,2,0,80,10.1.1.102,192.168.1.202,example-user,Threat-Id: 54321",
    "timestamp": "2025-06-16T17:04:00Z"
  }
  {
    "_ob": {
      "log_path": "message",
      "source": "100000000000000472",
      "ts": "2025-06-16T17:04:00Z"
    },
    "message": "Feb 26 12:35:10 PA-FW1 1,2024/02/26 12:35:10,THREAT,command-and-control,1,2024/02/26 12:35:10,10.1.1.103,192.168.1.203,0.0.0.0,0.0.0.0,app-http,vsys1,Trust,Untrust,ethernet1/1,ethernet1/2,Log-Forwarding,1004,1,13579,80,TCP,block,1,c2-server.com,1,2,0,5,10.1.1.103,192.168.1.203,example-user,Threat-Id: 98765",
    "timestamp": "2025-06-16T17:04:00Z"
  }
  {
    "_ob": {
      "log_path": "message",
      "source": "100000000000000472",
      "ts": "2025-06-16T17:04:00Z"
    },
    "message": "Feb 26 12:35:15 PA-FW1 1,2024/02/26 12:35:15,THREAT,phishing,1,2024/02/26 12:35:15,10.1.1.104,192.168.1.204,0.0.0.0,0.0.0.0,app-ftp,vsys1,Trust,Untrust,ethernet1/1,ethernet1/2,Log-Forwarding,1005,1,98765,21,TCP,block,1,phishing-site.com,1,2,0,60,10.1.1.104,192.168.1.204,unknown-user,Threat-Id: 11223",
    "timestamp": "2025-06-16T17:04:00Z"
  }

Results: This conversion facilitates threat detection and analysis in SIEM systems, ensuring critical threat details are effectively captured and communicated.

Convert Windows Security Log to CEF

Scenario: Convert Windows Security Log to CEF format.

  • CEF Field Name: Message

  • Syslog Fragment Field: _ob.cef.syslog_part.

Input

Feb 26 12:40:00 WIN-SERVER1 Security 4624 An account was successfully logged on. Subject: Security ID: S-1-5-18 Account Name: SYSTEM Logon ID: 0x3E7. New Logon: Security ID: S-1-5-21-123456789-123456789-123456789-1001 Account Name: user1 Logon Type: 2 Source Network Address: 192.168.1.100
Feb 26 12:41:05 WIN-SERVER1 Security 4625 An account failed to log on. Account Name: user2 Logon Type: 3 Failure Reason: Unknown user name or bad password Source Network Address: 192.168.1.101
Feb 26 12:42:10 WIN-SERVER1 Security 4670 Object access attempted. Object Name: C:\SensitiveFile.txt Accesses: WRITE_OWNER, WRITE_DAC Account Name: admin1
Feb 26 12:43:15 WIN-SERVER1 Security 4688 A new process has been created. New Process Name: C:\Windows\System32\cmd.exe Creator Process Name: C:\Windows\explorer.exe Account Name: user3
Feb 26 12:44:20 WIN-SERVER1 Security 4720 A user account was created. Account Name: new_user Account Enabled: Yes Creator: admin2

Output

  {
    "_ob": {
      "log_path": "message",
      "source": "100000000000000472",
      "ts": "2025-06-16T17:12:32Z"
    },
    "message": "Feb 26 12:40:00 WIN-SERVER1 Security 4624 An account was successfully logged on. Subject: Security ID: S-1-5-18 Account Name: SYSTEM Logon ID: 0x3E7. New Logon: Security ID: S-1-5-21-123456789-123456789-123456789-1001 Account Name: user1 Logon Type: 2 Source Network Address: 192.168.1.100",
    "timestamp": "2025-06-16T17:12:32Z"
  }
  {
    "_ob": {
      "log_path": "message",
      "source": "100000000000000472",
      "ts": "2025-06-16T17:12:32Z"
    },
    "message": "Feb 26 12:41:05 WIN-SERVER1 Security 4625 An account failed to log on. Account Name: user2 Logon Type: 3 Failure Reason: Unknown user name or bad password Source Network Address: 192.168.1.101",
    "timestamp": "2025-06-16T17:12:32Z"
  }
  {
    "_ob": {
      "log_path": "message",
      "source": "100000000000000472",
      "ts": "2025-06-16T17:12:32Z"
    },
    "message": "Feb 26 12:42:10 WIN-SERVER1 Security 4670 Object access attempted. Object Name: C:\\SensitiveFile.txt Accesses: WRITE_OWNER, WRITE_DAC Account Name: admin1",
    "timestamp": "2025-06-16T17:12:32Z"
  }
  {
    "_ob": {
      "log_path": "message",
      "source": "100000000000000472",
      "ts": "2025-06-16T17:12:32Z"
    },
    "message": "Feb 26 12:43:15 WIN-SERVER1 Security 4688 A new process has been created. New Process Name: C:\\Windows\\System32\\cmd.exe Creator Process Name: C:\\Windows\\explorer.exe Account Name: user3",
    "timestamp": "2025-06-16T17:12:32Z"
  }
  {
    "_ob": {
      "log_path": "message",
      "source": "100000000000000472",
      "ts": "2025-06-16T17:12:32Z"
    },
    "message": "Feb 26 12:44:20 WIN-SERVER1 Security 4720 A user account was created. Account Name: new_user Account Enabled: Yes Creator: admin2",
    "timestamp": "2025-06-16T17:12:32Z"
  }

Results: This CEF transformation makes Windows security logs more SIEM-friendly, aiding in threat detection, compliance, and incident response.

CEF Serialization Best Practices

CEF Serialization is crucial for ensuring that logs are easy to analyze, search, and integrate into various SIEM and security analytics tools. Here are some best practices for CEF log serialization:

  1. Ensure Field Mapping Consistency: Properly map log fields to CEF key names such as src, dst, spt, dpt, cs* to maintain data integrity across SIEM systems.

  2. Standardize Time Format: Use UTC timestamps in rt (receipt time) and ensure they follow the correct format to avoid time zone misinterpretations.

  3. Limit Field Lengths: Adhere to CEF field length limitations such as msg should be concise to prevent truncation and data loss in downstream processing.

  4. Use Custom Fields for Additional Data: Utilize CEF custom fields (cs1Label=customField1 cs1=value) to preserve vendor-specific metadata while keeping default fields intact.

  5. Escape Special Characters: Encode or escape characters like |, = and \ within field values to avoid CEF format corruption such as replace | with \|.

  6. Maintain Readability and Structure: Keep log messages structured, removing redundant or excessive details to optimize storage and indexing efficiency.

  7. Test with SIEM Solutions: Validate CEF output compatibility with SIEM platforms such as Splunk, ArcSight, QRadar to ensure proper parsing and event correlation.

  8. Minimize Unnecessary Fields: Exclude empty or redundant fields to reduce log size and improve log processing performance.

  9. Include Threat Intelligence Data: If applicable, append threat intelligence context such as Threat ID, URL reputation, risk score for enriched security analytics.

  10. Monitor for Data Quality Issues: Regularly review CEF logs for formatting errors, missing fields, or data inconsistencies to maintain high-quality event ingestion.

By following these best practices, CEF logs remain structured, efficient, and SIEM-compatible, ensuring reliable security monitoring and incident response.

Related Functions

  • Windows Serializer: Converts Windows event logs into a structured format like CEF, JSON, or Syslog, ensuring compatibility with targeted destinations.

  • CEF (Parser): Extracts and normalizes fields from CEF-formatted logs, enabling efficient search, correlation, and analysis in SIEM systems.

PreviousSerializersNextCisco Serializer

Last updated

Was this helpful?