CrowdStrike Next-Gen SIEM

CrowdStrike Next-Gen SIEM (formerly Humio) is a scalable, cloud-native log management platform that processes massive log and event data volumes for real-time analytics, threat detection, and observability. This document provides step-by-step instructions for configuring Observo AI's CrowdStrike Next-Gen SIEM destination to ingest logs using the Splunk-compatible HEC (HTTP Event Collector) ingestion protocol. The integration enables secure, scalable delivery of observability data from Observo AI pipelines to CrowdStrike Next-Gen SIEM for centralized log management and security analytics.

Purpose

The Observo AI CrowdStrike Next-Gen SIEM destination enables organizations to stream security logs, threat intelligence, and operational data to CrowdStrike's Next-Gen SIEM platform for comprehensive security monitoring and analysis. This integration leverages CrowdStrike's advanced analytics engine and correlation capabilities while benefiting from Observo AI's data optimization to reduce ingestion costs and improve signal-to-noise ratios.

Prerequisites

Before configuring the CrowdStrike Next-Gen SIEM destination in Observo AI, ensure the following requirements are met to facilitate seamless data ingestion:

  • CrowdStrike Next-Gen SIEM Account:

    • Access to the SIEM instance with appropriate administrative privileges.

    • Access to the SIEM instance with appropriate administrative privileges.

    • Configured repository for log ingestion

  • Ingest Token:

    • Generate an Ingest Token in the CrowdStrike Next-Gen SIEM platform to authenticate data ingestion.

    • Ensure the token has permissions to ingest logs into the target repository.

  • Network Access:

    • Verify that the Observo AI instance can communicate with the CrowdStrike Next-Gen SIEM endpoint.

    • Check for firewall rules or network policies that may block outbound HTTPS traffic to the endpoint on port 443.

Prerequisite
Description
Notes

CrowdStrike Next-Gen SIEM Account

Active CrowdStrike Next-Gen SIEM instance

Active CrowdStrike Next-Gen SIEM instance

Ingest Token

Authenticates log data ingestion

Securely store the Ingest Token

Network Access

Enables communication with CrowdStrike Next-Gen SIEM

Ensure HTTPS connectivity to the endpoint

Integration

The Integration section outlines default configurations for the CrowdStrike Next-Gen SIEM destination. To tailor the setup to your environment, consult the Configuration Parameters section in the CrowdStrike Next-Gen SIEM documentation for advanced options.

To configure CrowdStrike Next-Gen SIEM as a destination in Observo AI, follow these steps:

1. Log in to Observo AI

  • Navigate to the Destinations section.

  • Click the Add Destinations button and select Create New.

  • Choose CrowdStrike Next-Gen SIEM from the list of available destinations to begin configuration.

2. General Settings

Configure the core connection parameters:

  • Name: Provide a descriptive, unique identifier for this destination

    • Examples: crowdstrike-siem-production, cs-security-events, siem-firewall-logs

  • Description (Optional): Document the purpose and scope of this destination

    • Example: "Production security events to CrowdStrike Next-Gen SIEM for threat detection and compliance monitoring"

  • Endpoint: The base URL of your CrowdStrike Next-Gen SIEM instance

    • Must include the scheme (http or https)

    • Do not include path components - CrowdStrike uses standard Splunk HEC-compatible endpoints

    Examples

    https://siem.crowdstrike.com

    https://siem.yourcompany.com

    http://127.0.0.1 (testing only)

  • Token: The ingest token from CrowdStrike Next-Gen SIEM

    • Generate in Settings → API Tokens within the SIEM interface

    • Use environment variables or secrets management for production

    Examples

    ${CROWDSTRIKE_INGEST_TOKEN}

    A94A8FE5CCB19BA61C4C08

  • Event Type (Optional): Specifies the parser CrowdStrike should use for data processing

    • Default: none

    Examples

    json

    cef

    none

    {{ event_type }}

  • Host Key (Optional): Field name containing the hostname in your security events

    • Default: host

    • Override if your logs use different field naming conventions

    • Examples: hostname, source_host, device_name

  • Index (Optional): Repository name for log ingestion

    • In cloud deployments: must match the repository associated with the ingest token

    • In self-managed(private) deployments: can be configured separately

    Examples

    {{ host }}

    security-events

    firewall-logs

  • Timestamp Key (Optional): Field name containing the event timestamp

    • Default: timestamp

    • Set to empty string ("") to omit timestamp assignment

    Examples

    timestamp

    @timestamp

    event_time

    log_timestamp

  • Acknowledgements Enabled: Enable end-to-end delivery guarantees

    • Default: false

    • When enabled: sources wait for SIEM confirmation before acknowledging receipt

    • Recommended: Enable for critical security logs requiring guaranteed delivery

3. Encoding

Select and configure the data serialization format:

  • Encoding Codec: Choose the format for encoding security events

    • Default: JSON Encoding

    • Available options: JSON, logfmt, Avro, NDJSON, Raw, Text, Parquet, CEF, CSV, Protobuf, GELF

JSON Encoding (Recommended for Most Use Cases)

  • Pretty JSON (Boolean, Default: false)

    • Enable: Format with indentation for debugging

    • Disable: Compact format for production efficiency

  • Encoding Metric Tag Values (Default: single)

    • single: Display only the last non-bare tag value

    • full: Expose all metric tags as separate assignments

  • Encoding Timestamp Format

    • RFC 3339: ISO 8601 format (2025-10-17T14:30:00Z)

    • Unix: Epoch timestamp in seconds

logfmt Encoding

Structured key-value format for cloud-native applications.

  • Encoding Metric Tag Values (Default: single)

  • Encoding Timestamp Format

    • RFC 3339: ISO 8601 format (2025-10-17T14:30:00Z)

    • Unix: Epoch timestamp in seconds

Apache Avro Encoding

Binary serialization with schema validation.

  • Avro Schema (Required): JSON schema definition

    • Example:

    {
      "type": "record",
      "name": "security_event",
      "fields": [
        { "name": "event_type", "type": "string" },
        { "name": "severity", "type": "int" },
        { "name": "source_ip", "type": "string" },
        { "name": "timestamp", "type": "long" }
      ]
    }

  • Encoding Metric Tag Values (Default: single)

  • Encoding Timestamp Format

    • RFC 3339: ISO 8601 format (2025-10-17T14:30:00Z)

    • Unix: Epoch timestamp in seconds

Newline Delimited JSON (NDJSON)

Streaming JSON format with one object per line.

  • Encoding Metric Tag Values (Default: single)

  • Encoding Timestamp Format

    • RFC 3339: ISO 8601 format (2025-10-17T14:30:00Z)

    • Unix: Epoch timestamp in seconds

No Encoding (Raw Message)

Pass data without transformation.

  • Encoding Metric Tag Values (Default: single)

  • Encoding Timestamp Format

    • RFC 3339: ISO 8601 format (2025-10-17T14:30:00Z)

    • Unix: Epoch timestamp in seconds

Plain Text Encoding

Simple text format without structure.

  • Encoding Metric Tag Values (Default: single)

  • Encoding Timestamp Format

    • RFC 3339: ISO 8601 format (2025-10-17T14:30:00Z)

    • Unix: Epoch timestamp in seconds

Parquet

Columnar storage format optimized for analytics.

  • Include Raw Log (Boolean, Default: false)

    • When enabled: Adds observo_record field with complete original event

    • Useful for preserving raw data alongside structured schema

  • Parquet Schema (Required): Schema definition in Parquet message format

    • Example:

    message security_event {
      optional binary event_id;
      optional binary timestamp;
      optional int32 severity;
      optional group source {
        optional binary ip_address;
        optional binary hostname;
      }
      optional group detection {
        optional binary rule_name;
        optional binary signature_id;
      }
    }

  • Encoding Metric Tag Values (Select, Default: single)

  • Encoding Timestamp Format

    • RFC 3339: ISO 8601 format (2025-10-17T14:30:00Z)

    • Unix: Epoch timestamp in seconds

Common Event Format (CEF)

Industry-standard security event format.

  • CEF Device Event Class ID (Required): Event type identifier (max 1023 chars)

    • Examples: intrusion-detected, authentication-failure, policy-violation

  • CEF Device Product (Required): Product name generating events (max 63 chars)

    • Examples: Next-Gen Firewall, Web Application Firewall, Endpoint Security

  • CEF Device Vendor (Required): Vendor name (max 63 chars)

    • Examples: Observo, Security Solutions Inc

  • CEF Device Version (Required): Product version (max 31 chars)

    • Examples: 1.0.0, 2.5.3, 3.1.0-beta

  • CEF Extensions (Key-Value Pairs): Custom security event fields

    • Define additional attributes specific to your security events

    • Examples: src, dst, suser, duser, act, outcome

  • CEF Name (Required): Human-readable event description (max 512 chars)

    • Example: Suspicious network activity detected from external IP

  • CEF Severity (Required): Event importance level (0-10)

    • 0-3: Low severity (informational)

    • 4-6: Medium severity (warning)

    • 7-8: High severity (critical)

    • 9-10: Very high severity (emergency)

  • CEF Version (Select, Default: V0): CEF specification version

    • V0: CEF specification version 0.1

    • V1: CEF specification version 1.x

CSV Format

Comma-separated values for structured tabular data.

  • CSV Fields (Required): Array of field names for columns

    • Order determines column sequence in output

    • Examples: ["timestamp", "source_ip", "destination_ip", "event_type", "severity"]

  • CSV Buffer Capacity (Number, Default: 8192)

    • Internal buffer size in bytes (minimum: 1024)

  • CSV Delimiter (String, Default: ,)

    • Character separating fields

    • Alternatives: \t (tab), | (pipe), ; (semicolon)

  • Enable Double Quote Escapes (Boolean, Default: true)

    • Enabled: Escape quotes by doubling ("")

    • Disabled: Use escape character

  • CSV Escape Character (String, Default: ")

    • Character for escaping when double quote disabled

  • CSV Quote Character (String, Default: ")

    • Character for quoting field values

  • CSV Quoting Style (Select, Default: necessary)

    • always: Quote all fields

    • necessary: Quote only when required

    • never: Never quote fields

    • non_numeric: Quote all non-numeric fields

Protocol Buffers (Protobuf)

Binary serialization using Protocol Buffers.

  • Protobuf Message Type (Required): Fully qualified message type name

    • Example: com.company.security.SecurityEvent

  • Protobuf Descriptor File (Required): Path to compiled .desc file

    • Example: /etc/observo/schemas/security_events.desc

  • Encoding Metric Tag Values (Select, Default: single)

  • Encoding Timestamp Format

    • RFC 3339: ISO 8601 format (2025-10-17T14:30:00Z)

    • Unix: Epoch timestamp in seconds

Graylog Extended Log Format (GELF)

JSON format designed for Graylog and similar platforms.

  • Encoding Metric Tag Values (Default: single)

  • Encoding Timestamp Format

    • RFC 3339: ISO 8601 format (2025-10-17T14:30:00Z)

    • Unix: Epoch timestamp in seconds

4. Request Configuration (Optional)

Fine-tune HTTP request behavior for optimal performance:

  • Request Concurrency: Control parallel request processing

    • Default: Adaptive concurrency

    • Options:

      • Adaptive concurrency: Dynamically adjusts based on system load

      • A fixed concurrency of 1: Sequential processing for strict ordering

  • Request Rate Limit Duration Secs: Time window for rate limiting

    • Default: 1 second

    • Defines the period for rate_limit_num calculations

  • Request Rate Limit Num: Maximum requests per time window

    • Default: Unlimited

    • Set to control throughput (e.g., 100 requests per second)

  • Request Retry Attempts: Maximum retry attempts for failed requests

    • Default: Unlimited

    • Recommended: 5-10 for production deployments

  • Request Retry Initial Backoff Secs: Initial retry delay

    • Default: 1 second

    • Subsequent retries use Fibonacci sequence for exponential backoff

  • Request Retry Max Duration Secs: Maximum time between retries

    • Default: 3600 seconds (1 hour)

    • Caps the maximum backoff duration

  • Request Timeout Secs: Request timeout before abortion

    • Default: 60 seconds

    • Should exceed CrowdStrike's internal timeout to prevent orphaned requests

5. Batching Configuration

Optimize data transmission through intelligent batching:

  • Batch Max Bytes: Maximum batch size in bytes (uncompressed)

    • No default (unlimited)

    • Recommended: 1000000 (1 MB) for balanced performance

    • Adjust based on network capacity and event size

  • Batch Max Events: Maximum number of events per batch

    • No default (unlimited)

    • Recommended: 1000 events for typical security logs

    • Lower values reduce latency, higher values improve throughput

  • Batch Timeout Secs: Maximum batch age before flushing

    • Default: 1.0 second

    • Ensures timely delivery even during low-volume periods

    • Balance between latency and batching efficiency

6. TLS Configuration (Optional)

Configure secure communication:

  • TLS CA: CA certificate in PEM format

    • Required when using custom certificate authority

    • Necessary for self-signed certificates

  • TLS CRT: Client certificate in PEM format

    • Required for mutual TLS (mTLS) authentication

  • TLS Key: Private key in PEM format

    • Must correspond to TLS CRT

    • Store securely and never commit to version control

  • TLS Verify Certificate: Enable certificate verification

    • Default: false

    • Critical: Set to true in production environments

    • Verifies certificate validity and trusted issuer chain

  • TLS Verify Hostname: Enable hostname verification

    • Default: false

    • Critical: Set to true in production environments

    • Prevents man-in-the-middle attacks by verifying hostname matches certificate

7. Advanced Settings

Additional configuration options for specialized use cases:

  • Compression: Data compression algorithm

    • Default: No compression

    • Options:

      • No compression: Fastest, no CPU overhead

      • Gzip compression: Good balance of speed and compression ratio

      • Zlib compression: Similar to Gzip, uses DEFLATE algorithm

    • Recommended: Gzip for bandwidth-constrained environments

  • Source (Optional): Identifies the log source origin

    • Typically the originating system, file, or network source

    • If unset, CrowdStrike collector will attempt to set automatically

    Examples

    {{ file }}

    /var/log/security.log

    firewall-01

    UDP:514

8. Save and Test Configuration

  • Save the configuration settings in Observo AI.

  • Send sample security events to validate end-to-end data flow

  • Monitor the CrowdStrike repository to confirm successful ingestion

  • Review parsed fields and verify correct event formatting

Troubleshooting

If issues arise with the CrowdStrike Next-Gen SIEM destination, use the following steps to diagnose and resolve them:

Verify Configuration Settings

  • Ensure all fields, such as Endpoint, Token, and Event Type, are correctly entered and match the CrowdStrike Next-Gen SIEM account configuration.

  • Confirm that the Index (if set) matches the repository associated with the Ingest Token.

Check Authentication

  • Verify that the Ingest Token is valid and has not been revoked or expired.

  • Regenerate the token in CrowdStrike Next-Gen SIEM if necessary and update the Observo AI configuration.

Monitor Logs

  • Check Observo AI logs for errors or warnings related to log data transmission to CrowdStrike Next-Gen SIEM.

  • In the CrowdStrike Next-Gen SIEM platform, navigate to the target repository to confirm that logs are arriving with the expected tags or index.

Validate Network Connectivity

  • Ensure that the Observo AI instance can reach the CrowdStrike Next-Gen SIEM endpoint.

  • Check for firewall rules or network policies blocking HTTPS traffic on port 443.

Test Data Flow

  • Send sample log data through Observo AI and monitor its arrival in CrowdStrike Next-Gen SIEM's repository.

  • Use the Analytics tab in the targeted Observo AI pipeline to monitor data volume and ensure expected throughput.

Check Quotas and Limits

  • Verify that the CrowdStrike Next-Gen SIEM account is not hitting ingestion limits or quotas.

  • Adjust batching settings such as Batch Max Bytes, Batch Max Events if backpressure or slow data transfer occurs.

Common Issues

Issue
Possible Cause
Resolution

Logs not appearing in CrowdStrike Next-Gen SIEM

Incorrect Endpoint, Token, or Index

Verify Endpoint, Token, and Index in configuration

Authentication errors

Expired or invalid Ingest Token

Regenerate token and update configuration

Connection failures

Network or firewall issues

Check network policies and HTTPS connectivity

Slow log transfer

Backpressure or rate limiting

Adjust batching settings or check quotas

Common Errors

Error Message
Possible Cause
Solution

Connection refused

Endpoint unreachable

Verify endpoint URL and network connectivity

401 Unauthorized

Invalid or expired token

Regenerate and update ingest token

403 Forbidden

Insufficient token permissions

Verify token has write access to repository

404 Not Found

Repository doesn't exist

Create repository or correct index name

413 Payload Too Large

Batch size exceeds limits

Reduce Batch Max Bytes and Batch Max Events

429 Too Many Requests

Rate limit exceeded

Implement rate limiting or increase quotas

500 Internal Server Error

CrowdStrike SIEM issue

Check CrowdStrike status, contact support

503 Service Unavailable

SIEM temporarily unavailable

Increase retry attempts and backoff duration

SSL certificate verify failed

Certificate validation error

Configure TLS CA or fix certificate issues

Timeout

Request duration exceeded

Increase Request Timeout Secs or reduce batch size

Parser error

Event format doesn't match parser

Verify Event Type matches configured parser

Resources

For additional guidance and detailed information, refer to the following resources:

PreviousBlackholeNextCrowdstrike LogScale Logs

Last updated

Was this helpful?