CrowdStrike EventStreams

The CrowdStrike EventStreams Source in Observo AI enables the ingestion of security and audit events from CrowdStrike's streaming data feeds, facilitating continuous collection of threat intelligence, user activity, authentication events, and API audit logs for enhanced security monitoring, threat detection, and compliance reporting.

Purpose

The purpose of the CrowdStrike EventStreams source is to enable users to ingest real-time security events and audit logs from CrowdStrike Falcon platform into Observo AI for advanced analysis and correlation. It facilitates the collection of structured security events through CrowdStrike's EventStreams API, allowing organizations to streamline security data pipelines, enhance threat visibility, and support use cases such as security incident detection, threat hunting, compliance auditing, user behavior analytics, and real-time security alerting by processing continuous event streams from CrowdStrike endpoints and security controls.

Prerequisites

Before configuring the CrowdStrike EventStreams source in Observo AI, ensure the following requirements are met to facilitate seamless data ingestion:

• CrowdStrike Falcon Platform:

  • Active CrowdStrike Falcon subscription with EventStreams API access enabled.

  • Identify your CrowdStrike cloud region (e.g., us-1, us-2, eu-1) for API endpoint configuration.

  • Verify access to the CrowdStrike EventStreams API endpoints for your region.

• API Credentials:

  • Obtain OAuth2 API credentials from CrowdStrike Falcon console:

    • Client ID: API client identifier with appropriate permissions.

    • Client Secret: Corresponding secret for the API client.

  • Ensure the API client has the necessary scopes/permissions for EventStreams access.

  • Document the OAuth2 token URL for your CrowdStrike region.

• Event Stream Configuration:

  • Determine which event types to collect (e.g., UserActivityAuditEvent, AuthActivityAuditEvent, APIActivityAuditEvent).

  • Decide whether to collect historical events or start from the latest events only.

• Network and Connectivity(Skip this section for SaaS deployments):

  • Ensure Observo AI can establish outbound HTTPS connections to CrowdStrike API endpoints.

  • Verify firewall rules allow connections to api.<region>.crowdstrike.com on port 443.

  • Check for proxy settings or network policies that may affect API connectivity.

  • Verify DNS resolution for the CrowdStrike API hostname.

Prerequisite	Description	Notes
CrowdStrike Subscription	Active Falcon subscription with EventStreams API access	Verify region and API endpoint availability
OAuth2 Credentials	Client ID and Client Secret from Falcon console	Obtain from CrowdStrike API Clients & Keys section
API Permissions	Sufficient scopes for EventStreams access	Verify permissions include event stream read access
Network Connectivity(Not relevant for SaaS deployments)	Outbound HTTPS access to CrowdStrike API	Check firewall rules, DNS resolution, and network policies

Integration

The Integration section outlines the configurations for the CrowdStrike EventStreams source. To configure the CrowdStrike EventStreams source in Observo AI, follow these steps to set up and test the data flow:

1. Log in to Observo AI:

   • Navigate to the Sources tab.

   • Click the Add Source button and select Create New.

   • Choose CrowdStrike EventStreams from the list of available sources to begin configuration.

2. General Settings:

   • Source Type: CrowdStrike EventStreams (pre-selected)

   • Name: A unique identifier for the source, such as crowdstrike-security-events-prod.

   • Description (Optional): Provide a description for the source.

   • API Base URL: The base HTTPS URL for your CrowdStrike region's API endpoint. Do no include path and query to the base URL. Default: https://api.<region>.crowdstrike.com

     Example
     https://api.us-2.crowdstrike.com
     https://api.eu-1.crowdstrike.com

   • App ID: Client identifier that uniquely identifies this collector instance consuming the event stream.

     Example
     observo
     observo_pipeline

   • Liveness (seconds): Stream-processor heartbeat interval that helps determine if a consumer is active on a data-feed partition. In a low-throughput stream, this also governs frequency of commiting consume offset Default: 60 (Range: 10-3600)

     Example
     60

   • Event type (Optional): Specific event type to collect. Leave blank to collect all event types.

     Example
     UserActivityAuditEvent
     AuthActivityAuditEvent
     APIActivityAuditEvent

   • Start at the latest event: When enabled, starts consuming events from the current point forward, ignoring historical events. Default: Enabled

     Options
     Enabled
     Disabled

   • Collection interval (seconds): Trigger interval for collection cycles. Should be greater than twice the liveness interval. Default: 130 (Range 30-7201)

     Example
     130

3. Authentication:

   • Client ID: OAuth2 client ID for CrowdStrike API authentication.

     Example
     a1b2c3d4e5f6g7h8i9j0k1l2m3n4o5p6

   • Client Secret: OAuth2 client secret for CrowdStrike API authentication. (Masked field)

     Example
     AbCdEfGhIjKlMnOpQrStUvWxYz0123456789AbCdEfGh

   • Token URL: OAuth2 token endpoint URL for your CrowdStrike region.

     Example
     https://api.us-2.crowdstrike.com/oauth2/token
     https://api.eu-1.crowdstrike.com/oauth2/token

   • Scopes (Optional): OAuth2 scopes to request during authentication.

     Example
     event-streams

   • Token Refresh Margin (seconds): Time buffer before token expiry to trigger refresh. Default: 60 (Range: 10-3600)

     Example
     60

4. Advanced Settings:

   • Feed refresh safety margin (seconds): Safety margin for refreshing data feeds before they expire. CrowdStrike feeds are valid for a limited time (typically 30 minutes). Default: 60 (Range: 10-300)

   • Force commit offsets after (events): Forces the collector to commit offset after processing N events. Upper-bounds replay on failure. Default: 10000 (Range: 1-1000000)

   • Collector Script (Lua): Collection script that discovers available data feeds and process streams. The script contains core logic for stream management, offset tracking , and event emission. It is a pre-populated lua script which can be customized.

   • The number of runtimes to utilize: Number of concurrent runtime instances for processing. Default: 2

   • The number of fetchers to utilize: Number of concurrent fetchers for API calls. Default: 8

   • Fetch request queue depth: Depth of the queue for outgoing fetch requests. Default: 1000

   • Fetch response queue depth: Depth of the queue for incoming fetch responses. Default: 10

   • Fetch completion queue depth: Depth of the queue for completed fetch operations. Default: 10

   • Event batch queue depth: Depth of the queue for batched events. Default: 10

   • Stream Queue: Depth of the queue for combining data from concurrent streams within a runner instance. Default: 64

   • Streams Per Fetcher: Maximum number of concurrent streams each fetcher can handle. Default: 1

   • Start routine to be executed at the beginning of the collection cycle: Name of the Lua function to execute at the beginning of each collection cycle. Default: start

   • The amount of time to wait between two consecutive collections in seconds: Initial delay between retries. Default: 1 (Range: 1-60)

   • The maximum delay between retries in seconds Maximum delay between retries. Default: 5 (Range: 2-60)

   • The maximum delay between retries.: Maximum number of retry attempts. Default: 4 (Range: 1-10)

   • The backoff factor by which the delay increases after each retry: Multiplier for exponential backoff between retries. Default: 2.0 (Minimum: 1.0)

5. Save and Test:

   • Click Save to store the configuration.

   • Monitor the source connection status to verify successful stream initialization.

   • Verify event ingestion by checking the Analytics tab in the Observo AI pipeline for event counts and data throughput.

Troubleshooting

If issues arise with the CrowdStrike EventStreams source in Observo AI, use the following steps to diagnose and resolve them:

• Verify Configuration Settings:

  • Ensure the API Base URL is correctly entered for your CrowdStrike region (e.g., https://api.us-2.crowdstrike.com) and ensure there is no path or query included.

  • Confirm the OAuth2 Token URL matches your region's endpoint.

  • Verify the App ID is unique and follows naming conventions.

  • Check that the Collection interval is greater than twice the Liveness interval.

• Check Authentication:

  • Verify the Client ID and Client Secret are correct and copied completely without extra spaces.

  • Ensure the OAuth2 credentials have not been revoked or expired in CrowdStrike Falcon console.

  • Verify the API client has the necessary permissions/scopes for EventStreams access.

  • Confirm system time is synchronized, as OAuth2 token validation depends on accurate timestamps.

• Validate Network Connectivity(Not relevant for SaaS deployments):

  • Test connectivity to the CrowdStrike API endpoint using curl or similar tools.

  • Check for firewall rules or network policies that may block outbound HTTPS connections to CrowdStrike.

  • Verify DNS resolution for the CrowdStrike API hostname.

  • Ensure proxy settings, if applicable, are configured correctly for HTTPS traffic.

• Stream Processing Issues:

  • Check for stream timeout errors if liveness interval is too short or network latency is high.

  • Verify offset checkpointing is occurring regularly (check logs for checkpoint update messages).

  • Review feed refresh logs to ensure streams are refreshed before expiry.

• Performance and Scaling:

  • If event ingestion is slow, consider increasing the number of fetchers or runtimes.

  • Adjust scaling parameters based on observed event volume and stream count.

• Common Error Messages:

  • "Authentication failed": OAuth2 credentials are invalid, expired, or lack necessary permissions. Verify Client ID and Client Secret.

  • "CrowdStrike discover failed with status: XXX": Stream discovery API call failed. Check network connectivity and API endpoint URL.

  • "CrowdStrike discover response is missing resources": API response format issue. Verify API compatibility and check for service disruptions.

  • "CrowdStrike poll interval too small": Collection interval is less than twice the liveness interval. Increase collection interval or decrease liveness interval.

  • "Connection timeout": Network latency or API unavailability. Check network conditions and CrowdStrike service status.

  • "Token refresh failed": OAuth2 token renewal failed. Verify token URL and credentials.

• Monitor Logs and Data:

  • Check the source connection status in Observo AI to verify active stream processing.

  • Monitor the Analytics tab in the targeted Observo AI pipeline for event volume and throughput.

  • Review Observo AI logs for errors, warnings, stream lifecycle events, and checkpoint updates.

  • Track authentication success/failure rates and token refresh frequency.

Issue	Possible Cause	Resolution
No events ingested	Stream discovery failed or auth error	Verify API Base URL, credentials, and network connectivity
Authentication errors	Invalid or expired OAuth2 credentials	Check Client ID and Client Secret in CrowdStrike console
Stream discovery failures	Incorrect API endpoint or network issue	Verify API Base URL matches region and check connectivity
"Poll interval too small"	Collection interval < 2× liveness	Increase collection interval or decrease liveness interval
Connection timeouts	Network latency or API unavailability	Check network conditions and CrowdStrike service status
Token refresh failures	Invalid token URL or credentials	Verify Token URL and OAuth2 credentials
Duplicate events	Multiple collectors with same App ID	Use unique App ID for each collector instance
Missing events after restart	Offset checkpointing not working	Check logs for checkpoint errors and verify storage
Feed expiry errors	Refresh safety margin too small	Increase feed refresh safety margin
Slow ingestion	Insufficient fetchers or runtimes	Increase scaling parameters based on event volume
Stream not starting	Already active consumer on partition	Normal behavior; collector will take over if other consumer fails
High replay after failure	Max checkpoint offset lag too high	Decrease force commit offset threshold

Resources

For additional guidance and detailed information, refer to the following resources:

• CrowdStrike Documentation:

  • CrowdStrike Falcon EventStreams API Documentation

  • CrowdStrike API Clients & Keys Management

  • CrowdStrike OAuth2 Authentication Guide

  • CrowdStrike Event Types Reference

• Security and Best Practices:

  • OAuth 2.0 Framework (RFC 6749)

  • API Security Best Practices - OWASP

  • Stream Processing Patterns and Best Practices