Snyk

The Snyk Issues API Collector source enables the ingestion of security vulnerability and issue data from Snyk's REST API, facilitating continuous security monitoring and compliance tracking. This document provides step-by-step instructions for configuring Snyk Issues API Collector as a source within Observo AI.

Purpose

The purpose of the Observo AI Snyk Issues API Collector source is to enable users to ingest security issue data from Snyk organizations or groups into the Observo AI platform for real-time analysis and processing. It facilitates the collection of vulnerability data, allowing organizations to streamline security data pipelines, enhance observability, and support use cases such as vulnerability tracking, compliance reporting, security event correlation, and risk-based prioritization by processing security findings from diverse Snyk-monitored projects and repositories.

Prerequisites

Before configuring the Snyk Issues API Collector source in Observo AI, ensure the following requirements are met to facilitate seamless data ingestion:

• Snyk Account Configuration:

  • Identify the Snyk Organization or Group from which to collect security issues.

  • Obtain the Organization or Group UUID from Snyk account settings.

  • Ensure your Snyk account has appropriate permissions to access the Snyk API.

• Authentication:

  • Prepare authentication credentials as required by the Snyk API:

    • API Token: Generate a valid API token from your Snyk account settings.

  • The API token must have read access to issues within your organization or group.

  • Refer Authentication for Snyk API documentation for specific authentication requirements.

• API Access Configuration:

  • Ensure API access is enabled for your Snyk organization or group.

  • Verify API rate limits for your Snyk subscription plan.

  • Confirm access to the Snyk REST API endpoint (https://api.snyk.io).

• Network and Connectivity:

  • Ensure Observo AI can establish outbound HTTPS connections to the Snyk API endpoint.

  • Check for firewall rules, proxy settings, or network policies that may affect connectivity.

Prerequisite	Description	Notes
Snyk Account	Active Snyk account with organization/group access	Required for API access and issue retrieval
Entity ID	UUID of Snyk organization or group	Format: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
API Token	Valid Snyk API authentication token	Obtain from Snyk account settings → API Token section
API Rate Limits	Understanding of Snyk API rate limits	Varies by subscription plan; affects collection frequency
Network Connectivity	Outbound HTTPS access to api.snyk.io	Check firewall rules, DNS resolution, and network policies

Integration

The Integration section outlines the configurations for the Snyk Issues API Collector source. To configure the source in Observo AI, follow these steps to set up and test the data flow:

1. Log in to Observo AI:

   • Navigate to the Sources tab.

   • Click the Add Source button and select Create New.

   • Choose Snyk Issues API Collector from the list of available sources to begin configuration.

2. General Settings:

   • Source Type: Snyk Issues API Collector (pre-selected)

   • Name: A unique identifier for the source, such as snyk-security-issues-prod.

   • Description (Optional): Provide a description for the source.

   • API Base URL: The Snyk REST API endpoint to connect to. Default is the public Snyk API.

     Default Value
     https://api.snyk.io

   • API Version: The version of the Snyk REST API to use.

     Default Value
     2024-05-23

   • Entity Type: Select whether collecting from a Snyk Organization or Group.

     Options
     Organization
     Group

   • Entity ID: The UUID of your Snyk organization or group. Required field.

     Example
     1234abcd-5678-efgh-9012-ijklmnopqrst

   • Starting Date Time for Collecting Issues: Initial timestamp for collecting issues (used only on the first run). Leave empty to start from 24 hours ago.

     Example
     2025-11-03T00:44:33

   • Time in Seconds to Pause Between Script Executions: Interval between consecutive collection cycles. Default: 300 seconds (5 minutes)

     Range	Recommended Value
     10 - 86401	300 - 900

3. Authentication:

   • Auth Token: Your Snyk API authentication token. (Masked field)

     Example
     abcd1234-efgh-5678-ijkl-9012mnopqrst

4. Advanced Settings:

   • Page Size: Number of results returned per API page. Controls the amount of data retrieved in each API call. Default: 100

     Range	Recommended Value
     10-100	100

   • Max Page Calls Per Run: Upper limit on the number of paginated API requests per collection run. Protects against API rate limits. Default: 10

     Example	Notes
     10	Adjust based on data volume and rate limit constraints

   • Lua Script: The collection logic script that handles API interactions, pagination, and data emission. This is pre-populated with the default collection script.

   Scaling Parameters:

   • The Number of Runtimes to Utilize: Controls parallel execution capacity. Default: 2

   • The Number of Fetchers to Utilize: Number of concurrent HTTP request handlers. Default: 8

   • Fetch Request Queue Depth: Buffer size for outgoing requests. Default: 1000

   • Fetch Response Queue Depth: Buffer size for incoming responses. Default: 10

   • Fetch Completion Queue Depth: Buffer for completed fetch operations. Default: 10

   • Event Batch Queue Depth: Buffer for batched events. Default: 10

   • Stream Queue Depth: Data stream buffer size. Default: 64

   • Streams Per Fetcher: Concurrent streams per fetcher instance. Default: 1

   Retry Configuration:

   • The Amount of Time to Wait Between Two Consecutive Collections in Seconds: Initial backoff delay before retry. Default: 1

   • The Maximum Delay Between Retries in Seconds: Maximum backoff delay cap. Default: 5

   • The Maximum Delay Between Retries: Maximum number of retry attempts. Default: 4

   • The Backoff Factor by Which the Delay Increases After Each Retry: Exponential backoff multiplier. Default: 2.0

   Start Routine:

   • Start Routine to Be Executed at the Beginning of the Collection Cycle: Function name in Lua script. Default: start

     Example
     start

5. Save and Test:

   • Click "Save" to store the configuration settings in Observo AI.

   • Verify the source is appearing in the Destinations list.

Detailed Steps to Create a Pipeline with Snyk OCSF Serializer in Observo

Step 1: Access the Pipelines Section

1. Log into your Observo account.

2. Navigate to the “Pipelines” section from the left-hand panel.

Step 2: Create a New Pipeline

1. Click on “Add Pipeline”, add a new pipeline.

2. In Add Source section, select the Snyk API collector source previously configured.

3. In the Destination section, select the destination for these logs to be ingested.

Step 3: Add OCSF Serializer and Other Transforms

1. Click on the + sign to add transforms.

2. Select and add transforms to optimize and enrich your data. Select Serializer -> OCSF serializer

3. Add a name and Select the Synk(1.3.0) from the serializer dropdown.

Step 4: Deploy the Pipeline

1. After configuring the pipeline (source, transforms, OCSF serializer, and destination), click “Deploy Pipeline”.

2. Monitor logs and statuses to ensure data flows without issues.

Troubleshooting

• Verify Configuration Settings:

  • Ensure the Entity ID is correctly entered and matches the format: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx.

  • Confirm Entity Type (Organization or Group) matches the Entity ID provided.

  • Verify API Base URL is set to https://api.snyk.io (or your custom endpoint if applicable).

• Check Authentication:

  • Verify the API token is valid and not expired. Snyk API tokens do not expire by default but can be revoked.

  • Ensure the API token has read access to issues within the specified organization or group.

  • Check Observo AI logs for authentication failure messages.

  • Verify the token was correctly copied without extra spaces or characters.

• Validate Network Connectivity:

  • Test connectivity to the Snyk API endpoint using tools like curl or wget.

  • Ensure firewall rules allow outbound HTTPS connections to api.snyk.io on port 443. Verify DNS resolution for the endpoint.

  • Verify DNS resolution for api.snyk.io.

  • Ensure proxy settings, if applicable, are configured correctly.

• Rate Limiting Issues:

  • Monitor for HTTP 429 "Rate limit exceeded" errors in logs.

  • Reduce Max Page Calls Per Run if rate limits are being hit frequently.

  • Increase execution interval (Time in Seconds to Pause Between Script Executions) to spread API calls over time.

  • Review your Snyk subscription plan's API rate limits.

  • Check Snyk's rate limit headers in API responses for quota information.

• Monitor Logs and Data:

  • Check the source connection status in Observo AI to verify active data collection.

  • Monitor the Analytics tab in the targeted Observo AI pipeline for data volume and throughput.

Issue	Possible Cause	Resolution
No data collected	Invalid Entity ID or authentication	Verify Entity ID format and API token validity
Authentication errors	Invalid or revoked API token	Check token in Snyk settings and update in Observo AI
"Entity ID must be provided"	Missing Entity ID configuration	Enter valid UUID in Entity ID field
"Authentication failed"	Invalid token or insufficient permissions	Verify token has read access to specified entity
"Access forbidden"	Token lacks permissions	Check token permissions in Snyk admin console
"Bad request" (400)	Invalid Entity ID or parameters	Verify Entity ID format and API version settings

Resources

• Snyk API Documentation

• Snyk REST API Reference

• Snyk Organizations and Groups

• Snyk API Rate Limiting