New Relic

The New Relic destination facilitates the forwarding of observability data (logs, metrics, traces) to New Relic for robust monitoring and analysis. This integration allows users to leverage New Relic's powerful analytics capabilities, gaining insights into system performance and user behavior. Configuring the New Relic destination ensures that data flows seamlessly and securely from your observability pipeline.

Purpose

The Observo AI New Relic destination enables users to send observability data, including logs, metrics, and traces, to New Relic for comprehensive monitoring and analysis. This integration leverages New Relic’s analytics capabilities to provide insights into system performance and user behavior. It ensures seamless and secure data flow from your observability pipeline to New Relic’s platform for centralized monitoring.

Prerequisites

Before configuring the New Relic destination in Observo AI, ensure the following requirements are met to facilitate seamless data ingestion:

  • New Relic Account:

    • Create a New Relic account if one does not already exist. This account serves as the hub for your observability data.

    • Ensure the account is active and configured to accept data via the Events, Logs, or Metrics API..

    • Note the Account ID and region (e.g., US, EU) for your New Relic account.

  • License Key:

    • Generate a New Relic License Key (or Ingestion Key) in the New Relic platform to authenticate data ingestion.

    • Navigate to API Keys in the New Relic UI, create a new key, and securely store its value.

    • Ensure the License Key has permissions to send data to New Relic.

  • Network Access:

    • Verify that the Observo AI instance can communicate with the New Relic API endpoints (e.g., metric-api.newrelic.com, log-api.newrelic.com)

    • Check for firewall rules or network policies that may block outbound HTTPS traffic to New Relic’s endpoints on port 443.

  • New Relic Tags (Optional):

    • Prepare tags for organizing data in New Relic (e.g., env:production, service:webapp).

    • Tags can be configured in Observo AI to align with New Relic’s tagging conventions.

Prerequisite
Description
Notes

New Relic Account

Hub for observability data

Must be active with Account ID and region

License Key

Authenticates data ingestion

Securely store the License Key

Network Access

Enables communication with New Relic

Ensure HTTPS connectivity to New Relic endpoints

New Relic Tags

Organizes data in New Relic

Optional, but recommended for filtering

Integration

The Integration section outlines default configurations for the New Relic destination. To configure New Relic as a destination in Observo AI, follow these steps:

  1. Log in to Observo AI:

    • Navigate to the Destinations tab.

    • Click the Add Destinations button and select Create New.

    • Choose New Relic from the list of available destinations to begin configuration.

  2. General Settings:

    • Name: Add a unique identifier, such as new-relic-observability-1.

    • Description (Optional): Provide a description, such as “Sends logs and metrics to New Relic.”

    • Account ID: Enter the New Relic Account ID

      Example

      4198765

    • License Key (Optional): A key that authenticates requests to the New Relic API. This key should be kept secure to prevent unauthorized access to your account. Refer New Relic API Keys for more information on this.

      Examples

      ${NEW_RELIC_LICENSE_KEY_ENV_VAR}

      ea1b2c3d4ef2cec871ab94901812d0fdFABCDEF

    • API Endpoint: Select the appropriate endpoint. Default: Events API.

      Options
      Description

      Events API

      for sending event data

      Logs API

      for sending log data

      Metrics API

      for sending metrics data

    • Region: Select the region (US, EU). Default: US

      Regions
      Description

      US

      European data centers

      EU

      United States data centers

  3. Batching Configuration:

    • Batch Max Bytes: Specify the maximum number of bytes to include in a batch. The size of an event is computed prior to compression. Once this limit is reached, the batch is flushed to New Relic. Default: 1000000

    • Batch Max Events: Specify the maximum number of events to include in a batch. Once this limit is reached, the batch is flushed to New Relic. Default: 100

    • Batch Timeout Secs: Specify the maximum number of seconds to wait before sending a batch to New Relic. Default: 1

  4. Encoding:

    • Fields to exclude: Specify a list of fields to exclude when encoding an event.

      Examples

      Message.key

      log.message

    • Timestamp Format: Format used for timestamp fields.

      Options
      Description
      Example

      RFC 3339

      ISO 8601 date/time format with timezone.

      2024-10-12T14:35:00Z

      Unix Time

      Seconds since Jan 1, 1970 (UTC)

      1634059200

  5. Buffering Configuration:

    • Buffer Type (Empty): Specifies the buffering mechanism for event delivery.

      Options
      Description

      Memory

      High-Performance, in-memory buffering Max Events: The maximum number of events allowed in the buffer. Default: 500 When Full: Event handling behavior when a buffer is full. Default: Block - Block: Wait for free space in the buffer.This applies backpressure up the topology, signalling that sources should slow down the acceptance/consumption of events. This means that while no data is lost, data will pile up at the edge. - Drop Newest: Drop the event instead of waiting for free space in the buffer. The event will be intentionally dropped. This mode is typically used when performance is the highest priority, and it is preferable to temporarily lose events rather than cause a slowdown in the acceptance/consumption of events.

      Disk

      Lower-Performance, Less-costly, on disk buffering Max Bytes Size: The maximum number of bytes size allowed in the buffer. Must be at-least 268435488 When Full: Event handling behavior when a buffer is full. Default: Block - Block: Wait for free space in the buffer. This applies backpressure up the topology, signalling that sources should slow down the acceptance/consumption of events. This means that while no data is lost, data will pile up at the edge. - Drop Newest: Drop the event instead of waiting for free space in the buffer. The event will be intentionally dropped. This mode is typically used when performance is the highest priority, and it is preferable to temporarily lose events rather than cause a slowdown in the acceptance/consumption of events.

  6. Advanced Settings:

    • Compression: Select the compression algorithm. Default: GZIP.

      Options
      Description

      GZIP

      Widely used DEFLATE-based compression format

      None

      No compression applied to data

      ZLIB

      DEFLATE-based, lightweight compression library

      ZSTD

      Fast, high-ratio compression by Facebook

    • Enable Proxy: (False) Defines whether to use a proxy to connect to New Relic. If set to true, the proxy settings must be configured. If enabled:

      • Proxy HTTP Endpoint: Specify the HTTP proxy endpoint.

        Example

        http://proxy.example.com:8080

      • Proxy HTTPS Endpoint: Specify the HTTPS proxy endpoint.

        Example

        https://proxy.example.com:8080

      • Proxy Bypass List (Add as needed): Hosts to avoid connecting through the proxy.

        Example

        https://proxy.example.com:8080

  7. Save and Test Configuration:

    • Save the configuration settings in Observo AI.

    • Send sample web data and verify that it appears in the New Relic ui.

Example Scenario

To illustrate the New Relic destination’s functionality, consider a scenario where you configure Observo AI to send application logs to New Relic for monitoring a web application:.

Standard New Relic Destination Setup

Here is a standard New Relic Destination configuration example. Only the required sections and their associated field updates are displayed in the table below:

General Settings

Field
Value
Description

Name

new-relic-webapp-logs

Unique identifier for the destination.

Description

Send application logs to New Relic for monitoring a web application

Provides context for the destination's purpose.

Account ID

Account ID

Specify the New Relic account ID.

License Key

Your New Relic License Key

Specifies the New Relic license key.

Pipeline

web-event-pipeline

Specifies the New Relic ingest pipeline for processing incoming data.

API Endpoint

Logs API

Select the New Relic API endpoint to send data to.

Region

US

Select the New Relic Region to send data to.

Batching Configuration

Field
Value
Description

Batch Max Bytes

1048576 ( 1 MB) To optimize data delivery.

Specify the maximum number of bytes to include in a batch. The size of an event is computed prior to compression. Once this limit is reached, the batch is flushed to New Relic.

Batch Max Events

1000 Balance throughput and performance.

Specify the maximum number of events to include in a batch. Once this limit is reached, the batch is flushed to New Relic.

Batch Timeout Secs

5 secondsTo ensure timely delivery.

Specify the maximum number of seconds to wait before sending a batch to New Relic.

Encoding

Field
Value
Description

Fields to Exclude from Serialization

Specify fields to omit (e.g., message.key, log.message).

Specify a list of fields to exclude when encoding an event.

Encoding Timestamp Format

RFC 3339

Format used for timestamp fields.

Buffering Configuration

Field
Value
Description

Buffer Type

Memory

Specifies High-Performance, in-memory buffering mechanism for event delivery.

Max Events

1000

The maximum number of events allowed in the buffer.

When Full

Block

Event handling behavior when a buffer is full. Wait for free space in the buffer and apply backpressure.

Advanced Settings

Field
Value
Description

Compression

GZIP

Use GZIP compression algorithm

Enable Proxy

False

Disable traffic routing through a proxy.

Test Configuration:

  • Save settings, send sample data, verify ingestion in New Relic.

  • Saves configuration, tests data flow, and confirms data appears in New Relic with the correct pipeline processing.

Notes:

  • Confirm HTTPS connectivity and check firewall rules to allow traffic from Observo AI to the New Relic endpoint.

  • Monitor Observo’s Notifications tab and New Relic’s interface to verify data ingestion and troubleshoot errors.

  • Ensure the event_id field is present in the data to serve as the unique identifier, and the host field is correctly excluded from the payload.

Troubleshooting

If issues arise with the New Relic destination, use the following steps to diagnose and resolve them:

  • Verify Configuration Settings:

    • Ensure all fields, such as Account ID, License Key, API Endpoint, and Region, are correctly entered and match the New Relic account configuration.

    • Confirm that the API Endpoint (e.g., Logs API) and Region (e.g., US) align with your data type and account.

  • Check Authentication:

    • Verify that the License Key is valid and has not been revoked or expired.

    • Regenerate the License Key in New Relic if necessary and update the Observo AI configuration.

  • Monitor Logs and Traces:

    • Check Observo AI logs for errors or warnings related to data transmission to New Relic.

    • In the New Relic platform, navigate to Metrics Explorer, Logs, or APM to confirm that data is arriving with the expected tags.

  • Validate Network Connectivity:

    • Ensure that the Observo AI instance can reach the New Relic endpoint (e.g., log-api.newrelic.com).

    • Check for firewall rules or network policies blocking HTTPS traffic on port 443.

    • If proxy is enabled, verify proxy settings (hostname, port, credentials) and ensure HTTPS traffic on port 443 is allowed.

  • Test Data Flow:

    • Send sample data through Observo AI and monitor its arrival in New Relic’s platform.

    • Use the Analytics tab in the targeted Observo AI pipeline to monitor data volume and ensure expected throughput.

  • Check Quotas and Limits:

    • Verify that the New Relic account is not hitting data ingestion limits or quotas (refer to New Relic documentation).

    • Adjust batching settings such as Batch Max Bytes, Batch Max Events if backpressure or slow data transfer occurs.

Issue
Possible Cause
Resolution

Data not appearing in New Relic

Incorrect Account ID, License Key, or API Endpoint

Verify Account ID, License Key, and API Endpoint

Authentication errors

Expired or invalid License Key

Regenerate License Key and update configuration

Connection failures

Network, firewall, or proxy issues

Check network policies, proxy settings and HTTPS connectivity

Slow trace transfer

Backpressure or rate limiting

Adjust batching settings or check New Relic quotas

Resources

For additional guidance and detailed information, refer to the following resources:

PreviousLokiNextSocket

Last updated

Was this helpful?