Data Security

Data at Rest

Observo implements the following strategies to secure data at rest:

Manager

The Manager securely stores a wide array of control plane data for all managed sites. This encompasses user settings, source and destination configurations, pipeline configurations, as well as internal service logs and metrics. Additionally, the Manager retains log patterns and optionally stores log snippets for enhanced functionality.

• Database: The Manager employs encryption to safeguard configurations for sources, transforms and destinations, as well as any other files containing sensitive information.

• Authentication: User authentication is managed by a third-party authentication provider, which supports both social logins and traditional email/password methods. In this setup, user information is handled by the third-party provider.

Additionally, we offer integration with customer-specific Single Sign-On (SSO) providers. When configured, the SSO provider handles authentication instead of the third-party service, with authentication requests being redirected accordingly.

We also support local authentication, where user emails and securely hashed passwords are stored within the Manager's database.

Data ingested from sources and written to destinations are not stored within the Manager cluster. However, some data may be retained for use for log patterns and log preview functionality.

• Patterns: Patterns emitted from sources are stored in the Manager, and the Site sends this data to the Manager via API calls. This feature is optional and enabled only if selected by the user.

• Log Preview: When users opt to preview logs from their pipelines, the Site captures a small snippet, typically 100 lines, of log data from the source and stores it in the Manager for log preview functionality.

No data leaves the customer’s network other than those described above.

For private cloud deployment, no data of any kind leaves the customer's network, and Observo does not store any data.

Data in Transit

To ensure secure communication while data is in transit, Observo implements the following measures:

Manager

The Manager adopts a passive role in communication and does not initiate contact with the Site. Instead, the Site always initiates requests to the Manager through the API gateway. The API gateway authenticates and authorizes all incoming requests. Furthermore, all incoming and outgoing communication is consistently encrypted using TLS/SSL protocols.

Site

The Site communicates with the Manager through authenticated API requests, with each Site possessing a periodically rotated token for authentication. This token is utilized to ensure the validity and authentication of requests made to the Manager.

Furthermore, log data traversing through the Site within a pipeline, from the user’s source to destination, can be encrypted if the user opts to configure the sources and destinations with TLS settings. This encryption extends to both ingested data and data transmitted to destinations, offering an additional layer of security.

Moreover, the Site has the capability to authenticate with the source using mutual TLS, in addition to the standard API key, token, or other configurations required for authentication.

For private cloud deployment, no data of any kind leaves the customer's network, and Observo does not store any data.

Data Retention

Private Cloud Deployment

In a private cloud deployment, no data ever leaves the customer's network, and no data related to the customer is stored by Observo at any time.

Hybrid Deployment

In a hybrid deployment, the management plane (also known as the control plane) is hosted and managed by Observo, while the site is deployed within the customer’s network. The site communicates with the control plane to manage configuration and operations. The following data is securely stored on Observo, encrypted using FIPS-compliant encryption:

• Source, Sink, and Transform Configuration Files: These files contain metadata related to pipeline configurations. Retention: Data is retained while the site remains active.

• Log Patterns and Insights: These consist of patterns and metrics for each log source. As telemetry data is dynamic, these patterns evolve over time. Observo only stores this data if configured by the customer. Retention: Data is retained while the site remains active.

• Log Snippets: These are stored only when the customer uses the "Log Preview" feature for a given source. Retention: Data is stored for 30 days.

• System-Level Metrics and Logs: These are stored to provide analytics and operational insights.

All metadata, patterns, and log snippets stored by Observo are encrypted using FIPS-compliant encryption. No other data is stored by Observo, and log data processing occurs locally within the customer's site.

Archival & Hydration

Hydration refers to the process of replaying source logs that are stored on your side within a configured pipeline. This process is only activated when you initiate it, ensuring full control. Importantly, Observo does not retain any data from the hydration process or the original source from which it is replayed, maintaining your data's privacy and security.

Observo also provides a powerful Archival feature that allows users to store raw source data in a configured destination, such as AWS S3, Google Cloud Storage (GCS), or Azure Blob Storage. When a user sets up an archival destination, all source data is automatically sent to the specified location for safekeeping.

Archival works seamlessly with Hydration, enabling efficient data retrieval and processing. Hydration jobs pull data from the configured archival destinations and import it into customer data-plane for processing.

\