Data Collection Process

Data Sources

Secberus ingests data from data sources. Supported data sources include cloud service providers such as AWS, Azure, and GCP, as well as SaaS solutions like Cloudflare and Okta. An individual instance, account, or subscription for one of these providers is referred to as a data source. An authenticated data source is used by the Secberus platform to collect provider information for analysis.

Authentication

Each Data Source provider has its own mechanisms to allow authentication from third-party applications.

  • AWS uses AWS access key for role-based authentication
  • Microsoft Azure allows third-party authentication through its Application Registrations.
  • Google Cloud allows authentication through its service accounts, which allows the account owner to specify which resources the service account is allowed to access.
  • Cloudflare authentication is provided via a generated token.
  • Okta authentication is provided via a generated token.

GCP

In order to create a GCP data source you will need to create a service account and key for that service account on your GCP platform. Using the service account credentials and the required permissions, GCP will allow read access to the resource configurations needed for auditing.

For details on creating a GCP data source, click here.

Azure

To create an Azure data source on the Secberus platform you’ll need to create an Application Registration in your Azure Active Directory. Using the Application Registration and the required permissions, Azure will allow read access to the resource configurations needed for auditing.

For detailed steps on creating an Azure data source click here.

AWS

AWS Provides two different ways of authentication:

Role-Based Authentication requests temporary credentials from AWS to perform read operations on the required resources for a policy.

AWS user credentials authorizes a user to perform read operations on the required resources for a policy.

For detailed steps on creating an AWS data source click here.

Cloudflare

To create a Cloudflare data source you need to generate an API token with the proper permissions.

For detailed steps on creating an Cloudflare data source click here.

Okta

To create an Okta data source you need to know your Okta domain and you need to generate an API token with the proper permissions.

For detailed steps on creating an Okta data source click here.

Collection

Once a data source is created, the data collection can be achieved using by two different processes.

Event-Based Collection

Event-Based Collection continuously monitors event logs that modify resources within the data source account. We parse these logs to extract which resources were altered and trigger a collection for the modified resources.

Time-Based Collection

Time-Based Collection schedules an entire data source account for collection at regular intervals. These intervals will trigger collections for all available resources in the specified data source.

Evaluation

Once the data is collected, we run the data against active managed and custom policies to determine if there are any violations.

If a violation has occurred, those violations will be added to the dashboard and any integrations previously configured will be triggered.