The Carbon Black Cloud Forwarder lets you send data about alerts and events to an AWS S3 bucket where it can be reconfigured to port into other applications in your security stack, such as Splunk. The Event Forwarder is recommended over APIs for obtaining large amounts of data from Carbon Black Cloud in real time.
Watch the video tutorial or follow the step-by-step guide for enabling the Carbon Black Cloud Event Forwarder using Postman. The guide walks you through the following:
If you already have an AWS S3 bucket or are looking for additional ways to configure your bucket for varying levels of access and other use-cases, see here.
Currently the Carbon Black Cloud Event Forwarder only supports output to AWS S3 buckets provided by customers. The Event Forwarder outputs gzip compressed single-event-per-line JSON format (sometimes known as “JSONL”) to the customer’s S3 bucket with the following folder structure:
org_key={org_key}/year={year}/month={month}/day={day}/hour={hour}/minute={minute}/second={second}/{uuid}.jsonl.gz
Therefore, if a batch of events occurred on August 27, 2019 at 12:36:53UTC for the organization withorg_key
ABCD1234 can be located in:
org_key=ABCD1234/year=2019/month=8/day=27/hour=12/minute=36/second=53/ac203140-f0e2-48fe-90cf-05eb7289f628.jsonl.gz
You can find the Event and Alert schema as well as sample data in the Data Mapping Guide.
EDR provides tremendous visibility into everything happening across your endpoints. This data may be prohibitively expensive in licensing or scale for some downstream SIEM, SOAR, or data lake solutions. To reduce the load, one or more filters can be applied to an event forwarder configuration.
Data Filtering is available for forwarders of type “endpoint.event”.
If you want to remove or modify an existing filter then edit the configuration with the desired filter change. To remove all filters for a configuration set the filter property to an empty array.
Field | Definition | Data Type | Values |
---|---|---|---|
attribute |
The field from the endpoint.event to filter. | String | event_origin , type , alert_id , or sensor_action |
equals |
Filters the endpoint.events based on the attribute equalling the specified value. | String | See below |
not_equals |
Filters the endpoint.events based on the attribute not equalling the specified value. | String | See below |
match_any_bits |
Filters the endpoint.events based on the attribute equalling one of the specified values in the array. | Array | See below |
The following values are supported based on what attribute is provided and the filters that are supported.
Attribute | Values | Filter |
---|---|---|
event_origin |
NGAV , EDR |
equals , not_equals , match_any_bits |
type |
endpoint.event.procstart , endpoint.event.procend , endpoint.event.apicall , endpoint.event.netconn , endpoint.event.netconn_proxy , endpoint.event.filemod , endpoint.event.regmod , endpoint.event.moduleload , endpoint.event.crossproc , endpoint.event.scriptload , endpoint.event.fileless_scriptload |
equals , not_equals , match_any_bits |
alert_id |
empty string "" |
not_equals |
sensor_action |
ACTION_ALLOW , ACTION_BLOCK , ACTION_BREAK , ACTION_SUSPEND , ACTION_TERMINATE |
equals , not_equals , match_any_bits |
Use the following information for authentication, and see the Carbon Black Cloud Authentication Guide for full instructions.
Create, Read, Update, Delete
(or see each call below for individual requirements)event_forwarder_config/v1/orgs/{org_key}/configs
The following API calls can be used for either “endpoint.event” or “alert” forwarder types.
Use this call to create a new forwarder. The API will then make calls to check whether the Forwarder can write to the specified S3 Bucket using the configuration supplied. It will attempt to write a test message called healthcheck.json
to the specified bucket under a subfolder called healthcheck
. If the bucket is misconfigured (i.e. incorrect permissions, principle arn, etc.) or the configuration is incorrect (i.e. bucket prefix doesn’t match path specified in policy), the API will respond with a 400
error and message with information about what was incorrect and how to fix the issue.
If you want to forward both alert type data and endpoint.event type data, you should create a separate forwarder for each. The forwarder should be configured to send the data to its own subfolder in the S3 bucket using the S3 prefix property. The subfolder you configure will be automatically added to the S3 bucket.
RBAC Permissions Required
Permission (.notation name) | Operation(s) |
---|---|
event-forwarder.settings |
CREATE |
Request
POST <cbc-hostname>/event_forwarder_config/v1/orgs/{org_key}/configs
Request Body
Required fields: Name, s3_bucket_name, s3_prefix, type
{
"name": "<string>",
"s3_bucket_name": "<string>",
"s3_prefix": "<string>",
"type": "<string>",
"enabled": "<boolean>",
"filters": [{
"attribute": "<string>",
"equals": "<string>",
"not_equals": "<string>",
"match_any_bits": [
"<string>",
"<string>"
]
}]
}
Body Schema
Field | Definition | Data Type | Values |
---|---|---|---|
name REQUIRED
|
Defined name for the specific event or alert forwarder | String | N/A |
s3_bucket_name REQUIRED
|
Configured unique name for s3 bucket | String | N/A |
s3_prefix REQUIRED
|
Defined folder structure the forwarder will write events or alerts to | String | N/A |
type REQUIRED
|
The datastream type that is to be forwarded. | Sting | endpoint.event , alert |
enabled |
Boolean flag inidicating whether to create the event forwarder as enabled | Boolean | true, false |
filters |
A list of filters to apply to the data being forwarded. Use only one of equals , not_equals , or match_any_bits per filter in the list. Only supported when type equals endpoint.event |
Array |
|
Response
Use the following to troubleshoot errors. For more troubleshooting tips, see below.
Code | Description | Content-Type | Content |
---|---|---|---|
200 | Successful creation of forwarder | application/json | View example response below |
400 | Bad request. The JSON body was malformed, or the configuration is invalid (misconfigured bucket or configuration value) | application/json | View example response below |
404 | Resource not found | N/A | N/A |
500 | Internal server error | N/A | N/A |
Example
Request
POST https://defense-eap01.conferdeploy.net/event_forwarder_config/v1/orgs/ABCD1234/configs
Request Body
{
"name": "DevRelTest",
"s3_bucket_name": "DevRelTestBucket",
"s3_prefix": "test",
"type": "endpoint.event",
"enabled": false,
"filters": [
{
"attribute": "type",
"match_any_bits": [
"endpoint.event.netconn",
"endpoint.event.procstart"
]
},
{
"attribute": "event_origin",
"equals": "EDR"
}
]
}
Response for valid configuration
{
"id": "58aa259f-f42c-11e9-bd5d-2eca45f7dc1c",
"org_key": "ABC123",
"name": "DevRelTest",
"enabled": false,
"s3_bucket_name": "DevRelTestBucket",
"s3_prefix": "test",
"type": "endpoint.event",
"filters": [
{
"attribute": "type",
"match_any_bits": [
"endpoint.event.netconn",
"endpoint.event.procstart"
]
},
{
"attribute": "event_origin",
"equals": "EDR"
}
],
"create_time": "2019-10-21T17:58:11Z",
"update_time": "2019-10-21T17:58:11Z"
}
Response for invalid configuration
If you receive an error indicating your configuration is invalid, either NoSuchBucket
or AccessDenied
, see the Troubleshooting Errors section below.
This call can be used to run a healthcheck on an Forwarder to determine if the forwarder is able to write to the specified S3 Bucket using the existing configuration. The healthcheck can indicate if the bucket is misconfigured (i.e. incorrect permissions, principle arn, etc.), if the configuration is incorrect (i.e. bucket prefix doesn’t match path specified in policy), or if the forwarder is working as expected. If successful, the healthcheck will attempt to write a test message called healthcheck.json
to your S3 bucket under a subfolder called healthcheck
and respond with a 200
, regardless of whether the API was able to write to the bucket.
RBAC Permissions Required
Permission (.notation name) | Operation(s) |
---|---|
event-forwarder.settings |
READ |
Request
GET <cbc-hostname>/event_forwarder_config/v1/orgs/{org_key}/configs/{config_id}/health_check
Response
Use the following to troubleshoot errors. For more troubleshooting tips, see below.
Code | Description | Content-Type | Content |
---|---|---|---|
200 | Healthcheck completed successfully | application/json | View example response below |
404 | Resource not found | N/A | N/A |
500 | Internal server error | N/A | N/A |
Example
Request
GET
https://defense-eap01.conferdeploy.net/event_forwarder_config/v1/orgs/ABCD1234/configs/57b09141-f4e8-11e9-83de-22656feed3f2/health_check
Successful Response
Response for a successful healthcheck that was able to write to the S3 bucket using the specified configuration:
{
"id": "57b09141-f4e8-11e9-83de-22656feed3f2",
"enabled": true,
"update_time": "2019-10-22T16:59:41Z",
"status": "Valid Bucket Configuration for Bucket: DevRelTestBucket with Prefix: test",
"error": ""
}
Successful Response with Error Status
Example response for successful healthcheck that was unable to write to the S3 bucket using the specified configuration:
{
"id": "57b09141-f4e8-11e9-83de-22656feed3f2",
"enabled": true,
"update_time": "2019-10-22T16:59:41Z",
"status": "Invalid Bucket Configuration for Bucket: IncorrectBucket with Prefix: test",
"error": "AccessDenied"
}
If you received a NoSuchBucket
error or AccessDenied
error, see the Troubleshooting Errors section below for suggested fixes.
Use this call to delete a forwarder.
RBAC Permissions Required
Permission (.notation name) | Operation(s) |
---|---|
event-forwarder.settings |
DELETE |
Request
DELETE <cbc-hostname>/event_forwarder_config/v1/orgs/{org_key}/configs/{config_id}
Response
Use the following to troubleshoot errors. For more troubleshooting tips, see below.
Code | Description | Content-Type | Content |
---|---|---|---|
200 | Successful deletion of forwarder | application/json | View example response below |
400 | Resource not found | N/A | N/A |
500 | Internal server error | N/A | N/A |
Example
Request
DELETE https://defense-eap01.conferdeploy.net/event_forwarder_config/v1/orgs/ABCD1234/configs/58aa259f-f42c-11e9-bd5d-2eca45f7dc1c
Response
{}
Get all configured forwarders and their information
RBAC Permissions Required
Permission (.notation name) | Operation(s) |
---|---|
event-forwarder.settings |
READ |
Request
GET <cbc-hostname>/event_forwarder_config/v1/orgs/{org_key}/configs
Response
Use the following to troubleshoot errors. For more troubleshooting tips, see below.
Code | Description | Content-Type | Content |
---|---|---|---|
200 | Successful acquiring of all configured forwarders | application/json | View example response below |
400 | Resource not found | N/A | N/A |
500 | Internal server error | N/A | N/A |
Example
Request
GET https://defense-eap01.conferdeploy.net/event_forwarder_config/v1/orgs/ABCD1234/configs
Response
[{
"id": "57b09141-f4e8-11e9-83de-22656feed3f2",
"org_key": "ABCD1234",
"name": "DevRelTest",
"enabled": true,
"s3_bucket_name": "DevRelTestBucket",
"s3_prefix": "test",
"type": "endpoint.event",
"filters": [
{
"attribute": "type",
"match_any_bits": [
"endpoint.event.netconn",
"endpoint.event.procstart"
]
},
{
"attribute": "event_origin",
"equals": "EDR"
}
],
"create_time": "2019-10-22T16:23:55Z",
"update_time": "2019-10-22T16:59:41Z"
}]
This call is used to edit an existing forwarder. You can edit the Enabling/Disabling functionality or change the s3 bucket name.
When editing an existing forwarder, the API will make additional calls to check whether the Carbon Black Cloud event or alert forwarder can write to the specified S3 Bucket using the configuration supplied. It will attempt to write a test message called healthcheck.json
to the specified bucket under a subfolder called healthcheck
. If the bucket is misconfigured (i.e. incorrect permissions, principle arn, etc.) or the configuration is incorrect (i.e. bucket prefix doesn’t match path specified in policy or bucket does not exist), the Event Forwarder Configuration API will respond with a 400
and message including information regarding what was incorrect, providing the customer with feedback that enables them to fix issues as needed.
If you want to remove filters that are applied to your configuration then update the configuration with an empty array for the filters property.
RBAC Permissions Required
Permission (.notation name) | Operation(s) |
---|---|
event-forwarder.settings |
UPDATE |
Request
PUT <cbc-hostname>/event_forwarder_config/v1/orgs/{org_key}/configs/{config_id}
Request Body
Required fields: Name, s3_bucket_name, s3_prefix, type
{
"name": "<string>",
"s3_bucket_name": "<string>",
"s3_prefix": "<string>",
"type": "<string>",
"filters": [{
"attribute": "<string>",
"equals": "<string>",
"not_equals": "<string>",
"match_any_bits": [
"<string>",
"<string>"
]
}]
}
Body Schema
Field | Definition | Data Type | Values |
---|---|---|---|
name REQUIRED
|
Defined name for the specific event or alert forwarder | String | N/A |
s3_bucket_name REQUIRED
|
Configured unique name for s3 bucket | String | N/A |
s3_prefix REQUIRED
|
Defined folder structure the forwarder will write events or alerts to | String | N/A |
type REQUIRED
|
The datastream type that is to be forwarded. | Sting | endpoint.event , alert |
filters |
A list of filters to apply to the data being forwarded. Use only one of equals , not_equals , or match_any_bits per filter in the list. Only supported when type equals endpoint.event |
Array |
|
Response
Use the following to troubleshoot errors. For more troubleshooting tips, see below.
Code | Description | Content-Type | Content |
---|---|---|---|
200 | Successful editing of forwarder | application/json | View example response below |
400 | Bad request. The JSON body was malformed, or some part of the JSON body included an invalid value | application/json | View example response below |
404 | Resource not found | N/A | N/A |
500 | Internal server error | N/A | N/A |
Example
Request
PUT https://defense-eap01.conferdeploy.net/event_forwarder_config/v1/orgs/ABCD1234/configs/57b09141-f4e8-11e9-83de-22656feed3f2
Request Body
{
"name": "DevRelTest",
"s3_bucket_name": "DevRelTestBucket",
"s3_prefix": "test",
"type": "endpoint.event",
"filters": [
{
"attribute": "alert_id",
"not_equals": ""
}
],
"enabled": false
}
Response for valid configuration
{
"id": "57b09141-f4e8-11e9-83de-22656feed3f2",
"org_key": "ABCD1234",
"name": "DevRelTest",
"enabled": false,
"s3_bucket_name": "DevRelTestBucket",
"s3_prefix": "test",
"type": "endpoint.event",
"filters": [
{
"attribute": "alert_id",
"not_equals": ""
}
],
"create_time": "2019-10-22T16:23:55Z",
"update_time": "2019-10-22T16:26:01Z"
}
Response for invalid configuration
If you receive a response for invalid configuration, see the Troubleshooting Errors section below for suggested fixes.
Request for removing filters
PUT https://defense-eap01.conferdeploy.net/event_forwarder_config/v1/orgs/ABCD1234/configs/57b09141-f4e8-11e9-83de-22656feed3f2
Request Body
{
"name": "DevRelTest",
"s3_bucket_name": "DevRelTestBucket",
"s3_prefix": "test",
"type": "endpoint.event",
"filters": [],
"enabled": true
}
Response for removing filters
{
"id": "57b09141-f4e8-11e9-83de-22656feed3f2",
"org_key": "ABCD1234",
"name": "DevRelTest",
"enabled": true,
"s3_bucket_name": "DevRelTestBucket",
"s3_prefix": "test",
"type": "endpoint.event",
"filters": [],
"create_time": "2019-10-22T16:23:55Z",
"update_time": "2019-10-22T16:26:01Z"
}
Error Code | Reason For Error | Suggested Fix |
---|---|---|
NoSuchBucket |
Configuration is invalid. The specified S3 bucket name doesn’t exist | Follow the Quick Start Guide to create an S3 bucket for use by the CBC Event Forwarder |
AccessDenied |
Either: Bucket was created in the wrong region OR The policy is malformed | Ensure that your bucket is created in the same region as the where the Event Forwarder is deployed. Refer to the Event Forwarder and S3 Policy document for policy troubleshooting and configuration instructions |
400 |
The reason varied depending on the API call | Refer to the individual API call you are making to validate the JSON body |
401 |
Problem using the specified token to authenticate | Check that your X-Auth-Token matches the format secret_key/api_id and that the values are correct |
404 |
This could be due to a specified resource could not be found, an incorrect org key in the path, or incorrect permissions on the API key’s access level | Verify that you specified the correct values (path, org key, forwarder id, etc.) and your forwarder configuration is using the correct Authentication information |
Quick Setup in Postman & S3 Bucket Configuration
Carbon Black Cloud Authentication Guide
Writing a Bucket Policy for the CBC Event Forwarder
Carbon Black Cloud Event Forwarder Data Mapping
Deconstructing the Process GUID
CBC: What URLs are used to access the APIs?
.jsonl.gz
) files are supported.