Carbon Black Cloud Enterprise EDR (Endpoint Detection and Response) is the new name for the product formerly called CB ThreatHunter.
The unified binary store (UBS) is a centralized service that is part of the Carbon Black Cloud. UBS is responsible for storing all binaries and corresponding metadata for those binaries. UBS comes packaged with Enterprise EDR. To obtain the maximum benefit of UBS, organizations must opt-in to binary uploads on the Policies page.
Enterprise EDR customers are able to utilize the UBS APIs to download binaries observed on their endpoints, as well as to retrieve the corresponding metadata. At this time, the data retention for UBS is infinite - so as long as an organization has ever observed execution of that binary on one or more endpoints, that organization will be able to retrieve the associated metadata (and optionally the binary itself, if binary uploads were opted in at the time of execution).
UBS obtains data on first execution of a binary for any device on which the Carbon Black Cloud sensor is running and Enterprise EDR has been subscribed - both metadata about the binary, and if uploads are enabled for the organization, the binary itself. This has two consequences:
For binaries less than 25 MB, the sensor uploads the entire file. For binaries over 25 MB, the sensor uploads the first 25 MB of the binary file.
At this time UBS is supported on the Windows CBC sensor, so all binaries available in UBS are Windows PE files.
UBS is a RESTful service, utilizing HTTP as the transport protocol. All routes to the UBS take the following format:
<cbc-hostName>/ubs/<versionId>/orgs/{org_key}/…
The following table explains the components of the above route:
Variable | Description | Example |
---|---|---|
cbc-hostName | The URL of the environment where your organization is hosted. | https://defense-prod05.conferdeploy.net |
versionId | The version of the UBS API that is being targeted. This document specifically documents v1. | v1 |
org_key | The organization key. | ABCD123 |
The org_key
may be obtained from the API Access page under Settings in the Carbon Black Cloud console.
POST routes must have the Content-Type header set to application/json.
All successful API calls will result in returning an HTTP 200 status code. The result will be returned in the body of the response. The specific responses will be detailed in subsequent sections, for each API.
Whenever an HTTP status code not equal to 200 is returned, the response will be as follows:
{
"message": <string>
"error_code": <string>
}
error_code
is an enum, with values that will be defined below.
This field may be used for programmatically handling errors.
message
is a short explanation as to why an error occurred.
This field is intended to be interpreted for debugging purposes, and not to be consumed programmatically.
The following table outlines the expected error HTTP status codes:
Status Code | Reason | error_code |
---|---|---|
400 | Invalid user input. Ensure that any SHA-256s are valid. | BAD_REQUEST |
403 | Authentication error. Ensure that both the API key andorg_key are correct. Check the response message field for a more detailed error. |
FORBIDDEN |
404 | Requested resource not found. | |
500 | Server error. This is typically a transient error and the request may be retried, ideally with an exponential backoff. | INTERNAL_SERVER_ERROR |
Unless otherwise stated, all dates/timestamps will be strings, formatted as ISO 8601 UTC, i.e. YYYY-MM-DDThh:mm:ss.sssZ
POST routes MUST have the conventional header Content-Type: application/json
or the API will return a 500 error.
These APIs are specifically for interacting with files stored within the UBS.
This API provides a means to download files. The files are able to be downloaded via AWS S3 pre-signed URLs. The URLs enables the client to perform a GET on the provided URLs and download the files. The links will automatically expire after one hour, unless a different expiration time is requested.
CAUTION - Anyone who has these links will be able to download the files until the requested expiration time (default one hour after the request). Ensure that care is taken when utilizing this API.
RBAC Permissions Required
Identity Manager | Permission (.notation name) | Operation(s) | Environment |
---|---|---|---|
Carbon Black Cloud | ubs.org.file |
READ |
Majority of environments |
VMware Cloud Services Platform | _API.Unified.Binary.Store:ubs.Org.File.read |
N/A - included in permission name | Prod UK |
The following table describes the inputs required to utilize this API.
Description | Value |
---|---|
Route | <cbc-hostname>/ubs/<versionId>/orgs/{org_key}/file/_download |
HTTP verb | POST |
The body for the request should be as follows:
{
"sha256": [
<string>,
...
],
"expiration_seconds": <int>
}
The following table explains the above data:
Item | Data Type | Description | Required |
---|---|---|---|
sha256 | array of strings | An array of SHA-256 hashes as ASCII hex strings, with a limit of 100. More than 100 items will result in an HTTP 400. | Yes |
expiration_seconds | integer | The number of seconds to make the download URLs available for. The default is 3600 (1 hour). | No |
The body for the response is as follows:
{
"found": [
{
"sha256": <string>,
"url": <string>
},
...
],
"not_found": [
<string>,
...
],
"error": [
<string>,
...
]
}
The following table explains the above response:
Item | Data Type | Description |
---|---|---|
found | array of JSON object | Details about the files that were found to be available for download |
found[ix].sha256 | string | SHA-256 hash of file that is available to be downloaded |
found[ix].url | string | An AWS S3 presigned URL for this file. Perform a GET on this URL to download the file. |
not_found | array of strings | The SHA-256 hashes that were not found. |
error | array of strings | The SHA-256 hashes that had an intermittent error. These hashes should be retried. |
Set AUTHENTICATION to your APIKEY/ConnectorID and the HOST, ORGID and TARGETSHA variables appropriately.
curl -XPOST https://"$HOST"/ubs/v1/orgs/"$ORGID"/file/_download -H 'Content-Type: application/json' -H "X-Auth-Token: $AUTHENTICATION" -d '{"sha256": ["87976f3430cc99bc939e0694247c0759961a49832b87218f4313d6fc0bc3a776"]}'
{"found":[
{"sha256":"87976f3430cc99bc939e0694247c0759961a49832b87218f4313d6fc0bc3a776",
"url":"https://cdc-file-storage-staging-us-east-1.s3.amazonaws.com/87/97/6f/34/30/cc/99/bc/93/9e/06/94/24/7c/07/59/96/1a/49/83/2b/87/21/8f/43/13/d6/fc/0b/c3/a7/76/87976f3430cc99bc939e0694247c0759961a49832b87218f4313d6fc0bc3a776.zip?AWSAccessKeyId=ASIAQZRW24JUWSSITSS4&Signature=FUVo3yW9DTj8iAZckynCCAXg10U%3D&x-amz-security-token=FQoGZXIvYXdzELn%2F%2F%2F%2F%2F%2F%2F%2F%2F%2FwEaDNnE2ZGKw4sKvDQgzCK3A27%2BmyKzV%2BQcYEdnqUhW8WJ6L9GbiIsDL0rv6uK46DCkdwvMPYDdBoWitDaSkb6KXzbOpItfPDnnpPAjswzdeh8nr4LHjkcaZUtYBBw2L1W33Ee0A%2F7RgY%2Fw75nWtgbnamxWWlNq9kapELBV6v1ohJzAiq%2FwRgZKCNeGSToT%2BR0RVjo5wJFFoKM7C4eVfZxIZyaKi2eA4gMdrmDhXvhBi%2Bphdk0ER1%2FkW8Q9XvtePjvypKCc%2BDwAXNA0BNa4pmeqmOQofUzzBD2FPdbuucR0W5LG7ajYCCrM1S5LcbatwTmqtZ8HbVB7ZNX%2BSxuyb4yMYNnBj22bldsjmUTuY6Ya%2FokZVuCRoTqi59PVB9D3PB%2FD4dk5RuRnQaCzz0TSnaYFTN9qznbWAhF%2FynUiyDC%2FozX2b%2BbpNcawixTSrnp76rEj8OLYm8Yrl9DqPs14Tw8WanULirzgtcOwvrMBioR0PBIYbInKNEhWhzq2D%2FdMkqyAxcasbR%2BFN7CVzXAGXjsFE3yVnyXz7CTQaOfowj1HNoj%2FA3dIdLZzSiaDD1nj6r5e81V7OphQFFfp%2BFB5Io82mwBuw%2FcH8jooj7nV4wU%3D&Expires=1551201116"}]
,"not_found":[],
"error":[]
}
This data is specific to the binary and is not a function of organization data.
This API returns all of the metadata for the specified binary identified by the SHA-256 hash.
RBAC Permissions Required
Identity Manager | Permission (.notation name) | Operation(s) | Environment |
---|---|---|---|
Carbon Black Cloud | ubs.org.sha256 |
READ |
Majority of environments |
VMware Cloud Services Platform | _API.Unified.Binary.Store:ubs.Org.Sha256.read |
N/A - included in permission name | Prod UK |
The following table describes the inputs required to utilize this API.
Description | Value |
---|---|
Route | <cbc-hostname>/ubs/<version_id>/orgs/{org_key}/sha256/<sha256>/metadata |
HTTP verb | GET |
sha256 | The requested SHA-256 hash to obtain metadata information. |
The body for the response is as follows:
{
"sha256": <string>,
"md5": <string>,
"file_available": <boolean>,
"available_file_size": <long>,
"file_size": <long>,
"os_type": <string>,
"architecture": [
<string>
],
"lang_id": <int>,
"charset_id": <int>,
"internal_name": <string>,
"product_name": <string>,
"company_name": <string>,
"copyright": <string>,
"trademark": <string>,
"file_description": <string>,
"file_version": <string>,
"comments": <string>,
"original_filename": <string>,
"product_description": <string>,
"product_version": <string>,
"private_build": <string>,
"special_build": <string>
}
The following table explains the above response:
Item | Data Type | Description |
---|---|---|
sha256 | string | The SHA-256 hash of the file. |
md5 | string | The MD5 hash of the file. |
file_available | boolean | If true, the file is available for download. |
available_file_size | long | The size of the file, that is available for download. If the file is unavailable the size will be zero. In the case of the null file, then the size will be zero and file_available will be true. Note that files may be truncated, before upload, which is why this number may be less that file_size . |
file_size | long | The size of the actual file. This is the size of the file represented by this hash. |
os_type | string | The OS that this file is designed for. This may contain one or more of the following values: WINDOWS, ANDROID, MAC, IOS, LINUX, and OTHER. |
architecture | array of string | The set of architectures that this file was compiled for. This may contain one or more of the following values: none, x86, amd64, and arm64. |
lang_id | unsigned int | The Language ID value from the Windows VERSIONINFO resource. See this link on MSDN for more information. |
charset_id | unsigned int | The Character set ID value from the Windows VERSIONINFO resource. See this link on MSDN for more information. |
internal_name | nullable string | If present, Internal name from FileVersionInformation |
product_name | nullable string | If present, Product name from FileVersionInformation |
company_name | nullable string | If present, Company name from FileVersionInformation |
copyright | nullable string | If present, Legal copyright from FileVersionInformation |
trademark | nullable string | If present, Legal trademark from FileVersionInformation |
file_description | nullable string | If present, File description from FileVersionInformation |
file_version | nullable string | If present, File version from FileVersionInformation |
comments | nullable string | If present, comments from FileVersionInformation |
original_filename | nullable string | If present, Original filename from FileVersionInformation |
product_description | nullable string | If present, Product description from FileVersionInformation |
product_version | nullable string | If present, Product version from FileVersionInformation |
private_build | nullable string | If present, Private build from FileVersionInformation |
special_build | nullable string | If present, Special build from FileVersionInformation |
Set the TARGETSHA, ORGID, and HOST envars appropriately.
curl -H "X-Auth-Token: $AUTHENTICATION" https://$HOST/ubs/v1/orgs/"$ORGID"/sha256/"$TARGETSHA"/metadata
{"sha256":"87976f3430cc99bc939e0694247c0759961a49832b87218f4313d6fc0bc3a776",
"architecture":["amd64"],
"available_file_size":96672,
"charset_id":1200,
"comments":null,
"company_name":null,
"copyright":"\u00a9 Microsoft Corporation. All rights reserved.",
"file_available":true,
"file_description":"Runtime Broker",
"file_size":96672,
"file_version":"10.0.16299.15 (WinBuild.160101.0800)",
"internal_name":"RuntimeBroker.exe",
"lang_id":1033,
"md5":"bd4401441a21bf1abce6404f4231db4d",
"original_filename":"RuntimeBroker.exe",
"os_type":"WINDOWS",
"private_build":null,
"product_description":null,
"product_name":"Microsoft\u00ae Windows\u00ae Operating System",
"product_version":"10.0.16299.15",
"special_build":null,
"trademark":null
}
These APIs return summarized information about a given SHA-256 hash for the target organization.
This API returns a summarized view of the device details. Use this API to get an overview of the devices that executed the file.
RBAC Permissions Required
Identity Manager | Permission (.notation name) | Operation(s) | Environment |
---|---|---|---|
Carbon Black Cloud | ubs.org.sha256 |
READ |
Majority of environments |
VMware Cloud Services Platform | _API.Unified.Binary.Store:ubs.Org.Sha256.read |
N/A - included in permission name | Prod UK |
The following table describes the inputs required to utilize this API.
Description | Value |
---|---|
Route | <cbc-hostname>/ubs/<versionId>/orgs/{org_key}/sha256/<sha256>/summary/device |
HTTP verb | GET |
sha256 | The requested SHA-256 hash to obtain information for. |
The body for the response is as follows:
{
"sha256": <string>,
"num_devices": <int>,
"first_seen_device_timestamp": <string>,
"first_seen_device_id": <int>,
"first_seen_device_name": <string>,
"last_seen_device_timestamp": <string>,
"last_seen_device_id": <int>,
"last_seen_device_name": <string>
}
The following table explains the above response:
Item | Data Type | Description |
---|---|---|
sha256 | string | The SHA-256 hash of the file. |
num_devices | long | The total number of devices, for this organization, that have observed this file. |
first_seen_device_timestamp | string | The time that this file was first seen, for this organization. |
first_seen_device_id | long | The Device ID of the device that first saw this file. |
first_seen_device_name | string | The name of the device that first saw this file. |
last_seen_device_timestamp | string | The time that this file was most recently seen, for this organization. |
last_seen_device_id | long | The Device ID of the device that most recently saw this file. |
last_seen_device_name | string | The name of the device that last saw this file. |
Set AUTHENTICATION, HOST, ORGID and TARGETSHA variables appropriately.
curl -H "X-Auth-Token: $AUTHENTICATION" https://"$HOST"/ubs/v1/orgs/"$ORGID"/sha256/"$TARGETSHA"/summary/device
{"sha256":"87976f3430cc99bc939e0694247c0759961a49832b87218f4313d6fc0bc3a776",
"num_devices":3,
"first_seen_device_timestamp":"2018-12-10T15:55:51.828771Z",
"first_seen_device_id":716428,
"first_seen_device_name":"CRAIG_WIN10_X64",
"last_seen_device_timestamp":"2018-12-28T16:49:30.185259Z",
"last_seen_device_id":787514,
"last_seen_device_name":"CRAIG_VM_WIN10_X64"}
This API will return a summary of the observed digital signature results for a given SHA-256 hash.
The digital signature information for a binary may vary from one machine to another based on a variety of factors,
including the presence of an up-to-date signature catalog on the host, system clock variations, ability to reach OCSP servers,
custom root trust anchors, and more. Therefore, the results are ordered by prevalence,
such that the most observed signatures will be returned first.
The number of results are configurable, up to a max of 100 entries.
Digital signatures can be recorded in a separate file (known as a “catalog” file) or embedded inside of the binary itself (an “embedded” signature).
This signature API will capture the results of the endpoint’s verification of the digital signature associated with a given SHA-256 hash,
stating whether that signature validation was based on a catalog file or an embedded signature. It is not uncommon for a catalog signature to be frequently updated, such that this API can return a wide variety of sign_timestamp
values for a given SHA-256 hash.
RBAC Permissions Required
Identity Manager | Permission (.notation name) | Operation(s) | Environment |
---|---|---|---|
Carbon Black Cloud | ubs.org.sha256 |
READ |
Majority of environments |
VMware Cloud Services Platform | _API.Unified.Binary.Store:ubs.Org.Sha256.read |
N/A - included in permission name | Prod UK |
The following table describes the inputs required to utilize this API.
Description | Value |
---|---|
Route | <cbc-hostname>/ubs/<versionId>/orgs/{org_key}/sha256/<sha256>/summary/signature?rows=<rows> |
HTTP verb | GET |
sha256 | The requested SHA-256 hash to obtain information for. |
rows | The number of results to return. The default is 5 and the max is 100. |
The body for the response is as follows:
{
"sha256": <string>,
"total_signatures_count": <int>,
"signatures_count": <int>,
"signatures": [
{
"count": <int>,
"is_catalog_signature": <boolean>,
"publisher_name": <string>,
"issuer_name": <string>,
"sign_timestamp": <string>,
"sign_states": [
{
"device_count": <int>,
"sign_state": [
<string>
],
"first_seen_timestamp": <string>
},
...
]
},
...
]
}
The following table explains the above response:
Item | Data Type | Description |
---|---|---|
sha256 | string | The SHA-256 hash of the file. |
total_signatures_count | long | The total number of signatures that have been observed by this organization for this file. This is the sum, over all devices that have executed this file, of the count of signatures for that file observed by each device. |
signatures_count | long | The total number of unique signatures that have been observed, by this organization, for this file. This field represents only unique occurrences, not total existence. Assuming that each device observed the same signature details for the file, then the following should be true: total_signatures_count = (number of devices that executed file) * signatures_count |
signatures | array of JSON objects | The signature details. The length of the array is limited based off the provided input parameter “rows” with a maximum of 100. |
signatures[ix].count | long | The total number of occurrences where this signature was observed, by this organization, for this file. This field is similar to total_signatures_count; however, it represents the count at a per signature-level instead of at the file-level. The sum of signatures[ix].count (over all signatures) should be equal to total_signatures_count. |
signatures[ix].is_catalog_signature | boolean | If true, then this signature is a catalog signature. |
signatures[ix].publisher_name | nullable string | The name of the company who published the file, as recorded in the digital certificate. |
signatures[ix].issuer_name | nullable string | The name of the company who issued the signature. This is the company who signed the certificate. This may be an intermediate or the root certificate authority. |
signatures[ix].sign_timestamp | nullable string | The date which the signature was issued. |
signatures[ix].sign_states | array of JSON object | The signature states for the signatures. The length of this array is unbounded and will be as big as needed to return all available signature states. |
signatures[ix].sign_states[ix2].device_count | long | The number of devices, for this organization, that observed this signature, for this file, with this specific signature result. |
signatures[ix].sign_states[ix2].sign_state | array of strings | The human readable signature states. This may contain one or more of the following values: invalid, signed, verified, not_signed, unknown, chained, trusted, os, and catalog_signed. Some of those values are mutually exclusive, e.g. not_signed and signed will never both appear in the same array. Important note - a state of signed does not mean that the signature is valid. Both signed and valid must be within the array to indicated that it is valid. A value of unknown is used when signature verification errors occur and is also the initial state of all files. A value of chained means that the signature chains to a locally trusted root certificate. A value of os means that the file is a component of the operating system. A value of catalog_signed means that the signature was in a catalog file instead of within the file. A valid of trusted means that the local machine trusts the signing chain. |
signatures[ix].sign_states[ix2].first_seen_timestamp | string | The date which a device first executed a file with this signature and signature state. |
curl -H "X-Auth-Token: $AUTHENTICATION" https://"$HOST"/ubs/v1/orgs/"$ORGID"/sha256/"$TARGETSHA"/summary/signature
{"sha256":"87976f3430cc99bc939e0694247c0759961a49832b87218f4313d6fc0bc3a776",
"total_signatures_count":51,
"signatures_count":1,
"signatures":[
{"count":51,
"is_catalog_signature":false,
"publisher_name":"Microsoft Windows",
"issuer_name":"Microsoft Windows Production PCA 2011",
"sign_timestamp":"2017-09-29T04:30:49.456000Z",
"sign_states":[{"device_count":51,
"sign_state":["signed","verified","trusted","os"],
"first_seen_timestamp":"2018-12-11T01:40:04.827586Z"}]
}]
}
This API will return a summary of the observed file paths. The results are ordered by prevalence, such that the most observed file path will be returned first. The number of results are configurable, up to a max of 100 entries.
The following table describes the inputs required to utilize this API.
Description | Value |
---|---|
Route | <cbc-hostname>/ubs/<versionId>/orgs/{org_key}/sha256/<sha256>/summary/file_path?rows=<rows> |
HTTP verb | GET |
sha256 | The requested SHA-256 hash to obtain information for. |
rows | The number of results to return. The default is 5 and the max is 100. |
RBAC Permissions Required
Identity Manager | Permission (.notation name) | Operation(s) | Environment |
---|---|---|---|
Carbon Black Cloud | ubs.org.sha256 |
READ |
Majority of environments |
VMware Cloud Services Platform | _API.Unified.Binary.Store:ubs.Org.Sha256.read |
N/A - included in permission name | Prod UK |
The body for the response is as follows:
{
"sha256": <string>,
"total_file_path_count": <int>,
"file_path_count": <int>,
"file_paths": [
{
"count": <int>,
"file_path": <string>,
"first_seen_timestamp": <string>
},
...
]
}
The following table explains the above response:
Item | Data Type | Description |
---|---|---|
sha256 | string | The SHA-256 hash of the file. |
total_file_path_count | long | The total number of file paths that have been observed, by this organization, for this file. |
file_path_count | long | The total number of unique file paths that have been observed, by this organization, for this file. |
file_paths | array of JSON objects | The file path details. The length of the array is limited based off the provided input parameter. |
file_paths[ix].count | long | The number of occurrences where this signature was observed, by this organization, for this file. |
file_paths[ix].file_path | string | The file path. Note that only the file path for the first execution for a particular device is captured, as such each device will only ever have a single file path. |
file_paths[ix].first_seen_timestamp | string | The time that this file path was first observed. |
Set the AUTHENTICATION, HOST, ORGID, and TARGETSHA variables appropriately.
curl -H "X-Auth-Token: $AUTHENTICATION" https://"$HOST"/ubs/v1/orgs/"$ORGID"/sha256/"$TARGETSHA"/summary/file_path
{"sha256":"87976f3430cc99bc939e0694247c0759961a49832b87218f4313d6fc0bc3a776",
"total_file_path_count":3,
"file_path_count":1,
"file_paths":[{"count":3,"file_path":"c:\\windows\\system32\\runtimebroker.exe",
"first_seen_timestamp":"2018-12-10T15:55:51.828771Z"}]}