Unified Binary Store REST API Reference

Carbon Black Cloud Enterprise EDR (Endpoint Detection and Response) is the new name for the product formerly called CB ThreatHunter.

Version: v1

Overview

The unified binary store (UBS) is a centralized service that is part of the Carbon Black Cloud. UBS is responsible for storing all binaries and corresponding metadata for those binaries. UBS is included with all Carbon Black Cloud subscriptions that include Enterprise EDR. To obtain the maximum benefit of UBS, organizations must opt-in to binary uploads in each of their Policies used to configure Windows assets.

Enterprise EDR customers are able to utilize the UBS APIs to download previously-uploaded binaries observed on their endpoints, as well as to retrieve the corresponding metadata.

For all API endpoints except Summary API, UBS data retention (binaries and metadata) for each binary is guaranteed for a year from the time the first execution of that binary was reported on any endpoint in their organization. This is true for all org-wide data from the File, Metadata, Signature and File Path API endpoints.

These UBS APIs return metadata for retained binaries, as well as the binary itself from the File API endpoint (if the binary was executed on any asset whose Policy enabled the “Upload new binaries and their metadata to Carbon Black for later analysis and download” policy setting).

This data will be further retained if, after 12 months since the first observed execution, the binary is executed on any endpoint in the previous 3 months. Otherwise, binaries and their metadata for such “older” binaries will no longer be available from the UBS API (or the Binary Details console page that visualises this API). Once this cutoff occurs, the data and binaries are subject to deletion, and any subsequent executions will start fresh.

The UBS Summary API has more narrow retention constraints: it summarises per-device metadata for any binary executions observed in the past 3 months. For example, if a binary was executed on “Asset A” 6 months ago and 5 months ago, and was also executed on “Asset B” 5 months ago, 2 months ago and 1 month ago the Summary API would report for that binary: first_seen_device_timestamp = 2 months ago, last_seen_device_timestamp = 1 month ago and num_devices = 1.

UBS obtains data on first execution of a binary for any device on which the Carbon Black Cloud Windows sensor is running, for any organization subscribed to Enterprise EDR. This data includes metadata about the binary, and if uploads are enabled for the affected assets' Policy, the binary itself. This has two consequences:

  1. If the binary did not successfully execute, no metadata or binary will be uploaded to the Carbon Black Cloud, and UBS will not have any data related to that binary’s non-execution.
  2. If any metadata about the binary changes after initial execution (e.g. file path, signature state), UBS will not receive that updated binary metadata.

For binaries less than 25 MB, the sensor uploads the entire file. For binaries over 25 MB, the sensor uploads the first 25 MB of the binary file.

At this time UBS is supported on the Windows CBC sensor version 3.4 or later, which means that all binaries available in UBS are Windows PE files.

Authentication

Determine whether you use Carbon Black Cloud or VMware Cloud Services Platform to manage identity and authorization, or see the Carbon Black Cloud API Access Guide for complete instructions.


Carbon Black Cloud Managed Identity and Authentication
Customize your access to the Carbon Black Cloud APIs with Role-Based Access Control; All APIs and Services authenticate via API Keys. To access the data in Carbon Black Cloud via API, you must set up a key with the correct permissions for the calls you want to make and pass it in the HTTP Headers.

Environment
Available on majority of environments; Use the Carbon Black Cloud Console URL, as described here.

API Route
Replace the {cbc-hostname} and {org_key} with the URL of your Environment and the org_key for your specific Org.
  • Download: {cbc-hostname}/ubs/v1/orgs/{org_key}/file/_download
  • Hash information: {cbc-hostname}/ubs/v1/orgs/{org_key}/sha256/{sha256}

Access Level
Before you create your API Key, you need to create a "Custom" Access Level including each category:
  • ubs.org.file, allow permission to READ
  • ubs.org.sha256, allow permission to READ

API Key
When creating your API Key, use the Access Level Type of "Custom" and select the Access Level you created. Details on constructing and passing the API Key in your requests are available here.


Overall Response Details

POST routes must have the Content-Type header set to application/json.

Success Responses

All successful API calls will result in returning an HTTP 200 status code. The result will be returned in the body of the response. The specific responses will be detailed in subsequent sections, for each API.

Error Responses

Whenever an HTTP status code not equal to 200 is returned, the response will be as follows:

{
    "message": "<string>",
    "error_code": "<string>"
}

error_code is an enum, with values that will be defined below. This field may be used for programmatically handling errors. message is a short explanation as to why an error occurred. This field is intended to be interpreted for debugging purposes, and not to be consumed programmatically.

The following table outlines the expected error HTTP status codes:

Status Code Reason error_code
400 Invalid user input. Ensure that any SHA-256s are valid. BAD_REQUEST
403 Authentication error. Ensure that both the API key andorg_keyare correct. Check the response message field for a more detailed error. FORBIDDEN
404 Requested resource not found.
500 Server error. This is typically a transient error and the request may be retried, ideally with an exponential backoff. INTERNAL_SERVER_ERROR

Additional Notes

Unless otherwise stated, all dates/timestamps will be strings, formatted as ISO 8601 UTC, i.e. YYYY-MM-DDThh:mm:ss.sssZ

POST requests MUST include the header Content-Type: application/json or the API will return a 500 error.

File API

The File API is specifically for interacting with files stored within the UBS.

Download

This API provides a means to download previously-uploaded files. Once this API is successfully called, the files become available to be downloaded via AWS S3 pre-signed URLs. These URLs enable the client to perform a GET on the provided URLs and download the files. The links will automatically expire after one hour, unless a different expiration time is requested.

Files are available for download per the UBS data retention policy described in the Overview.

CAUTION - Anyone who has these links will be able to download the files until the requested expiration time (default one hour after the request). Ensure that care is taken when utilizing this API to prevent unauthorized acccess to your organization’s binaries.

API Permissions Required

Identity Manager Permission (.notation name) Operation(s) Environment
Carbon Black Cloud ubs.org.file READ Majority of environments

Request

The following table describes the inputs required to utilize this API.

Description Value
Route {cbc-hostname}/ubs/v1/orgs/{org_key}/file/_download
HTTP verb POST

The body for the request should be as follows:

{
    "sha256": [
        "<string>",
        ...
    ],
    "expiration_seconds": <int>
}

The following table explains the above data:

Item Data Type Description Required
sha256 array of strings An array of SHA-256 hashes as ASCII hex strings, with a limit of 100. More than 100 items will result in an HTTP 400. Yes
expiration_seconds integer The number of seconds to make the download URLs available for. The default is 3600 (1 hour). No

Response

The body for the response is as follows: 

{
    "found": [
        {
            "sha256": "<string>",
            "url": "<string>"
        },
       ...
    ],
    "not_found": [
        "<string>",
       ...
    ],
    "error": [
        "<string>",
       ...
    ]
}

The following table explains the above response:

Item Data Type Description
found array of JSON object Details about the files that were found to be available for download
found[ix].sha256 string SHA-256 hash of file that is available to be downloaded
found[ix].url string An AWS S3 presigned URL for this file. Perform a GET on this URL to download the file.
not_found array of strings The SHA-256 hashes that were not found.
error array of strings The SHA-256 hashes that had an intermittent error. These hashes should be retried.

CURL Example - Download File by Hash

Set AUTHENTICATION to your APIKEY/ConnectorID and the HOST, ORGID and TARGETSHA variables appropriately.

curl -XPOST https://"$HOST"/ubs/v1/orgs/"$ORGID"/file/_download  -H 'Content-Type: application/json' -H "X-Auth-Token: $AUTHENTICATION"  -d '{"sha256": ["87976f3430cc99bc939e0694247c0759961a49832b87218f4313d6fc0bc3a776"]}'

{"found":[
    {"sha256":"87976f3430cc99bc939e0694247c0759961a49832b87218f4313d6fc0bc3a776",
    "url":"https://cdc-file-storage-staging-us-east-1.s3.amazonaws.com/87/97/6f/34/30/cc/99/bc/93/9e/06/94/24/7c/07/59/96/1a/49/83/2b/87/21/8f/43/13/d6/fc/0b/c3/a7/76/87976f3430cc99bc939e0694247c0759961a49832b87218f4313d6fc0bc3a776.zip?AWSAccessKeyId=ASIAQZRW24JUWSSITSS4&Signature=FUVo3yW9DTj8iAZckynCCAXg10U%3D&x-amz-security-token=FQoGZXIvYXdzELn%2F%2F%2F%2F%2F%2F%2F%2F%2F%2FwEaDNnE2ZGKw4sKvDQgzCK3A27%2BmyKzV%2BQcYEdnqUhW8WJ6L9GbiIsDL0rv6uK46DCkdwvMPYDdBoWitDaSkb6KXzbOpItfPDnnpPAjswzdeh8nr4LHjkcaZUtYBBw2L1W33Ee0A%2F7RgY%2Fw75nWtgbnamxWWlNq9kapELBV6v1ohJzAiq%2FwRgZKCNeGSToT%2BR0RVjo5wJFFoKM7C4eVfZxIZyaKi2eA4gMdrmDhXvhBi%2Bphdk0ER1%2FkW8Q9XvtePjvypKCc%2BDwAXNA0BNa4pmeqmOQofUzzBD2FPdbuucR0W5LG7ajYCCrM1S5LcbatwTmqtZ8HbVB7ZNX%2BSxuyb4yMYNnBj22bldsjmUTuY6Ya%2FokZVuCRoTqi59PVB9D3PB%2FD4dk5RuRnQaCzz0TSnaYFTN9qznbWAhF%2FynUiyDC%2FozX2b%2BbpNcawixTSrnp76rEj8OLYm8Yrl9DqPs14Tw8WanULirzgtcOwvrMBioR0PBIYbInKNEhWhzq2D%2FdMkqyAxcasbR%2BFN7CVzXAGXjsFE3yVnyXz7CTQaOfowj1HNoj%2FA3dIdLZzSiaDD1nj6r5e81V7OphQFFfp%2BFB5Io82mwBuw%2FcH8jooj7nV4wU%3D&Expires=1551201116"}]
    ,"not_found":[],
    "error":[]
}

Metadata API

The Metadata API returns all of the metadata for the specified binary identified by the SHA-256 hash.

This data is specific to the binary and is not limited to the data reported by your organization’s assets.

Metadata for each binary are available for retrieval per the UBS data retention policy described in the Overview.

Retrieve Metadata

API Permissions Required

Identity Manager Permission (.notation name) Operation(s) Environment
Carbon Black Cloud ubs.org.sha256 READ Majority of environments

Request

The following table describes the inputs required to utilize this API.

Description Value
Route {cbc-hostname}/ubs/v1/orgs/{org_key}/sha256/{sha256}/metadata
HTTP verb GET
sha256 The requested SHA-256 hash to obtain metadata information.

Response

The body for the response is as follows: 

{
    "sha256": "<string>",
    "md5": "<string>",
    "file_available": <boolean>,
    "available_file_size": <long>,
    "file_size": <long>,
    "os_type": "<string>",
    "architecture": [
        "<string>"
    ],
    "lang_id": <int>,
    "charset_id": <int>,
    "internal_name": "<string>",
    "product_name": "<string>",
    "company_name": "<string>",
    "copyright": "<string>",
    "trademark": "<string>",
    "file_description": "<string>",
    "file_version": "<string>",
    "comments": "<string>",
    "original_filename": "<string>",
    "product_description": "<string>",
    "product_version": "<string>",
    "private_build": "<string>",
    "special_build": "<string>"
}

The following table explains the above response:

Item Data Type Description
sha256 string The SHA-256 hash of the file.
md5 string The MD5 hash of the file.
file_available boolean If true, the file is available for download.
available_file_size long The size of the file, that is available for download. If the file is unavailable the size will be zero. In the case of the null file, then the size will be zero and file_available will be true. Note that files may be truncated, before upload, which is why this number may be less that file_size.
file_size long The size of the actual file. This is the size of the file represented by this hash.
os_type string The OS that this file is designed for. This may contain one or more of the following values: WINDOWS, ANDROID, MAC, IOS, LINUX, and OTHER.
architecture array of string The set of architectures that this file was compiled for. This may contain one or more of the following values: none, x86, amd64, and arm64.
lang_id unsigned int The Language ID value from the Windows VERSIONINFO resource. See this link on MSDN for more information.
charset_id unsigned int The Character set ID value from the Windows VERSIONINFO resource. See this link on MSDN for more information.
internal_name nullable string If present, Internal name from FileVersionInformation
product_name nullable string If present, Product name from FileVersionInformation
company_name nullable string If present, Company name from FileVersionInformation
copyright nullable string If present, Legal copyright from FileVersionInformation
trademark nullable string If present, Legal trademark from FileVersionInformation
file_description nullable string If present, File description from FileVersionInformation
file_version nullable string If present, File version from FileVersionInformation
comments nullable string If present, comments from FileVersionInformation
original_filename nullable string If present, Original filename from FileVersionInformation
product_description nullable string If present, Product description from FileVersionInformation
product_version nullable string If present, Product version from FileVersionInformation
private_build nullable string If present, Private build from FileVersionInformation
special_build nullable string If present, Special build from FileVersionInformation

CURL Example - Get File Metadata

Set the TARGETSHA, ORGID, and HOST envars appropriately.

curl -H "X-Auth-Token: $AUTHENTICATION" https://$HOST/ubs/v1/orgs/"$ORGID"/sha256/"$TARGETSHA"/metadata
{"sha256":"87976f3430cc99bc939e0694247c0759961a49832b87218f4313d6fc0bc3a776",
"architecture":["amd64"],
"available_file_size":96672,
"charset_id":1200,
"comments":null,
"company_name":null,
"copyright":"\u00a9 Microsoft Corporation. All rights reserved.",
"file_available":true,
"file_description":"Runtime Broker",
"file_size":96672,
"file_version":"10.0.16299.15 (WinBuild.160101.0800)",
"internal_name":"RuntimeBroker.exe",
"lang_id":1033,
"md5":"bd4401441a21bf1abce6404f4231db4d",
"original_filename":"RuntimeBroker.exe",
"os_type":"WINDOWS",
"private_build":null,
"product_description":null,
"product_name":"Microsoft\u00ae Windows\u00ae Operating System",
"product_version":"10.0.16299.15",
"special_build":null,
"trademark":null
}

Summary API

The Summary API returns summarized information about a given SHA-256 hash for the target organization.

This API returns a summary of data available from binary executions reported by all of your devices over the last three months. For example, if a binary was executed on “Asset A” 6 months ago and 5 months ago, and was also executed on “Asset B” 5 months ago, 2 months ago and 1 month ago the Summary API would report for that binary: first_seen_device_timestamp = 2 months ago, last_seen_device_timestamp = 1 month ago and num_devices = 1.

Device

This API returns a summarized view of the device details. Use this API to get an overview of the devices that executed the file.

API Permissions Required

Identity Manager Permission (.notation name) Operation(s) Environment
Carbon Black Cloud ubs.org.sha256 READ Majority of environments

Request

The following table describes the inputs required to utilize this API.

Description Value
Route {cbc-hostname}/ubs/v1/orgs/{org_key}/sha256/{sha256}/summary/device
HTTP verb GET
sha256 The requested SHA-256 hash to obtain information for.

Response

The body for the response is as follows:

{
    "sha256": "<string>",
    "num_devices": <int>,
    "first_seen_device_timestamp": "<string>",
    "first_seen_device_id": <int>,
    "first_seen_device_name": "<string>",
    "last_seen_device_timestamp": "<string>",
    "last_seen_device_id": <int>,
    "last_seen_device_name": "<string>"
}

The following table explains the above response:

Item Data Type Description
sha256 string The SHA-256 hash of the file.
num_devices long The total number of devices, for this organization, that have observed this file.
first_seen_device_timestamp string The time that this file was first seen, for this organization.
first_seen_device_id long The Device ID of the device that first saw this file.
first_seen_device_name string The name of the device that first saw this file.
last_seen_device_timestamp string The time that this file was most recently seen, for this organization.
last_seen_device_id long The Device ID of the device that most recently saw this file.
last_seen_device_name string The name of the device that last saw this file.

CURL Example - Get Devices for SHA256

Set AUTHENTICATION, HOST, ORGID and TARGETSHA variables appropriately.

curl -H "X-Auth-Token: $AUTHENTICATION" https://"$HOST"/ubs/v1/orgs/"$ORGID"/sha256/"$TARGETSHA"/summary/device

{"sha256":"87976f3430cc99bc939e0694247c0759961a49832b87218f4313d6fc0bc3a776",
"num_devices":3,
"first_seen_device_timestamp":"2018-12-10T15:55:51.828771Z",
"first_seen_device_id":716428,
"first_seen_device_name":"CRAIG_WIN10_X64",
"last_seen_device_timestamp":"2018-12-28T16:49:30.185259Z",
"last_seen_device_id":787514,
"last_seen_device_name":"CRAIG_VM_WIN10_X64"}

Signature API

The Signature API returns a summary of the observed digital signature results for a given SHA-256 hash. The digital signature information for a binary may vary from one machine to another based on a variety of factors, including the presence of an up-to-date signature catalog on the host, system clock variations, ability to reach OCSP servers, custom root trust anchors, and more. Therefore, the results are ordered by prevalence, such that the most observed signatures will be returned first.

The number of results are configurable, up to a max of 100 entries.

Digital signatures can be recorded in a separate file (known as a “catalog” file) or embedded inside of the binary itself (an “embedded” signature).

The Signature API will capture the results of the endpoint’s verification of the digital signature associated with a given SHA-256 hash, stating whether that signature validation was based on a catalog file or an embedded signature. It is not uncommon for a catalog signature to be frequently updated, such that this API can return a wide variety of sign_timestamp values for a given SHA-256 hash.

Signature data for each binary are available for retrieval per the UBS data retention policy described in the Overview.

API Permissions Required

Identity Manager Permission (.notation name) Operation(s) Environment
Carbon Black Cloud ubs.org.sha256 READ Majority of environments

Request

The following table describes the inputs required to utilize this API.

Description Value
Route {cbc-hostname}/ubs/v1/orgs/{org_key}/sha256/{sha256}/summary/signature?rows={rows}
HTTP verb GET
sha256 The requested SHA-256 hash to obtain information for.
rows The number of results to return. The default is 5 and the max is 100.

Response

The body for the response is as follows:

{
    "sha256": "<string>",
    "total_signatures_count": <int>,
    "signatures_count": <int>,
    "signatures": [
        {
            "count": <int>,
            "is_catalog_signature": <boolean>,
            "publisher_name": "<string>",
            "issuer_name": "<string>",
            "sign_timestamp": "<string>",
            "sign_states": [
                {
                    "device_count": <int>,
                    "sign_state": [
                        "<string>"
                    ],
                    "first_seen_timestamp": "<string>"
                },
                ...
            ]
        },
        ...
    ]
}

The following table explains the above response:

Item Data Type Description
sha256 string The SHA-256 hash of the file.
total_signatures_count long The total number of signatures that have been observed by this organization for this file. This is the sum, over all devices that have executed this file, of the count of signatures for that file observed by each device.
signatures_count long The total number of unique signatures that have been observed, by this organization, for this file. This field represents only unique occurrences, not total existence. Assuming that each device observed the same signature details for the file, then the following should be true: total_signatures_count = (number of devices that executed file) * signatures_count
signatures array of JSON objects The signature details. The length of the array is limited based off the provided input parameter “rows” with a maximum of 100.
signatures[ix].count long The total number of occurrences where this signature was observed, by this organization, for this file. This field is similar to total_signatures_count; however, it represents the count at a per signature-level instead of at the file-level. The sum of signatures[ix].count (over all signatures) should be equal to total_signatures_count.
signatures[ix].is_catalog_signature boolean If true, then this signature is a catalog signature.
signatures[ix].publisher_name nullable string The name of the company who published the file, as recorded in the digital certificate.
signatures[ix].issuer_name nullable string The name of the company who issued the signature. This is the company who signed the certificate. This may be an intermediate or the root certificate authority.
signatures[ix].sign_timestamp nullable string The date which the signature was issued.
signatures[ix].sign_states array of JSON object The signature states for the signatures. The length of this array is unbounded and will be as big as needed to return all available signature states.
signatures[ix].sign_states[ix2].device_count long The number of devices, for this organization, that observed this signature, for this file, with this specific signature result.
signatures[ix].sign_states[ix2].sign_state array of strings The human readable signature states. This may contain one or more of the following values: invalid, signed, verified, not_signed, unknown, chained, trusted, os, and catalog_signed. Some of those values are mutually exclusive, e.g. not_signed and signed will never both appear in the same array. Important note - a state of signed does not mean that the signature is valid. Both signed and valid must be within the array to indicated that it is valid. A value of unknown is used when signature verification errors occur and is also the initial state of all files. A value of chained means that the signature chains to a locally trusted root certificate. A value of os means that the file is a component of the operating system. A value of catalog_signed means that the signature was in a catalog file instead of within the file. A valid of trusted means that the local machine trusts the signing chain.
signatures[ix].sign_states[ix2].first_seen_timestamp string The date which a device first executed a file with this signature and signature state.

CURL Example - Get Signature

curl -H "X-Auth-Token: $AUTHENTICATION" https://"$HOST"/ubs/v1/orgs/"$ORGID"/sha256/"$TARGETSHA"/summary/signature
{"sha256":"87976f3430cc99bc939e0694247c0759961a49832b87218f4313d6fc0bc3a776",
"total_signatures_count":51,
"signatures_count":1,
"signatures":[
    {"count":51,
    "is_catalog_signature":false,
    "publisher_name":"Microsoft Windows",
    "issuer_name":"Microsoft Windows Production PCA 2011",
    "sign_timestamp":"2017-09-29T04:30:49.456000Z",
    "sign_states":[{"device_count":51,
                    "sign_state":["signed","verified","trusted","os"],
                    "first_seen_timestamp":"2018-12-11T01:40:04.827586Z"}]
    }]
 }

File Path API

The File Path API returns a summary of the observed file paths. The results are ordered by prevalence, such that the most observed file path will be returned first. The number of results are configurable, up to a max of 100 entries.

File Path data for each binary are available for retrieval per the UBS data retention policy described in the Overview.

Input

The following table describes the inputs required to utilize this API.

Description Value
Route {cbc-hostname}/ubs/v1/orgs/{org_key}/sha256/{sha256}/summary/file_path?rows={rows}
HTTP verb GET
sha256 The requested SHA-256 hash to obtain information for.
rows The number of results to return. The default is 5 and the max is 100.

API Permissions Required

Identity Manager Permission (.notation name) Operation(s) Environment
Carbon Black Cloud ubs.org.sha256 READ Majority of environments

Response

The body for the response is as follows: 

{
    "sha256": "<string>",
    "total_file_path_count": <int>,
    "file_path_count": <int>,
    "file_paths": [
        {
            "count": <int>,
            "file_path": "<string>",
            "first_seen_timestamp": "<string>"
        },
        ...
    ]
}

The following table explains the above response:

Item Data Type Description
sha256 string The SHA-256 hash of the file.
total_file_path_count long The total number of file paths that have been observed, by this organization, for this file.
file_path_count long The total number of unique file paths that have been observed, by this organization, for this file.
file_paths array of JSON objects The file path details. The length of the array is limited based off the provided input parameter.
file_paths[ix].count long The number of occurrences where this signature was observed, by this organization, for this file.
file_paths[ix].file_path string The file path. Note that only the file path for the first execution for a particular device is captured, as such each device will only ever have a single file path.
file_paths[ix].first_seen_timestamp string The time that this file path was first observed.

CURL Example - Get FilePath information

Set the AUTHENTICATION, HOST, ORGID, and TARGETSHA variables appropriately.

curl -H "X-Auth-Token: $AUTHENTICATION" https://"$HOST"/ubs/v1/orgs/"$ORGID"/sha256/"$TARGETSHA"/summary/file_path

{"sha256":"87976f3430cc99bc939e0694247c0759961a49832b87218f4313d6fc0bc3a776",
"total_file_path_count":3,
"file_path_count":1,
"file_paths":[{"count":3,"file_path":"c:\\windows\\system32\\runtimebroker.exe",
"first_seen_timestamp":"2018-12-10T15:55:51.828771Z"}]}

Last modified on February 19, 2025