Job Service API

Introduction

The Job Service API tracks the execution of long running tasks making it easier to monitor progress of asynchronous jobs and retrieve the jobs’ output once completed and prevents important tasks from timing out mid execution. It can be used with other APIs, such as the Live Query REST API - Get Query Run Results, to asynchronously export large amounts of data.

More about this API:

  • Long running or slow jobs tracked through the jobs service will complete without timing out
  • Downloads can be repeated without having to run the underlying job multiple times
  • Jobs can only be seen by the connector or user who created them
  • Jobs are saved for 30 days

Requirements

  • Any of the Carbon Black Cloud products
  • All API calls require an API key with appropriate permissions, see Authentication for details
  • Must have a Job Investigate Page Export or Live Query to get the status of other asynchronous jobs

Authentication

Determine whether you use Carbon Black Cloud or VMware Cloud Services Platform to manage identity and authorization, or see the Carbon Black Cloud API Access Guide for complete instructions.


Carbon Black Cloud Managed Identity and Authentication
Customize your access to the Carbon Black Cloud APIs with Role-Based Access Control; All APIs and Services authenticate via API Keys. To access the data in Carbon Black Cloud via API, you must set up a key with the correct permissions for the calls you want to make and pass it in the HTTP Headers.

Environment
Available on majority of environments; Use the Carbon Black Cloud Console URL, as described here.

API Route
Replace the {cbc-hostname} and {org_key} with the URL of your Environment and the org_key for your specific Org.
  • Jobs: {cbc-hostname}/jobs/v1/orgs/{org_key}/jobs/

Access Level
Before you create your API Key, you need to create a "Custom" Access Level including each category:
  • Background Tasks > Status > jobs.status, allow permission to READ

API Key
When creating your API Key, use the Access Level Type of "Custom" and select the Access Level you created. Details on constructing and passing the API Key in your requests are available here.


Cloud Services Platform Managed Identity and Authentication
Customize your access to the Carbon Black Cloud APIs with OAuth Access Control; API access is controlled using OAuth apps or User API Tokens. This is currently limited to the UK Point of Presence and AWS GovCloud (US).

Environment
Available on Prod UK and AWS GovCloud (US). Full list of environments is available here; Use the Carbon Black Cloud Console URL from Cloud Services Platform, as described here.

API Route
Replace the {cbc-hostname} and {org_key} with the URL of your Environment and the org_key for your specific Org.
  • Jobs: {cbc-hostname}/jobs/v1/orgs/{org_key}/jobs/

Access Level
Before you create your OAuth App, you need to create a custom Role with the following permissions under IDENTITY & ACCESS MANAGEMENT > Roles > VMware Carbon Black Cloud:
  • _API.Background.Tasks:jobs.Status, allow permission to READ

API Authentication
The Cloud Services Platform supports several authentication options, Access Token, API Token, and for backward compatibility, X-Auth-Token. To learn about the differences or how to use the authentication methods see the Authentication Guide.


Quick Setup

Typical use of the API follows this sequence:

  1. Call a route in a service that utilizes the job service (eg. /jobs/v1/orgs/{org_key}/jobs/start/event_export or /livequery/v1/orgs/{org_key}/runs/{id}/results/_search?format=csv&async=true)
  2. Receive the job ID
  3. Call /jobs/v1/orgs/{org_key}/jobs/{job_id}/progress route to get job progress
  4. Repeat 3 until the status shows COMPLETED (or FAILED)
  5. If successful, call the /jobs/v1/orgs/{org_key}/jobs/{job_id}/download route to retrieve the output of the job

Helpful tips:

  • Use the /jobs/{job_id}/progress route to check the progress of long running exports. Shorter jobs may not update the progress before they complete.
  • You cannot call GET /jobs/{job_id}/status. You can retrieve status from the GET /jobs/{job_id} route
  • All time stamps are formatted as ISO 8601 UTC, i.e. YYYY-MM-DDThh:mm:ss.sssZ. This includes create_time, last_update_time, and time_range.

List All Available Jobs

Lists all available jobs. A user or connector can only see jobs assigned to themselves.

RBAC Permissions Required

Identity Manager Permission (.notation name) Operation(s) Environment
Carbon Black Cloud jobs.status READ Majority of environments
VMware Cloud Services Platform _API.Background.Tasks:jobs.Status.read N/A - included in permission name Prod UK and AWS GovCloud (US)

Request

GET {cbc-hostname}/jobs/v1/orgs/{org_key}/jobs

Response

Code Description Content-Type Content
200 Successful Request application/json View example response below
400 The JSON body was malformed, or some part of the JSON body included an invalid value N/A N/A
401 Unauthorized N/A N/A
403 Forbidden N/A N/A
404 Not found N/A N/A
500 Internal Server Error N/A N/A

Response Body

{
  "num_found": "<integer>",
  "results": [
    {
      "connector_id": "<string>",
      "create_time": "<string>",
      "errors": "<string>",
      "id": "<integer>",
      "job_parameters": {},
      "last_update_time": "<string>",
      "org_key": "<string>",
      "owner_id": "<integer>",
      "progress": {
        "num_total": "<integer>",
        "num_completed": "<integer>",
        "message": "<string>"
      },
      "status": "<string>",
      "type": "<string>"
    }
  ]
}

Additional response parameter details

Parameter Possible Values
["progress"]["message"] QUEUED, IN_PROGRESS, FINISHED
["status"] CREATED, FAILED, COMPLETED
["type"] event_export, livequery_export

If a parameter in the response is Null, it will not appear in the response. For example, if a job does not have any errors, the errors parameter will not appear in the response, as shown in the example below.

Example

Request

GET https://defense.conferdeploy.net/jobs/v1/orgs/ABCDEF1/jobs

Response

{
  "num_found": 1,
  "results": [
    {
      "id": 1,
      "type": "event_export",
      "job_parameters": {
        "job_parameters": {
          "query": {
            "query": "(TTP:SUSPICIOUS_BEHAVIOR OR TTP:PACKED_CALL)",
            "time_range": {
              "start": "2020-03-16T13:06:06-04:00"
            },
            "rows": 10000,
            "fields": [
              "document_guid"
            ],
            "sort": [
              {
                "field": "device_timestamp",
                "order": "DESC"
              }
            ]
          }
        },
        "api_resource": "ENRICHED_EVENTS",
        "version": "v2"
      },
      "connector_id": "2SP9TZA1LL",
      "org_key": "ABCDEF1",
      "status": "COMPLETED",
      "progress": {
        "num_total": 0,
        "num_completed": 0,
        "message": "FINISHED"
      },
      "create_time": "2020-05-19T18:08:00.690Z",
      "last_update_time": "2020-05-19T18:08:12.183Z"
    }
  ]
}

Get Job Details

Get the details of a specific job.

RBAC Permissions Required

Identity Manager Permission (.notation name) Operation(s) Environment
Carbon Black Cloud jobs.status READ Majority of environments
VMware Cloud Services Platform _API.Background.Tasks:jobs.Status.read N/A - included in permission name Prod UK and AWS GovCloud (US)

Request

GET {cbc-hostname}/jobs/v1/orgs/{org_key}/jobs/{job_id}

Response

Code Description Content-Type Content
200 Successful Request application/json View example response below
400 The JSON body was malformed, or some part of the JSON body included an invalid value N/A N/A
401 Unauthorized N/A N/A
403 Forbidden N/A N/A
404 Not found N/A N/A
500 Internal Server Error N/A N/A

Response Body

{
  "connector_id": "<string>",
  "create_time": "<string>",
  "errors": "<string>",
  "id": "<integer>",
  "job_parameters": {},
  "last_update_time": "<string>",
  "org_key": "<string>",
  "owner_id": "<integer>",
  "progress": {
    "num_total": "<integer>",
    "num_completed": "<integer>",
    "message": "<string>"
  },
  "status": "<string>",
  "type": "<string>"
}

Additional response parameter details

Parameter Possible Values
["progress"]["message"] QUEUED, IN_PROGRESS, FINISHED
["status"] CREATED, FAILED, COMPLETED
["type"] event_export, livequery_export

Example

Request

GET https://defense.conferdeploy.net/jobs/v1/orgs/ABCDEF1/jobs/1

Response

{
  "id": 3039,
  "type": "event_export",
  "job_parameters": {
    "job_parameters": {
      "query": {
        "query": "(TTP:ADAPTIVE_WHITE_APP OR TTP:UNKNOWN_APP OR TTP:DETECTED_SUSPECT_APP OR TTP:DETECTED_PUP_APP OR TTP:DETECTED_BLACKLIST_APP OR TTP:DETECTED_MALWARE_APP)",
        "time_range": {
          "start": "2020-03-16T13:06:06-04:00"
        },
        "rows": 10000,
        "fields": [
          "*"
        ],
        "sort": [
          {
            "field": "device_timestamp",
            "order": "DESC"
          }
        ]
      }
    },
    "api_resource": "ENRICHED_EVENTS",
    "version": "v2"
  },
  "connector_id": "2SP9TZA1LL",
  "org_key": "ABCDEF1",
  "status": "COMPLETED",
  "progress": {
    "num_total": 10000,
    "num_completed": 10000,
    "message": "FINISHED"
  },
  "create_time": "2020-05-19T18:08:46.490Z",
  "last_update_time": "2020-05-19T18:08:46.490Z"
}

Download Job Output

Provides a redirect with a pre-signed S3 URL to download the output of the job.

If the job does not exist or the output is not yet available, this route will return 404.

RBAC Permissions Required

Identity Manager Permission (.notation name) Operation(s) Environment
Carbon Black Cloud jobs.status READ Majority of environments
VMware Cloud Services Platform _API.Background.Tasks:jobs.Status.read N/A - included in permission name Prod UK and AWS GovCloud (US)

Request

GET {cbc-hostname}/jobs/v1/orgs/{org_key}/jobs/{job_id}/download

Response

Code Description Content-Type Content
200 Successful Request Varies by job Output of job
400 The JSON body was malformed, or some part of the JSON body included an invalid value N/A N/A
401 Unauthorized N/A N/A
403 Forbidden N/A N/A
404 Not found N/A N/A
500 Internal Server Error N/A N/A

Example

Request

GET https://defense.conferdeploy.net/jobs/v1/orgs/ABCDEF1/jobs/1/download

Get Job Progress

This route only returns the progress of a specific job. This route is preferred over listing the entire job for checking the progress.

RBAC Permissions Required

Identity Manager Permission (.notation name) Operation(s) Environment
Carbon Black Cloud jobs.status READ Majority of environments
VMware Cloud Services Platform _API.Background.Tasks:jobs.Status.read N/A - included in permission name Prod UK and AWS GovCloud (US)

Request

GET {cbc-hostname}/jobs/v1/orgs/{org_key}/jobs/{job_id}/progress

Response

Code Description Content-Type Content
200 Successful Request application/json View example response below
400 The JSON body was malformed, or some part of the JSON body included an invalid value N/A N/A
401 Unauthorized N/A N/A
403 Forbidden N/A N/A
404 Not found N/A N/A
500 Internal Server Error N/A N/A

Response Body

{
    "num_total": "<integer>",
    "num_completed": "<integer>",
    "message": "<string>"
}

Additional response parameter details

Parameter Possible Values
message QUEUED, IN_PROGRESS, FINISHED

Example

Request

GET https://defense.conferdeploy.net/jobs/v1/orgs/ABCDEF1/jobs/1

Response

{
    "num_total": 3142,
    "num_completed": 3142,
    "message": "FINISHED"
}

Last modified on September 6, 2022