Job Service API
Introduction
The Job Service API tracks the execution of long running tasks making it easier to monitor progress of asynchronous jobs and retrieve the jobs’ output once completed and prevents important tasks from timing out mid execution. It can be used with other APIs, such as the Live Query REST API - Get Query Run Results, to asynchronously export large amounts of data.
More about this API:
- Long running or slow jobs tracked through the jobs service will complete without timing out
- Downloads can be repeated without having to run the underlying job multiple times
- Jobs can only be seen by the connector or user who created them
- Jobs are saved for 30 days
Requirements
- Any of the Carbon Black Cloud products
- All API calls require an API key with appropriate permissions, see Authentication for details
- Must have a Job Investigate Page Export or Live Query to get the status of other asynchronous jobs
Authentication
Determine whether you use Carbon Black Cloud or VMware Cloud Services Platform to manage identity and authorization, or see the Carbon Black Cloud API Access Guide for complete instructions.Carbon Black Cloud Managed Identity and Authentication
Customize your access to the Carbon Black Cloud APIs with Role-Based Access Control; All APIs and Services authenticate via API Keys. To access the data in Carbon Black Cloud via API, you must set up a key with the correct permissions for the calls you want to make and pass it in the HTTP Headers.
Environment
Available on majority of environments; Use the Carbon Black Cloud Console URL, as described here.
API Route
Replace the {cbc-hostname} and {org_key} with the URL of your Environment and the org_key for your specific Org.
- Jobs: {cbc-hostname}/jobs/v1/orgs/{org_key}/jobs/
Access Level
Before you create your API Key, you need to create a "Custom" Access Level including each category:
- Background Tasks > Status > jobs.status, allow permission to
READ
API Key
When creating your API Key, use the Access Level Type of "Custom" and select the Access Level you created. Details on constructing and passing the API Key in your requests are available here.
Cloud Services Platform Managed Identity and Authentication
Customize your access to the Carbon Black Cloud APIs with OAuth Access Control; API access is controlled using OAuth apps or User API Tokens. This is currently limited to the UK Point of Presence and AWS GovCloud (US).
Environment
Available on
Prod UK
and AWS GovCloud (US)
. Full list of environments is available here; Use the Carbon Black Cloud Console URL from Cloud Services Platform, as described here.
API Route
Replace the {cbc-hostname} and {org_key} with the URL of your Environment and the org_key for your specific Org.
- Jobs: {cbc-hostname}/jobs/v1/orgs/{org_key}/jobs/
Access Level
Before you create your OAuth App, you need to create a custom Role with the following permissions under IDENTITY & ACCESS MANAGEMENT > Roles > VMware Carbon Black Cloud:
- _API.Background.Tasks:jobs.Status, allow permission to
READ
API Authentication
The Cloud Services Platform supports several authentication options, Access Token, API Token, and for backward compatibility, X-Auth-Token. To learn about the differences or how to use the authentication methods see the Authentication Guide.
Quick Setup
Typical use of the API follows this sequence:
- Call a route in a service that utilizes the job service (eg.
/jobs/v1/orgs/{org_key}/jobs/start/event_export
or/livequery/v1/orgs/{org_key}/runs/{id}/results/_search?format=csv&async=true
) - Receive the job ID
- Call
/jobs/v1/orgs/{org_key}/jobs/{job_id}/progress
route to get job progress - Repeat 3 until the status shows COMPLETED (or FAILED)
- If successful, call the
/jobs/v1/orgs/{org_key}/jobs/{job_id}/download
route to retrieve the output of the job
Helpful tips:
- Use the
/jobs/{job_id}/progress
route to check the progress of long running exports. Shorter jobs may not update the progress before they complete. - You cannot call GET
/jobs/{job_id}/status
. You can retrieve status from the GET/jobs/{job_id}
route - All time stamps are formatted as ISO 8601 UTC, i.e. YYYY-MM-DDThh:mm:ss.sssZ. This includes
create_time
,last_update_time
, andtime_range
.
List All Available Jobs
Lists all available jobs. A user or connector can only see jobs assigned to themselves.
RBAC Permissions Required
Identity Manager | Permission (.notation name) | Operation(s) | Environment |
---|---|---|---|
Carbon Black Cloud | jobs.status |
READ |
Majority of environments |
VMware Cloud Services Platform | _API.Background.Tasks:jobs.Status.read |
N/A - included in permission name | Prod UK and AWS GovCloud (US) |
Request
GET {cbc-hostname}/jobs/v1/orgs/{org_key}/jobs
Response
Code | Description | Content-Type | Content |
---|---|---|---|
200 | Successful Request | application/json | View example response below |
400 | The JSON body was malformed, or some part of the JSON body included an invalid value | N/A | N/A |
401 | Unauthorized | N/A | N/A |
403 | Forbidden | N/A | N/A |
404 | Not found | N/A | N/A |
500 | Internal Server Error | N/A | N/A |
Response Body
{
"num_found": "<integer>",
"results": [
{
"connector_id": "<string>",
"create_time": "<string>",
"errors": "<string>",
"id": "<integer>",
"job_parameters": {},
"last_update_time": "<string>",
"org_key": "<string>",
"owner_id": "<integer>",
"progress": {
"num_total": "<integer>",
"num_completed": "<integer>",
"message": "<string>"
},
"status": "<string>",
"type": "<string>"
}
]
}
Additional response parameter details
Parameter | Possible Values |
---|---|
["progress"]["message"] |
QUEUED, IN_PROGRESS, FINISHED |
["status"] |
CREATED, FAILED, COMPLETED |
["type"] |
event_export, livequery_export |
If a parameter in the response is Null, it will not appear in the response. For example, if a job does not have any errors, the errors
parameter will not appear in the response, as shown in the example below.
Example
Request
GET https://defense.conferdeploy.net/jobs/v1/orgs/ABCDEF1/jobs
Response
{
"num_found": 1,
"results": [
{
"id": 1,
"type": "event_export",
"job_parameters": {
"job_parameters": {
"query": {
"query": "(TTP:SUSPICIOUS_BEHAVIOR OR TTP:PACKED_CALL)",
"time_range": {
"start": "2020-03-16T13:06:06-04:00"
},
"rows": 10000,
"fields": [
"document_guid"
],
"sort": [
{
"field": "device_timestamp",
"order": "DESC"
}
]
}
},
"api_resource": "ENRICHED_EVENTS",
"version": "v2"
},
"connector_id": "2SP9TZA1LL",
"org_key": "ABCDEF1",
"status": "COMPLETED",
"progress": {
"num_total": 0,
"num_completed": 0,
"message": "FINISHED"
},
"create_time": "2020-05-19T18:08:00.690Z",
"last_update_time": "2020-05-19T18:08:12.183Z"
}
]
}
Get Job Details
Get the details of a specific job.
RBAC Permissions Required
Identity Manager | Permission (.notation name) | Operation(s) | Environment |
---|---|---|---|
Carbon Black Cloud | jobs.status |
READ |
Majority of environments |
VMware Cloud Services Platform | _API.Background.Tasks:jobs.Status.read |
N/A - included in permission name | Prod UK and AWS GovCloud (US) |
Request
GET {cbc-hostname}/jobs/v1/orgs/{org_key}/jobs/{job_id}
Response
Code | Description | Content-Type | Content |
---|---|---|---|
200 | Successful Request | application/json | View example response below |
400 | The JSON body was malformed, or some part of the JSON body included an invalid value | N/A | N/A |
401 | Unauthorized | N/A | N/A |
403 | Forbidden | N/A | N/A |
404 | Not found | N/A | N/A |
500 | Internal Server Error | N/A | N/A |
Response Body
{
"connector_id": "<string>",
"create_time": "<string>",
"errors": "<string>",
"id": "<integer>",
"job_parameters": {},
"last_update_time": "<string>",
"org_key": "<string>",
"owner_id": "<integer>",
"progress": {
"num_total": "<integer>",
"num_completed": "<integer>",
"message": "<string>"
},
"status": "<string>",
"type": "<string>"
}
Additional response parameter details
Parameter | Possible Values |
---|---|
["progress"]["message"] |
QUEUED, IN_PROGRESS, FINISHED |
["status"] |
CREATED, FAILED, COMPLETED |
["type"] |
event_export, livequery_export |
Example
Request
GET https://defense.conferdeploy.net/jobs/v1/orgs/ABCDEF1/jobs/1
Response
{
"id": 3039,
"type": "event_export",
"job_parameters": {
"job_parameters": {
"query": {
"query": "(TTP:ADAPTIVE_WHITE_APP OR TTP:UNKNOWN_APP OR TTP:DETECTED_SUSPECT_APP OR TTP:DETECTED_PUP_APP OR TTP:DETECTED_BLACKLIST_APP OR TTP:DETECTED_MALWARE_APP)",
"time_range": {
"start": "2020-03-16T13:06:06-04:00"
},
"rows": 10000,
"fields": [
"*"
],
"sort": [
{
"field": "device_timestamp",
"order": "DESC"
}
]
}
},
"api_resource": "ENRICHED_EVENTS",
"version": "v2"
},
"connector_id": "2SP9TZA1LL",
"org_key": "ABCDEF1",
"status": "COMPLETED",
"progress": {
"num_total": 10000,
"num_completed": 10000,
"message": "FINISHED"
},
"create_time": "2020-05-19T18:08:46.490Z",
"last_update_time": "2020-05-19T18:08:46.490Z"
}
Download Job Output
Provides a redirect with a pre-signed S3 URL to download the output of the job.
If the job does not exist or the output is not yet available, this route will return 404.
RBAC Permissions Required
Identity Manager | Permission (.notation name) | Operation(s) | Environment |
---|---|---|---|
Carbon Black Cloud | jobs.status |
READ |
Majority of environments |
VMware Cloud Services Platform | _API.Background.Tasks:jobs.Status.read |
N/A - included in permission name | Prod UK and AWS GovCloud (US) |
Request
GET {cbc-hostname}/jobs/v1/orgs/{org_key}/jobs/{job_id}/download
Response
Code | Description | Content-Type | Content |
---|---|---|---|
200 | Successful Request | Varies by job | Output of job |
400 | The JSON body was malformed, or some part of the JSON body included an invalid value | N/A | N/A |
401 | Unauthorized | N/A | N/A |
403 | Forbidden | N/A | N/A |
404 | Not found | N/A | N/A |
500 | Internal Server Error | N/A | N/A |
Example
Request
GET https://defense.conferdeploy.net/jobs/v1/orgs/ABCDEF1/jobs/1/download
Get Job Progress
This route only returns the progress of a specific job. This route is preferred over listing the entire job for checking the progress.
RBAC Permissions Required
Identity Manager | Permission (.notation name) | Operation(s) | Environment |
---|---|---|---|
Carbon Black Cloud | jobs.status |
READ |
Majority of environments |
VMware Cloud Services Platform | _API.Background.Tasks:jobs.Status.read |
N/A - included in permission name | Prod UK and AWS GovCloud (US) |
Request
GET {cbc-hostname}/jobs/v1/orgs/{org_key}/jobs/{job_id}/progress
Response
Code | Description | Content-Type | Content |
---|---|---|---|
200 | Successful Request | application/json | View example response below |
400 | The JSON body was malformed, or some part of the JSON body included an invalid value | N/A | N/A |
401 | Unauthorized | N/A | N/A |
403 | Forbidden | N/A | N/A |
404 | Not found | N/A | N/A |
500 | Internal Server Error | N/A | N/A |
Response Body
{
"num_total": "<integer>",
"num_completed": "<integer>",
"message": "<string>"
}
Additional response parameter details
Parameter | Possible Values |
---|---|
message |
QUEUED, IN_PROGRESS, FINISHED |
Example
Request
GET https://defense.conferdeploy.net/jobs/v1/orgs/ABCDEF1/jobs/1
Response
{
"num_total": 3142,
"num_completed": 3142,
"message": "FINISHED"
}