Back to Blogs

Announcing the XDR Events Data Forwarder!

Posted on December 12, 2023

You’ve adopted Carbon Black Cloud XDR. You see the obvious benefits of the integrated network telemetry in your endpoint security stack. You want this rich network data in your SIEM, security lake or custom security platform. One problem: the data isn’t coming through your CBC Data Forwarder stream.

That changes now. Welcome to the Data Forwarder for Endpoint Events version 1.1.0 - now with XDR support!


What’s changed?
  • New data fields!
    • Data Forwarder adds support for all the new endpoint event fields that come with CBC XDR.
    • As with other Data Forwarder fields, these new fields only get sent if the data exists in the underlying CBC data; for example, no XDR, no netconn_request_headers field
    • Exception: the new version field is included with all new Data Forwarder versions, denoting the version of the data schema used to generate the data fields and values in forwarded data
  • Filtering!
    • All the new fields are fully supported by Endpoint Event filtering, can be used in INCLUDE/EXCLUDE filters and can use wildcarding
  • Support for “SemVer” (semantic versioning)!
    • When configuring new Endpoint Event forwarders to consume the new schema version, you can select the v1.1.0 schema using a variety of combinations of new configuration options on the Add Forwarders page - see below for details
  • Schema version switching!
    • Along with semver support, Forwarder allows you to manually update your Endpoint Event forwarder anytime after new schema versions are made available - see below for details
  • New API response content!
    • For Data Forwarders of type = endpoint.event, the Data Forwarder API now returns the following values:
      • Existing Endpoint Event forwarders: version = 1.0.0, version_constraint = 1.0.*
      • New Endpoint Event forwarders: version = one of 1.0.0 or 1.1.0, version_constraint = one of 1.0.0, 1.1.0, 1.0., 1.1. or 1.., depending on user’s selected configuration
  • New console inputs!
    • Schema updates, Schema version/constraint for Data Forwarders of type Alert or Endpoint event
  • New console defaults!
    • Schema updates = Minor (automatic), Schema version/constraint = 1.. (latest) for Data Forwarders of type Endpoint event

How do I configure an Endpoint Event forwarder?

When configuring new Endpoint Event forwarders to consume the new schema version, you can select the v1.1.0 schema with one of four configuration approaches on the Add Forwarders page (options 1-3) or in the API (option 4):

  • Option 1: select “Minor (automatic)” from Schema updates
  • Option 2: select “Patch (automatic)” from Schema updates and “1.1.*” fromSchema version/constraint
  • Option 3: select “Pinned (manual)” from Schema updates and “1.1.0” from Schema version/constraint
  • Option 4 (API): select 1.1.0 or 1.1.* version_constraint in the Create Forwarder API or Edit Forwarder API

What effect will each combination have?

Depending on which “Schema updates” selection you make, Carbon Black Cloud will take different automated actions for your forwarder as we release future data schema updates:

  • If your forwarder was configured for Minor (1..), every future update on the 1.x.y versions will automatically be applied
  • If you selected Patch (1.1.*), every future update to 1.1.y will be automatically applied, but any v1.2.y or beyond versions will not be automatically applied
  • If you selected Pinned (1.1.0), no future updates will be automatically applied

You can also continue to consume version 1.0.0, the original Endpoint Event schema by

  • selecting Pinned from Schema Updates dropdown and Version 1.0.0 from Schema dropdown; or
  • requesting version_constraint:"1.0.0" in your POST request to the Create Forwarder API endpoint

As part of the new semantic versioning support, Forwarder allows you to manually update new or existing Endpoint Event forwarders anytime after new schema versions are made available.

For example, your existing (Version 1.0.0) Endpoint Event forwarder can be updated to use Version 1.1.0 schema by

  • editing your Endpoint Event forwarder and selecting Version 1.1.* from the Schema dropdown; or
  • by selecting Minor from the Schema updates dropdown which will automatically upgrade your Endpoint Event forwarder to the latest released 1.x.y schema version

Note: Endpoint Event filters that use fields only available in later versions of a schema will not be supported with Endpoint Event forwarders that are configured to use an earlier schema version

  • e.g. filtering on netconn_tls_cipher:TLS_ECDHE* with your v1.1.0 Endpoint Event forwarder will not be supported on an Endpoint Event forwarder configured to use the 1.0.0 schema version. It will drop data that does not explicitly match this filter term
  • In the future Carbon Black Cloud will automatically prevent such combinations from being set when adding or editing Endpoint Event forwarders either from the console or the API

What’s Required

The Carbon Black Cloud Extended Detection and Response (XDR) add-on must be enabled in your CBC Enterprise EDR-enabled organization, for the Data Forwarder to include the new XDR fields in a Version 1.1.0 Endpoint Event forwarder

Otherwise a version 1.1.0 Endpoint Event forwarder has the same requirements as the existing Data Forwarder.


How does v1.1.0 data compare with v1.0.0 data?
  • The v1.1.0 Endpoint Event forwarder builds on the existing v1 Endpoint Event forwarder schema, which follows the Semantic Versioning principles outlined on the Developer Network blog
  • There are no breaking changes to existing fields
  • v1.1.0 Endpoint Event forwarder schema adds the new XDR fields to the v1.0.0 Endpoint Event forwarder schema
  • See the full schema including definitions in the Endpoint Event Schema 1.1.0

The newly-added fields are:

  1. netconn_application_protocol
  2. netconn_bytes_received
  3. netconn_bytes_sent
  4. netconn_community_id
  5. netconn_last_packet_timestamp
  6. netconn_first_packet_timestamp
  7. netconn_request_method
  8. netconn_request_url
  9. netconn_request_headers
  10. netconn_response_headers
  11. netconn_response_status_code
  12. netconn_ja3_local_fingerprint
  13. netconn_ja3_remote_fingerprint
  14. netconn_ja3_local_fingerprint_fields
  15. netconn_ja3_remote_fingerprint_fields
  16. netconn_tls_certificate_issuer_name
  17. netconn_tls_certificate_subject_name
  18. netconn_server_name_indication
  19. netconn_tls_certificate_subject_not_valid_before
  20. netconn_tls_certificate_subject_not_valid_after
  21. netconn_tls_version
  22. version

What will happen with my existing Endpoint Event forwarder?
  • TL;DR: no problem, no change
  • Your existing Endpoint Event forwarder will now return version = 1.0.0 and version_constraint = 1.0.* in both the console and in responses on the v2 Forwarder API
  • Your existing “v1” Endpoint Event forwarders will only be automatically upgraded to any bug fix updates to the v1.0.x Endpoint Event schema. There are none planned at this time.
  • If you wish to adopt the newer data schema, and if you wish to automatically adopt future data schema updates, you will have to manually reconfigure your Endpoint Event forwarder instance (see above)

What changes will be automatically applied to my Endpoint Event forwarder?
  • For any Config API request that omits the optional version_constraint request parameter, Data Forwarder will set that property of your Forwarder to the lowest supported version (i.e. presently 1.0.*)
  • This protects system integrators who haven’t instrumented support for the newer schema fields and aren’t yet including version_constraint in their API requests, while giving other integrators the ability to automatically select any supported schema versions by optionally specifying the version_constraint
  • As and when you’re ready, you are free to update your Forwarder instance to set Schema (in UI) or version_constraint (in API) at any time

If my CBC org doesn’t subscribe to XDR, but I configure a v1.1.0 Endpoint Event Forwarder, will my data take up any more space than the same events from v1.0.0 Endpoint Event Forwarder?
  • Each v1.1.0 event from a non-XDR-enabled org will include the extra bytes necessary to assert the “version”:“1.1.0” value
  • Since the Carbon Black Cloud Data Forwarder by design does not emit fields that have no value, even if they’re defined as part of the associated schema, you should not expect to see any of the XDR-only fields in your v1.1.0 data
  • The same will hold true for future Forwarder schema updates

Resources

Have questions or feedback?
  • Subscribe to the Developer Network Newsletter