Change Data Capture. A quick overview

A couple of months ago I had the chance to collaborate in Change Data Capture pilot program thanks to a workmate. It was a great experience and I felt like a little boy with new toys.

Change Data Capture (CDC) caught my attention in DF17. It is a really powerful feature but I think it was mostly because of several problems I faced a long time ago while I was working as an ABAP developer and we tried to integrate SAP with Salesforce. Because when CDC will be finally released this kind of things will be a piece of cake.

So, what is CDC? Well, we always can go to the Wikipedia. But also we can say that CDC is the easiest way to keep updated any external database with the same info you have in your ORG in real time. This is possible because of CDC is based in event-driven architecture and so, it uses the power of Salesforce platform events.

What Salesforce has created is a new type of event called Change Event (CE). Those new events will be available for any custom object defined in your org, and auto enabled for a subset of standard objects. Not all standard objects enjoy this feature but I bet the number of objects will increase. What this new feature does is fire an event with a JSON format anytime we modify, create, delete or undelete a record in Salesforce in case CDC is enabled for this object. And due to the publisher is automatically created, all you will need to do for a correct tracking of changes in our Salesforce org is create the necessary listeners to catch the changes.

As we still not having a user interface for the enabling or disabling of CDC for custom objects has to be done through metadata API. But it will be as simple as set to true a new parameter. I asked about the UI for CDC and a nice interface with two columns where you will easily see what custom objects have this available and what not is under development, but so far what we need is to add a new value to the XML object definition. In the following code block, the label <enableChangeDataCapture> will be doing the trick.

<?xml version="1.0" encoding="UTF-8"?>
<CustomObject xmlns="http://soap.sforce.com/2006/04/metadata">
    <description>CDC example</description>
    <enableChangeDataCapture>true</enableChangeDataCapture>
    <fields>
        <fullName>CDCexample__c</fullName>
        ...
        ...
    </fields>
</CustomObject>

A significant difference with a regular Platform Event is the amount of information sent. As the CDC still a pilot program the labels shown may not be the final version, but this how a CE looks right now.

image2018-1-22 17_11_53

From my view, the most relevant labels are:

  • Schema: A tricky one. This can be used for retrieving the full event schema using the REST API. for more info about how to do this you can check this and this.
  • RecordIds: Typically, this field contains one record ID. But if the same change occurred in multiple records (fieldA is updated to valueA) of the same object type during the same second, Salesforce groups the change notifications and sends one change event for all affected records. In this case, the recordIds field contains an array of record IDs for all records that have the same change.
  • ChangeOrigin: Who to blame! The name of the system that caused the change. really useful if you want to detect if it was your APP who initiates the change and so, to do not run it again and fall in a painful loop of changes. Review Call option header o explain how to set up this.
  • TransactionKey: A globally unique identifier to know the transaction where this change is included.
  • SequenceNumber: The sequence within a transaction. Perfect for replicate, in an external DB, the order of the changes done in SF.
  • Fields in Change Event Body: Include system, custom, and standard fields. Null fields what correspond to empty fields what were unchanged won’t show here. just those that were set to null along the update process. In an update, only the modified stuff will appear.

 

I hope you liked this quick overview of this powerful feature. Trust me when I say that will easier your life while connecting Salesforce with external databases and in order to have everything up to date in both systems. In my next post, I will talk about my experiences when playing with CDC.

See you soon!!

 

 

 

Advertisements

One thought on “Change Data Capture. A quick overview

  1. Pingback: My Top 5 Change Data Capture Highlights – Pedro M. Molina

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.