So in this short post, I’ll cover a few common mistakes that you might make when starting out configuring your DEF project, I know I certainly did and a fellow colleague hit the same issues on a recent project he just started. Hopefully, it will prevent you from falling into the same trap and banging your head against the wall.
Error An Item with the same key has already been added.
Symptoms: You see the following error being logged when you run your pipeline batch and you have a Resolve Sitecore Item Pipeline Step. You will probably see something like this in your Sitecore logs:
[Date/Time] ERROR An item with the same key has already been added.
[Date/Time] ERROR Pipeline step processing will abort because a critical error occurred during processing.
Possible Cause: You see this occur during the Resolve Sitecore Item pipeline step processor. It usually means you have duplicate field name on the item you are trying to create in Sitecore. Check the template of the Sitecore item you are attempting to create including any inherited templates and resolve any duplicate fields names.
Item is created in Sitecore but the fields are not populated.
Symptoms: You have a pipeline batch with pipeline steps that read data from a source, iterates the items and create an item in Sitecore. The Item is created with the correct identifier, however, the fields on the Sitecore item are not populated.
Possible Cause: Most likely this is an issue with Data Location field has been set incorrectly on a pipeline step. Go through each step and check the location value.
Hope this helps!
With the General Data Protection Regulation (GDPR) about to go into effect on May 25th, you should be aware of what Personally Identifiable Information (PII) could be stored in Sitecore 9 xConnect.
These fields are easily identifiable if you have installed Sitecore Data Exchange Framework 2.0+ and the xConnect Provider (provides the ability to read and write data to the collection and reference data services of Sitecore xConnect).
When transferring data between systems while some data will map directly you inevitably will come across data objects that need be transformed from the source format into a format that is compatible with the target object. Sitecore Data Exchange framework provides a mechanism and flexibility to reuse to Transform source data objects into the required format for the target data object.
A Value Reader allows you to read a value from a Source object. While it can read the value as is, they can also be used to transform the value. With each version of DEF Sitecore has provided a number Value Readers OOTB.
With DEF version 1.4.1 you have access to the following values readers:
With Sitecore 9.0.1 having just been released I was pretty excited to have a look at some of the improvements covered in the release notes. Not too mention a newer version of Sitecore Data Exchange Framework 2.0.1 and the OOTB providers.
Having a Sitecore 9 instance already installed and all its prerequisites means installing Sitecore 9.0.1 quick and easy to get up and running.
- Downloaded Resource Packages for XP Single (XP0) Instance configuration and extracted it to a resource folder c:\Sitecore\901\resourcefiles
- Scanned the 9.0.1 Installation Guide for changes or additional prerequisites and the only thing that jumped out was the updated version of Sitecore Installation Framework 1.1. It contains new features and some bug fixes covered in the release notes. I updated it by running the following cmdlet in Powershell:
- Copied the Powershell Installation script from chapter 4 and compared it to the script provided for the initial release. They are almost identical except for the updated package parameters used to configure xConnect and Sitecore:
Package = "$PSScriptRoot\Sitecore 9.0.1 rev. 171219 (OnPrem)_xp0xconnect.scwdp.zip"
Package = "$PSScriptRoot\Sitecore 9.0.1 rev. 171219 (OnPrem)_single.scwdp.zip"
- When defining the parameters at the top of the script I used the same values as Sitecore 9.0 script except for $prefix & $FilesRoot.
- Kicked off the script and watched it run along without any errors and it installed in 6 mins 46 secs.
Don’t forget to run the applicable Post Installation Steps covered in Chapter 6.
Now, where did I put that DEF 2.0.1 package download!!
If you are familiar with Data Exchange Framework you will know that apart from being able to run a number of pipelines, a Pipeline Batch can also provide summary information about the last time the batch was run:
- Date & time the batch was last started.
- Data & time the batch last finished.
- Log Messages that were captured during the last time the batch ran.
- Log Levels to determine what is written to the message log.
Prior DEF 2.0 the log messages were written to the Sitecore log file (by default) and written to the Messages field. You could also control the maximum number of messages that were written and displayed in the messages field.
With Sitecore 9.0 and the introduction xConnect you might be wondering what you are going to do with all that analytics data sitting in MongoDB, when you upgrade your Sitecore 8.x instance to Sitecore 9. Well, Sitecore have created a tool called the xDB migration tool. The tool uses the Sitecore Data Exchange Framework 2.0 and it provides the ability to read from MongoDB (source system) and write to the xConnect server (target system). If you are not familiar with the Data Exchange Framework I have covered it in an earlier post.
The xDB migration Tool provides a pre-configured mapping of data from the MongoDB database to the xConnect server in the form of a DEF Tenant. The Tool includes two data migration processes.
- MongoDB Contacts to xConnect Contacts – Reads contact data and related device and interaction data for each contact – from a MongoDB database and writes it to xConnect.
- MongoDB Reference Data to xDB – Reads geo IP and location data from a MongoDB database and writes it to xConnect.
I recently presented at Atlanta Sitecore Usergroup and gave a demo using Data Exchange Framework (DEF) 2.0 and the SQL Provider – to read customer records from a SQL table and create items in Sitecore. You can read more about the DEF 2.0 here. In this follow-up post, I’ll walk through the DEF configuration required for the demo.