Instrument Sitecore SQL Container for Prometheus

In the previous post in this series on monitoring Sitecore containers I showed you how to instrument your Solr container, in this post, I’ll show you how to instrument the Sitecore SQL container and expose metrics for Prometheus to scrape.

SQL Exporters

There are several SQL exporters available in the Prometheus community some of these are SQL agnostic. Those that support MSSQL, either obtain their metrics via performance counters or by querying SQL directly. I recommend you review the different exporters and then choose an exporter that meets your specific requirements or you create your own custom exporter.

Lets take a look some of these exporters and see if they would be a good fit for exporting metrics from Sitecore SQL container.

Continue reading

Prometheus File Based Service Discovery

One of the challenges with having a pull model like Prometheus is that it needs to know where the targets are located so it can scrape the metrics. While we can configure static scrape targets in the Prometheus configuration file directly for our local environment I discovered anytime I want to make changes to these setting or add a new target I must restart Prometheus. This can be very annoying, as compared to my Sitecore containers, the Prometheus container takes much longer to stop and restart. So I needed to find a better solution for configuring these targets locally.

Service Discovery

Service Discovery helps solve the problem of identifying scrape targets which is really useful in an orchestrated environment as it will dynamically discover targets. There is support for several common services like:

There are several other methods of Service Discovery supported by Prometheus:

Continue reading

Sitecore Personalization Rules Vs Caching

I recently ran into an issue with a personalization rule on a rendering that changed the data source for mobile devices, a common scenario if you want to reduce payload and improve mobile experience. However, I discovered if I hit a page with the component on a mobile device I was seeing the desktop version.

Using my SPE script to list component cache settings I discovered the component had “Vary By Data” enabled – could this be the problem? As it turns out it it was. You see Personalization rules enable dynamically changing of components and their data sources. However the “Vary By Data” option causes Sitecore to render cached html markup for the component without performing an actual rendering process. For the majority of scenarios applying personalization rules before retrieving output from cache would make caching almost pointless. But there are certain scenarios, like the one I described above, where this might be beneficial and improve performance. Thankfully, there is a solution using a custom “Vary By Personalized Data” covered here by Ahmed Okour.

It does feel like a gap in Sitecore’s implementation so you either Cache a component or you Personalize it. So this needs to be captured when defining requirements.

Is this component going to be personalized?

  • No – implement caching.
  • Yes – don’t implement caching.
  • Not sure – implement caching and disable it when the component needs to be personalized.
  • Need both – customize with Vary By Personalized Data.

Useful Info

Reset Sitecore Publish Service tables

We ran into an issue with the Sitecore publishing service whereby the following exceptions were being logged:

[Information] Executing Cleanup Task : "PublishOperationCleanupTask-60832bb9509e4a37855285a8346a6a53"

[Error] There was an error adding 1 publisher operations. - Error : "Execution Timeout Expired. The timeout period elapsed prior to completion of the operation or the server is not responding." 

Following some investigation we discovered this was caused by the PublishOperationCleanUpTask running on the PublishService.

The PublishOperationCleanupTask works like the following:

  1. The “Publishing_TargetSyncState” table contains entries containing a publishing target, language, and timestamp
  2. These entries are updated when one of the following publishes is done:
    a. Incremental publish
    b. Publishing the root “sitecore” item with subitems
  3. The PublishOperationCleanupTask checks the entries in the “Publishing_PublisherOperation” table, and removes entries that are older than the timestamp in the “Publishing_TargetSyncState” specific to the language.
  4. This task runs by default once a day but can be configured to run as frequently as required.

So this job is suppose to cleanup Publishing_PublisherOperation table, however on inspection of this table we discovered it contained much older records that had not yet been removed. Indicating the job was having problems completing its task or at least it was not able to cleanout the records.

Part of the solution to resolve the Timeout Expired exception was to increase the <CommandTimeout> setting located in the sc.publishing.sqlazure.xml file. The default value is set to 120.

But what if you wanted to just cleanup the publishing service tables? The following tables are created in the Sitecore master database when you install and setup the publishing service:

  • Publishing_ActivationLock
  • Publishing_Data_Params_FieldIds
  • Publishing_Data_Params_Languages
  • Publishing_JobManifest
  • Publishing_JobMetadata
  • Publishing_JobQueue
  • Publishing_ManifestOperationResult
  • Publishing_ManifestStatus
  • Publishing_PublisherOperation
  • Publishing_TargetSyncState

To reset these tables you can run the following command from the Publish Service webroot: Sitecore.Framework.Publishing.Host schema reset –force

NOTE: The only risk in running this would be if you have any publishing jobs that are in-flight will need to be republished.

Useful Links

Happy Publishing!

Instrument Sitecore Solr Container for Prometheus

So far in this series I have provided a brief introduction to Prometheus and shown you how you can configure Prometheus to monitor Docker and the HostOS metrics and visualize performance metrics using Grafana. In this post I’ll show how to instrument the Sitecore Solr Container to expose performance metrics for Prometheus to scrape display those metrics in Grafana. I’ll go into a bit more detail on Solr metrics, than what I shared during my Sitecore Symposum presentation: Insufficient facts always invite danger, Captain!, which is still available on demand.

Continue reading

Build Coveo for Sitecore Docker image variants

I’ve been developing with Sitecore on Docker for almost a year now… WOW time flies when you are having fun! While the Sitecore Docker image repo is awesome and provides a range of Sitecore variants, you will probably discover the need for additional image variants, depending on your project. As you often require additional modules to be installed as your base Sitecore install, like the Coveo for Sitecore module! Rather than your team having to install this module manually when they docker compose up, it would much quicker and easier they have a variant image they can use with it already installed. In this post, I will show you how to can build a docker image variant for Coveo for Sitecore v5 using the Sitecore 9.3 images.

Continue reading

Tips for creating useful Release Notes

You’ve just come to the end of your sprint and it’s time to ship a new release to the QA environment for testing. But what about the release notes? In this post I’ll share some tips on creating useful release notes your team and customer will actually read and use. I explain why they are important and how they can help ensure a smoother release through to production.

Why do we create release notes?

Release notes are a way to convey to the business as well as the QA team what changes are included in the upcoming release. They are usually generated by Dev Team Lead as he/she is the gatekeeper for dev and responsible for what is allowed to leave development and go to higher environments.

Continue reading

Tips for Monitoring and Tuning Sitecore Cache

Utilizing caching improves Sitecore performance and there are plenty of posts on this topic. While doing some performance analysis on an inherited solution I was surprised to find that caching was not being utilized to its full potential. In this post, i’ll share some useful scripts and tools that make the task of monitoring and tuning cache settings easier.

Continue reading

Handy JMeter Script to crawl your site

JMeter is my tool of choice for load testing. This isn’t necessarily a load test but rather a useful script to crawl site urls using a sitemap. By hitting every accessible url allows Sitecore to populate all the various caches and provides you with a sense of the potential sizes of the various Sitecore caches and helps you identify any tuning that may be required.

Continue reading