In the previous post in this series on monitoring Sitecore containers I showed you how to instrument your Solr container, in this post, I’ll show you how to instrument the Sitecore SQL container and expose metrics for Prometheus to scrape.
There are several SQL exporters available in the Prometheus community some of these are SQL agnostic. Those that support MSSQL, either obtain their metrics via performance counters or by querying SQL directly. I recommend you review the different exporters and then choose an exporter that meets your specific requirements or you create your own custom exporter.
Lets take a look some of these exporters and see if they would be a good fit for exporting metrics from Sitecore SQL container.
One of the challenges with having a pull model like Prometheus is that it needs to know where the targets are located so it can scrape the metrics. While we can configure static scrape targets in the Prometheus configuration file directly for our local environment I discovered anytime I want to make changes to these setting or add a new target I must restart Prometheus. This can be very annoying, as compared to my Sitecore containers, the Prometheus container takes much longer to stop and restart. So I needed to find a better solution for configuring these targets locally.
Service Discovery helps solve the problem of identifying scrape targets which is really useful in an orchestrated environment as it will dynamically discover targets. There is support for several common services like:
There are several other methods of Service Discovery supported by Prometheus:
In the previous post I shared how you create Coveo for Sitecore variant docker images. In this post i’ll share some tips when running Coveo for Sitecore in a Container how to configure and persist your trial organization details.
I’ve been developing with Sitecore on Docker for almost a year now… WOW time flies when you are having fun! While the Sitecore Docker image repo is awesome and provides a range of Sitecore variants, you will probably discover the need for additional image variants, depending on your project. As you often require additional modules to be installed as your base Sitecore install, like the Coveo for Sitecore module! Rather than your team having to install this module manually when they docker compose up, it would much quicker and easier they have a variant image they can use with it already installed. In this post, I will show you how to can build a docker image variant for Coveo for Sitecore v5 using the Sitecore 9.3 images.
Performance tools, like dotTrace profiler are really useful when trying identify potential bottlenecks in your Sitecore application. Normally, attaching to the w3wp.exe process and profiling your local Sitecore instance is relatively straightforward, however now that we are taking advantage of Docker and running our instances in containers the process is slightly different. In this post I’ll show you how to profile your Sitecore instance running in containers.
In part 1 of this series, I introduced Prometheus and lay some foundations for monitoring a Sitecore container environment using Prometheus. In this post, I will show you how to collect and monitor the Host OS and container metrics exposed by Docker.
Why should we care about Docker the Host OS Metrics?
To monitor your containerized Sitecore application effectively you need to understand the health of the underlying system your containers are running on. To achieve this, you have to monitor the system metrics like CPU, memory, network, and disk also docker is starting and stopping containers so you need to know the state of your docker resource.
How to collect Docker performance metrics
Docker supports Prometheus OOTB and provides performance metrics you can collect and monitor, however it is disabled by default.
1. Enabling Docker Metrics.
First things first lets enable the metrics. Open the daemon.json file and add the metrics-address setting:
“metrics-addr” : “127:0:0:1:9323”
Apply and restart docker. Now you should be able browse to the docker metrics 127.0.0.1:9323/metrics and verify metrics are exposed.
In a previous post I provided you with some techniques to help you monitor your Containerized Sitecore instance using native tools. Over the next few posts, I will show some of the common tools and techniques used for monitoring applications running in containers. When I initially started thinking about this I considered using InfluxDB as my time series DB to store the performance metrics and Grafana for visualizing and alerting. As both of these were already in my tool-set for load testing. However, the more I dug into monitoring container performance I quickly realized, Prometheus has established itself as the leading tool in this space and Docker also provides support for Prometheus – more on that later. So let’s start with an introduction to Prometheus and lay some foundations for a monitoring platform.
What is Prometheus?
Prometheus is an open-source application for monitoring systems and generating alerts. It can monitor almost anything, from servers to applications, databases, or even a single process. Prometheus monitors your defined targets by scraping metric data in a simple text format that is exposed by the target. Prometheus stores this metric data in a multi-dimensional data model by metric name and key/value in its timeseries database which can then be queried and retrieved using its own query language PromQL, in a nutshell.
Let’s take a quick look at the the main components and architecture that comprise of the Prometheus platform.
Over the course of several months I have had the opportunity to introduce the use of docker within development teams as an alternative approach for developing with Sitecore. Developing with Sitecore on Docker requires a bit of a mind shift from your normal way of working with Sitecore. But if offers many benefits like the ability to stand up a fully scaled instance of Sitecore in seconds rather than hours. This blog provides a introduction to docker and getting started developing with Sitecore on Docker.
What is Docker
Docker is a platform that enables you to develop, ship, and run applications as containers. The Docker Daemon process (or engine) runs on your host OS which manages images and containers.
The Command Line Interface (CLI) communicates with the docker daemon via rest API.
Containers vs VMs
Containers are an abstraction at the app layer that packages code and dependencies together. Multiple containers can run on the same machine and share the OS kernel with other containers, each running as isolated processes.
take up less space.
more portable and efficient.
require fewer VMs and Operating systems.
reduce cost as fewer licenses are required.
start in seconds.
are ephemeral – treated like cattle in your DevOps service model.
VMs are an abstraction of physical hardware turning one server into many servers. The hypervisor allows multiple VMs to run on a single machine.
includes a full copy of an operating system, the application, necessary binaries and libraries – are much larger in size.
VMs can also be slow to boot.
start in minutes
tend to hang around longer – treated more like pets in your DevOps service model.
By combining Containers and VMs in your infrastructure provide a great deal of flexibility in deploying and managing your application.
Sitecore Docker Repo
The Sitecore Docker Repo contains docker files for a plethora of Sitecore versions and their variants including: JSS, SXA, SPE. Also necessary scripts, tools and supporting documentation to help you build Sitecore Docker images and ultimately run a containerized instance of Sitecore.
Lets backup for those of you that may not be familiar with some of these terms:
Dockerfile – is a text document that contains all the commands you would normally execute manually in order to build a Docker image.
Docker can build images automatically by reading the instructions from a Dockerfile.
Docker Images – are the basis of containers. An Image is an ordered collection of root filesystem changes and the corresponding execution parameters for use within a container runtime. An image typically contains a union of layered filesystems stacked on top of each other. An image does not have state and it never changes.
Docker Container– is a runtime instance of a docker image.
Thanks to the hardwork and dedication to the individuals in our community that contribute to the Sitecore Docker repo. Which is continually evolving.
The ability to monitor your Sitecore instances is essential. Knowing your application is available and performing in production as expected is critical. To be proactive and respond to changes in performance as opposed to reactive after an incident has occurred, you need a continuous overview of the state of the application and the underlying infrastructure. This involves gathering metrics like CPU, memory usage, and storage consumption as well as any application-related metrics. However, when monitoring a containerized Sitecore instance we cannot apply our usual methods and tools to provide us with all the data we need. In this post, I will show you some of the native tools to help you monitor Sitecore containers.
The docker stats command display a live stream (updated every second) of running containers resource usage statistics docker stats:
CPU % – the percentage of the host’s CPU the container is using.
PRIV WORKING SET – refers to the total physical memory (RAM) used by the process.
NET I/O – the amount of data the container has sent and received over its network interface.
BLOCK I/O – the amount of data the container has read to and written from block devices on the host.