At the end of 2016, I embarked on a project with Greg Gonzalez to incorporate Columnstore into the SentryOne database. Larger customers were experiencing increasing storage needs while also needing faster query response times. Most of our larger customers were monitoring between 200 and 250 targets per SentryOne database before performance became an issue. We wanted to break through that barrier. Microsoft had recently made Enterprise edition-only features such as Columnstore, available across all editions in 2016 SP1.
In early 2017, we were encouraged by the query response times we experienced with Columnstore in our test environment. We innovated a creative solution with Columnstore, In-Memory OLTP, and Partitioning. This combination optimized query response times while reducing the storage, memory, overall compute, and even database maintenance requirements. Microsoft’s decision to unlock that massive feature set allowed us to deliver a better monitoring solution for our customers.
By the middle of 2017, we implemented this solution in our production environment as well as the environments of some of our customers. We were all blown away by the results. We continued to tweak things and optimize the solution. Remember that ceiling of 200 to 250 targets? Well, we now have customers with over 800 targets on a single SentryOne database and we still haven’t hit a new performance barrier.
Here’s what the SentryOne database system looks like when monitoring 800 targets on a commodity VM — as you can see there is headroom to spare in all resource categories:
The SentryOne dashboard showing how its own SQL Server database performs while monitoring over 800 targets, running on a virtual machine with only 16 CPUs and 88GB of RAM.
Recently, Microsoft contacted us to write about our successful solution. You can read our story on their website, “Solution provider breaks through performance ceiling with data monitoring innovation.”
