When you use the self-hosted version of Matomo on-premise instead of the Matomo cloud-hosted solution, you may experience some traffic peaks on your Matomo server when the traffic volume on your websites increases. For example, every day at a certain time you might receive two or three times the amount of traffic that usually visits your website. This can have many negative impacts, including:
- Slow loading time for your JavaScript tracker (piwik.js) which in turn may slow down your website giving your users a poor experience. Also you may see less page views in Matomo because by the time the tracker is loaded on your website, the user has already moved on to another page.
- Some tracking requests might be simply ignored at some point because your server might not be able to handle any tracking requests anymore which results in many untracked visits and page views.
- You may need additional servers only to handle traffic peaks which results in increased server costs, maintenance work and maintenance costs.
The solution
Handling traffic peaks has been possible with Matomo for years using the Queued Tracking plugin. When this feature is enabled, tracking requests are put into a queue instead of being processed immediately. Then when a job is running separately it takes the requests out of the queue and processes them. This brings various benefits.
Faster tracking
It improves the tracking speed on your server by a factor of 5 to 15. So for example, instead of a tracking request taking 50ms, it takes only 5ms. This means your server will be able to handle a lot more concurrent requests compared to the traditional tracking and is likely to survive traffic peaks much more likely without any trouble at all.
Faster processing
When a request is queued, the request still needs to be processed eventually. Because the Queued Tracking solution can take multiple tracking requests out of the queue at once and process them in one go, the processing speed increases massively as well. This is because by default each tracking request has to bootstrap Matomo and do a lot of things again and again which takes quite a bit of time (you’d be surprised). Instead, many things can now be cached and don’t have to be done multiple times. As a result, your server can process tracking requests much faster and needs less resources overall which in turn reduces cost and trouble.
Queued Tracking is now easier to set up
In the background, Queued Tracking has been using Redis, an in-memory database. While Redis is very fast, it’s not simple to setup and maintain it. Especially when it comes to making Redis “highly available” and when you need to scale your Redis. Also, your servers will need a lot more memory for Redis as all queued tracking requests are stored in memory.
One click setup
We have now added support for a MySQL database so you can activate Queued Tracking with a simple click. What used to take hours or maybe even weeks to set up and a lot of maintenance, can now be cut down to seconds. Queued Tracking will then simply reuse the database that you have been using all along for storing all your visits. A side benefit is that your server won’t need more memory and all queued tracking requests even survive a server reboot.
Both Redis and MySQL are now supported in Queued Tracking. If you do have experience with managing Redis, we still recommend using this solution as it’s likely a bit faster. However, in most cases the MySQL solution should work just as well.
Further improvements
We have made various other improvements for Queued Tracking that increases the performance and you can now be notified when the number of queued tracking requests reaches a certain threshold. View the changelog for a list of all changes.
Learn more
We have been setting up Queued Tracking multiple times when it comes to high volume traffic or dealing with peaks and are amazed by the results. Often, we can even reduce the overall amount of needed servers.
If this sounds like something that could be beneficial to you, we recommend you have a look at the Queued Tracking page and also check out the FAQ. You might be also interested in learning how to configure Matomo for speed.
Need help with setting up, maintaining, or scaling Matomo? Get in touch now.
