Fixing High CPU Usage in Logstash

March 31, 2015

This was originally posted to engineering.blopboard.com which is now defunct.

I blogged previously about our Logstash installation. Some time after we’d set it up, we noticed the CPU usage on the Logstash server was constantly at or near 100%, even with only minor load. Unfortunately, this wasn’t the type of issue that was easy to google for and it took some time to track down a working solution, so I thought I’d share it here.

The source of the solution ended up being a thread on a Google Group for Logstash users. The issue seemed to be that the logstash-web service was continuously trying and failing to bind to its TCP port, which may have been due to the port lockdown that was part of our ElasticSearch vulnerability response.

Since we weren’t actually using the logstash-web service, the simplest solution seemed to be what the thread suggested: shut the service down and configure it not to start automatically. Per this message in the thread, this was as easy as running these commands:

sudo service logstash-web stop
echo manual | sudo tee /etc/init/logstash-web.override

Once we did this, our CPU usage dropped off immensely, as shown in the screenshot below.

This is our CPU decreasing after logstash-web was stopped

Hope this solution helps others. Thanks for reading!

About the Author

Matthew Turland is a Senior Engineer at Blopboard. He has been working with PHP since 2002 and is the author of "php|architect's Guide to Web Scraping with PHP" and co-author of SitePoint's "PHP Master: Write Cutting-Edge Code." In his spare time, he likes to bend PHP to his will to scrape web pages and run IRC bots.