Over the last two years, Salesforce has been developing an open-source machine learning library called Merlion that runs time-series analysis with machine learning.
It was originally designed to help solve the challenge that Salesforce’s application performance management team faced. Merlion is an end-to-end Python library for many time-series tasks, he explained, including anomaly detection as well as forecasting.
Salesforce’s application performance management team faced a challenge in late 2020, they needed to improve anomaly detection algorithms. The performance management team was monitoring the health of Salesforce’s data centres, which emit a lot of metrics in real-time, including CPU utilisation for any particular service. Those metrics generate what is known as time-series data. The Merlion project was started as a collaboration between Salesforce’s research teams in Palo Alto and Singapore.
A common challenge with any machine learning library is integration into production environments. That includes making the machine learning tool able to get the data in the way it needs, with access to the necessary compute resources and the ability to read the data back out to the appropriate places. Salesforce said the Merlion project has added some default options to deal with that challenge, providing users with a good starting point. The project makes the overall workflow easier to have operations be more automatic.
Merlion isn’t the first open-source project to attempt to help solve the challenge of time series analysis. Among the most popular is the Facebook-led Prophet project, which provides forecasting capabilities for time series data. Prophet didn’t meet Salesforce’s needs, as it only has a subset of Merlion’s features, including pre-processing, modelling, evaluation, and post-processing. That’s why Salesforce decided to build its project and then open-source it.
As an open-source project, Merlion can be used internally at Salesforce and by anyone looking for a time-series data machine learning analysis framework.
