WSO2 APIM Analytics 2.6.0

WSO2 API Manager team will be announcing the release of the 2.6.0 version of its popular API Management product soon. As most of you might know, every release of API Manager comes along with a corresponding analytics distribution which is responsible for batch and real-time analytics. This release is not an exception, but there has been a considerable amount of change on the analytics component from its previous versions. In this article, we will be focusing on what has changed and how you could better leverage APIM Analytics 2.6.0 to cater your analytics requirements.

What has Changed?

• From APIM Analytics 2.0.0 to 2.5.0, the distribution was based on WSO2 Data Analytics Server(Carbon4). APIM Analytics 2.6.0 is based on WSO2 Stream Processor(Carbon5).
• All batch and real-time analytics are now done via WSO2 Siddhi whereas previous versions relied on Apache Spark for batch analytics.
• Simplified alerts story by giving more control to the user on configuring when the alerts should be triggered.
• Microgateway Analytics bundled in.
• Redesigned stream publishing mechanism from Gateway to Analytics Server to yield a higher performance.
• No more database sharing between Analytics Server and API Manager.
• Configuring template parameters for alerts is now done at the Analytics Server instead of the APIM admin portal.

Let’s look at each of the points mentioned above in detail.

Moving Analytics to WSO2 Stream Processor

WSO2 Stream Processor is the replacement introduced for WSO2 Data Analytics Server to strengthen the analytics component of the WSO2 platform. This product is based on a new version of Carbon Framework (Carbon 5) which is more lightweight than its predecessor Carbon4. With its highly improved startup time and its profile story, WSO2 Stream Processor provides many more features to support today’s enterprise analytics requirements.

API Manager team decided to inherit these features for its API Manager Analytics distribution and has migrated all APIM Analytics artefacts to work on top of WSO2 Stream Processor.

WSO2 Stream processor introduces few profiles(OSGI runtimes). WSO2 APIM Analytics Server makes use of the worker profile in normal use, and the dashboard profile when Business Rules Manager UI needs to be accessed.

Therefore starting the APIM Analytics Server in normal use has to be done via the following command.

Linux -> sh worker.sh

Windows-> worker.bat — run

More info in can be found here.

Siddhi Replacing Spark

WSO2 DAS used siddhi SQL for real-time processing and Apache Spark for batch processing. With its completely redesigned syntax and architecture, now WSO2 SIddhi supports batch processing as well. Therefore WSO2 API Manager Analytics 2.6.0 has completely eliminated the dependency on Spark and rewritten all the aggregation logic using Siddhi only.

Alerts — Real Time Analytics

One of the complications API Manager Analytics had on Alerts is that the threshold values for some of the alerts were dependent on percentile values which were calculated iteratively by the Analytics Server. This resulted in a model where generating alerts were not that straight-forward and the user story was a bit complex to understand for a newbie. This release has taken these points into account and completely eliminated the percentile values and replaced it using user-configured values as thresholds for triggering alerts. This means now you can generate an alert if the response time for your API is higher than a specific value which can be configured separately for each API. And if no value is configured, there will be no alerts of that alert type for this API. Yes, that simple.

These configurations can be done in both Publisher and Store portals by clicking on Manage Alert Types and then clicking on the settings icon on the right corresponding to the alert type. Following alert types support this feature.

• Abnormal Request Count (Store)
• Abnormal Response Time (Publisher)
• Abnormal Backend Time (Publisher)

The user stories for each alert type have been revisited and modified to make it more user-friendly. The redesigned alert types documentation can be found here.

Microgateway Analytics

APIM Analytics 2.6.0 comes with micro-gateway analytics support by default whereas in 2.5.0 this was supported by adding extensions to the Analytics Server. You can use the same analytics server to publish events from the normal gateway(Synapse based) and the micro-gateway(Ballerina based). Publisher and Store portal analytics will be showing you an aggregated view of statistics of both types of gateways.

WSO2 Admin Portal has a new feature where you could download a PDF report of the API traffic for all of your micogateways. If your deployment consists of multiple microgateways, this report will provide you with an aggregated request count for all microgateways given the year and month. The motivation for this feature is to use this report for billing purposes.

Event Publishing Revamped

Previous versions of APIM used to publish 3 events to the analytics server for each API call. Namely the request, response and the execution-time events. This model resulted in the Analytics Server requiring to support 3x times of throughput than the gateways which publish events to this server. With this release, now all three events are combined into one single stream by carefully looking into the common properties published by each event stream. Therefore with the revamped model, each API call will only publish a single event to the analytics server for each API call. This is a huge advantage for high traffic APIM deployments. This event publishing mechanism is supported in both Synapse and microgateway.

The definitions of each event stream published from the gateway to the Analytics server can be found here.

Database Sharing — Not Needed

In previous versions, the statistic charts fetched data from the STATS_DB which was a shared database between the API Manager publisher/Store nodes and the Analytics Server. Analytics Server was responsible to write summarized data into this DB, and the APIM portals were responsible for fetching data from this and rendering out various charts. For demo purposes, it was required to unpack both APIM and Analytics Server to the same directory so that it could share the common STATS_DB h2 database referenced from each distribution(Although this was not required if you are using any other DB server for this database). Version 2.6.0 no more fetches data from this DB, rather it uses a REST API exposed in Analytics Server to fetch this data. This will also simplify analytics configurations since there will be no more data-source configurations related to analytics on APIM distribution.

Template Manager → Business Rules Manager

If you have worked with Analytics, you might be aware of the APIM Admin Portal UI which was responsible to configure server vide alert parameters. This was called the template manager(because it manages templated analytics parameters). All previous versions had this UI in both admin portal and Analytics Server Management Console. In 2.6.0 this UI is removed from the admin portal and moved to the Analytics Server dashboard profile to make it the single place to manage server vide alerts parameters. Additionally, the naming for this configuration UI is now changed to Business Rules Manager.

These parameters are per alert type. In order to view this dashboard, you need to start the APIM Analytics Server in the dashboard profile as follows and visit https://localhost:9643/business-rules in your browser.

sh dashboard.sh

More information can be found here.

I hope this article helped you to understand how APIM Analytics 2.6.0 stands out from its previous versions. Give it a try, and let us know what you think. The links to the latest packs are given below.

APIM 2.6.0 RC3 https://github.com/wso2/product-apim/releases/tag/v2.6.0-rc3

APIM Analytics 2.6.0 RC3 https://github.com/wso2/analytics-apim/releases/tag/v2.6.0-rc3