AI Driven API Analytics
In an era where artificial intelligence (also known as machine intelligence) is creeping into many software systems, enterprises are trying to leverage them by introducing these techniques to its own enterprise systems. Conventional software is programmed to work with static rules whereas software based on Artificial Intelligence provides a unique ability to act according to rules which are constructed by itself. This powerful method provides systems to be able to develop algorithms which are considered to be hard problems for conventional algorithms — detecting abnormal activity, predicting future activities, facial recognition etc.
As a complementary technology for API Management, WSO2 API Manager introduced several intelligence-based features to its analytics offering WSO2 API Manager Analytics : starting from version 2.0.0. As a result of introducing these features to WSO2 API Manager, its analytics capabilities have been strengthened to learn and detect abnormal activities. Once an abnormal activity is detected, stakeholders (API developers, application developers or system admins) will be alerted depending on the type of event. Thereby providing the ability to take precautions to protect their systems and also to take reactive measures to minimize user dissatisfaction. These features have been developed in a way, where learning occurs continuously and evolves with time (online learning) whereas a typical machine learning model is trained with only a set of data and its intelligence will be static. This ensures that the learning algorithms used in WSO2 API Manager Analytics, learns and adjusts decisions it takes with more data it acquires.
Let’s explore these features by taking a conceptual airline ticketing application.
FlySmart Inc is an organization which owns a set of APIs which can talk to several airlines to gather information on various flights. This organization decides to develop a mobile application where its users can search flight details and purchase them via the application. The CEO of FlySmart Inc assigns the task of developing this application to its lead developer — Jack. Since FlySmart Inc uses WSO2 API Manager as its API Management platform to expose its APIs, Jack can create an application in WSO2 API Manager developer portal and subscribe to the necessary APIs via this application. He designs the application in a way where all API calls from the mobile application will flow through WSO2 API Manager. Jack launches the mobile application FlySmart, to the public and suddenly people from all over the world start using it.
FlySmart app, mainly makes use of the following API resources to perform its tasks.
- /search — searches for flight information
- /buy — purchase tickets
- /cancel — cancels a purchased ticket
Tom is a travel blogger, who loves to travel to various destinations all over the world. He starts using the FlySmart application and loves all its features.
Incident 1 — Abnormal Resource Access Pattern
Tom usually purchases all his flight tickets via this app, and very rarely cancels a ticket he has already purchased. Therefore most of the time the applications calls the /search API and followed by /buy API. Seldom, he may cancel the purchased ticket, requesting for a refund.
Since all API calls flow through WSO2 API Manager, it’s learning algorithms will start learning the resource access pattern for user Tom and all other end users. During this period it will build a Markov Chain Model for each end user. Until a specific number of requests are made from Tom’s application, API Manager will not register an activity as abnormal since it is too early to decide whether an event is abnormal. Once it receives a considerable amount of requests (configurable), it will then start detecting abnormal patterns and alerting the application developer (or any email address provided) providing information about the activity. The relevant stakeholders can then contact the user via email or any other medium informing about the suspicious activity.
Unfortunately, Tom had written his user credentials on a note beside his table, and an intruder steals this information and logs in to the application using Tom’s credentials. The intruder decides he is going to purchase a ticket and then cancels it, and does it all over again, many numbers of times.
WSO2 API manager detects this abnormality using its learning features and sends out an alert to all stakeholders.
It may also decide to temporary block the user based on the severity of the attack. Jack receives an alert with this information and contacts Tom via email informing about the abnormal access pattern received from his mobile application. Upon receiving the mail Tom realizes his credentials have been stolen and changes his credentials with immediate effect. The intruder will not be able to perform the bogus calls anymore.
Incident 2 — Abnormal Overall Response Time
Usually, the search API responds to the mobile client within 300 ms of latency. Occasionally the value may rise up to 500ms, but this happens very rarely. API Manager learns this and adjusts its alert thresholds to a suitable value. An engineer in FlySmart Inc decides to add some transformation logic in the API Management layer, and this results in the average response time to rise up to 600 ms. The change has been done by a team member of FlySmart app without informing the original API developer. Once the change is in effect, the original API developer or any stakeholder configured to receive alerts for this API will receive an alert informing about the abnormal response time for search API.
Upon discussing with the development team, they realize the delay is introduced by the transformation logic and may decide to live with the new latency as the transformation is important.
API Manager Analytics will learn that the new API calls almost always have a latency of 600 ms and raise its own alerting threshold to an upper value without any human involvement.
Incident 3 — Abnormal Response Time from Tagert Servers
Due to a package update of the Linux operating system on the backend servers, the latency of API calls processed by the target servers has increased. API Manager will detect this abnormality and figure out the delay is actually from the target servers and not within the API Management layer. Then it will send out an alert to the stakeholders with the relevant information.
Stakeholders may decide to downgrade the package and stabilize the delay. If such an action was not taken, suppose that the package update was because of a security vulnerability and cannot be downgraded, then API Manager Analytics will learn that the new latency is becoming normal and stop alerting for the earlier abnormal value. It will raise the abnormal threshold value to a new value just as in incident 2.
Incident 4 — Abnormal number of requests being processed.
Some attackers create an internet bot to repetitively call the search API and stress the backend servers. API Manager will detect that the request count for this API has suddenly spiked up and alert the stakeholders.
Upon receiving the alert, they can take precautions to block the suspicious calls. It also may, depending on the severity of the attacking, block certain IP addresses and/or applications making the denial of service attacks.
However, the request counts may rise seasonally, such as when holiday seasons are nearing the number of tickets being bought may rise. In such a case, API Manager will learn the new request pattern and adjust its alert thresholds accordingly.
Summary
WSO2 API Manager Analytics can perform AI driven analytics by analyzing API usage patterns and take necessary precautions when abnormal activity is detected within your enterprise API Management layer. These learning algorithms have the ability to evolve with time and adjust its alert thresholds to new values by taking into account the recent data as well. Following are the alert types which involves learning.
- Abnormal Request Patterns.
- Abnormal Overall Response Times.
- Abnormal Response time from Target Servers.
- Abnormal number of requests being processed.
All of these alert types and a few more have been available since API Manager version 2.0.0. All intelligent learning and decision making is carried out by WSO2 Siddhi and Apache Spark which are built-in components of WSO2 API Manager Analytics. Enterprises can also build their own intelligent algorithms by implementing new siddhi execution plans or spark scripts and define new alerts based on them.
We hope this article illustrated how your enterprise can benefit from introducing intelligence into API Management. For more information please refer to the following links.
https://docs.wso2.com/display/AM250/Alert+Types
https://docs.wso2.com/display/AM250/Subscribing+for+Alerts
