LAMA: The Brokerage Firm’s Framework for Staying Ahead of the Curve

Brokerage firms are constantly under pressure to stay ahead of the competition. They need to make sure that they are using the latest technology and techniques to provide their clients with the best possible service. With constant advancements of technologies and integrations used by these brokerage systems, technical issues do arise.

With the increase in volumes of trading with retail trading, the National Stock Exchange of India (NSE) – one of the largest stock exchanges of the world, needed a system to prevent technical incidents and outages across different market participants.

To address these technical operational incidents Securities and Exchange Board of India (SEBI) formed a working group. This SEBI working group proposed proactive monitoring and problem detection system – Log Analytics and Monitoring Application (LAMA).

What is LAMA?

LAMA is an API based performance data gathering and early warning system that is designed to help brokerage firms identify and address potential problems before they cause an outage. It collects data from the broker’s trading systems and uses machine learning to identify patterns that could indicate a problem. If LAMA detects a potential problem, it will generate an alert so that the brokerage firm and exchange can take preventive corrective action. It is a cost-effective way to prevent outages and reduce the risk of financial losses.

NSE India has adopted LAMA and LAMA reporting is a requirement for brokerage firms. There are heavy penalties for non-compliance with LAMA. Non-compliance is NOT an option.

LAMA Key Metrics
Application Hardware Database Network
Log Monitoring CPU Replication Status Bandwidth
Request/Second – throughput Memory Replication Queue Size Packet Error Count
Response Time Disk Replication Bandwidth
Trading API Failure Count Uptime Replication Latency
Client Authentication Failure Count
Historical Requests/Second
Historical Response Time

Reporting to Exchange

All the LAMA Key metrics details reporting is to be done at every 5 minutes for 24 hours on all trading days.

LAMA compliance challenges

Time-consuming:  Time consuming manual posting of data to LAMA is not even an option because of the frequency and the extent of data reporting requirement.

Inadequate monitoring data:  Most brokerage firms don’t have adequate monitoring on all the required systems to be able to report required data to NSE at required intervals.

How Applicare helps?

Applicare provides an out of the box implementation of LAMA that can be deployed in any brokerage firm’s environment in a matter of hours. With our current banking and brokerage customers, Applicare has been reporting data to NSE since NSE LAMA has gone live. We have worked closely with our customers, NSE LAMA team and our partner TA3S in developing and fine tuning Applicare LAMA implementation.

Applicare is scalable to meet the needs of any stock broker, regardless of their size. This means that it can be used by both large and small stock brokers.

The Applicare is an asset for any organization that wants to improve its IT operations. It helps you in identifying and troubleshooting problems, improve performance, and make better decisions about your IT infrastructure. It is built from ground up by administrators and developers who faced these challenges every day and wanted to automate the repetitive tasks.

Applicare provides full stack observability, self-detects problems and becomes foundation for self healing.

 

Applicare Agent

The Applicare agent is a key component of the Applicare monitoring solution. The agent is easy to install and use. It collects data from your virtual machines, systems and applications, and then sends it to the Applicare controller for processing and analysis.

Applicare Controller

The Applicare controller posts the data to LAMA API. All the posted data is saved for the historical purpose. Historical records can be used to identify patterns and trends that can help brokerage firms make better decisions. By analyzing them, brokerage firms can identify patterns that may indicate a problem with their trading system.

LAMA Key Metrics Dashboard

Application

 

 

 

 

Hardware

 

 

 

Database 

 

 

 

Network 

 

 

 

LAMA API Failure Notifications

If there was a failure while posing the key metrics to the LAMA API gateway, the Applicare notification service will notify the brokerage firms with its efficient logging mechanism. The notification service can be customized to meet the specific needs of the customers.

 

 

 

 

 

Conclusion

Brokerage firms are responsible for the financial losses of their clients. If an outage occurs, the firm may be liable for the losses incurred by their clients. Applicare can help protect the firm’s reputation by preventing outages and keeping their trading systems running smoothly. Applicare is known for its flexibility and ease of use, which allows it to be customized to meet the specific needs of each customer. Applicare makes the LAMA data posting process hassle-free and user-friendly for the brokerage firms.

Getting started with Applicare is easy. Simply register and get access to free on prem or in cloud Applicare.

 

 

A frequent scenario and a quick root cause analysis with Applicare

This morning when I logged in, I saw an alert from a test environment where we are constantly testing Applicare. I was informed of slowness in JPetStore transaction’s response time in the middle of the night.

 

I drilled down into those transactions and immediately saw that any backends/databases were not the cause of the issue. This helped me immediately eliminate them from the root cause analysis.

 

I could also see from Top Business Transactions that SQLs were getting processed fairly quickly and hence weren’t causing the delay.

 

By focusing on the JPetStore app server and host I could clearly see that # of processes running on the host has gone up during that period. I also saw disk transfers/sec has gone up significantly.

 

That made me look at what processes were running on the machine and which one of those processes was consuming resources. I saw MsMpEng – Windows Security component formerly called Windows Defender is consuming the resources and that caused the slowness in the transactions.

 

This is just an example scenario that we see in real life many times. We could have gotten to the same root cause results many other ways too e.g. from a higher CPU alert right in my inbox that includes top processes at the time of alert. But I choose the longer route to show the root cause analysis process which will come handy in many other scenarios.

Happy Applicare-ing for your apps and systems 🙂

 

Is just systems monitoring good enough?


We are often asked this question – we are monitoring our systems and able to keep their uptime high. Isn’t that enough?

Unfortunately that only provides one side of the story. Yes, they are up and resource utilization on them may be well within the limits. But it doesn’t tell us

    • How are the services running on those systems performing?
    • How are the apps that use those services performing?
    • How is the overall users experience for the apps/services hosted on those systems?

In fact, low resource utilization is very misleading because low utilization also happens when things are stuck waiting on external services response and nothing is processing.

Even when services’ performance is monitored via synthetic transactions we don’t get to know

    • Which users were impacted?
    • What is the response time for every request to the app or services?
    • What caused the service to fail?
    • What code/method caused the slowness/failure?
    • Is the issue happening at database, app server or external services level?

End to end visibility is a must to catch problems early on, and be able to take timely action and avoid outages. Say for example an external service that is being called to process credit card transactions is not performing well, then the caller server will have to just wait on process credit card transaction and its resource utilization would be low. But the end users will feel the pain of this slowness and may even abandon the carts. It will not only be important to know why the issue is occurring but also it will also be valuable to know what users are impacted. In this situation with only systems monitoring things will look very good because systems are up and resource utilization is low 🙂

APPLICARE comes to rescue

Applicare provides end2end monitoring and answers the questions that are unanswered with just systems monitoring.

Applicare allows you to

    • see the response time of every incoming request
    • see the detailed drill down into performance of every request including requests that span multiple servers
    • see the exact component or method that caused the slowness or errors
    • see what resource became an issue on what system and how it impacted the services performance and finally users experience.

Applicare cuts down the MTTR significantly by providing the visibility into every method, every component, every external service called and deep dive into systems performance & health.

Although server monitoring is an important part of monitoring but we shouldn’t consider ‘mission complete’ with just servers monitoring in the environment. Deeper dive available with Applicare is a must for any business critical application.

Download Applicare

Continuous Database Monitoring

Continuous Database Monitoring is a very important aspect of enterprise applications monitoring.  Database is the foundation of any application. If the performance of the database is not good then every user request can be impacted. Continuous database monitoring does provide very quick ROI. Tweaking the time consuming SQLs and any other database bottlenecks have impact on performance, scalability and availability of the entire application.

Applicare nicely ties database internals monitoring with systems monitor to be able to see the root cause of the problems e.g. disk IO becoming an issue and causing the SQLs to be slow or CPU starvation by another process causing a negative impact on database performance.  One agent to deploy and one console to use simplifies things from deployment of agents to root cause analysis.

ORACLE DATABASE MONITORING WITH APPLICARE

Applicare provides a very powerful continuous monitoring solution for Oracle along with many other most popular industry databases. No more dealing with AWRs.

Applicare Oracle Monitoring Dashboard
Oracle Monitoring Dashboard

Applicare monitors 100s of database metrics including

    • Load – Transactions/minute
    • Top SQLs & who is executing them
    • Active Sessions and Session Details
    • SQL Tuning Advice
    • Query Plan to ensure query is using most appropriate plan and access path
    • Locking Detection
    • Database growth
    • Oracle RAC Monitoring
    • Oracle SGA Monitoring
    • Oracle ASM Monitoring
    • Along with many other database metrics 
Database Growth
Database Growth
Session Details
Session Details
SQL Monitoring
SQL Monitoring
SQL Tuning Advisor
SQL Tuning Advisor
SQL Query Plan
SQL Query Plan

 

Additional Matrices Available
OS Monitoring

 

CPU Cores Monitoring

 

Network Monitoring

 

Disk Monitoring

Get your free trial https://support.arcturustech.com/register.do

Logs monitoring

Logs Monitoring plays an important role in analyzing, troubleshooting and alerting on problems. Organizations have lots of data in logs that should be mined to get valuable insights on users, applications and systems behavior.

Real time alerting from logs monitoring does help in identifying the problems early. It also provides security related insights when someone is trying to break into your organization.

LOGS MONITORING WITH APPLICARE

Applicare offers an extremely easy to use logs monitoring solution that doesn’t require a separate monitoring agent deployment. Applicare agent includes logs monitoring along with APM level application and systems monitoring. This simplifies things from deployment and analysis of data prospective – one agent to deploy and one console to use.

Applicare allows you to

    • access, search, visualize, analyze and correlate logs data
    • access logs data without requiring access to the system
    • convert searches into real time alerts in just a few clicks
    • monitor a single file or directories, one time snap or create watcher

Easy configuration

Easy Navigation

Easy Drill Down

For further reading on Logs Monitoring with Applicare, go to: https://helpdesk.arcturustech.com/hc/en-us/community/posts/360047713252-How-to-use-Log-Monitor-in-Applicare

 

Apache Monitoring with Applicare

Because of its customizable modular architecture catering to a multitude of needs, Apache is a widely used cross-platform web server. Although it has highly stable nature, yet, performance bottlenecks in Apache web server lead to slowness in the web pages. Inability to see the number of web server requests and traffic spikes to the website makes server capacity management difficult. Poor performance of the associated IT infrastructure—operating system, database, virtual machines, hardware, etc.—causes website downtime. Hence, it is important to implement a robust Apache web server monitoring system in order to identify potential problems before they turn into something big.

With Applicare’s Apache Web Server monitoring agent, one can monitor Apache Web Server in real-time, and diagnose performance bottlenecks while running in a live production or development environment. Applicare’s Apache agent monitors the below performance monitoring metrics:

Idle Threads: It is the total number of idle workers/idle processes waiting for an HTTP request. If there are very few or no idle processes, that means the server is using up all the processes and the new requests have to be on hold until the older requests are completed.

Busy Threads show the total number of processes actively processing an HTTP request.

Bytes per Request: This metric gives the average number of bytes being transferred per HTTP request.

Bytes per Second records the total amount of data the web server is transferring per second.

Throughput records the total number of HTTP requests the web server is processing per minute.

The Server Analyzer module of the agent monitors other aspects of the OS such as CPU%, Process, Physical free memory, system load, swap memory usage, TCP Connections etc.

Apache User Experience Analyzer dashboard gives details of types of transactions, number of transactions coming in, number of errors in those transactions, load at any point of time and average response time. Alerts can be generated on transaction failures & SLA violations.

Individual transaction details are available in the Transactions tab.

The module for monitoring Apache Web Server also supports the IBM HTTP Server.

Applicare Apache Monitoring Setup

User Experience Monitoring

“I am going through few videos of [our competitor]…And I must congratulate UI/UX Team of Arcturus Technologies for doing fabulous Applicare dashboards.”

This feedback from a happy customer encouraged us to write & publish about Applicare’s User Experience Monitoring, which is going to be amazingly useful for anyone in the industry.

End User Experience Monitoring enables teams to monitor the impact of application and device performance from the end user’s point of view. End User Experience Analyzer is used to monitor performance of your web applications as experienced by actual end users. This is achieved by automatically injecting Javascript code to your JSP pages which reports back the total time it took to load the page from the time a user clicked the link, network time, server processing and browser rendering times, browser type, user location, and various other data. User Experience Analyzer displays historical data related to end-user performance broken down by different aspects in separate tabs as shown below:

The world map on the Overview tab shows the performance breakdown for each country. One can view top transactions by different criteria such as page count, average page load time etc. The chart shows the performance data over the selected time period / for a specific country selected on the world map.

Transactions Summary view displays performance statistics of transactions for the selected time period  / for the selected transaction, along with the timeline chart.

Data is displayed on the Realtime Transactions tab as soon as users start accessing the application being monitored. This view not only displays the performance stats of the last n number of individual recorded transactions, but also the location and browser information of the users accessing the application. One can reach to the root cause of a slow transaction by drilling down that transaction to find slow methods / SQL queries.

Slow Transactions are determined based on the ‘Saved Slow Transactions Percentage’ specified in UEM Slow Transactions configuration. The ‘Time Distribution’ picture of any transaction helps in identifying where the slowness is, in a matter of a minute.

Browsers tab displays performance data for each client browser processing the transactions.

Servers tab displays performance data of the backend servers processing the transactions.

OS tab shows the OS used by the users of the application being monitored.

Analytics tab provides analysis of the application usage, e.g. Total visits, average page view duration, unique visitors, bounce rate etc.

Interested in trying out Applicare’s UEM? Register here now and get access to the trial version.

Health Check With “Applicare”

‘An ounce of prevention is better than a pound of cure.’

Because Ben Franklin knew a lot about not letting small problems become big headaches, he would have loved Applicare!

Discovering trouble after it happens is too late. The only way to survive in business is to stay ahead of what tomorrow may bring. Arcturus Technologies’ health check service can keep your application environment tuned-up and running smoothly.

Applicare™ is the market’s easiest-to-use next-generation Application Performance Management (APM) and Business Transactions Management (BTM) solution. Its embedded AI monitoring capabilities can improve your app’s health, user satisfaction, and business transactions. Applicare lets you know if something is wrong in real-time. Its innovative alert system enables enterprises to fix problems quickly before they have a chance to become major challenges. Arcturus Technologies is proud to provide application performance monitoring with advanced analytics and data integration platform — the Applicare Platform — that lets organizations to take maximum advantage of their business-critical data.

There is no better way to discover and remediate where enterprise level applications are missing the mark. Applicare delivers instantly! By uniquely combining consulting expertise and Applicare capabilities, we offer a comprehensive health check to review your applications’ environment, configuration, and performance. Register here to schedule an ROI rich “Health Check Engagement”.