The following technology stacks shows our strengths & expertise in creating value for our customers with exceptional quality. Our experience in delivering visualization software, server monitoring software, server automation & training is a key component to our progress in a fast evolving world.
Nagios Monitoring with Grafana visualization is a powerful combination. We have implemented over 6000+ Linux & Windows end-points with dozens of grafana dashboards. Each Grafana dashboard is equipped to visualize a single environment for aesthetic preferences.
The following end-points illustrates the various capture devices.
- All OS matrices monitoring (CPU, Memory, Disk, Swap, Process counts),
- Redis Caching
- Email Queues
- Apache Server Performance
- RabbitMQ Queue
- ActiveMQ Queue
- Oracle monitoring
- JBoss health
- NGinX health
- MySQL connection counts and health
- Postgres connection & health (bloating, size)
- REST application performance
- HaProxy/Apache configuration checks alert (tamper proof)
- Load-Balancer health
- Application-logic-health (memory foot print / Java)
- Folder/File sizes alerts
- TCP end-to-end pipeline monitoring
- LogStash – ElasticSearch simple pipeline monitoring
- Kibana monitoring
- Temperature/Humidity monitors (using Arduino units)
- Custom REST monitoring (app performance of TCP/ authorized apps)
- Custom Selenium scripts (automate login with credentials)
- Microservice REST end-point monitoring
Grafana Analytic Graphs
The Above Grafana dashboard displays the data obtained from the previous mentioned Nagios dashboard. Nagios+Grafana can be implemented on any Linux cloud server that has access to the internet and end-points. With Grafana, the team was able to visualize KPI’s that otherwise would have gone unnoticed and caused consistent losses that would last for years.
Time To Deliver – One Month to create Nagios Monitor with 10 monitored servers. Two production Grafana dashboard displayed important KPI. Currently we’re supporting this as an on-going project for customers.
ElasticSearch, LogStash & Kibana (ELK) are vital components in the big-data analytical world. We have implemented ELK Stacks from Version 2.1v to 7.5v using both manual & ansible automated environments.
ELK for Linux can be automated to include clustering and application performance analysis using custom application logs. We have also implemented TCP based message transports (for C#, PHP & Java). The ELK was also implemented for clients running 100% windows servers. Our latest addition to the ELK stack was the geo-graphing & geo-region mapping.
LogStash Architecture Implemented
Multiple LogStash architectures were implemented on aws-cloud in order to maintain low costs for QA Environments, while consistently maintaining high availability in production environments. The following low-cost architecture was implemented for the development/QA teams to gather error data, SIEM & performance data. The application performance was gathered by custom log files ingested through logstash.
The following Logstash/Elasticsearch architecture was implemented to overcome the challenges of decentralized logging. This production architecture had high fault tolerance with load-balanced Elasticsearch clusters.
Ansible automation is a push configuration management tool introduced to a major online reservation company in 2015. Using ansible, we were able to automate the delivery of nagios plugins, application configurations, load-balancing configurations and system configurations. Ansible effectively reduced a 8 hour deployment cycle to several minutes.
Challenges Faced With Automation
The strongest challenges faced in automation came from the deficit in knowledge among the team members. We began training the teams to improve their knowledge in order to speed up the automation process. When the team knowledge was fully saturated, the automation of the system improved significantly. These instructions includes but not limited to:
- Ansible automation training
- App configuration & log-file documentation
- File backup instruction before uploads
- Application compilation instructions
- Application config file management instructions
- Database table upload instructions (import, export, flush etc)
- Caching instructions
- Instructions on stopping & starting dependent services
- Email generation upon completion
- Application initialization instructions
Load-Balancing is a key to scaling application in a cloud environment. With poorly implemented load-balancing, companies could face loss of business and probably total collapse of the company over time. In order to avoid such disasters, We have implemented load-balanced systems that handled over 100 Million+ xml requests/day (3 billion requests/month). In order to achieve this, We used a Big/IP F5 Load-balancer with 5x HaProxy servers to distribute the load to 20x Linux cloud servers. Each cloud server comprised of apache-server which load-balances multiple Tomcat-Nodes. HaProxy was used to load-balance 60 replicated postgres db-slaves. On the Windows environment, the same was accomplish by configuring NGinX instead of HaProxy.
Individual Application Server Architecture
Apache is a high-performance web-server with Load-balancing features built in. It can also serves as an Application-Server with the appropriate plug-ins. The following diagram shows the traffic flow within the apache load-balancing architecture that was implemented when handling 100 Million+ xml transactions.
Note: The ELK Architectures that required UserName/Passwd access was provided by HaProxy & NGinX basic-auth over SSL/TLS. (Xpack was not implemented at the time of deployment).