How can a Hybrid Cloud Monitoring Solution help my team monitor the cloud?

By Kevin Cox, AWS SAA, CCSK
Lead Consultant, Impact Makers

Existing Monitoring Systems

There are hundreds of monitoring products in the marketplace that cover monitoring from enterprise scale to small and medium businesses. How can a monitoring system help your team? It is imperative that an IT team know the state of the environment and quickly respond to issues.

Most IT teams have a monitoring system, or several monitoring systems. These systems monitor applications, services, operating systems, network devices, and technology infrastructure. This monitoring results in the collection of valuable systems metrics that can be presented in a central dashboard or graphical representation of the environments.

Multiple System and Integrated Monitoring

Many larger enterprises may have multiple monitoring systems that watch separate areas. A product like PRTG may be dedicated to network monitoring, Solarwinds may be dedicated to systems monitoring, Oracle Enterprise Manager may be used for Oracle database monitoring, and IBM Tivoli could be used for application monitoring. The monitoring systems can be integrated with service desk products like ServiceNow or BMC Remedy.

Alerts and Notifications

In practice, monitoring systems send out alert notifications that are triggered based on thresholds and escalations defined within them—the alerts themselves have the potential to be sent using email, SMS, voice call, or even mobile application push notifications.

Some sample alerts and graphical reports are shown below. These images were captured from a Nagios monitoring system dashboard.

Nagios Cloud Monitoring Examples

Images captured from a Nagios cloud monitoring system dashboard.

The Cloud Challenge with Monitoring Systems

A common situation faced by IT organizations is the extension of monitoring capabilities, or the replacement of this capability, with a cloud native system. All the functionality of an on-premise monitoring system is generally required in cloud environments. This can apply to Software as a Service (SaaS), Platform as a Service (PaaS), or Infrastructure as a Service (IaaS) cloud computing service models. If an organization is beginning the cloud journey, this can be a daunting task. This extension of on-premise monitoring can be extremely complex when many monitoring systems are in use.

Specific challenges are introduced with SaaS, PaaS, or IaaS cloud computing service models. The monitoring responsibility across these different models is segregated between customer and cloud provider depending on the cloud service in question. In the following representational graphic, the responsibilities are organized by the cloud computing service model.

Learn more about the cloud computing service models in our blog on patching in the cloud.

Separation of Responsibilities by Cloud Computing Service Model

Responsibilities organized by cloud computing service model – on-premises, IaaS, PaaS, and SaaS.

On-premise Monitoring

The on-premise model is where the monitoring system is traditionally deployed. Any decisions, configuration, and usage are entirely controlled by IT operations staff. In this situation, IT Operations staff retains full responsibility for the health, metric collection, and incident response for all resources in the environment.

IaaS Monitoring

The IaaS (Infrastructure as a Service) cloud computing model is the most like on-premise environments. Extending an on-premise monitoring system is usually the most simple and direct method of providing monitoring capabilities to a cloud environment.

In AWS, monitoring agent software can be installed to EC2 virtual machine instances to extend monitoring into a cloud environment. However, detailed visibility into the core network, storage, physical servers, or hypervisor cannot be duplicated like an on-premise datacenter. A customer can monitor many elements in AWS using APIs or leverage cloud native services like CloudWatch for monitoring services such as Elastic Load Balancing (ELB), CloudFront, EBS, or EFS.

PaaS Monitoring

The PaaS (Platform as a Service) cloud computing model limits the visibility of the typical on-premises monitoring system. Assets like servers or virtual machines, network devices, databases, and other infrastructure are typically masked by the platform to the point where they cannot be monitored in any meaningful way. An effective monitoring solution in this situation is to use cloud native tools coupled with APIs or other integration options to view the aggregate data in a on-premises monitoring solution. Alternatively, many cloud native monitoring solutions offer enough functionality to meet monitoring needs without the need for additional integration.

AWS Relational Database Service is a commonly used PaaS. It provides cost-efficient and resizable capacity while automating time-consuming administration tasks such as hardware provisioning, database setup, patching and backups. RDS has a monitoring system provided with the service that delivers visibility to network throughput, I/O for read, write, and/or metadata operations, client connections, and other info. More monitoring can be viewed using the performance insights tool. AWS RDS monitoring is a cloud native tool. Additionally, the data from RDS can be pulled into common third-party tools like Solarwinds or Idera monitoring.

AWS Relational Database Service Monitoring

A screenshot of the monitoring of an AWS Relational Database Server instance. This is a PaaS offering from AWS of a managed MySQL database.

SaaS Monitoring

The SaaS (Software as a Service) cloud computing model offers few points at which monitoring can be integrated, as the fundamental nature of this model is that the underlying systems providing the functionality are not visible to the customer. Monitoring in this type of environment is usually restricted to verifying pings and HTTP status codes of endpoints available or utilizing the cloud service provider’s API to capture service status.

Cloud and Hybrid Monitoring Impacts

Monitoring has a wide-ranging scope that spans many areas in technology such as release management, operations, CI/CD, IT security management, successful implementation of projects, and more. Monitoring is important across hybrid cloud environments, cloud native environments, and multi-cloud environments. Planning, consideration, and proactive adjustment is required to successfully monitor a cloud or hybrid cloud environment.

The costs for monitoring software, how to leverage a cloud provider’s APIs, the integration of cloud monitoring data with an on-premise monitoring system, and the limitations of SaaS visibility should factor into any decisions regarding the architecture of a monitoring system. It is also significant that it is entirely possible for a company to transition to a cloud monitoring system for on-premise resources and cloud resources. There are many factors that must be considered to make an informed decision and ensure that your IT team is responsive to quickly resolve issues.

The Impact Makers Solution

Impact Makers is an AWS Advanced Consulting Partner. We leverage comprehensive and mature practices to enable customers to see all facets of their AWS ecosystem. Every project has unique elements that must be incorporated into a comprehensive strategy in addition to identification and execution of technical work. As Advanced AWS consulting partners, we recognize the impact of comprehensive hybrid monitoring for the future of your cloud journey and we will help you optimize your present and future usage of cloud. Our comprehensive framework includes the AWS Well Architected Framework and industry best practices in addition to elements like compliance , asset and metadata management, business strategy alignment, service portfolio management, support model definition, service design and deploy, CloudOps and much more. We work with our customers to deliver and enable strategic business advantage with cloud services.

To learn more, contact us.