Unpacking network monitoring

Nov 4th, 2019

By Chris Muyaruka: Turrito Operation Centre Lead

What is Network Monitoring?

Network monitoring is a key IT process that involves data collection from critical IT infrastructure components including but not limited to routers, firewalls, switches, servers and to some extent end user devices.  Data is collected from these devices for fault and performance monitoring, these data sets are continuously evaluated to maintain and optimize the availability of components and ultimately services on the network.

Data collection from network devices can either be Passive or Active.  With Passive Monitoring, techniques such as SNMP polling, packet capture, log file analysis and flow monitor are deployed to discover and provide diagnostics on key IT infrastructure components.  Active monitoring or Synthetic monitoring involves emulation of user activity and this emulation happens in real time.   Active monitoring allows Network administrators to be proactive and be able to detect potential problems before they occur.  Active monitoring provides a complete picture of performance between clients and servers.

The important aspects of Network monitoring are:

  • Monitoring critical and essentials Infrastructure components
  • Optimised monitoring intervals
  • Selecting the correct monitoring protocol
  • Setting realistic thresholds
  • Data analysis and Dashboards

Why is Network Monitoring becoming increasingly important?

The picture of the enterprise network has radically changed over the last 10 years, with user applications increasingly being hosted in consolidated data centers or by external SaaS and IaaS providers.

This paradigm shift of Data Center to Cloud has resulted in the emergence of a more complex network that has the internet at its core.  Applications being used by employees and customers now heavily rely on a predictable and optimized network to work properly and network architects have had to design networks that adapt to increasing bandwidth demands and budgets that are shrinking year on year.  Network monitoring is key in ensuring that key components in the network core performs within the parameters dictated by SaaS and IaaS based applications so as to avoid downtime and loss of productivity.

Without proper modern Network monitoring solutions IT administrations will be unable to be proactive and meet service metrics leading to unpredictable behavior of the network, and thus poor application performance and ultimately productivity loss due to downtime or slow performance.  In short Networking monitoring in this new environment is a pillar that impacts user experience if not implemented correctly or not implemented at all.

What are the risks of not having Network Monitoring

Network infrastructure without some form of monitoring brings with it huge risks that include:

  • Zero Visibility – Without monitoring IT administrators have no way of knowing what is happening in their infrastructure, who is connected to what and how applications are performing, the outcome of this is that Application performance becomes unpredictable, residual risks such as Security and Downtime become inherent.
  • High probability of unscheduled downtime – No Network monitoring means network administrators can only attend problems with the Infrastructure when they occur, Network administration and management becomes highly reactive with a direct consequence of increased downtime ranging from a few hours to days.
  • Manual intervention in diagnosing problems – No network monitoring means no data to analyse when trying to troubleshoot a problem within the infrastructure, the only feasible option IT Admins have is to look for highly specialized IT professionals to dig into the mountains of log data and make sense of it and find the root causes, most enterprises do not have these specialized skills in house or accessible easily.
  • Security Threats – Network security can only work if there is some sort of system collecting data on activities happening in the network, if data collection malfunctions for some reasons there will be no way of knowing this without Network Monitoring, thus your systems are at a hug risk of being compromised
  • Excessive residual IT Costs – Downtime from not having a proper monitoring solution will be a huge by product for any business, downtime means no production and ultimately less profit. With Network Monitoring IT administrators can be more proactive and be able to diagnose problems more quickly and thus reducing downtime and increasing productivity.

How is BYOD adding to this challenge?

BYoD provides device independence to end-users, the consumerisation IT has greatly facilitated this way of working and enterprises have had to inevitably accommodate BYoD into their network design and operating strategies.  However, although BYoD comes with a lot of benefits for the Enterprise and Users, managing risks associated with BYoD presents a lot of challenges for IT Administrators.  With respect to Network monitoring the challenges with BYoD include:

  • Malicious or untrusted applications – As personal devices are plugged on to the network, IT Administrators have the daunting task of having Network Monitoring systems that are also application aware the reason being that, end user devices are vulnerable to malware and malicious apps, if these devices are not monitored correctly they can become gateways for outsiders to access critical corporate data that lead to a PR crisis or serious financial loss for the business. 
  • Lost, Stolen Devices and Fired Employees – Network monitoring in a BYoD enabled environment should also be able to quickly disable a lost or stolen device so as to prevent sensitive corporate data from falling into the wrong hands or unauthorized access by Employees that no longer have any contractual relationship with the enterprise.

What are some of the solutions that companies can implement

When selecting a good Network monitoring solution Enterprise IT administrator should make sure that the following pre-requisites are met by the solutions:

  • Simple to implement and manage – Network monitoring software should give the admin time to focus on other critical tasks instead of having to keep a permanent eye on the infrastructure and connected systems. In short, the solution works automatically after a simple installation.
  • Relevant Protocols – The solution should support monitoring protocols that are relevant to the business and should be able to centrally monitor distributed locations with least effort and administrative overhead.
  • Simple customization of Thresholds and Alerting – Thresholds limits vary from device to device and the solution should have the ability to adjust these with little or no effort. With regards to Alerts the solution should be able to integrate easily into your existing ticketing system.
  • Dashboards – A monitoring solution is useless if it cannot analyse and display information for IT admins to make proactive decisions to prevent downtime. A network dashboard should provide at-a-glance overview on the current status of the network

Can machine learning play a role? How would this work?

AI by its very nature is extraordinarily data intensive, as both machine learning and deep learning algorithms must process massive quantities of data in order to make intelligent inferences on what the data represents.  Network Monitoring processes harvest vast amounts of data as input streams are processed thus making Network monitoring a prime candidate for data driven automation implementation.

AI is becoming the cornerstone facilitator of Network Monitoring given the sheer number and variety of devices at the far edge of the network, machine learning is increasingly becoming the only practical way to identify potential problems and threats – separating the interesting traffic for the noise.  Data becomes useful only when it is presented clearly to the right audience, machine learning plays an important role in quickly analyzing important metrics and highlighting recommendations.