High-speed, low latency networks vital for dealing with vast data volumes

Data volumes have exploded over the last few years, with digital information being generated by a wide variety of connected equipment.

January 17, 2013

By Raul Del Fabbro, Manager: Storage Solutions Division at Drive Control Corporation

Data volumes have exploded over the last few years, with digital information being generated not only by computers but also a wide variety of other connected equipment such as aeroplanes to medical equipment, factory floors to cars and even televisions. This data explosion has made vital analysis and intelligence gathering from data a challenging task, and even storage and data processing have become more complex in light of the sheer amount of information that needs to be dealt with. As a result, high speed EtherNet and InfiniBand networks, which has traditionally been the exclusive domain of high-performance computing environments, are fast gaining traction in mainstream enterprise grids, data centres, enterprise servers, storage and virtualisation environments, delivering the fast throughput and low latency needed to cope with big data challenges.

The pace of business has increased dramatically and the competitive edge of many organisations now relies on their ability to quickly analyse and gain intelligence from their data. However, this has become increasingly challenging as the volumes, variety and velocity of this data has increased. The reality is that the typical 100 Megabit per second (Mbps) network connection, and even faster 1 Gigabit per second (Gbps) connection, is simply no longer fast enough for data transfer and processing in many environments. Although many enterprises still use a 1Gbps network backbone, the performance of these networks is increasingly coming under pressure as data volumes grow, the number of transactions increases and more and more data needs to be moved into the data centre. Data becomes bottlenecked, reducing performance, and the only solution is to move to a higher speed Ethernet connection.

The growth of cloud services and infrastructure too has highlighted the need for faster networks, as virtualised servers and applications which are delivered to multiple users require maximum performance. Cloud providers looking to offer infrastructure as a service also need to take the performance and more importantly, latency of the network into account, and require the fastest possible speeds across the network backbone. As a result of these and other factors, high speed Ethernet and InfiniBand, which havetypically only been used in very specialist applications, is beginning to grow in the mainstream market. High-speed EtherNet solutions offer 10Gbps and 40Gbps connections, while InfiniBand solutions offer 40Gbps and 56Gbps connectivity, greatly improving performance, lowering network latency, thus reducing bottlenecks for extreme network performance.

High-speed Ethernet and InfiniBand solutions deliver high performance with low latency and efficient CPU utilisation. This enables enterprises to consolidate data environments, boost performance, improve manageability and even incorporate network virtualisation. This allows for fast response times and even real-time analytics, helping to boost the competitive edge of any business.

Fast throughput and data transfer capability, along with low latency, is critical in applications such as on stock exchange trading floors. These live environments rely on real-time, split second timing and data transfers as if there is a lag of even a few seconds, the financial implications can be significant.

Aside from this specialist application, high-speed Ethernet is also applicable in the data centre environments of many organisations. By incorporating high-speed technology, current data centre performance can be vastly improved for today, and the data centre can be future proofed for the requirements of tomorrow. This technology facilitates any standard networking, clustering, storage and management protocols to operate seamlessly over a converged network, and can provide lower cost, lower power consumption, better CPU utilisation and lower latency for Ethernet-based solutions in both blade and standard rack and tower data centre environments.

InfiniBand technology is of course also applicable in high-performance computing environments, including supercomputer environments. It is also useful in Solid State based appliance environments, ensuring that bottlenecks do not occur on the appliance access points into the data centre. For embedded applications, InfiniBand also provides the ideal solution, providing the necessary reliability, availability and scalability, as well as for any high-performance computing clusters.

As data volumes continue to explode, and the need for ever faster networking grows, InfiniBand and high-speed Ethernet are growing in applications from specialised fields to more mainstream areas. Demand for high performance, reliable connectivity is also growing unabated, and the price of technology is continually on the decrease. These factors combined will see the adoption of more and more high-speed networking solutions across a broader range of industries, from medical research to climate modelling, and even mission critical financial services data. This technology will eventually also filter down to enterprise data centres and businesses, delivering fast data transfer to keep pace with insatiable demand and explosive data growth.