By Sujal Das
published: Thursday, March 13 2008
Business process management (BPM)
and deployment of service-oriented architectures (SOAs) are the new trends in
using information technology to gain a competitive advantage; and total cost of
ownership and green initiatives are major spending drivers as data centers are
being refreshed to support these new IT models. In this environment, data
center cost structures are shifting from equipment-based to space, power and
personnel-based.
I/O technology plays a key role during this process of
IT evolution. I/O technology can help data center managers address many of the
spending drivers in this data center transition. I/O technology impacts the
data center manager's ability to provision capacity for future growth, to efficiently
scale compute, LAN and SAN capacity; to reduce space and power requirements,
and to reduce TCO while enhancing data center agility.
I/O unification is now
available as a means of reducing I/O space, power, and management overhead
while bringing far more resource management flexibility and reducing costs. In
this article, we'll look at the challenges that have made I/O unification a
solution to data center scaling challenges, and we will compare technologies
available for unifying I/O.
The Need for More I/O Capacity
While most servers today are equipped with Gigabit
Ethernet or 1 Gb/s bandwidth, four key trends are fast driving the need for
more I/O bandwidth per server, typically up to 10Gb/s or higher:
- Adoption of multi-core CPUs in servers: Dual- and quad-core CPUs are becoming common in
servers, allowing more applications to run on each server and driving the need
for 10 Gb/s of I/O bandwidth per server. Being able to allocate I/O bandwidth
and other services to applications is critical for enabling applications to
deliver required services that meet BPM and SOA goals.
- Adoption of SAN: Storage area networking requires storage access and services to be
delivered over the I/O devices on the server. Popular database servers and
server virtualization software utilize SANs to deliver value-added and
cost-effective services. SANs demand
high levels of bandwidth and have low tolerance for data loss.
- Server Virtualization: When servers are virtualized, multiple virtual
machines and guest operating systems execute on each server. This enhances data
center agility, but requires I/O bandwidth in excess of 10 Gb/s to meet the
needs of LAN and SAN capacity shared across multiple virtual machines. High I/O
performance is also required for virtual machine and storage or file system
mobility-related - functions provided by server virtualization software
providers.
- Server resource sharing: In N-tier server farms, servers were dedicated for
specific applications, i.e., front-end web services, middleware applications,
database and storage servers. These server silos have typically required varied
levels of I/O capacity, with the storage server tiers demanding the most
capacity and the front-end web servers requiring the least. With server
virtualization and virtual machine mobility, servers are shared across all
application tiers, and I/O capacity provisioned per server is dictated by the
most demanding apps, which may reside on any server.
Figure 1: Driving demand for higher I/O bandwidth
Reducing Space and Power Requirements
Servers are the fastest-growing power consumer in the
data center. According to Electronics Cooling Magazine (February, 2007), server
energy costs exceed server purchase costs in today's data centers. There are two primary approaches to power reduction:
- Reduce power consumption per server: Blade servers are the fastest-growing server market
segment (according to IDC), and they are fulfilling the promise to deliver
lower power and space-saving solutions.
- Reduce the number of servers needed: Server clustering enables use of cheap commodity
servers to deliver SMP- and mainframe-like performance and capacity scaling.
When compute capacity can be scaled linearly with the addition of servers to
the cluster, the highest levels of clustered server efficiency can be achieved.
High-performance I/O plays a
key role in both of these approaches. Blade servers lack I/O real estate (PCI
slots), so they require higher-bandwidth I/O PCI adapters that can carry all
data center traffic types and consume the least power. Efficient clustering
requires high-bandwidth, low-latency I/O to enable higher levels of clustering
efficiency.
Service Levels for Data
Center Traffic Types
Data center servers need to handle three classes of
traffic - LAN/WAN (Local and Wide Area Networking), SAN (Storage Area
Networking) and IPC (Inter Processor Communication or server-to-server
messaging). Each of these traffic classes requires different I/O service
levels. Today, the prevailing I/O technologies are Gigabit Ethernet, 10-Gigabit
Ethernet, Fibre Channel and InfiniBand. Each of these I/O technologies delivers
varying degrees of LAN/WAN, SAN and IPC services:
- Gigabit Ethernet is moderately suitable for LAN/WAN traffic, but because of its low
reliability (data loss and software-based retransmissions) and high latency
characteristics, it is not suitable for SAN and IPC traffic.
- 10 Gigabit Ethernet can be excellent for LAN/WAN
connectivity and with use of TOE (TCP
Offload Engines) can deliver iSCSI-based SAN
traffic. However, it still suffers from high per-port costs (especially in the
infrastructure), unreliability, and inefficient scalability (i.e., spanning
tree restrictions), making it unsuitable for Fibre Channel storage connectivity
over SAN and IPC.
- Fibre Channel
delivers excellent services for SAN connectivity, but lacks the capabilities
needed for LAN/WAN and IPC traffic.
- InfiniBand
delivers very high bandwidth, low latency, and reliability that:
- Matches or
exceeds what Fibre Channel provides
for SAN,
- Matches or
exceeds what Gigabit Ethernet and 10
Gigabit Ethernet provides for LAN/WAN, and
- Is best in class
for IPC traffic.
Figure 2: I/O fragmentation
In most cases, two (if not three) of these I/O
technologies are used to meet service levels of the different traffic classes.
As a result, IT managers must purchase, deploy and manage two or three
different types of I/O adapters, networks, cables and switches. This approach
causes higher management overhead, higher power and space consumption, and
higher equipment costs. These problems are further exacerbated with blade servers,
which are compact and lack power and space for multiple I/O adapters.
Unifying I/O
I/O unification is a new technique that delivers all
three traffic classes (LAN/WAN, IPC, and SAN) over the same I/O adapter and the
same wire. This is an important solution that can simplify infrastructure,
reduce space, power, and management costs; and increase resource management
flexibility. So the question is, which of the interconnect technologies is best
suited for I/O unification?
We can eliminate Gigabit
Ethernet because it simply lacks the bandwidth to serve as a unified I/O
technology. As for 10 Gigabit Ethernet, there are a number of IEEE initiatives
- some in fledgling stages - under the umbrella name of Converged Enhanced
Ethernet (CEE). These initiatives aim to address the feature gaps required for
delivering unified I/O over 10 Gigabit Ethernet. Deployment of such features
would require significant overhaul of data center networks using new equipment from
system manufacturers, and in any event, cost-effective deployment of CEE is not
expected until 2011 or later according to many industry experts.
Fibre Channel has never been
seriously considered as a unified I/O technology, being predominantly a SAN-only technology. On the other hand, InfiniBand
(which provides the same or higher bandwidth as 10 Gigabit Ethernet but
significantly higher levels of service, meeting needs of all three classes of
data center traffic,) has already gone through multiple product generations and
is ready for cost-effective production deployment using servers and network
equipment from Tier 1 OEMs (see Figure 3).
Figure 3: I/O technologies
compared.
As we can see, InfiniBand is the best choice - today
and in the near future - as a unified I/O technology throughout the data
center. With InfiniBand, IT managers can use a single 20 Gb/s InfiniBand
adapter in servers, and they can use modular InfiniBand-to-Ethernet or InfiniBand-to-Fibre
Channel gateways for connectivity to Ethernet-based LAN/WAN and Fibre
Channel-based SAN networks, enabling cost-effective SAN and LAN capacity
provisioning and scaling. Emerging
native IB-attached storage systems will eliminate the need for gateways,
enabling end-to-end I/O unification using InfiniBand.
InfiniBand I/O adapters, switches and gateways deliver
cost-effective, low power, efficient, lossless and reliable connectivity for servers
and storage. InfiniBand is the highest performing industry-standard I/O
solution, with 20 Gb/s products in deployment since 2006 and 40 Gb/s products
expected in late 2008. InfiniBand
protocol software, available in all popular operating systems, delivers
industry-standard application interfaces like IP, sockets, SCSI, iSCSI, NFS,
etc., which makes application deployment transparent to the underlying fabric.
I/O unification is a significant strategy for reducing
data center space, power, management costs and complexity. For IT managers
investigating solutions for I/O unification, InfiniBand offers important
advantages.
Related Links:
Electronics Cooling Magazine , Converged Enhanced
Ethernet , Mellanox News
Sujal Das is Senior Director of Product Management at Mellanox Technologies. For
more than twelve years, he has held senior management roles in companies such
as Unisys, AMD, and Marvell
Semiconductor. He has a strong track record of driving both commercial and open
source-based software alliances, and marrying those with internal software and
hardware development initiatives to grow business for existing semiconductor
products into new or adjacent enterprise and consumer networking markets. Sujal
has a BS EE and MBA from Santa Clara
University, California.
He can be reached at:
This e-mail address is being protected from spam bots, you need JavaScript enabled to view it
.
|
