Server Security: the Time is Now By Greg Ness published: Monday, June 18 2007
The Case for Server Shields
When it comes to data center security, servers have been in the closet for years. Security pros have wisely focused their attention and budgets on clients and public-facing network equipment, and for good reason. Since the early days of computing, the vast majority of servers have operated within well-protected internal networks accessed by controlled populations of users. Hackers have similarly targeted the large populations of more accessible desktop computers and edge appliances.
So before I make my case regarding the server security problem, let me say that historically servers have been among the safest of IT assets. Yet that quiet history has been a double-edged sword for security and data center operations pros as organizations embrace the enterprise web and expose servers to the public-facing network.
The ever-present security and operational issues that forced client computing into some degree of standardization and patch automation -and also drove the development of perimeter security architectures- had very little influence on the development of the data center. For example, while personal computers are upgraded on a fairly regular basis with completely new sets of standardized hardware and operating systems and applications, data centers have typically evolved in layers. A typical data center today contains multiple operating systems and applications and often multiple versions of each on multiple generations of hardware.
There is no one to blame for the complex state of affairs. It was a natural outcome of rapid technological change on an increasingly critical infrastructure required to operate 24/7 without interruption and with increasing numbers of access points. Real-time enterprise initiatives came along in the last five years as organizations tried to replicate the power of the consumer web for their employees and supply chains. These initiatives boosted employee productivity while exposing these once secure havens to new and increasing populations of users with an increasing variety of devices.
Standardization bred evolution in client machines and ultimately spurred innovation in networks and traffic management. Complexity and operational demands in the data center, however, have forced innovation into niches… often creating more complexity.
As a result, server security has been left in the dark ages compared to the rest of the infrastructure. Some of the brightest minds in IT developed solutions to secure the network perimeter from now primitive, yet malicious and effective client attacks. High profile hacker successes forced the market to innovate, and client standardization simplified protection (e.g. widespread adoption of anti-virus and anti-spam solutions).
Yet very little was done to protect servers, the critical machines that keep organizations running. They were assumed to be adequately protected by the perimeter. That is, until the hackers started looking for fortune over fame and started attacking servers and databases by exploiting the complexity of the data center, its varied applications and protocols and the inability of many operations teams to keep up with known vulnerabilities in their software.
Updating servers is still a primitive process relative to desktop updates: each machine must be manually updated. Because of data center complexity, some updates even carry “house of cards” risks; that is, they can bring down more than the updated server. And as I mentioned, the most important job for servers is staying available for any potential request. While laptops are turned on and off every day, servers often run 24/7 until they need to be updated.
THE AVAILABILITY REQUIREMENT
Availability is a unique, yet key server security requirement. Effective server security requires minimal invasiveness and minimal downtime. Making code changes in complex server environments introduces undesired downtime and reboot risks.
When servers go down, the pain can be felt throughout an organization, from users to partners to bank accounts and boardrooms. A server outage is a high profile event with sometimes ugly bottom line consequences. While availability has rarely been as important an issue for individual desktops, it has always been a key issue for servers. Host-based security, thusly, has fared better in the desktop than server world because it involves code changes.
When general purpose network security appliances were architected to fight off fame-seeking hackers, availability wasn’t even on the security agenda. A PC going down is and may always be acceptable collateral damage to spare other PCs and servers from attack. The brute force blocking of suspicious attacks, therefore, took precedence over any requirements for maintaining service availability. Perimeter-driven security solutions have as a result relied almost completely on monitoring and blocking suspicious traffic, to the unfortunate detriment of availability.
Yet blocking suspicious traffic is not a satisfactory solution for protecting a server from attack; it jeopardizes server availability and, perhaps worse, database accuracy. As a result, network security vendors and analysts have advised security pros to tune their systems very carefully, in order to only selectively block (when they know they’re under apparent attack). Please reread the words within the last set of parentheses for essence of the tuning paradox.
Yet blocking even a known bad email attack on an Exchange server, for example, is a problem. The server may keep trying to re-initiate the bad email, which in effect ends up blocking good traffic.
Perimeter solutions have evolved as servers have become part of the security challenge; they’ve added new features to improve their chances of catching the bad guys, like advanced traffic anomaly and protocol decoding capabilities. These capabilities increase accuracy, partially as an effort to make blocking less painful.
Because these solutions were not architected originally with enough intelligence to be highly accurate (they didn’t have to when protecting desktop PCs against well-known attacks) their promises had asterisks and footnotes leading to plenty of fine print as customers started turning on blocking for server traffic. Highly respected analysts, including Gartner, have published reports about the importance of tuning (and the careful use of “high fidelity signatures”) for less disruptive network security. Hint: High maintenance and high accuracy are not necessarily synonymous.
ACCURACY MATTERS
After availability, accuracy is also an obvious requirement for server security. Yet high availability requires high accuracy. If you’re going to interrupt server traffic you better be sure it’s malicious traffic and the act of blocking on its own won’t set up undesirable and highly visible side effects, from the self-inflicted denial of service attack we discussed earlier to database corruption. When data flows to databases are interrupted, their data can be rendered inaccurate, yet another potential high profile risk for the security team. Inaccurate data can have substantial operational side effects for real-time enterprises.
Perimeter security solutions evolved at a time when fuzzy accuracy was not only acceptable, it was pretty much all that was available. Network traffic was examined packet by packet and powerful processors conducted pattern matching in an effort to identify a malicious attack. While they do protect broad swaths of the network infrastructure from a variety of attacks, they simply do not have the up stack intelligence to be accurate enough to detect application and protocol layer attacks, the kind used to get to servers.
Hackers attacking in the application layer can evade static signature detection by mutating their signatures and avoiding the suspicious pattern match detection. Black Hat demonstrations last summer demonstrated how signatures could be evaded with polymorphic attacks… that simply disguise themselves (sometimes even in the midst of an attack) so as to evade the pattern-matching capabilities of leading intrusion prevention systems. Signatures, therefore, are no longer accurate enough for server protection, even if blocking is turned off. They shift the security burden into an operational burden by generating false alarms while allowing some types of attacks through. You can get distracted and evaded at the same time.
This complexity and noise factor has helped to create new categories of appliances dedicated to managing and evaluating alarms across vast network vistas. Understandably threat/noise management is growing at a fast pace. Yet threat management devices are still dependent on the data they’re sifting through: garbage in, garbage out. If perimeter devices don’t detect attacks, the management devices will be hard pressed to manage the alerts. If perimeter devices don’t understand the traffic protocols passing through them, they won’t be well positioned to detect today’s more sophisticated attacks.
COMPREHENSIVENESS MATTERS
Because of data center complexity a typical data center could be communicating in more than 100 protocols at any point in time. Each protocol could support multiple pathways, multiple attack vectors. Blocking or detecting a partial list of protocols takes us back to the hilarious desert toll booth strategy from Blazing Saddles; anyone with the skill to get in won’t use protected protocols. This wasn’t much of a problem for the desktop security world, but for data centers solutions need to be comprehensive from a protocol standpoint. If they aren’t then they’re still dependent on fall alarms, false blocking and missing sophisticated attacks.
BLOCKING IS UNDESIRABLE
We discussed blocking earlier as a part of the availability challenge. Call it an interdependent capability: without accuracy blocking server traffic is high risk behavior. Reliance on blocking is a very basic, primitive vestige of the pre-server security era. It’s a simple tactic in a complex environment; a scenario often replete with undesirable, unintended consequences.
Yet to this day it’s the best that most perimeter security solutions can do. Adding inline correction to a low layer intrusion prevention appliance would be about as easy as teaching a rhino to waltz across Texas. It requires intelligence, timing and comprehensive coverage… and a deep knowledge of application and protocol behavior. I think that is the prime reason why perimeter security appliances have not been able to evolve up close to data center servers and successfully protect them from specialized, intelligent, mutating attacks.
NOW FOR THE BAD NEWS
I’ve spent more than my usual share of words talking about the inherent challenges and difficulties of server security and why the status quo is in a state somewhere between cash burn, operational shock and risk denial. It wasn’t until Allwyn and I took the stage at the Best of Interop awards late last month that I realized that the world is coming around. Yet plenty of work lies ahead.
Large enterprises are spending inordinate sums on manual server updates in reaction to vendor patch announcements because they’ve realized just how vulnerable their servers have become. Many organizations have already been hit by multiple polymorphic attacks that merely mutated around their traditional network defenses and compromised their servers.
Every new update either increases the costs of IT or opens the server attack vulnerability window even further. That means, with every update more organizations either pay up or open up those windows of vulnerability and hope for the best. Now for the second half of the bad news: vulnerabilities are increasing and exploits are increasing exponentially.
Those older signature-based systems were created to block known exploits. As we said earlier, they weren’t developed to truly eliminate software vulnerabilities but rather to detect and block known exploits/attacks against those vulnerabilities. A single vulnerability can produce dozens of unique exploits over time. Rising quantities of vulnerabilities can produce exponential increases in signatures required for effective defense, as mutating worms create wave after wave of zero day attacks on a known software vulnerability.
VULNERABILITY WINDOWS ARE GROWING
The problem is that known software vulnerabilities continue to climb to record levels just as hackers get faster, more sophisticated and increasingly profit-motivated. While a range of patch management/update tools exist for clients, deploying rapid updates for servers is downright troublesome (as we’ve discussed). Again we’re back to the availability challenge. Changing server code, testing configurations, reboots and careful deployment can be time-consuming and introduce high downtime risks to business-critical operations.
That is why the increased pace of software vulnerability announcements is particularly painful for operations teams. While they can keep up with client patches to keep them protected from attacks against known vulnerabilities, server vulnerability windows are growing. Enterprises can be tempted to accept larger windows of opportunity for hackers as a necessary evil, because of the undesirable financial and operational side-effects of patching too soon. This “tolerable exposure” combined with the increased pace of vendor vulnerability announcements is grinding down effective server security practices.
Forget the media drumbeat of vanishing laptop stories; a compromised server can give a hacker access to core databases and applications, stockpiles of proprietary information. A vanishing laptop is more likely to give the recipient some new hardware or pawn cash. Someone who has gained access to critical data is more likely to be there for the data, not the hardware.
SUMMARY
Critical enterprise servers are more vulnerable than ever, facing exponential increases in attacks at a time when traditional perimeter security appliances have failed to address their unique availability, accuracy and coverage requirements. The result: operational costs for effective server security are skyrocketing as increasingly profit-minded hackers continue to innovate. Large enterprises with deep IT pockets have insulated themselves from the short term risks by throwing resources and alarms at the problem while more frugal organizations are accepting longer server vulnerability windows.
That status quo simply cannot continue. IT needs specialized server security appliances that co-exist with their other layers of protection. Blocking and tackling of suspicious traffic when it comes to servers is not acceptable. The limited protocol fluency offered by older security appliances is a (no pun intended) token effort. It’s time for the emergence of server vulnerability shields that offer high availability and accuracy and low latency.
As usual
This e-mail address is being protected from spam bots, you need JavaScript enabled to view it
. By the way, you can listen to a recent podcast interview about how these issues relate to virtualization by visiting the newsroom at www.bluelane.com and clicking on the IT Conversations link. In the newsroom you’ll find links to multiple podcasts, articles and perspectives on these issues.
Greg is VP of Marketing for Blue Lane Technologies, a winner of the 2007 InfoWorld Technology of the Year and 2007 Best of Interop in security. He has been a marketing executive at Juniper Networks, Redline Networks, IntruVert Networks and ShoreTel.
|
