“Taking Off” with Linux Clusters

A new computing architecture allows Orbital Sciences to complete jobs 30 times faster
Eric Pitcher

Since its beginning in 1982, Orbital Sciences has become a pioneer in developing small rockets and satellite systems for such diverse purposes as intercepting hostile missiles launched against the U.S. or launching satellites for better cellular phone reception. With a range of space and satellite systems, plus the added responsibility of supporting virtually all of our country's major missile defense programs, Orbital relies on massive amounts of computing power to continually optimize and simulate launch vehicles for reliability and accuracy.

Like many organizations in the aerospace industry, the company depends on computational fluid dynamics (CFD) to predict how rockets and satellites will perform in various environments. CFD works by turning the region of flow into discrete volumes, generally cubic cells. The governing fluid equations are then used to set up a large matrix that can be solved using a high performance computer system. Orbital relies on CFD software from Fluent Inc. to simulate many aspects of the motion of fluid within or around its launch vehicles.


The challenge: fast and reliable CFD
CFD image of Orbital Science's Hyper-X vehicle at a Mach 7 test condition with the engine operating. Photo courtesy of NASA. 
Orbital's existing computer system was continually bogged down and unable to produce fast, accurate simulations. As a result, there was a significant slowdown in arriving at solutions for CFD problems over one million cells. Engineers were limited in the number and variety of CFD problems they could process. To help meet deadlines, they often had to outsource smaller simulation problems, which could cost up to $50,000 per job and $300,000 annually.

"In the case where a larger problem was deemed mission critical, there invariably was a need to justify the cost of outsourcing smaller problems that had to be completed, including projected payback on the outsourced engineering work," said Richard Straka, vice president and chief engineer of Orbital's Launch Systems Group (LSG). "Ultimately these justification efforts pulled engineering talent from its core competencies to convince management to spend the money necessary to support the outsourced CFD."


Investigating Linux clusters
The inefficiencies and frustration caused by its existing system led Orbital to investigate new computing architectures. The company wanted to migrate to a Linux cluster because of their reputation for providing optimal performance for CFD applications.

"We wanted a system that scaled linearly with additional CPUs and could grow with us as computational demands increased. Linux clusters provided us with that capability," said Straka.

Orbital engineers were experienced in using UNIX and had dabbled in the Linux operating system for years, so they were well aware of the advantages Linux offered. Although Orbital hadn't deployed Linux on a large scale yet, they were drawn to the cost benefits and flexibility of Linux.

Orbital engineers were experienced in using UNIX and had dabbled in the Linux operating system for years, so they were well aware of the advantages Linux offered. Although Orbital hadn't deployed Linux on a large scale yet, they were drawn to the cost benefits and flexibility of Linux.

The Fluent factor
Since Fluent is such a major component of Orbital's design process, it was important that the cluster vendor they chose could meet the following criteria:
1. understand the CFD and aerospace industry
2. experience with Fluent Inc. software
3. experience deploying Fluent-based clusters
4. compelling price/performance ratio

Fluent simulation of a missile drop similar to the types of simulations performed on Orbital's CFD cluster. Photo courtesy of Fluent. 
Orbital approached Linux Networx after learning about a system that was successfully installed at Fluent's facility to run CFD jobs for its customers at their Remote Simulation Facility (RSF). Orbital was impressed with the knowledge-base of Linux Networx engineers gleaned from previous CFD installations at other customer sites such as The Boeing Company. However, the deciding factor was the partnership that had formed between Linux Networx and Fluent to provide an optimized Evolocity cluster system pre-loaded with Fluent software for the CFD market.

"We want our customers to have a successful, seamless experience when deploying a new compute environment and the Evolocity CFD cluster provides this. Problems with deployment represent a barrier to expanded use of our software and incur costs to the customer," said Paul Bemis, vice president of marketing for Fluent. "The partnership between Linux Networx and Fluent provides end-users with a 'sure bet' in terms of successful deployment."

To be certain the Evolocity CFD solution would meet Orbital's needs, Linux Networx ran several Fluent benchmark tests on a cluster at Linux Networx's Solution Center, an in-house testing and validating facility that allows customers to try different cluster configurations. The Solutions Center cluster ran problems with very fine granularity in decomposition, which Orbital's previous system was unable to handle in a timely manner. The results proved that an optimized CFD cluster could produce results much faster than the previous solution and led to Orbital's purchase of an Evolocity CFD cluster with 24 Intel Xeon processors.

The Solutions Center where Orbital performed Fluent benchmarks to test the capabilities of the Linux Networx CFD Cluster. 
"The major reason why we chose Linux Networx was because of their intimate experience with Fluent," said Vince Allen, manager of aerodynamics. "We wanted a turn-key type system with Fluent loaded and ready to be used - which is what the Linux Networx CFD system offered."


The cluster solution
The Evolocity CFD cluster was an ideal fit for Orbital's environment, not only because it was pre-loaded with Fluent and optimized for CFD applications, but because the hardware is designed for maximum uptime. Linux Networx's Active Cooling technology provides cool air to critical circuitry deep inside the node and cools the most compute-intensive processors. This helps to avoid hotspots or pressure buildups and ensures each node provides maximum cooling capabilities. The result is longer node life, less unplanned downtime, and more productivity for their cluster system.

With a 24-processor cluster, Orbital engineers needed to know that each node was operating at maximum efficiency, but didn't want to spend time and resources having administrators monitor the cluster for optimal performance. They were able to easily address this problem through the suite of total cluster management tools included with the system. Evolocity systems include Clusterworx, a comprehensive cluster management software solution, and Icebox, a cluster management hardware appliance that fully integrates with Clusterworx to increase system uptime and track cluster performance, allowing Orbital to run more jobs and to simplify system administration.

"The management tools allow complete, comprehensive and effective remote management, including environmental and process monitoring, threshold reporting, remote reboots, power downs and remote console," said Sam Hutton, principal systems analyst in Orbital's IS department. "Clusterworx and Icebox are a very robust set of management and proactive monitoring tools."


Delivering the cluster
Pegasus, a space launch vehicle developed by Orbital to deploy small satellites, was engineered with the help of the Linux Networx CFD Cluster. 
Installing a cluster system can be a long, tedious task as miles of cable, hundreds of hardware components, and various software must come together to create a high performance computer. Linux Networx removed this complexity by completely integrating the cluster at its staging facility prior to delivery. Once the system was completely built up, the cluster underwent testing and validation to ensure it could be delivered to the Orbital site problem-free. As a result, the installation was quick and seamless.

"The racks were laid out perfectly with regard to cable placement and management. The onsite installation engineer was knowledgeable and focused," said Sam Hutton, principal systems analyst in Orbital's IS department. "All in all, we experienced a very neat, quick and professional installation."


Since implementing the optimized CFD cluster, Orbital is seeing significant performance improvements and cost savings. Jobs can be completed 30 times faster than with their previous system, which significantly reduces the amount of third-party outsourcing. Orbital engineers estimate the Linux Networx CFD cluster has more than paid for itself within the first year because of its fast results and the reduction in outsourcing needs - a savings of over $130,000.

"The Linux Networx CFD cluster allows us to run bigger problems than ever before and more numerous design variations on smaller cases, allowing us to refine our analytical predictions to levels that were not attainable at Orbital before," said Allen.

The Linux Networx CFD Cluster helped Orbital complete jobs 30 times faster than their previous solution. 
More importantly, with the power and capabilities of the cluster, Orbital engineers now have the luxury of routinely running small tasks and achieving much more focused and valid results than they were able to accomplish previously.

"The mere fact that we have the opportunity to make adjustments to a CFD calculation and to resubmit it for more specific analysis makes nearly every run a candidate for more refined runs," said Straka. "The biggest benefit to us is that even large tasks can run and be refined as needed, resulting in reduced risk of anomalous performance, which increases the confidence of the company in its products."

Eric Pitcher is VP of Technical Marketing at Linux Networx. He may be contacted at