Linux is ready - Linux scalability example

Saw an interesting blog late last week from Paul Murphy from over on ZDNet[1]. While comparing AIX and Solaris offerings, he shifts to quickly assess the possibilities of Linux running on the Power systems.

He seems to miss some of the flexibility of Linux in that we believe it's pretty trivial to re-build applications from one Linux base or platform to another, he makes the observation:

• "Right now getting Linux code to run on Linux for PPC isn’t very difficult - it’s very inefficient, because of the endian change and the inappropriateness of the x86 specific ideas embedded in just about everything Linux, but it’s relatively easy to do and it works."

With today's software vendors, most applications are designed very reasonably and thoughtfully with respect to cross-platform issues, which includes of course endian issues. So code built with Linux should be efficient with respect to that hardware base, and in fact, a lot of what Linux operating system programmers focus is exactly the notion of being efficient and effective across the platforms, be it x86, Intel, AMD, zSeries, and Power to name a few. There are of course more platforms, which highlights the advantages of leveraging the Linux base.

In working on Power systems, and seeing work on other platforms, it's clear that non-x86 design points and flexible approaches are encouraged in the operating system. This allows programmers across the industry to easily adapt and leverage Linux to provide leadership performance or scalability for that hardware platform.

In some of the "talkback" comments from Paul's post, there were several posts which were flash-backs to Linux of many years ago, where people would refer to experiences where Linux didn't scale, Linux didn't perform, or Linux wasn't ready. There have been examples for years now which consistently demonstrate that Linux is scalable and does perform well.

From two and a half years ago, Sandor Szabo [2] took the two Linux distributions, SLES 9 and RHEL 4 at the time, and showed the features of what could be exploited in commercial grade Linux. Both SUSE and Redhat have updated their distro versions, moving up to SLES 10 and RHEL 5, both of which continue to significantly improve on the scalability and performance.

In the summer of 2006, just 15 months go, the Kernel Summit specifically covered Linux scalability [3]. This was a good article that summarized nicely the state of Linux scaling across the systems emerging in the industry. I found it interesting that one of the focus items was shifting to scaling issues for systems with 1024 to 4096 processors.

On a practical level, we see Linux scaling and performing fine from 4-core to 16-core to 64-core systems that we test on. For example, we tried SPEC.org's SPECompM2001 workload on a recent new Power 6 16-core system [4] and were able to get the OpenMP threads performing where we wanted it to. The breadth of software support on Linux extends to platform specific compilers, and in that publish the library libhugetlbfs [4] was used to take transparently take advantage of larger memory pages by the system. This library is supported across several architectures and is being actively updated and extended. Each workload could be compiled 32-bit or 64-bit mode, with varying optimization levels.

Just a couple of examples of Linux being ready to perform and to scale today

- - - - - - - - - - - - - -
References...

[1] Linux: it’s the last word on AIX vs. Solaris by ZDNet's Paul Murphy -- With Linux on Power you get open source, you get a hot chipset, you get that IBM relationship, you get a clear future direction, you get a solid development community, you get access to lots of applications, you get a free any time escape to the cheaper Lintel world.

[2] Exploit the power of Linux with Informix Dynamic Server 10.0

[3] Kernel Summit 2006: Scalability

[4] SPECompM2001 - 16 core publish

[5] Sourceforge.net - libhugetlbfs project