Various thoughts on the process of improving performance on a Linux system - in a mode of discovering just how much there is to learn. Customers use their systems uniquely - some care passionately about performance, some just want and expect the best "out-of-the-box" experience with no tweaking. I have observed that people in search of performance answers generally want the simple answer, but the practiced answer to any real performance question is: "Well, it depends..." - Bill Buros

Monday, October 27, 2008

So many cores... so few students...

So it's that time of year when we return to the semi-serious thought process regarding next year's college interns and college hires. Looking down the road in the industry and for Linux, the new servers coming have so many processor cores, requiring new programming models and approaches, that the students coming have got to know how to dive into this new world. And recalling the way-too-many resumes dug through over 2008, there's an amazing disconnect coming.

Which leads me to the three key questions..
  1. Can the schools actually shift gears to teach more about parallel programming?
  2. Or do we need to do this ourselves in the industry and in our example the Linux community itself?
  3. Can the students really cope with so many cores? They will eventually be the users of all of these cores.
I suspect we'll all be chasing just a handful of students who have the background to help the industry down a new path of effective and efficient parallel programming.

As SC08 looms in less than a month here in Austin, the many topics around super-computing keep coming up. One of the most biggest pieces we're interested in is this transition to the massive number of cores coming down the pipe across the hardware architectures and systems. This is especially apparent as our performance focus has branched into clusters as the systems scale up and scale out. And Linux plays on them all. HPC, commercial, educational, personal - the systems are going to have an amazing number of processor cores ready to do useful work.

For myself, it's been an interesting personal journey this year as I've learned more about cluster performance and the different ways that customers and performance benchmarks parallel'ize HPC work. I certainly have a long way to go, but now know enough to see that many cluster products and cluster programming techniques make things way too hard for the end user and the system admins.

Getting ready for SC08, there are a number of sessions, panels, and birds-of-a-feather sessions emerging on this topic. For example, Paul Steinberg over at Intel popped a blog up on "Sequential programming is dead. So stop teaching it!". Interesting theme. Paul's blog has a number of pointers to related information and papers.

Paul McKenney in the LTC nicely hooked me up with this effort so I'll be spending more time on this over the next month. This looks like a really good time to snag the universities and prod the educational processes to shift gears. And, I'm hoping I'll learn enough about parallel programming to know what to look for in next year's students. We need the skills. Today.

0 comments:

One of many bloggers.

Bill Buros

Bill leads an IBM Linux performance team in Austin Tx (the only place really to live in Texas). The team is focused on IBM's Power offerings (old, new, and future) working with IBM's Linux Technology Center (the LTC). While the focus is primarily on Power systems, the team also analyzes and improves overall Linux performance for IBM's xSeries products (both Intel and AMD) , driving performance improvements which are both common for Linux and occasionally unique to the hardware offerings.

Performance analysis techniques, tools, and approaches are nicely common across Linux. Having worked for years in performance, there are still daily reminders of how much there is to learn in this space, so in many ways this blog is simply another vehicle in the continuing journey to becoming a more experienced "performance professional". One of several journeys in life.

The Usual Notice

The postings on this site are my own and don't necessarily represent IBM's positions, strategies, or opinions, try as I might to influence them.