High Performance Computing - Charles Severance [0]
By: Charles Severance and Kevin Dowd
Online: This selection and arrangement of content as a collection is copyrighted by Charles Severance and Kevin Dowd. It is licensed under the Creative Commons Attribution License: http://creativecommons.org/licenses/by/3.0/ Collection structure revised: 2010/08/25 For copyright and attribution information for the modules contained in this collection, see the "Attributions" section at the end of the collection. High Performance Computing Table of Contents Introduction to the Connexions Edition Introduction to High Performance Computing 1. Modern Computer Architectures 2. Programming and Tuning Software 3. Shared-Memory Parallel Processors 4. Scalable Parallel Processing 5. Appendixes A. Attributions High Performance Computing By: Charles Severance and Kevin Dowd Online: This selection and arrangement of content as a collection is copyrighted by Charles Severance and Kevin Dowd. It is licensed under the Creative Commons Attribution License: http://creativecommons.org/licenses/by/3.0/ Collection structure revised: 2010/08/25 For copyright and attribution information for the modules contained in this collection, see the "Attributions" section at the end of the collection. Introduction to the Connexions Edition The purpose of this book has always been to teach new programmers and scientists about the basics of High Performance Computing. Too many parallel and high performance computing books focus on the architecture, theory and computer science surrounding HPC. I wanted this book to speak to the practicing Chemistry student, Physicist, or Biologist who need to write and run their programs as part of their research. I was using the first edition of the book written by Kevin Dowd in 1996 when I found out that the book was going out of print. I immediately sent an angry letter to O'Reilly customer support imploring them to keep the book going as it was the only book of its kind in the marketplace. That complaint letter triggered several conversations which let to me becoming the author of the second edition. In true "open-source" fashion - since I complained about it - I got to fix it. During Fall 1997, while I was using the book to teach my HPC course, I re-wrote the book one chapter at a time, fueled by multiple late-night lattes and the fear of not having anything ready for the weeks lecture. The second edition came out in July 1998, and was pretty well received. I got many good comments from teachers and scientists who felt that the book did a good job of teaching the practitioner - which made me very happy. In 1998, this book was published at a crossroads in the history of High Performance Computing. In the late 1990's there was still a question a to whether the large vector supercomputers with their specialized memory systems could resist the assault from the increasing clock rates of the microprocessors. Also in the later 1990's there was a question whether the fast, expensive, and power-hungry RISC architectures would win over the commodity Intel microprocessors and commodity memory technologies. By 2003, the market had decided that the commodity microprocessor was king - its performance and the performance of commodity memory subsystems kept increasing so rapidly. By 2006, the Intel architecture had eliminated all the RISC architecture processors by greatly increasing clock rate and truly winning the increasingly important Floating Point Operations per Watt competition. Once users figured out how to effectively use loosely coupled processors, overall cost and improving energy consumption of commodity microprocessors became overriding factors in the market place. These changes led to the book becoming less and less relevant to the common use cases in the HPC field and led to the book going out of print - much to the chagrin of its small but devoted fan base. I was reduced to buying used
1. Introduction to the Connexions Edition