The past twenty years have shown much progress in improving the computational speed of machines for scientific and engineering applications. This has led to significant advances in system architectures, floating point performance, languages, algorithms and scalable mathematical libraries optimized for these numerically intensive applications. However, similar studies are needed to improve the performance of data intensive applications having their foundations in discrete mathematics, artificial intelligence, fuzzy logic, decision theory, and human cognition rather than numerical methods.
Our goal is to initiate systematic studies to determine the characteristics of data traffic for both data and numerically intensive applications, identify the data bottlenecks and use the results to help guide the development of new classes of improved system architectures, algorithms and software libraries that effectively support both number and data "crunching" applications. Such systems are needed since organizations and governments are collecting tera- and even petabytes of data that could be "mined" to discover previously unknown but useful information for business decision making and for new breeds of verification or discovery-driven decision-support systems. Those organizations that learn how to effectively use such emerging systems will have an overwhelming economic advantage over those that do not.
This award uses certain IBM equipment under a Shared University Research Grant from IBM to install and integrate into the Albuquerque High Performance Computing Center (AHPCC) and the Maui High Performance Computing Center (MHPCC), both research centers within the University Of New Mexico's High Performance Computing, Education, and Research Center (HPCERC).
The Albuquerque High Performance Computing Center focuses on education and research with faculty, staff, and students of the University of New Mexico, from eleven academic departments involved in scientific and engineering research wherein computation is playing a major role. The AHPCC currently has approximately 7 faculty, 10 full-time technical staff, approximately 25 student research assistants, and over 15 associated faculty in 11 areas of research interest. The Center operates as its main computational and visualization platforms an IBM SP-2 with 128 nodes, an SGI Onyx 2 Reality Engine and associated SGI workstations with graphics pipes to support its Visualization and Virtual Reality Laboratory, and a Linux-based SuperCluster (RoadRunner) consisting of 64 dual Pentium processor nodes interconnected with fast (Myrinet) switching. The SuperCluster is jointly funded by UNM and the NCSA Alliance.
The Maui High Performance Computing Center is a national supercomputing center established in 1993 by the University of New Mexico under a Cooperative Agreement with the Air Force Research Laboratory. A leader in scalable parallel computing, MHPCC supports a base of 1,200 users from Department of Defense (DoD), government, commercial, and academic organizations. MHPCC provides its users with an innovative environment for developing high performance computing applications. This includes: a stable and secure platform for migrating applications from research to production; a robust computing system for prototyping, benchmarking, and testing applications; classified and unclassified computing capabilities; and programming tools, and scientific and engineering software libraries and application software. MHPCC currently employs 55 staff and scientists in support of its mission. The primary computing power at MHPCC is a 603-node IBM SP-2.
UNM and HPCERC together have three major projects of relevance to this award. First Arthur B. Maccabe of the CS Department and Michael McGowen from Essential Communications and Sandia National Laboratories, are forming a "Scalable System Laboratory" at UNM. This laboratory, located within the Computer Science Department in the School of Engineering, is an effort encouraged and supported by AHPCC. The centerpiece of the is a research program in scalable communication technology including switches and interconnect technology, focused on the emerging SIO standard for system busses. The laboratory will be involved with the development of open drivers and tuning using open Linux systems. Much of the work is based on and relying on the upcoming standardization of network information cards, led by recent development of a Intel-based communication chip set.
Second, AHPCC is taking the lead in the NCSA Alliance with the RoadRunner SuperCluster to demonstrate the feasility and viability of production high performance clusters of COTS computers and high performance data communication networks for use in scientific and engineering computational research. The goal of this project is to show that such a machine can be used for production scientific computing requiring scalable computational solutions and at the same time provide scalable system administration at a reasonable cost. Currently, the RoadRunner SuperCluster has been in production delivering over 500 node hours per day since July 1 and is currently allocated over 86,000 node hours to two NSF-supported supercomputer applications over the next year.
Third, a AHPCC collaboration with one of the major partners, Argonne Mational Laboratory, of the NCSA Alliance on Internet telconferencing technology is also relevant to this award, and is called Information Access Grid. UNM is playing a research role in the development of tools and interfaces to support the NCSA Access Grid. This project involves the development of software to support multi-point, multi-user conferences and meetings the Internet. Although the emphasis currently is on video and audio from cameras and microphones from each site, UNM is interested in developing techniques for inserting scientific visualization from the SGE frame buffer into this collaboration environment. For such an effort to be effective, compression algorithms and other techniques to reduce the data transmitted over the Internet would be essential. As such, we propose to leverage the research and techniques development by the visualization research group at IBM T. J. Watson laboratory for this effort.