Basser Seminar Series

Distribution and Storage of Data on Local and Remote Disks in Multi-Use Clusters of PCs

Felix Rauch Valenti (NICTA / ETH Zurich)

Wednesday 20 April 2005, 2-3pm

Basser Conference Room (Madsen Room G92)

Abstract

The trend to use the same or similar commodity components in all new architectures from a laptop to the largest supercomputers caused a highly welcome acceleration of growth in performance and decrease in price due to the high volume of these components. The annual increase in the amount of compute performance and disk storage capacities at the same or lower cost is just incredible. It is therefore highly important that we learn how to design a system under the constraint of commodity components and that we aim to use the widely available compute and storage resources in a more flexible and intelligent way. In this talk the speaker postulates a generalised view of all installed IT resources (processors, storage and network) in the form of “multi-use clusters of commodity PCs”.

To achieve this vision, the operating system installation of the previously isolated computers must support a flexible and distributed resource management. In order to achieve this by scientific means, we introduce a few fundamental technologies including (a) an analytic model to optimise the streams while distributing large data sets in clusters of PCs, (b) a simple software tool to distribute disk images, which derived from the model and its principle, (c) an efficient storage scheme for archiving software-installation images in an operating-system-independent way and (d) several implementations and a rigorous classification of a variety of distributed storage solutions based on file systems and distributed devices. While the first technologies enable a more flexible management of multi-use clusters, the latter technologies allow for a more efficient use of all unused disk space in multi-use clusters.

Speaker's biography

Dr Felix Rauch Valenti is a researcher in National ICT Australia’s (NICTA) Embedded, Real-Time, and Operating Systems (ERTOS) research program since June 2004. His research interests include operating systems, distributed storage systems and parallel and distributed systems. Prior to joining NICTA, Dr Rauch Valenti worked almost seven years as a research assistant, lecturer and postdoctoral researcher in the parallel and distributed systems research group at the laboratory for computer systems at the swiss federal institute of technology (ETH) Zurich, Switzerland, where his research focused on performance aspects of clusters of commodity PCs. He earned his doctorate oftechnical sciences in 2003 from ETH Zurich, Switzerland, and a masters degree in computer science and engineering from the same institute in 1997.