I/O-Efficient Algorithms for Sparse Graphs

Abstract

Massive graphs arise naturally in many applications. Recent web crawls, for example, produce graphs with on the order of 200 million nodes and 2 billion edges. Recent research in web modelling uses depth-first search, breadth-first search, and the computation of shortest paths and connected components as primitive routines for investigating the structure of the web [158]. Massive graphs are also often manipulated in Geographic Information Systems (GIS), where many problems can be formulated as fundamental graph problems. When working with such massive data sets, only a fraction of the data can be held in the main memory of a state-of-the-art computer. Thus, the transfer of data between main memory and secondary, disk-based memory, and not the internal memory computation, is often the bottleneck. A number of models have been developed for the purpose of analyzing this bottleneck and designing algorithms that minimize the traffic between main memory and disk. The algorithms discussed in this chapter are designed and analyzed in the parallel disk model (PDM) of Vitter and Shriver [755]. For a definition and discussion of this model, the reader may refer to Chapter 1.

Part of this work was done when the second author was a Ph.D. student at the School of Computer Science of Carleton University, Ottawa, Canada.