Abstract
In this paper we present an experimental study of the properties of web graphs. We study a large crawl from 2001 of 200M pages and about 1.4 billion edges made available by the WebBase project at Stanford [19], and synthetic graphs obtained by the large scale simulation of stochastic graph models for the Webgraph. This work has required the development and the use of external and semi-external algorithms for computing properties of massive graphs, and for the large scale simulation of stochastic graph models. We report our experimental findings on the topological properties of such graphs, describe the algorithmic tools developed within this project and report the experiments on their time performance.
Partially supported by the Future and Emerging Technologies programme of the EU under contracts number IST-2001-33555 COSIN “Co-evolution and Self-organization in Dynamical Network” and IST-1999-14186 ALCOM-FT “Algorithms and Complexity in Future Technologies”, and by the Italian research project ALINWEB: “Algoritmica per Internet e per il Web”, MIUR – Programmi di Ricerca di Rilevante Interesse Nazionale.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Albert, R., Jeong, H., Barabasi, A.L.: Nature (401), 130 (1999)
Riordan, O., Bollobas, B.: Robustness and ulnerability of scale-free random graphs. Internet Mathematics 1(1), 1–35 (2003)
Barabasi, A.L., Albert, A.: Emergence of scaling in random networks. Science (286), 509 (1999)
Brin, S., Page, L.: The anatomy of a large-scale hypertextual Web search engine. Computer Networks and ISDN Systems 30(1–7), 107–117 (1998)
Broder, A., Kumar, R., Maghoul, F., Raghavan, P., Rajagopalan, S., Stata, S., Tomkins, A., Wiener, J.: Graph structure in the web. In: Proceedings of the 9th WWW conference (2000)
Cooper, C., Frieze, A.: A general model of undirected web graphs. In: Proc. of the 9th Annual European Symposium on Algorithms (ESA)
Erdös, P., Renyi, R.: Publ. Math. Inst. Hung. Acad. Sci. 5 (1960)
Haveliwala, T.H.: Efficient computation of pagerank. Technical report, Stanford University (1999)
Kleinberg, J.: Authoritative sources in a hyperlinked environment. Journal of the ACM 46(5), 604–632 (1997)
Kumar, R., Raghavan, P., Rajagopalan, S., Sivakumar, D., Tomkins, A., Upfal, E.: Stochastic models for the web graph. In: Proc. of 41st FOCS, pp. 57–65 (2000)
Kumar, R., Raghavan, P., Rajagopalan, S., Tomkins, A.: Trawling the web for emerging cyber communities. In: Proc. of the 8th WWW Conference, pp. 403– 416 (1999)
Laura, L., Leonardi, S., Caldarelli, G., De Los Rios, P.: A multi-layer model for the webgraph. In: On-line proceedings of the 2nd International Workshop on Web Dynamics (2002)
Laura, L., Leonardi, S., Millozzi, S.: A software library for generating and measuring massive webgraphs. Technical Report 05-03, DIS - University of Rome La Sapienza (2003)
Mitzenmacher, M.: A brief history of generative models for power law and lognormal distributions. Internet Mathematics 1(2) (2003)
Pandurangan, G., Raghavan, P., Upfal, E.: Using pagerank to characterize web structure. In: Ibarra, O.H., Zhang, L. (eds.) COCOON 2002. LNCS, vol. 2387, pp. 330–339. Springer, Heidelberg (2002)
Pennock, D.M., Flake, G.W., Lawrence, S., Glover, E.J., Giles, C.L.: Winners don’t take all: Characterizing the competition for links on the web. Proc. of the National Academy of Sciences 99(8), 5207–5211 (2002)
Sibeyn, J.F., Abello, J., Meyer, U.: Heuristics for semi-external depth first search on directed graphs. In: Proceedings of the fourteenth annual ACM symposium on Parallel algorithms and architectures (SPAA), pp. 282–292 (2002)
Vitter, J.: External memory algorithms. In: Bilardi, G., Pietracaprina, A., Italiano, G.F., Pucci, G. (eds.) ESA 1998. LNCS, vol. 1461, pp. 1–25. Springer, Heidelberg (1998)
The stanford webbase project, http://www-diglib.stanford.edu/~testbed/doc2/WebBase/
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2003 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Laura, L., Leonardi, S., Millozzi, S., Meyer, U., Sibeyn, J.F. (2003). Algorithms and Experiments for the Webgraph. In: Di Battista, G., Zwick, U. (eds) Algorithms - ESA 2003. ESA 2003. Lecture Notes in Computer Science, vol 2832. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-39658-1_63
Download citation
DOI: https://doi.org/10.1007/978-3-540-39658-1_63
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-20064-2
Online ISBN: 978-3-540-39658-1
eBook Packages: Springer Book Archive