Abstract
The Southern California Earthquake Center digital library publishes scientific data generated by seismic wave propagation simulations. The output from a single simulation may be as large as 50 Terabytes of data and 400,000 files. The total size of the digital library is over 130 Terabytes with nearly three million files. We examine the data Grid capabilities required to manage this scale of simulation output, and the implementation of these capabilities in a digital library built on top of the Storage Resource Broker data Grid. We also examine the capabilities that will be needed in the future as the scale of simulation output increases to petabytes of data.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
SCEC Community Modeling Environment Project: http://www.scec.org/cme
Rajasekar, A., Wan, M., Moore, R., Schroeder, W.: A prototype rule-based distributed data management system. In: HPDC workshop on “Next Generation Distributed Data Management,” May, Paris, France (2006)
GridSphere portal technology: http://www.gridsphere.org/
The SDSC Storage Resource Broker, http://www.sdsc.edu/srb/
Moore, R., Rajasekar, A., Wan, M.: Data Grids, digital libraries and persistent archives: An integrated approach to publishing, sharing and archiving data. Special issue of the Proceedings of the IEEE on Grid Computing. 93(3), 578–588 (2005)
DSpace digital library, http://www.dspace.org/
Fedora digital object repository middleware, http://www.fedora.info/
Olsen, K.B.: Simulation of three-dimensional wave propagation in the Salt Lake Basin. PhD Thesis, University of Utah (1994)
Minster, J.-B., Olsen, K. B., Moore, R. W., Day, S., Maechling, P., Jordan, T. H., Faerman, M., Cui, Y., Ely, G., Hu, Y., Shkoller, B., Marcinkovich, C., Bielak, J., Okaya, D., Archuleta, R., Wilkins-Diehr, N., Cutchin, S., Chourasia, A., Kremenek, G., Jagatheesan, A., Brieger, L., Majumdar, A., Chukkapalli, G., Xin, Q., Moore, R. L., Banister, B., Thorp, D., Kovatch, P., Diegel, L., Sherwin, T., Jordan, C., Thiebaux, M., Lopez, J.: The SCEC TeraShake Earthquake Simulation. Eos Trans. AGU, 85, 46, Fall Meet. Suppl., Abstract SF31B-05 (2004)
Scalable Visualization Toolkit, http://visservices.sdsc.edu/vistools/
HDF5 – Hierarchical Data Format version 5, http://hdf.ncsa.uiuc.edu/HDF5/
Open Grid Forum, http://www.ogf.org/
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Faerman, M., Moore, R., Cui, Y. et al. Managing Large Scale Data for Earthquake Simulations. J Grid Computing 5, 295–302 (2007). https://doi.org/10.1007/s10723-007-9072-x
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10723-007-9072-x