Skip to main content
SpringerLink
Log in

Review of data storage and management technologies for massive remote sensing data

  • Published:
Science China Technological Sciences Aims and scope Submit manuscript

Abstract

Aiming at the storage and management problems of massive remote sensing data, this paper gives a comprehensive analysis of the characteristics and advantages of thirteen data storage centers or systems at home and abroad. They mainly include the NASA EOS, World Wind, Google Earth, Google Maps, Bing Maps, Microsoft TerraServer, ESA, Earth Simulator, GeoEye, Map World, China Centre for Resources Satellite Data and Application, National Satellite Meteorological Centre, and National Satellite Ocean Application Service. By summing up the practical data storage and management technologies in terms of remote sensing data storage organization and storage architecture, it will be helpful to seek more suitable techniques and methods for massive remote sensing data storage and management.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Chen S P, Guo H D. Digital Earth and Earth observation (in Chinese). Acta Geogr Sin, 2000, 55(1): 8–14

    Google Scholar 

  2. Li D R. Development prospect of photogrammetry and remote sensing (in Chinese). Geom Inf Sci Wuh Univ, 2008, 33(12): 1211–1215

    Google Scholar 

  3. Li D R, Shao Z F. The new era for geo-information. Sci China Ser F-Inf Sci, 2009, 52(7): 1233–1242

    Article  MATH  Google Scholar 

  4. Wang J Y. Development trends of cartography and geographic information engineering (in Chinese). Acta Geod Cart, 2010, 39(2): 115–118

    Google Scholar 

  5. Chen J, Jiang J, Zhou X, et al. Design of national geo-spatial information service platform: overall structure and key components (in Chinese). Geom World, 2009, 6(3): 7–11

    Google Scholar 

  6. Cheng C Q, Guan L. The global subdivision grid based on extended mapping division and its address coding (in Chinese). Acta Geod Cart, 2010, 39(3): 295–302

    MathSciNet  Google Scholar 

  7. Cheng C Q, Lü X F, Guan L. Study on system architecture of subdi vision storage cluster for global spatial data (in Chinese). Acta Sci Nat Univ Pek, 2010, 47(1): 103–108

    MATH  Google Scholar 

  8. Esfandiari M, Ramapriyan H, Behnke J, et al. Evolving a ten year old data system. In: Proceedings of 2nd IEEE International Conference on Space Mission Challenges for Information Technology (SMC-IT′06). Pasadena, California: IEEE, 2006. 243–250

    Chapter  Google Scholar 

  9. Esfandiari M, Ramapriyan H, Behnke J, et al. Earth observing system (EOS) data and information system (EOSDIS)-evolution update and future. In: Proceedings of IEEE International Geoscience and Remote Sensing Symposium (IGARSS 2007). Barcelona: IEEE, 2007. 40005–40008

    Google Scholar 

  10. Behnke J, Watts T H, Kobler B, et al. EOSDIS petabyte archives: tenth anniversary. In: Proceedings of the 22nd IEEE/13th NASA Goddard Conference on Mass Storage Systems and Technologies (MSST′05). Monterey, California: IEEE, 2005. 81–93

    Chapter  Google Scholar 

  11. Ramapriyan H K, Pfister R, Weinstein B. An overview of the EOS data distribution systems. Rem Sens Dig Imag Proc, 2011, 11(3): 183–202

    Google Scholar 

  12. Ilg D, Wynne D, Gejjagaraguppe R, et al. HDF-EOS library user’s guide for the ECS project, volume 1: overview and examples. Tech Paper 170-TP-500-001, 1999

  13. Wei Y X, Di L P, Zhao B Z, et al. Transformation of HDF-EOS metadata from the ECS model to ISO 19115-based XML. Comp Geosci, 2007, 33: 238–247

    Article  Google Scholar 

  14. Mitchell A, Ramapriyan H, Lowe D. Evolution of Web services in EOSDIS-search and order metadata registry (ECHO). In: Proceedings of IEEE International Geoscience and Remote Sensing Symposium (IGARSS 2009). Cape Town, South Africa: IEEE, 2009. 371–374

    Google Scholar 

  15. Kempler S, Lynnes C, Vollmer B. Evolution of information management at the GSFC earth sciences (GES) data and information services center (DISC): 2006–2007. IEEE T Geosci Remote, 2009, 47(1): 21–28

    Article  Google Scholar 

  16. Boschetti L, Roy D P, Justice C O. Using NASA’s World Wind virtual globe for interactive internet visualization of the global MODIS burned area product. Int J Remote Sens, 2008, 29(11): 3067–3072

    Article  Google Scholar 

  17. Bell D G, Kuehnel F, Maxwell C, et al. NASA World Wind: opensource GIS for mission operations. In: Proceedings of IEEE Aerospace Conference. Big Sky, MT: IEEE, 2007. 1–9

    Google Scholar 

  18. Gibin M, Singleton A, Milton R, et al. An exploratory cartographic visualization of London through the Google Maps API. Appl Spat Anal Pol, 2008, 1(2): 85–97

    Article  Google Scholar 

  19. Sample J T, Loup E. Tile-base geospatial information system: principle and practices. New York: Springer, 2010. 23–200

    Book  Google Scholar 

  20. Guo W, Gong J Y, Jiang W S, et al. OpenRS-Cloud: a remote sensing image processing platform based on cloud computing environment. Sci China Tech Sci, 2010, 53(suppl 1): 221–230

    Article  Google Scholar 

  21. Ghemawat S, Gobioff H, Shun-Tak L. The Google file system. In: Proceedings of the Nineteenth ACM Symposium on Operating Systems Principles (SOSP′03). Bolton Landing, New York: IEEE, 2003. 1–15

    Google Scholar 

  22. Dean J, Ghemawat S. MapReduce: simplied data processing on large clusters. Commun ACM, 2008, 51(1): 107–113

    Article  Google Scholar 

  23. Chang F, Dean J, Ghemawat S, et al. Bigtable: a distributed storage system for structured data. ACM T Comput Syst, 2008, 26: 1–14

    Article  MATH  Google Scholar 

  24. Martino S D, Bimonte S, Bertolotto M, et al. Spatial online analytical processing of geographic data through the Google Earth interface. Geoc Sust Env Plann, 2011, 348: 163–182

    Article  Google Scholar 

  25. Dean J, Ghemawat S. MapReduce: a flexible data processing tool. Commun ACM, 2010, 53(1): 72–77

    Article  Google Scholar 

  26. Schwartz J. Bing Maps Tile System. http://msdn.microsoft.com/en-us/library/bb259689.aspx

  27. Pendleton C. The World according to Bing. IEEE Comp Gra Appl, 2010, 30(4): 15–17

    Article  Google Scholar 

  28. Redkar T. Windows Azure Platform. New York: Apress, 2009. 53–104

    Book  Google Scholar 

  29. Wu C, Fraundorfer F, Frahm J M, et al. Image localization in satellite imagery with feature-based indexing. In: Proceedings of the International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences. Beijing: ISPRS, 2008. 197–202

    Google Scholar 

  30. Barclay T, Gray J, Ekblad S, et al. Designing and building TerraService. IEEE Internet Comput, 2006, 10(5): 16–25

    Article  Google Scholar 

  31. Barclay T, Chong W, Gray J. “TerraServer Bricks-a high availability cluster alternative.” Microsoft Technical Report MSR-TR-2004-107, 2004

  32. Barclay T J G. TerraServer SAN-Cluster architecture and operations experience. Microsoft Technical Report MSR-TR-2004-67, 2004

  33. Barclay T, Gray J, Slutz D. Microsoft TerraServer: a spatial data warehouse. In: Proceedings of the ACM SIGMOD International Conference on Management of Data. Dallas, Texas: ACM, 2000. 29(2): 307–318

    Chapter  Google Scholar 

  34. Albani S, Giaretta D. Long-term preservation of Earth observation data and knowledge in ESA through CASPAR. Int J Dig Cur, 2009, 4(3): 4–16

    Google Scholar 

  35. Fusco L, Cossu R. Past and future of ESA Earth observation grid. Mem S A It, 2009, 80: 461–476

    Google Scholar 

  36. Beruti V, Forcada M E, Albani M, et al. ESA plans-a pathfinder for long term data preservation. In: Proceedings of the 7th International Conference on Preservation of Digital Objects (iPRES2010). Vienna, Austria: OAI, 2010. 1–6

    Google Scholar 

  37. Nakano T, Kawase Y, Yamaguchi T, et al. Parallel computing of magnetic field for rotating machines on the Earth Simulator. IEEE Trans Magn, 2010, 46(8): 3273–3276

    Article  Google Scholar 

  38. Nakajima K. Preconditioned iterative linear solvers for unstructured grids on the Earth Simulator. In: Proceedings of the Seventh International Conference on High Performance Computing and Grid in Asia Pacific Region (HPCAsia′04). Omiya Sonic City, Tokyo: IEEE, 2004. 150–159

    Chapter  Google Scholar 

  39. Peng X D, Li X L. Advances in the numerical method and technology of multi-scale numerical prediction (in Chinese). J Appl Met Sci, 2010, 21(2): 129–136

    Google Scholar 

  40. Habata S, Yokokawa M, Kitawaki S. The Earth Simulator system. NEC Res Dev, 2003, 44: 21–26

    Google Scholar 

  41. Itakura K. MDPS: The new mass data processing storage system for the Earth Simulator. J Earth Sim, 2006, 5: 20–24

    Google Scholar 

  42. Investor Presentation by GeoEye. http://www.geoeye.com/CorpSite/assets/docs/investor-relations/Investor_Presentation_JPMorgan_05_17_10.pdf

  43. Simplified Digital Archiving for Remote Sensing. http://www.quantum.com/Solutions/IndustrySolutions/Government/Index.aspx

  44. Gong J Y, Xiang L G, Chen J, et al. Multi-source geospatial information integration and sharing in Virtual Globes. Sci China Tech Sci, 2010, 53(suppl 1): 1–6

    Article  Google Scholar 

  45. Wu X, Guo J, Wallace J, et al. Evaluation of CBERS image data: geometric and radiometric aspects. Inn Rem Sens Phot, 2009, 2: 91–103

    Article  Google Scholar 

  46. Wang F. Design and implementation of ground data storage and management system for earth resources satellite (in Chinese). Space Eng, 2009, 18(3): 66–71

    Google Scholar 

  47. Zhao L C, Wang S J, Shi J M. Information sharing system and technology of the National Satellite Meteorological Center (in Chinese). J Appl Met Sci, 2002, 13(5): 627–632

    Google Scholar 

  48. Qian J M, Zheng X D. The satellite data archive system of National Satellite Meteorological Center (in Chinese). J Appl Met Sci, 2003, 14(6): 756–762

    Google Scholar 

  49. Yang J, Dong C H, Lu N M, et al. FY-3A: the new generation polar orbiting meteorological satellite of china (in Chinese). Acta Met Sin, 2009, 67(4): 501–509

    Google Scholar 

  50. Xu J M, Yang J, Zhang Z Q, et al. Chinese meteorological satellite, achievements and applications (in Chinese). Met Mon, 2010, 36(7): 94–100

    MathSciNet  Google Scholar 

  51. Jia S Z, Yang J, Shi J M, et al. Study and application of parallel scheduling algorithm in new generation meteorological satellite data processing system (in Chinese). Met Sci Techn, 2010, 38(1): 96–101

    Google Scholar 

  52. Bai Z G, Li Y F, Yang W T. Achievement and prospect of China ocean satellite technology (in Chinese). Space Eng, 2008, 17(4): 17–23

    MathSciNet  Google Scholar 

  53. Wang H, Peng H L. Research on long-term archive method of off-line data from HY-1 satellite (in Chinese). Mar Sci Bull, 2008, 27(4): 98–100

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Institute of Remote Sensing and GIS, Peking University, Beijing, 100871, China

    XueFeng Lü & ChengQi Cheng

  2. State Key Laboratory for Information Engineering in Surveying, Mapping and Remote Sensing, Wuhan University, Wuhan, 430079, China

    JianYa Gong

  3. Beijing Institute of Surveying and Mapping, Beijing, 100038, China

    Li Guan

Authors
  1. XueFeng Lü
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. ChengQi Cheng
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. JianYa Gong
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Li Guan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to ChengQi Cheng.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Lü, X., Cheng, C., Gong, J. et al. Review of data storage and management technologies for massive remote sensing data. Sci. China Technol. Sci. 54, 3220–3232 (2011). https://doi.org/10.1007/s11431-011-4549-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11431-011-4549-z

Keywords

Search

Navigation