Skip to main content
SpringerLink
Log in

Policy-Regulated Management of ETL Evolution

  • Chapter
Journal on Data Semantics XIII

Part of the book series: Lecture Notes in Computer Science ((JODS,volume 5530))

Abstract

In this paper, we discuss the problem of performing impact prediction for changes that occur in the schema/structure of the data warehouse sources. We abstract Extract-Transform-Load (ETL) activities as queries and sequences of views. ETL activities and its sources are uniformly modeled as a graph that is annotated with policies for the management of evolution events. Given a change at an element of the graph, our method detects the parts of the graph that are affected by this change and highlights the way they are tuned to respond to it. For many cases of ETL source evolution, we present rules so that both syntactical and semantic correctness of activities are retained. Finally, we experiment with the evaluation of our approach over real-world ETL workflows used in the Greek public sector.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Banerjee, J., Kim, W., Kim, H.J., Korth, H.F.: Semantics and implementation of schema evolution in object-oriented databases. In: Proc. ACM Special Interest Group on Management of Data, pp. 311–322 (1987)

    Google Scholar 

  2. Bellahsene, Z.: Schema evolution in data warehouses. Knowledge and Information Systems 4(3), 283–304 (2002)

    Article  Google Scholar 

  3. Bernstein, P., Levy, A., Pottinger, R.: A Vision for Management of Complex Models. SIGMOD Record 29(4), 55–63 (2000)

    Article  Google Scholar 

  4. Bernstein, P., Rahm, E.: Data warehouse scenarios for model management. In: Laender, A.H.F., Liddle, S.W., Storey, V.C. (eds.) ER 2000. LNCS, vol. 1920, pp. 1–15. Springer, Heidelberg (2000)

    Chapter  Google Scholar 

  5. Blaschka, M., Sapia, C., Höfling, G.: On Schema Evolution in Multidimensional Databases. In: Mohania, M., Tjoa, A.M. (eds.) DaWaK 1999. LNCS, vol. 1676, pp. 153–164. Springer, Heidelberg (1999)

    Google Scholar 

  6. Bouzeghoub, M., Kedad, Z.: A logical model for data warehouse design and evolution. In: Kambayashi, Y., Mohania, M., Tjoa, A.M. (eds.) DaWaK 2000. LNCS, vol. 1874, pp. 178–188. Springer, Heidelberg (2000)

    Chapter  Google Scholar 

  7. Fan, H., Poulovassilis, A.: Schema Evolution in Data Warehousing Environments – A Schema Transformation-Based Approach. In: Atzeni, P., Chu, W., Lu, H., Zhou, S., Ling, T.-W. (eds.) ER 2004. LNCS, vol. 3288, pp. 639–653. Springer, Heidelberg (2004)

    Google Scholar 

  8. Gupta, A., Mumick, I.S., Rao, J., Ross, K.A.: Adapting materialized views after redefinitions: Techniques and a performance study. Information Systems J. 26(5), 323–362 (2001)

    Article  MATH  Google Scholar 

  9. Golfarelli, M., Lechtenbörger, J., Rizzi, S., Vossen, G.: Schema Versioning in Data Warehouses. In: Atzeni, P., Chu, W., Lu, H., Zhou, S., Ling, T.-W. (eds.) ER 2004. LNCS, vol. 3288, pp. 415–428. Springer, Heidelberg (2004)

    Google Scholar 

  10. Kaas, C., Pedersen, T.B., Rasmussen, B.: Schema Evolution for Stars and Snowflakes. In: Sixth Int’l. Conference on Enterprise Information Systems (ICEIS 2004), pp. 425–433 (2004)

    Google Scholar 

  11. Liu, C.T., Chrysanthis, P.K., Chang, S.K.: Database schema evolution through the specification and maintenance of changes on entities and relationships. In: Loucopoulos, P. (ed.) ER 1994. LNCS, vol. 881, pp. 132–151. Springer, Heidelberg (1994)

    Google Scholar 

  12. Mohania, M., Dong, D.: Algorithms for adapting materialized views in data warehouses. In: Proc. International Symposium on Cooperative Database Systems for Advanced Applications (CODAS 1996), pp. 309–316 (1996)

    Google Scholar 

  13. Nica, A., Lee, A.J., Rundensteiner, E.A.: The CSV algorithm for view synchronization in evolvable large-scale information systems. In: Schek, H.-J., Saltor, F., Ramos, I., Alonso, G. (eds.) EDBT 1998. LNCS, vol. 1377, pp. 359–373. Springer, Heidelberg (1998)

    Chapter  Google Scholar 

  14. Papastefanatos, G., Vassiliadis, P., Vassiliou, Y.: Adaptive Query Formulation to Handle Database Evolution. In: Proc. Forum of the Eighteenth Conference on Advanced Information Systems Engineering (CAISE 2006) (2006)

    Google Scholar 

  15. Papastefanatos, G., Vassiliadis, P., Simitsis, A., Vassiliou, Y.: What-if analysis for data warehouse evolution. In: Song, I.-Y., Eder, J., Nguyen, T.M. (eds.) DaWaK 2007. LNCS, vol. 4654, pp. 23–33. Springer, Heidelberg (2007)

    Chapter  Google Scholar 

  16. Papastefanatos, G., Kyzirakos, K., Vassiliadis, P., Vassiliou, Y.: Hecataeus: A Framework for Representing SQL Constructs as Graphs. In: Proc. Tenth International Workshop on Exploring Modeling Methods in Systems Analysis and Design (held with CAISE) (2005)

    Google Scholar 

  17. Papastefanatos, G., Anagnostou, F., Vassiliadis, P., Vassiliou, Y.: Hecataeus: A What-If Analysis Tool for Database Schema Evolution. In: Proc. Twelfth European Conference on Software Maintenance and Reengineering (CSMR 2008) (2008)

    Google Scholar 

  18. Papastefanatos, G., Vassiliadis, P., Simitsis, A., Aggistalis, K., Pechlivani, F., Vassiliou, Y.: Language Extensions for the Automation of Database Schema Evolution. In: 10th International Conference on Enterprise Information Systems (ICEIS 2008) (2008)

    Google Scholar 

  19. Ra, Y.G., Rundensteiner, E.A.: A transparent object-oriented schema change approach using view evolution. In: Proc. Eleventh International Conference on Data Engineering (ICDE 1995), pp. 165–172 (1995)

    Google Scholar 

  20. Roddick, J.F., et al.: Evolution and Change in Data Management - Issues and Directions. SIGMOD Record 29(1), 21–25 (2000)

    Article  Google Scholar 

  21. Roddick, J.F.: A survey of schema versioning Issues for database systems. Information Software Technology J. 37(7) (1995)

    Google Scholar 

  22. Simitsis, A., Vassiliadis, P., Terrovitis, M., Skiadopoulos, S.: Graph-based modeling of ETL activities with multi-level transformations and updates. In: Tjoa, A.M., Trujillo, J. (eds.) DaWaK 2005. LNCS, vol. 3589, pp. 43–52. Springer, Heidelberg (2005)

    Chapter  Google Scholar 

  23. Tsichritzis, D., Klug, A.C.: The ANSI/X3/SPARC DBMS Framework Report of the Study Group on Database Management Systems. Information Systems 3(3), 173–191 (1978)

    Article  Google Scholar 

  24. Velegrakis, Y., Miller, R.J., Popa, L.: Preserving mapping consistency under schema changes. VLDB J. 13(3), 274–293 (2004)

    Article  Google Scholar 

  25. Zicari, R.: A framework for schema update in an object-oriented database system. In: Proc. Seventh International Conference on Data Engineering (ICDE 1991), pp. 2–13 (1991)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. National Technical University of Athens, Greece

    George Papastefanatos & Yannis Vassiliou

  2. University of Ioannina, Greece

    Panos Vassiliadis

  3. Stanford University, USA

    Alkis Simitsis

Authors
  1. George Papastefanatos
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Panos Vassiliadis
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Alkis Simitsis
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yannis Vassiliou
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. EPFL-IC, Database Laboratory, École Polytechnique Fédérale de Lausanne, 1015, Lausanne, Switzerland

    Stefano Spaccapietra

  2. Department of Computer and Decision Engineering,, Université Libre de Bruxelles, 50av. F.D. Roosevelt, 1050, Bruxelles, Belgium

    Esteban Zimányi

  3. College of Information Science and Technology,, Drexel University, 19104, Philadelphia, PA, USA

    Il-Yeol Song

Rights and permissions

Reprints and permissions

Copyright information

© 2009 Springer-Verlag Berlin Heidelberg

About this chapter

Cite this chapter

Papastefanatos, G., Vassiliadis, P., Simitsis, A., Vassiliou, Y. (2009). Policy-Regulated Management of ETL Evolution. In: Spaccapietra, S., Zimányi, E., Song, IY. (eds) Journal on Data Semantics XIII. Lecture Notes in Computer Science, vol 5530. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-03098-7_6

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-03098-7_6

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-03097-0

  • Online ISBN: 978-3-642-03098-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation