Abstract

KDDML (KDD Markup Language) is a middleware language and system designed to support the development of final applications or higher level systems which deploy a mixture of data access, data preprocessing, extraction and deployment of data mining models.

We present our three-years’ experience in the development of KDDML. The design principles are motivated by requirements derived from recurring patterns in the KDD process.

The KDDML language is XML-based, both for query syntax and data/model representation. A KDDML query is an XML-document where XML tags correspond to operations on data/models, XML attributes correspond to parameters of those operations and XML sub-elements define arguments passed to the operators. We present the operators for data access and preprocessing, model extraction and deployment, and control flow ones.

The core of the KDDML system is a KDDML language interpreter with modularity and extensibility requirements as the main goals. Additional data sources, and preprocessing and mining algorithms can be easily plugged in the system.

Keywords: Knowledge discovery in databases; Data mining; Query languages

Article Outline

1. Introduction

2. Background: Querying and processing data and models

2.1. Relational databases

2.1.1. Querying

2.1.2. Processing

2.2. XML

2.2.1. Querying

2.2.2. Processing

2.2.3. XML, relational data and SQL

2.3. Data mining

2.3.1. Querying

2.3.2. Processing

3. Modelling the KDD process: challenges and requirements

3.1. Requirements on accessing physical data

3.2. Requirements on specifying logical data

3.3. Requirements on mining model representation

3.4. Requirements on accessing mining models

4. KDDML: KDD Markup Language

4.1. Data access and preprocessing

4.1.1. Data format

4.1.2. Data access

4.1.3. Data preprocessing

4.2. Mining models

4.2.1. Model format

4.2.2. Model access

4.2.3. Model extraction

4.2.4. Model application and evaluation

4.2.5. Model (meta-)reasoning

4.3. Control flow and external programs

4.3.1. Calls to external programs/RDBMS

4.3.2. Calls of queries

4.3.3. Sequences and parallelism of queries

4.3.4. Decision and recursion

5. KDDML: system architecture

5.1. Repository layer

5.1.1. Data and models manager

5.1.2. Data and models factory

5.2. Operators and algorithms layer

5.3. Interpreter layer

5.4. User interface layer

6. Related work and conclusions

6.1. Related work

6.1.1. FlexiMine

6.1.2. Yale

6.2. Experiences with KDDML

6.3. Conclusion

Acknowledgements

References

Vitae