Fig. 1. Modular models.

Fig. 2. Block diagram of a committee machine. Flows of type A distribute data between the machines. Flows of type B appear when machines pass the statistically sampled data (each using different distribution) to the subsequent machines.

Fig. 3. The operation of regression and model (M5) trees. The tree, a hierarchy of tests (circular nodes) and modes (rectangular nodes), is the result of progressive splitting of input space. For the new instance (input vector), each test consists of comparing the value of the attribute indicated in the circle to a certain value; depending on the result, the instance is sorted left of right to undergo the next test, and, finally, is fed into one of the models in the tree leaves that predicts the output.

Fig. 4. Optimizing construction of hierarchical modular models: instead of using a greedy algorithm (M5) for the whole tree, a global optimization algorithm is applied to its upper part.

Fig. 5. Tree encoding and generation. An example shows a three-level tree with five nodes represented by the binary vector (1,1,1,1,0,0) and the corresponding integer vector (3,2,0,1,2) with the numbers of the corresponding split attributes.

Performance of M5′, M5opt and ANN on all data sets (RMSE)

The best performance of M5′, M5flex, M5opt and ANN for hydrological data sets (RMSE)

Scoring matrix for all 11 data sets (Verification), %

Scoring matrix for hydrological data sets (Verification), %