Abstract

A ridgelet kernel regression method is presented in this paper to approximate multi-dimensional functions, especially those with certain kinds of spatial inhomogeneities. This method is based on ridgelet theory, kernel and regularization techniques from which we can deduce a regularized kernel regression form. By representing this form with quadratic programming and taking the obtained solution to define a fitness function, we use particle swarm optimization to optimize the directions of ridgelets. The properties of ridgelet can guarantee the stability of this method in approximating multi-dimensional functions, as well as its superiority for functions with linear singularities. Additionally, the regularized technique employed in this model leads to smaller generalization error. Experiments in the tasks of regression and classification show its effectiveness.

Keywords: Kernal regression; Ridgelet; Partical swarn optimization

Abbreviations: BP, basis pursuit; BPDN, basis pursuit de-noising; ERM, empirical risk minimization; FNN, feed-forward neural network; FT, Fourier transform; GA, genetic algorithm; GSVM, Gaussian kernel SVM; IRLS, iterative reweighted least squares; KC, kernel clustering; KFD, kernel Fisher decision; KM, kernel machine; KNN, k-nearest neighbors; KPCA, kernel principal component analysis; LS-SVM, least square SVM; MDFA, multi-dimensional function approximation; MIMO, multi-input and multi-output systems; ML, machine learning; MP, marching pursuit; MSE, minimum squared error; OMP, orthogonal matching pursuit; OOMP, optimized orthogonal matching pursuit; RKHS, reproducing kernel Hilbert space; RMSE, root mean squared error; PPR, projection pursuit regression; PSO, particle swarm optimization; PSO-RKR, PSO-based ridgelet kernel regression; SRM, structure risk minimization; SVM, support vector machine; WSVM, wavelet kernel SVM; WT, wavelet transform

Article Outline

1. Introduction

2. RKR algorithm

2.1. Ridgelet and ridgelet regression

2.2. RKR model

2.3. MSE-based regularized kernel form

3. Optimization of directions of ridgelets

4. Simulation experiments

5. Conclusion

Acknowledgements

References

Vitae