Determining the variety of traditional Chinese medicine is the first step to ensure the quality of traditional Chinese medicine. In order to explore the rapid identification method of Chinese herbal medicine varieties, the classification and identification of six kinds of Chinese herbal medicine varieties are studied by terahertz spectroscopy combined with pattern recognition. Six kinds of commonly used Chinese herbal medicines such as Baifupian, Rhubarb, Dangshen, Tangerine Peel, Ophiopogon Japonicus and Gastrodia Elata are collected. A total of 420 groups of terahertz spectral data are obtained. Support Vector Machine(SVM), Principal Component Analysis(PCA) combined with SVM, Linear Discriminant Analysis(LDA) combined with SVM are employed in the 0.2-1.5 THz band to qualitatively identify six kinds of traditional Chinese medicine. The results show that the LDA-SVM Chinese herbal medicine variety recognition model based on terahertz spectral data combined with linear discriminant analysis and SVM is the best, with the accuracy of 100% and 98.41% for unknown samples. The LDA-SVM model in this paper bears good identification ability, can quickly and accurately identify the varieties of traditional Chinese medicine, and provides another identification means for the quality control of traditional Chinese medicine.