计算机工程与应用 ›› 2022, Vol. 58 ›› Issue (13): 135-145.DOI: 10.3778/j.issn.1002-8331.2112-0491

• 模式识别与人工智能 • 上一篇    下一篇

结合LDA与Word2vec的文本语义增强方法

唐焕玲,卫红敏,王育林,朱辉,窦全胜   

  1. 1.山东工商学院 计算机科学与技术学院,山东 烟台 264005 
    2.山东省高等学校协同创新中心:未来智能计算,山东 烟台 264005 
    3.山东省高校智能信息处理重点实验室(山东工商学院),山东 烟台 264005 
    4.山东工商学院 信息与电子工程学院,山东 烟台 264005 
    5.上海绘话智能科技有限公司,上海 200120
  • 出版日期:2022-07-01 发布日期:2022-07-01

Text Semantic Enhancement Method Combining LDA and Word2vec

TANG Huanling, WEI Hongmin, WANG Yulin, ZHU Hui, DOU Quansheng   

  1. 1.School of Computer Science and Technology, Shandong Technology and Business University, Yantai, Shandong 264005, China
    2.Co-innovation Center of Shandong Colleges and Universities:Future Intelligent Computing, Yantai, Shandong 264005, China
    3.Key Laboratory of Intelligent Information Processing in Universities of Shandong(Shandong Technology and Business University), Yantai, Shandong 264005, China
    4.School of Information and Electronic Engineering, Shandong Technology and Business University, Yantai, Shandong 264005, China 
    5.Shanghai Conversation Intelligence Co. Ltd., Shanghai 200120, China
  • Online:2022-07-01 Published:2022-07-01

摘要: 文本的语义表示是自然语言处理和机器学习领域的研究难点,针对目前文本表示中的语义缺失问题,基于LDA主题模型和Word2vec模型,提出一种新的文本语义增强方法Sem2vec(semantic to vector)模型。该模型利用LDA主题模型获得单词的主题分布,计算单词与其上下文词的主题相似度,作为主题语义信息融入到词向量中,代替one-hot向量输入至Sem2vec模型,在最大化对数似然目标函数约束下,训练Sem2vec模型的最优参数,最终输出增强的语义词向量表示,并进一步得到文本的语义增强表示。在不同数据集上的实验结果表明,相比其他经典模型,Sem2vec模型的语义词向量之间的语义相似度计算更为准确。另外,根据Sem2vec模型得到的文本语义向量,在多种文本分类算法上的分类结果,较其他经典模型可以提升0.58%~3.5%,同时也提升了时间性能。

关键词: LDA主题模型, Word2vec模型, 语义词向量, 语义相似度, 文本分类

Abstract: Text semantic representation is one of the most difficulty problems in natural language processing and machine learning. To solve the problem of semantic loss in text representation, this paper proposes a new text semantic representation method named Sem2vec(semantic to vector) model which is based on the LDA topic model and the Word2vec model. The topic similarity is calculated according to the word topic distribution obtained by the LDA model. Then the topic semantic word vectors are inputted into the Sem2vec model instead of the one-hot vector. Constrained by maximizing log-likelihood objective function, the parameters of the Sem2vec model are optimized. Finally, the semantic word vectors are learned by the Sem2vec model and the semantic representation of the text is further obtained. The experimental results on different datasets show that compared with the other classic models, the Sem2vec model is more accurate in calculating semantic similarity between words. Moreover, in different classification algorithms, the text semantic vectors generated by the Sem2vec model can improve the text classification results by 0.58%~3.5% and promote the time performance compared with the other classic models.

Key words: LDA topic model, Word2vec model, semantic word vector, semantic similarity, text categorization