王飞;陈立;易绵竹;谭新;张兴华;

• 1:信息工程大学洛阳校区
• 2:91709部队
• 3:68002部队

摘要(Abstract)：

自然语言处理发展过程中一直存在着基于规则和基于统计2种研究方向,2种研究方法在很长一段时间里都停滞不前,直到人工智能的进步带来了数据、计算能力和算法的巨大变化,从而推动了自然语言处理的发展.近年来,自然语言处理在语义分析、知识库建设和文本处理方面都有很大程度的进展,大部分进展源于深度学习带来的性能提升.本文梳理了深度学习驱动下自然语言处理的进步,系统地分析了进步的原因,指出深度学习本身的局限性.最后指出自然语言处理面临的挑战,并对该领域未来的研究方向提出了展望.

关键词(KeyWords)： 自然语言处理;深度学习;神经网络;机器学习;语义特征

Abstract:

Keywords:

基金项目(Foundation): 国家自然科学基金项目(编号:11590771)

作者(Authors): 王飞;陈立;易绵竹;谭新;张兴华;

DOI: 10.14188/j.1671-8844.2018-08-002

参考文献(References)：

• [1]Olive J,Christianson C,Mccary J.Handbook of Natural Language Processing and Machine Translation:DARPA Global Autonomous Language Exploitation[M].New York:Springer,2011.
• [2]Ortiz A M,Raskin V,Nirenburg S.New developments in ontological semantics[C]//Proceedings of LREC-02,2013.
• [3]Nirenburg S,Raskin V.Ontological Semantics(Language,Speech,and Communication)[M].The MIT Press,2004.
• [4]Mcshane M,Nirenburg S,Beale S,et al.Two kinds of paraphrase in modeling embodied cognitive agents[C]//National Conference on Artificial Intelligence,2008:87-94.
• [5]Niles I,Pease A.Towards a standard upper ontology[C]//International Conference on Formal Ontology in Information Systems,2001:2-9.
• [6]Chierchia G,Mcconnellginet S.Meaning and grammar:an introduction to semantics[J].Language,2000,67(67):481-502.
• [7]Fillmore C J.The Case for Case[M].Universals in Linguistic Theory,1968:1-88.
• [8]Katz J J,Fodor J A.The structure of a semantic theory[J].Language,1963,39(2):170-210.
• [9]Gontier E M.Web semantic and ontology[J].Advances in Internet of Things,2015,5(2):15-20.
• [10]Suarez-Figueroa M C,Gomez-Perez A,Motta E,et al.Ontology Engineering in a Networked World[M].Springer Publishing Company,Incorporated,2012.
• [11]Gruninger M U M.Ontologies:principles,methods and applications[J].Knowledge Engineering Review,2009,11(2):93-136.
• [12]Niu J,Issa R R A.Developing taxonomy for the domain ontology of construction contractual semantics[J].Advanced Engineering Informatics,2015,29(3):472-482.
• [13]Hinton G E,Salakhutdinov R R.Reducing the dimensionality of data with neural networks[J].Science,2006,313(5786):504-507.
• [14]Lecun Y,Bengio Y.Convolutional networks for images,speech,and time-series[C]//The Handbook of Brain Theory and Neural Networks,1995.
• [15]Lecun Y,Bottou L,Orr G B,et al.Efficient backProp[J].Neural Networks Ricks of the Trade,2000,1524(1):9-50.
• [16]Bengio Y.Learning deep architectures for AI[J].Foundations&Trends in Machine Learning,2009,2(1):1-127.
• [17]Bengio Y,Courville A,Vincent P.Representation learning:A review and new perspectives[J].IEEE Transactions on Pattern Analysis&Machine Intelligence,2013,35(8):1798-1828.
• [18]Bengio Y,Schwenk H,Senécal J S,et al.Neural probabilistic language models[J].Journal of Machine Learning Research,2001,3(6):1137-1155.
• [19]Mohamed A R,Dahl G E,Hinton G.Acoustic modeling using deep belief networks[J].IEEE Transactions on Audio Speech&Language Processing,2012,20(1):14--22.
• [20]Hinton G,Vinyals O,Dean J.Distilling the knowledge in a neural network[J].Computer Science,2015,14(7):38-39.
• [21]Deng L,Hinton G,Kingsbury B.New types of deep neural network learning for speech recognition and related applications:an overview[C]//IEEE International Conference on Acoustics,2013:8599-8603.
• [22]Feng M,Xiang B,Glass M R,et al.Applying deep learning to answer selection:A study and an open task[J].IEEE Automatic Speech Recognition and Understanding Workshop,2015:813-820.
• [23]Szegedy C,Liu W,Jia Y,et al.Going deeper with convolutions[C]//IEEE Conference on Computer Vision and Pattern Recognition.IEEE Computer Society,2014:1-9.
• [24]Long J,Shelhamer E,Darrell T.Fully convolutional networks for semantic segmentation[C]//IEEE Conference on Computer Vision and Pattern Recognition,2015:1337-1342.
• [25]Arel I,Rose D C,Karnowski T P.Deep machine learning-A new frontier in artificial intelligence research[research frontier][J].IEEE Computational Intelligence Magazine,2010,5(4):13-18.
• [26]Zheng X,Chen H,Xu T,et al.Deep learning for Chinese word segmentation and POS tagging[C]//Empirical Methods in Natural Language Processing,2013:647-657.
• [27]Zeng D,Liu K,Lai S,et al.Relation classification via convolutional deep neural network[C]//International Conference on Computational Linguistics,2014:2335-2344.
• [28]Santos C N,Xiang B,Zhou B,et al.Classifying relations by banking with convolutional neural networks[J].Meeting of the Association for Computational Linguistics,2015:626-634.
• [29]Chen Y,Xu L,Liu K,et al.Event extraction via dynamic multi-pooling convolutional neural networks[C]//Meeting of the Association for Computational Linguistics,2015:167-176.
• [30]张晴晴,刘勇,潘接林,等.基于卷积神经网络的连续语音识别[J].中国工程学报,2015,5(2):85-95.Zhang Qingqing,Liu Yong,Pan Jielin,et al.Continuous speech recognition by convolutional neural networks[J].Chinese Journal of Engineering,2015,5(2):85-95.
• [31]Xiong W,Droppo J,Huang X,et al.Achieving human parity in conversational speech recognition[J].IEEE/ACM Transactions on Audio Speech&Language Processing,2016:99.
• [32]Turian J,Ratinov L,Bengio Y.Word representations:a simple and general method for semi-supervised learning[C]//Proc of ACL,Stroudsburg,PA:ACL,2010:384-394.
• [33]Mikolov T,Chen K,Corrado G,et al.Efficient estimation of word representations in vector space[J].Computer Science,2013,arXiv:1301.3781v3.
• [34]Mikolov T,Wen-tau Yih,Zweig G,Linguistic regularities in continuous space word representations[C]//Proceedings of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies 2013,Atlanta,Georgia,2013:746-751.
• [35]Fu Ruiji,Guo J,Qin B,et al,Learning semantic hierarchies via word embeddings[C]//Proceedings of the52nd Annual Meeting of the Association for Computational Linguistics,Baltimore,Maryland,USA,2014:1199-1209.
• [36]Le Q V,Mikolov T.Distributed representations of sentences and documents[J].International Conference on Machine Learning,2014:1188-1196.
• [37]Wan S,Lan Y,Guo J,et al.A deep architecture for semantic matching with multiple positional sentence representations[J].National Conference on Artificial Intelligence,2016:2835-2841.
• [38]Pennington J,Socher R,Manning C.Glove:global vectors for word representation[C]//Conference on Empirical Methods in Natural Language Processing,2014:1532-1543.
• [39]Shazeer N,Doherty R,Evans C,et al.Swivel:improving embeddings by noticing what’s missing[J].Computation and Language,2016,arXiv:1602.02215.
• [40]Lai S,Liu K,He S,et al.How to generate a good word embedding[J].IEEE Intelligent Systems,2016,31(6):5-14.
• [41]Hamilton W L,Leskovec J,Jurafsky D,et al.Diachronic word embeddings reveal statistical laws of semantic change[J].Meeting of the Association for Computational Linguistics,2016:1489-1501.
• [42]Gershman S J,Tenenbaum J B.Phrase similarity in humans and machines[C]//Proceedings of the 37th Annual Conference of the Cognitive Science Society,2015:776-781.
• [43]Blacoe W,Lapata M.A comparison of vector-based representations for semantic composition[C]//Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning,2012:546-556.
• [44]Socher R,Huval B,Manning C D,et al.Semantic compositionality through recursive matrix-vector spaces[C]//Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning,2012:1201-1211.
• [45]Socher R,Bauer J,Manning C D,et al.Parsing with compositional vector grammars[C]//Meeting of the Association for Computational Linguistics,2013:455-465.
• [46]Manning C D.Computational linguistics and deep learning[J].Computational Linguistics,2015,41(4):699-705.