-
近年来,深度学习在计算机视觉感知[1]、语音识别[2]、文本理解[3]等领域取得巨大成功,引起了研究者的极大关注。然而,当前大部分深度学习方法需要海量的标注样本才能学习到泛化性较好的智能识别模型,单纯依靠数据驱动的建模方式使得基于深度学习的目标识别面临新的挑战。一方面,仅采用数据驱动方式难以解决标注样本较少的问题。对于很多实际问题,要么是很难采集到大量样本,要么是标注样本的成本极高,仅依靠少量标注数据,使用数据驱动建模很难得到可靠的模型;另一方面,模型的不稳定性和难解释性一直是深度学习理论面临的难点问题,伴随海量标注数据的大量噪声导致深度学习不稳定,深度学习模型提取的特征很难直观地理解和解释。这些问题和挑战限制了深度学习解决更复杂、更抽象问题的可能性。
导致这些问题和局限的根本原因在于当前人工智能方法与人类智能存在较大差异,人类自身学习识别并不需要大量的标注样本,而是通过已有知识、经验,对照少量样例归纳总结并进行分析与判断,实现目标的稳定识别。缩小人工智能与人类智能的鸿沟,仍然是现代人工智能面临的巨大挑战。
将外部可理解的语义空间知识引入识别建模过程,采用知识与数据联合驱动的方式进行智能模型构建是解决上述问题的一条重要途径。一方面知识与数据联合驱动建模,需要统一先验知识和数据信息的表征形式,实现相互补充,一定程度上解耦深度学习模型训练对海量数据的强依赖性,缓解小样本问题;另一方面,相比于数据,知识的稳定性和可靠性更高,基于知识与数据联合驱动模型更符合真实的人类思维与思考习惯,有利于提高识别算法的稳定性、可靠性与鲁棒性,能够进一步提高识别效果,为后续更上层的智能化应用(推理、决策等)提供基础感知模型。知识与数据联合驱动的识别建模能够突破当前基于深度学习的目标识别建模的瓶颈,解决深度学习在小样本、模型可解释性问题上的局限性。
如何摆脱深度学习模型对海量标注样本的依赖,突破人工智能在小样本问题上的瓶颈,提高模型可解释性,正逐渐成为重要的研究方向。本文首先以外部知识在智能识别模型构建中的引入方式为区分准则,提出一种模型构建方法的分类标准;然后对每类构建方法在解决小样本、模型可解释性问题方面的探索进行了综述总结;最后,提出了一种知识与数据联合驱动建模方式,并基于此提出了需要进一步研究的问题与未来的研究方向。
A Review of Modeling Techniques Jointly Driven by Knowledge and Data
-
摘要: 当前,基于深度学习的目标识别建模技术面临标注样本不足、模型可解释性不高、稳定性不够等新的挑战,限制了深度学习解决更复杂、更抽象问题的可能性。采用知识与数据联合驱动的方式进行智能模型构建是突破现有瓶颈的一条重要途径。该文以外部经验与认知知识在模型构建中的引入方式为区分准则,提出了模型构建方法的分类标准,包括基于显式知识的建模方法、基于隐式知识的建模方法以及基于融合知识的建模方法;然后围绕每类方法在解决小样本、模型可解释性等问题上的探索进行综述,并总结设想了一种未来的知识与数据联合驱动建模方式。这种方式吸取了不同建模方式的优点,通过解耦知识建模与数据建模,以无监督、弱监督为核心训练方式,可以有效解决小样本条件下模型构建问题,提高模型可解释性。最后,该文总结了需要进一步研究的问题和未来的研究方向,以促进目标识别模型构建技术的发展。Abstract: In recent years, object recognition modeling techniques based on deep learning face new challenges such as insufficient annotated samples, low interpretability of models, and insufficient stability. All these challenges limit the possibility of deep learning to solve more complex and abstract problems. Constructing intelligent model jointly driven by knowledge and data is an important way to break through the existing bottleneck. This paper presents a classification standard of model constructing methods according to the way of introduction of external experience and cognitive knowledge during model constructing, including modeling methods based on explicit knowledge, modeling methods based on implicit knowledge and modeling methods based on fusion knowledge. Then, following the proposed classification standard, the explorations in each class of methods about solving the problems of few samples and model interpretability are reviewed. Subsequently, taking advantage of different model constructing methods, a future model constructing method jointly driven by knowledge and data is proposed. The proposed method can effectively solve the problem of model construction under the condition of few samples and improve the interpretability of the model by decoupling knowledge modeling and data modeling and taking unsupervised and weak supervised training as the core training patterns. Finally, some research issues which need further study as well as future research directions are drawn in conclusion for promote the object recognition model constructing.
-
[1] KRIZHEVSKY A, SUTSKEVER I, HINTON G E. Imagenet classification with deep convolutional neural networks[J]. Advances in Neural Information Processing Systems, 2012, 25: 1097-1105. [2] HINTON G, DENG L, YU D, et al. Deep neural networks for acoustic modeling in speech recognition: The shared views of four research groups[J]. IEEE Signal Processing Magazine, 2012, 29(6): 82-97. doi: 10.1109/MSP.2012.2205597 [3] CONNEAU A, SCHWENK H, BARRAULT L, et al. Very deep convolutional networks for text classification[C]//European Chapter of the Association for Computational Linguistics. Stroudsburg, PA: ACL, 2017(1): 1107-1116. [4] SIMON H A. Models of man: Social and rational[M]. New York: Wiley & Sons, 1957. [5] NEWELL A, SIMON H A. Computer science as empirical inquiry: Symbols and search[J]. Communications of the ACM, 1976, 19(3): 113-126. doi: 10.1145/360018.360022 [6] NEWELL A. Physical symbol systems[J]. Cognitive Science, 1980, 4(2): 135-183. [7] FODOR J A. Methodological solipsism considered as a research strategy in cognitive psychology[J]. Behavioral and Brain Sciences, 1980, 3(1): 63-73. doi: 10.1017/S0140525X00001771 [8] MCCARTHY J, MINSKY M L, ROCHESTER N, et al. A proposal for the Dartmouth summer research project on artificial intelligence[J]. AI Magazine, 2006, 27(4): 12. [9] LINDSAY R K, BUCHANAN B G, FEIGENBAUM E A, et al. Applications of artificial intelligence for organic chemistry: The dendral project[M]. New York: McGraw-Hill Book Company, 1980. [10] BUCHANAN B G, SHORTLIFFE E H. Rule-based expert systems: The MYCIN experiments of the stanford heuristic programming project[M]. Boston: Addison Wesley, 1984. [11] MILLER E G, MATSAKIS N E, VIOLA P A. Learning from one example through shared densities on transforms[C]//IEEE Conference on Computer Vision and Pattern Recognition. Los Alamitos, CA: IEEE Computer Society, 2000(1): 464-471. [12] KWITT R, HEGENBART S, NIETHAMMER M. One-shot learning of scene locations via feature trajectory transfer[C]//IEEE Conference on Computer Vision and Pattern Recognition. Los Alamitos, CA: IEEE Computer Society, 2016: 78-86. [13] DIXIT M, KWITT R, NIETHAMMER M, et al. AGA: Attribute-guided augmentation[C]//IEEE Conference on Computer Vision and Pattern Recognition. Los Alamitos, CA: IEEE Computer Society, 2017: 7455-7463. [14] CHEN Z T, FU Y W, ZHANG Y D, et al. Multi-level semantic feature augmentation for one-shot learning[J]. IEEE Trans on Image Processing, 2019, 28(9): 4594-4605. doi: 10.1109/TIP.2019.2910052 [15] 汪航, 陈晓, 田晟兆, 等. 基于小样本学习的SAR 图像识别[J]. 计算机科学, 2020, 47(5): 124-128. doi: 10.11896/jsjkx.190400136 WANG H, CHEN X, TIAN S Z, et al. SAR image recognition based on few-shot learning[J]. Computer Science, 2020, 47(5): 124-128. doi: 10.11896/jsjkx.190400136 [16] LU J, LI J, YAN Z, et al. Attribute-based synthetic network (ABS-Net): Learning more from pseudo feature representations[J]. Pattern Recognition, 2018, 80: 129-142. doi: 10.1016/j.patcog.2018.03.006 [17] HARIHARAN B, GIRSHICK R. Low-shot visual recognition by shrinking and hallucinating features[C]//IEEE International Conference on Computer Vision. Los Alamitos, CA: IEEE Computer Society, 2017: 3018-3027. [18] SCHWARTZ E, KARLINSKY L, SHTOK J, et al. Delta-encoder: An effective sample synthesis method for few-shot object recognition[J]. Advances in Neural Information Processing Systems, 2018, 31: 2845-2855. [19] WANG Y X, GIRSHICK R, HEBERT M, et al. Low-shot learning from imaginary data[C]//IEEE Conference on Computer Vision and Pattern Recognition. Los Alamitos, CA: IEEE Computer Society, 2018: 7278-7286. [20] GAO H, SHOU Z, ZAREIAN A, et al. Low-shot learning via covariance-preserving adversarial augmentation networks[J]. Advances in Neural Information Processing Systems, 2018, 31: 975-985. [21] ANTONIOU A, STORKEY A, EDWARDS H. Data augmentation generative adversarial networks[EB/OL]. [2022-08-28]. https://arxiv.org/abs/1711.04340. [22] CHEN Z, FU Y W, WANG Y X, et al. Image deformation meta-networks for one-shot learning[C]//IEEE/CVF Conference on Computer Vision and Pattern Recognition. Los Alamitos, CA: IEEE Computer Society, 2019: 8680-8689. [23] LUO Q X, WANG L F, LV J G, et al. Few-shot learning via feature hallucination with variational inference[C]//IEEE Winter Conference on Applications of Computer Vision. Los Alamitos, CA: IEEE Computer Society, 2021: 3962-3971. [24] YANG S, WU S H, LIU T L, et al. Bridging the gap between few-shot and many-shot learning via distribution calibration[J]. IEEE Trans on Pattern Analysis and Machine Intelligence, 2022, 44(12): 9830-9843. doi: 10.1109/TPAMI.2021.3132021 [25] CHI Z Q, WANG Z, YANG M P, et al. Learning to capture the query distribution for few-shot learning[J]. IEEE Trans on Circuits and Systems for Video Technology, 2021, 32(7): 4163-4173. [26] LAZAROU M, STATHAKI T, AVRITHIS Y. Tensor feature hallucination for few-shot learning[C]//IEEE/CVF Winter Conference on Applications of Computer Vision. Piscataway, NJ: IEEE, 2022: 3500-3510. [27] TIAN S Z, CHEN D B. Attention based data augmentation for knowledge distillation with few data[C]//Journal of Physics: Conference Series. Bristol, England: IOP Publishing, 2022(2171): 012058. [28] SOHN K, BERTHELOT D, CARLINI N, et al. Fixmatch: Simplifying semi-supervised learning with consistency and confidence[J]. Advances in Neural Information Processing Systems, 2020, 33: 596-608. [29] CHEN T, KORNBLITH S, NOROUZI, et al. A simple framework for contrastive learning of visual representations[C]//International Conference on Machine Learning. New York: ACM, 2020: 1597-1607. [30] CHEN T, KORNBLITH S, SWERSKY K, et al. Big self-supervised models are strong semi-supervised learners[J]. Advances in Neural Information Processing Systems, 2020, 33: 22243-22255. [31] KOCH G, ZEMEL R, SALAKHUTDINOV R. Siamese neural networks for one-shot image recognition[C]//International Conference on Machine Learning. New York: ACM, 2015(2): 1-30. [32] YE M, GUO Y H. Deep triplet ranking networks for one-shot recognition[EB/OL]. [2022-09-15]. https://doi.org/10.48550/arXiv.1804.07275. [33] SNELL J, SWERSKY K, ZEMEL R. Prototypical networks for few-shot learning[C]//Conference on Neural Information Processing Systems. Cambridge, MA: MIT Press, 2017: 4077-4087. [34] DVORNIK N, SCHMID C, MAIRAL J. Diversity with cooperation: Ensemble methods for few-shot classification[C]//IEEE/CVF International Conference on Computer Vision. Los Alamitos, CA: IEEE Computer Society, 2019: 3723-3731. [35] HAO F S, CHENG J, WANG L, et al. Instance-level embedding adaptation for few-shot learning[J]. IEEE Access, 2019, 7: 100501-100511. doi: 10.1109/ACCESS.2019.2906665 [36] HU P, SUN X M, SAENKO K, et al. Weakly-supervised compositional feature aggregation for few-shot recognition[EB/OL]. [2022-09-15]. https://doi.org/10.48550/arXiv.1906.04833. [37] ZHENG Y, WANG R G, YANG J, et al. Principal characteristic networks for few-shot learning[J]. Journal of Visual Communication and Image Representation, 2019, 59: 563-573. doi: 10.1016/j.jvcir.2019.02.006 [38] VINYALS O, BLUNDELL C, LILLICRAP T, et al. Matching networks for one shot learning[C]//Conference on Neural Information Processing Systems. Cambridge, MA: MIT Press, 2016: 3630-3638. [39] BARTUNOV S, VETROV D. Few-shot generative modelling with generative matching networks[C]//International Conference on Artificial Intelligence and Statistics. Brookline, MA: Microtome Publishing, 2018: 670-678. [40] CAI Q, PAN Y W, YAO T, et al. Memory matching networks for one-shot image recognition[C]//IEEE/CVF Conference on Computer Vision and Pattern Recognition. Los Alamitos, CA: IEEE Computer Society, 2018: 4080-4088. [41] ZHANG L L, LIU J, LUO M N, et al. Scheduled sampling for one-shot learning via matching network[J]. Pattern Recognition, 2019, 96: 106962. doi: 10.1016/j.patcog.2019.07.007 [42] SUNG F, YANG Y X, ZHANG L, et al. Learning to compare: Relation network for few-shot learning[C]//IEEE/CVF Conference on Computer Vision and Pattern Recognition. Los Alamitos, CA: IEEE Computer Society, 2018: 1199-1208. [43] HILLIARD N, PHILLIPS L, HOWLAND S, et al. Few-shot learning with metric-agnostic conditional embeddings[EB/OL]. [2022-09-15]. https://doi.org/10.48550/arXiv.1802.04376. [44] ZHANG X T, SUNG F, QIANG Y T, et al. RelationNet2: Deep comparison network for few-shot learning[C]//International Joint Conference on Neural Networks. Los Alamitos, CA: IEEE Computer Society, 2020: 1-8. [45] LI W B, XU J L, HUO J, et al. Distribution consistency based covariance metric networks for few-shot learning[C]//AAAI Conference on Artificial Intelligence. Palo Alto, CA: AAAI, 2019, 33(1): 8642-8649. [46] LI W B, WANG L, XU J L, et al. Revisiting local descriptor based image-to-class measure for few-shot learning[C]//IEEE/CVF Conference on Computer Vision and Pattern Recognition. Los Alamitos, CA: IEEE Computer Society, 2019: 7260-7268. [47] ZHANG H G, KONIUSZ P. Power normalizing second-order similarity network for few-shot learning[C]//IEEE Winter Conference on Applications of Computer Vision. Los Alamitos, CA: IEEE Computer Society, 2019: 1185-1193. [48] HUI B Y, ZHU P F, HU Q H, et al. Self-attention relation network for few-shot learning[C]//IEEE International Conference on Multimedia and Expo Workshops. Piscataway, NJ: IEEE, 2019: 198-203. [49] LI W B, WANG L, HUO J, et al. Asymmetric distribution measure for few-shot learning[C]//International Joint Conference on Artificial Intelligence. San Francisco, CA: Margan Kaufmann, 2021: 2957-2963. [50] DONG C Q, LI W B, HUO J, et al. Learning task-aware local representations for few-shot learning[C]//International Joint Conference on Artificial Intelligence. San Francisco, CA: Margan Kaufmann, 2021: 716-722. [51] 汪航, 田晟兆, 唐青, 等. 基于多尺度标签传播的小样本图像分类[J]. 计算机研究与发展, 2022, 59(7): 1486-1495. WANG H, TIAN S Z, TANG Q, et al. Few-shot image classification based on multi-scale label propagation[J]. Journal of Computer Research and Development, 2022, 59(7): 1486-1495. [52] SANTORO A, BARTUNOV S, BOTVINICK M, et al. Meta-learning with memory-augmented neural networks[C]//International Conference on Machine Learning. New York: ACM, 2016: 1842-1850. [53] SHYAM P, GUPTA S, DUKKIPATI A. Attentive recurrent comparators[C]//International Conference on Machine Learning. New York: ACM, 2017: 3173-3181. [54] MISHRA N, ROHANINEJAD M, CHEN X, et al. A simple neural attentive meta-learner[EB/OL]. [2022-09-15]. https://doi.org/10.48550/arXiv.1707.03141. [55] ZHANG L, ZUO L Y, DU Y J, et al. Learning to adapt with memory for probabilistic few-shot learning[J]. IEEE Trans on Circuits and Systems for Video Technology, 2021, 31(11): 4283-4292. doi: 10.1109/TCSVT.2021.3052785 [56] WANG H, CHEN D B. Few-shot image classification based on ensemble metric learning[C]//Journal of Physics: Conference Series. Bristol, England: IOP Publishing, 2022(2171): 012027. [57] DW G D A. DARPA’s explainable artificial intelligence program[J]. AI Magazine, 2019, 40(2): 44. doi: 10.1609/aimag.v40i2.2850 [58] GUNNING D, VORM E, WANG J Y, et al. DARPA’s explainable AI (XAI) program: A retrospective[J]. Applied AI Letters, 2021, 2(4): e61. doi: 10.1002/ail2.61 [59] CLANCEY W J, HOFFMAN R R. Methods and standards for research on explainable artificial intelligence: Lessons from intelligent tutoring systems[J]. Applied AI Letters, 2021, 2(4): e53. doi: 10.1002/ail2.53 [60] MURDOCH W J, SINGH C, KUMBIER K, et al. Definitions, methods, and applications in interpretable machine learning[J]. Proceedings of the National Academy of Sciences, 2019, 116(44): 22071-22080. doi: 10.1073/pnas.1900654116 [61] TJOA E, GUAN C. A survey on explainable artificial intelligence (XAI): Toward medical XAI[J]. IEEE Trans on Neural Networks and Learning Systems, 2020, 32(11): 4793-4813. [62] HOU B J, ZHOU Z H. Learning with interpretable structure from gated RNN[J]. IEEE Trans on Neural Networks and Learning Systems, 2020, 31(7): 2267-2279. [63] VILONE G, LONGO L. Explainable artificial intelligence: A systematic review[EB/OL]. [2022-09-15]. https://doi.org/10.48550/arXiv.2006.00093. [64] ISLAM S R, EBERLE W, GHAFOOR S K, et al. Explainable artificial intelligence approaches: A survey[EB/OL]. [2022-09-22]. https://doi.org/10.48550/arXiv.2101.09429. [65] BURKART N, HUBER M F. A survey on the explainability of supervised machine learning[J]. Journal of Artificial Intelligence Research, 2021, 70: 245-317. doi: 10.1613/jair.1.12228 [66] AKULA A R, WANG K Z, LIU C S, et al. CX-ToM: Counterfactual explanations with theory-of-mind for enhancing human trust in image recognition models[J]. Iscience, 2022, 25(1): 103581. doi: 10.1016/j.isci.2021.103581 [67] CHEN X, DUAN Y, HOUTHOOFT R, et al. InfoGAN: Interpretable representation learning by information maximizing generative adversarial nets[C]//Conference on Neural Information Processing Systems. Cambridge, MA: MIT Press, 2016: 2172-2180. [68] LIU Y, WEI F Y, SHAO J, et al. Exploring disentangled feature representation beyond face identification[C]//IEEE/CVF Conference on Computer Vision and Pattern Recognition. Los Alamitos, CA: IEEE Computer Society, 2018: 2080-2089. [69] HIGGINS I, MATTHEY L, PAL A, et al. Beta-VAE: Learning basic visual concepts with a constrained variational framework[C]//International Conference on Learning Representations. Ithaca, NY: [s.n.], 2016: 1-12. [70] SIDDHARTH N, PAIGE B, DESMAISON A, et al. Inducing interpretable representations with variational autoencoders[EB/OL]. [2022-09-15]. https://doi.org/10.48550/arXiv.1611.07492. [71] LI C X, XU K, ZHU J, et al. Triple generative adversarial nets[C]//Conference on Neural Information Processing Systems. Cambridge, MA: MIT Press, 2017: 4088-4098. [72] HU X L, ZHANG J W, LI J M, et al. Sparsity-regularized HMAX for visual recognition[J]. PLoS One, 2014, 9: e81813. doi: 10.1371/journal.pone.0081813 [73] GRUBER T R. Toward principles for the design of ontologies used for knowledge sharing?[J]. International Journal of Human-Computer Studies, 1995, 43(5-6): 907-928. doi: 10.1006/ijhc.1995.1081 [74] LIU J F, DONG Y, LIU Z X, et al. Applying ontology learning and multi-objective ant colony optimization method for focused crawling to meteorological disasters domain knowledge[J]. Expert Systems with Applications, 2022, 198: 116741. doi: 10.1016/j.eswa.2022.116741 [75] 尹亮, 袁飞, 谢文波, 等. 关联图谱的研究进展及面临的挑战[J]. 计算机科学, 2018, 45(S1): 1-10. YIN L, YUAN F, XIE W B, et al. Research progress and challenges on association graph[J]. Computer Science, 2018, 45(S1): 1-10. [76] 尹亮, 何明利, 谢文波, 等. 装备-标准知识图谱的过程建模研究[J]. 计算机科学, 2018, 45(S1): 502-505. YIN L, HE M L, XIE W B, et al. Process modeling on knowledge graph of equipment and standard[J]. Computer Science, 2018, 45(S1): 502-505. [77] 尹亮, 何明利, 谢文波, 等. 基于装备标准关联图谱的标准化管控建模[J]. 装甲兵工程学院学报, 2018, 32(1): 38-41. YIN L, HE M L, XIE W B, et al. Standardization management and control modeling based on correlative graph of equipment standards[J]. Journal of Academy of Armored Force Engineering, 2018, 32(1): 38-41. [78] 史力晨, 赵俊严, 谢文波, 等. 装备标准关联图谱建模与可视化应用研究[J]. 系统仿真学报, 2017, 29(S1): 39-44. SHI L C, ZHAO J Y, XIE W B, et al. Research on modeling and visualization of equipment-standard association graph[J]. Journal of System Simulation, 2017, 29(S1): 39-44. [79] YIN L, SHI L C, ZHAO J Y, et al. Heterogeneous information network model for equipment-standard system[J]. Physica A: Statistical Mechanics and its Applications, 2018, 490: 935-943. doi: 10.1016/j.physa.2017.08.055 [80] YIN L, HE M L, XIE W B, et al. A quantitative model of universalization, serialization and modularization on equipment systems[J]. Physica A: Statistical Mechanics and its Applications, 2018, 508: 359-366. doi: 10.1016/j.physa.2018.05.120 [81] XIA C Y, ZHANG C W, YANG T, et al. Multi-grained named entity recognition[C]//Annual Meeting of the Association for Computational Linguistics. New York: ACM, 2019: 1430-1440. [82] HU A W, DOU Z C, NIE J Y, et al. Leveraging multi-token entities in document-level named entity recognition[C]//AAAI Conference on Artificial Intelligence. Palo Alto, CA: AAAI, 2020, 34(5): 7961-7968. [83] LI X Y, FENG J R, MENG Y X, et al. A unified MRC framework for named entity recognition[C]//Annual Meeting of the Association for Computational Linguistics. New York: ACM, 2020: 5849-5859. [84] SUN Y, WANG S H, LI Y K, et al. Ernie 2.0: A continual pre-training framework for language understanding[C]//AAAI Conference on Artificial Intelligence. Palo Alto, CA: AAAI, 2020, 34(5): 8968-8975. [85] QIU X, SUN T, XU Y, et al. Pre-trained models for natural language processing: A survey[J]. Science China Technological Sciences, 2020, 63(10): 1872-1897. [86] ZHANG Y H, QI P, MANNING C D. Graph convolution over pruned dependency trees improves relation extraction[C]//Conference on Empirical Methods in Natural Language Processing. Stroudsburg, PA: ACL, 2018: 2205-2215. [87] ZHAO Y, WAN H Y, GAO J W, et al. Improving relation classification by entity pair graph[C]//Asian Conference on Machine Learning. Cambridge, MA: MITPress, 2019: 1156-1171. [88] GUO Z J, ZHANG Y, LU W. Attention guided graph convolutional networks for relation extraction[C]//Annual Meeting of the Association for Computational Linguistics. Stroudsburg, PA: ACL, 2019: 241-251. [89] ZHANG N Y, DENG S M, SUN Z L, et al. Long-tail relation extraction via knowledge graph embeddings and graph convolution networks[C]//North American Chapter of the Association for Computational Linguistics. Stroudsburg, PA: ACL, 2019: 3016-3025. [90] LIU T Y, ZHANG X S, ZHOU W H, et al. Neural relation extraction via inner-sentence noise reduction and transfer learning[C]//Conference on Empirical Methods in Natural Language Processing. Stroudsburg, PA: ACL, 2018: 2195-2204. [91] LI Z G, CHEN H, QI R H, et al. DocR-BERT: document-level R-BERT for chemical-induced disease relation extraction via Gaussian probability distribution[J]. IEEE Journal of Biomedical and Health Informatics, 2022, 26(3): 1341-1352. doi: 10.1109/JBHI.2021.3116769 [92] GAO T Y, HAN X, LIU Z Y, et al. Hybrid attention-based prototypical networks for noisy few-shot relation classification[C]//AAAI Conference on Artificial Intelligence. Palo Alto, CA: AAAI, 2019, 33(1): 6407-6414. [93] QU M, GAO T Y, XHONNEUX L P, et al. Few-shot relation extraction via bayesian meta-learning on relation graphs[C]//International Conference on Machine Learning. New York: ACM, 2020: 7867-7876. [94] NGUYEN D Q, VERSPOOR K. End-to-end neural relation extraction using deep biaffine attention[C]//European Conference on Information Retrieval. Berlin: Springer, 2019: 729-738. [95] BEKOULIS G, DELEU J, DEMEESTER T, et al. Joint entity recognition and relation extraction as a multi-head selection problem[J]. Expert Systems with Applications, 2018, 114: 34-45. doi: 10.1016/j.eswa.2018.07.032 [96] LI X Y, YIN F, SUN Z J, et al. Entity-relation extraction as multi-turn question answering[C]//Annual Meeting of the Association for Computational Linguistics. Stroudsburg, PA: ACL, 2019: 1340-1350. [97] DING K, LIU S S, ZHANG Y H, et al. A knowledge-enriched and span-based network for joint entity and relation extraction[J]. CMC-Computers Materials & Continua, 2021, 68(1): 377-389. [98] XIONG W H, YU M, CHANG S Y, et al. One-shot relational learning for knowledge graphs[C]//Conference on Empirical Methods in Natural Language Processing. Stroudsburg, PA: ACL, 2018: 1980-1990. [99] LV X, GU Y X, HAN X, et al. Adapting meta knowledge graph information for multi-hop reasoning over few-shot relations[C]//Conference on Empirical Methods in Natural Language Processing. Stroudsburg, PA: ACL, 2019: 3376-3381. [100] CHEN M Y, ZHANG W, ZHANG W, et al. Meta relational learning for few-shot link prediction in knowledge graphs[C]//Conference on Empirical Methods in Natural Language Processing. Stroudsburg, PA: ACL, 2019: 4217-4226. [101] ZHANG C X, YAO H X, HUANG C, et al. Few-shot knowledge graph completion[C]//AAAI Conference on Artificial Intelligence. Palo Alto, CA: AAAI, 2020, 34(3): 3041-3048. [102] QIN P D, WANG X, CHEN W H, et al. Generative adversarial zero-shot relational learning for knowledge graphs[C]//AAAI Conference on Artificial Intelligence. Palo Alto, CA: AAAI, 2020, 34(5): 8673-8680. [103] BAEK J, LEE D B, HWANG S J. Learning to extrapolate knowledge: Transductive few-shot out-of-graph link prediction[C]//Conference on Neural Information Processing Systems. Cambridge, MA: MIT Press, 2020, 33: 546-560. [104] JI S X, PAN S R, CAMBRIA E, et al. A survey on knowledge graphs: Representation, acquisition, and applications[J]. IEEE Trans on Neural Networks and Learning Systems, 2021, 33(2): 494-514. [105] NAYYERI M, VAHDATI S, AYKUL C, et al. 5* knowledge graph embeddings with projective transformations[C]//AAAI Conference on Artificial Intelligence. Palo Alto, CA: AAAI, 2021, 35(10): 9064-9072. [106] CHEN M H, TIAN Y T, CHANG K W, et al. Co-training embeddings of knowledge graphs and entity descriptions for cross-lingual entity alignment[C]//International Joint Conference on Artificial Intelligence. San Francisco, CA: Margan Kaufmann, 2018: 3998-4004. [107] CHAMI I, WOLF A, JUAN D C, et al. Low-dimensional hyperbolic knowledge graph embeddings[C]//Annual Meeting of the Association for Computational Linguistics. Stroudsburg, PA: ACL, 2020: 6901-6914. [108] LU H N, HU H L, LIN X D. DensE: An enhanced non-commutative representation for knowledge graph embedding with adaptive semantic hierarchy[J]. Neurocomputing, 2022, 476: 115-125. doi: 10.1016/j.neucom.2021.12.079 [109] GUO J, KOK S. BiQUE: Biquaternionic embeddings of knowledge graphs[C]//Conference on Empirical Methods in Natural Language Processing. Stroudsburg, PA: ACL, 2021: 8338-8351. [110] SADEGHIAN A, ARMANDPOUR M, COLAS A, et al. ChronoR: Rotation based temporal knowledge graph embedding[C]//AAAI Conference on Artificial Intelligence. Palo Alto, CA: AAAI, 2021, 35(7): 6471-6479. [111] CAO Z S, XU Q Q, YANG Z Y, et al. Dual quaternion knowledge graph embeddings[C]//AAAI Conference on Artificial Intelligence. Palo Alto, CA: AAAI, 2021, 35(8): 6894-6902. [112] REN H Y, LESKOVEC J. Beta embeddings for multi-hop logical reasoning in knowledge graphs[C]//Conference on Neural Information Processing Systems. Cambridge, MA: MIT Press, 2020: 19716-19726. [113] WANG P F, HAN J L, LI C L, et al. Logic attention based neighborhood aggregation for inductive knowledge graph embedding[C]//AAAI Conference on Artificial Intelligence. Palo Alto, CA: AAAI, 2019, 33(1): 7152-7159. [114] LIU W J, ZHOU P, ZHAO Z, et al. K-bert: Enabling language representation with knowledge graph[C]//AAAI Conference on Artificial Intelligence. Palo Alto, CA: AAAI, 2020, 34(3): 2901-2908. [115] CHEN S X, LIU X D, GAO J F, et al. HittER: Hierarchical transformers for knowledge graph embeddings[C]//Conference on Empirical Methods in Natural Language Processing. Stroudsburg, PA: ACL, 2021: 10395-10407. [116] ZHANG Y, WANG J, YU L C, et al. MA-BERT: Learning representation by incorporating multi-attribute knowledge in transformers[C]//Annual Meeting of the Association for Computational Linguistics. Stroudsburg, PA: ACL, 2021: 2338-2343. [117] WANG P, XIE X, WANG X H, et al. Reasoning through memorization: Nearest neighbor knowledge graph embeddings[C]//International Conference on Natural Language Processing and Chinese Computing. Cham, Switzerland: Springer Nature, 2023: 111-122. [118] CHE F, ZHANG D, TAO J, et al. ParamE: Regarding neural network parameters as relation embeddings for knowledge graph completion[C]//AAAI Conference on Artificial Intelligence. Palo Alto, CA: AAAI, 2020, 34(3): 2774-2781. [119] YU E Y, FU Y, CHEN X, et al. Identifying critical nodes in temporal networks by network embedding[J]. Scientific Reports, 2020, 10(1): 1-8. doi: 10.1038/s41598-019-56847-4 [120] YU E Y, WANG Y P, FU Y, et al. Identifying critical nodes in complex networks via graph convolutional networks[J]. Knowledge-Based Systems, 2020, 198: 105893. doi: 10.1016/j.knosys.2020.105893 [121] MARINO K, SALAKHUTDINOV R, GUPTA A. The more you know: Using knowledge graphs for image classification[C]//IEEE Conference on Computer Vision and Pattern Recognition. Los Alamitos, CA: IEEE Computer Society, 2017: 2673-2681. [122] CHEN T S, LIN L, CHEN R Q, et al. Knowledge-embedded representation learning for fine-grained image recognition[C]//International Joint Conference on Artificial Intelligence. San Francisco, CA: Margan Kaufmann, 2018: 627-634. [123] XU H P, QI G L, LI J J, et al. Fine-grained image classification by visual-semantic embedding[C]//International Joint Conference on Artificial Intelligence. San Francisco, CA: Margan Kaufmann, 2018: 1043-1049. [124] JIANG C H, XU H, LIANG X D, et al. Hybrid knowledge routed modules for large-scale object detection[C]//Conference on Neural Information Processing Systems. Cambridge, MA: MIT Press, 2018: 1559-1570. [125] WANG X L, YE Y F, GUPTA A. Zero-shot recognition via semantic embeddings and knowledge graphs[C]//IEEE/CVF Conference on Computer Vision and Pattern Recognition. Los Alamitos, CA: IEEE Computer Society, 2018: 6857-6866. [126] WANG H, XU M H, NI B B, et al. Learning to combine: Knowledge aggregation for multi-source domain adaptation[C]//European Conference on Computer Vision. Berlin: Springer, 2020: 727-744. [127] GHOSH P, SAINI N, DAVIS L S, et al. All about knowledge graphs for actions[EB/OL]. [2022-09-15]. https://doi.org/10.48550/arXiv.2008.12432. [128] LUO R T, ZHANG N, HAN B Y, et al. Context-aware zero-shot recognition[C]//AAAI Conference on Artificial Intelligence. Palo Alto, CA: AAAI, 2020, 34(7): 11709-11716. [129] CHEN R Q, CHEN T S, HUI X L, et al. Knowledge graph transfer network for few-shot recognition[C]//AAAI Conference on Artificial Intelligence. Palo Alto, CA: AAAI, 2020, 34(7): 10575-10582. [130] CHEN T S, LIN L, HUI X L, et al. Knowledge-guided multi-label few-shot learning for general image recognition[J]. IEEE Trans on Pattern Analysis and Machine Intelligence, 2020, 44(3): 1371-1384. [131] HUANG H, CHEN Y W, TANG W, et al. Multi-label zero-shot classification by learning to transfer from external knowledge[EB/OL]. [2022-09-15]. https://doi.org/10.48550/arXiv.2007.15610. [132] PENG Z M, LI Z C, ZHANG J G, et al. Few-shot image recognition with knowledge transfer[C]//IEEE/CVF International Conference on Computer Vision. Los Alamitos, CA: IEEE Computer Society, 2019: 441-449. [133] KAMPFFMEYER M, CHEN Y B, LIANG X D, et al. Rethinking knowledge graph propagation for zero-shot learning[C]//IEEE/CVF Conference on Computer Vision and Pattern Recognition. Los Alamitos, CA: IEEE Computer Society, 2019: 11487-11496. [134] GARCIA V, BRUNA J. Few-shot learning with graph neural networks[EB/OL]. [2022-09-15]. https://doi.org/10.48550/arXiv.1711.04043. [135] YAO H X, ZHANG C X, WEI Y, et al. Graph few-shot learning via knowledge transfer[C]//AAAI Conference on Artificial Intelligence. Palo Alto, CA: AAAI, 2020, 34(4): 6656-6663. [136] LIU L, ZHOU T Y, LONG G D, et al. Attribute propagation network for graph zero-shot learning[C]//AAAI Conference on Artificial Intelligence. Palo Alto, CA: AAAI, 2020, 34(4): 4868-4875. [137] 张钹, 朱军, 苏航. 迈向第三代人工智能[J]. 中国科学: 信息科学, 2020, 50(9): 1281-1302. doi: 10.1360/SSI-2020-0204 ZHANG B, ZHU J, SU H. Toward the third generation of artificial intelligence[J]. Scientia Sinica Informationis, 2020, 50(9): 1281-1302. doi: 10.1360/SSI-2020-0204