Skip to main content
SpringerLink
Log in

Visualizing and Explaining Language Models

  • Chapter
  • First Online:
Integrating Artificial Intelligence and Visualization for Visual Knowledge Discovery

Part of the book series: Studies in Computational Intelligence ((SCI,volume 1014))

Abstract

During the last decade, Natural Language Processing has become, after Computer Vision, the second field of Artificial Intelligence that was massively changed by the advent of Deep Learning. Regardless of the architecture, the language models of the day need to be able to process or generate text, as well as predict missing words, sentences or relations depending on the task. Due to their black-box nature, such models are difficult to interpret and explain to third parties. Visualization is often the bridge that language model designers use to explain their work, as the coloring of the salient words and phrases, clustering or neuron activations can be used to quickly understand the underlying models. This paper showcases the techniques used in some of the most popular Deep Learning for NLP visualizations, with a special focus on interpretability and explainability.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 139.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 179.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 179.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    www.tableau.com.

  2. 2.

    Article [11] has garnered 149 citations at the moment of the submission, before being published in a conference or journal.

  3. 3.

    https://www.tensorflow.org/tensorboard.

  4. 4.

    https://neptune.ai/.

  5. 5.

    https://github.com/IDSIA/sacred.

  6. 6.

    https://www.comet.ml/site/.

  7. 7.

    https://www.wandb.com/.

  8. 8.

    Explainable points to the idea of describing or explaining in an intuitive manner, via charts or tables, the prediction of an algorithm.

  9. 9.

    https://github.com/marcoancona/DeepExplain.

References

  1. Abnar, S., Zuidema, W.H.: Quantifying attention flow in transformers. In: Jurafsky, D., et al. (eds.) Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, ACL 2020, Online, July 5–10, 2020, pp. 4190–4197. Association for Computational Linguistics (2020). ISBN: 978-1-952148-25-5

    Google Scholar 

  2. Agarwal, R., et al.: Neural additive models: interpretable machine learning with neural nets (2020). CoRR arXiv: abs/2004.13912

  3. Bostock, M., Ogievetsky, V., Heer, J.: D3 data-driven documents. IEEE Trans. Vis. Comput. Graph. 17(12), 2301–2309 (2011). https://doi.org/10.1109/TVCG.2011.185

    Article  Google Scholar 

  4. Brasoveanu, A., et al.: Framing named entity linking error types. In: Calzolari, N., et al. (eds.) Proceedings of the Eleventh International Conference on Language Resources and Evaluation, LREC 2018, Miyazaki, Japan, May 7–12, 2018. European Language Resources Association (ELRA) (2018)

    Google Scholar 

  5. Braşoveanu, A.M.P., Andonie, R.: Visualizing transformers for NLP: a brief survey. In: 2020 24th International Conference Information Visualisation (IV). IEEE. 2020, pp. 270–279. ISBN: 978-1-7281-9134-8. https://doi.org/10.1109/IV51561.2020

  6. Brown, T.B., et al.: Language models are few-shot learners. In: Larochelle, H., et al. (eds.) Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6–12, 2020, virtual (2020)

    Google Scholar 

  7. Cao, J., et al.: Behind the scene: revealing the secrets of pre-trained vision-and-language models. In: Vedaldi, A., et al. (eds.) Computer Vision—ECCV 2020—16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part VI. Lecture Notes in Computer Science, vol. 12351, pp. 565–580. Springer (2020). ISBN: 978-3-030-58538-9. https://doi.org/10.1007/978-3-030-58539-6_34

  8. Carlini, N., et al.: Extracting training data from large language models (2020). arXiv: 2012.07805

  9. Chen, S., et al.: Seeing things from a different angle: discovering diverse perspectives about claims. In: Burstein, J., Doran, C., Solorio, T. (eds.) Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2019, Minneapolis, MN, USA, June 2–7, 2019, Volume 1: Long and Short Papers, pp. 542–557. Association for Computational Linguistics (2019). ISBN: 978-1-950737-13-0. https://doi.org/10.18653/v1/n19-1053

  10. Chen, X., et al.: BadNL: backdoor attacks against NLP models (2020). arXiv: 2006.01043

  11. Clark, K., et al.: What does BERT look at? An analysis of BERT’s attention (2019). arXiv: 1906.04341

  12. Conneau, A., et al.: What you can cram into a single vector: probing sentence embeddings for linguistic properties. In: Gurevych, I., Miyao, Y. (eds.) Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, ACL 2018, Melbourne, Australia, July 15–20, 2018, Volume 1: Long Papers, pp. 2126–2136. Association for Computational Linguistics (2018). ISBN: 978-1-948087-32-2. https://doi.org/10.18653/v1/P18-1198

  13. DeRose, J.F., Wang, J., Berger, M.: Attention flows: analyzing and comparing attention mechanisms in language models. IEEE Trans. Vis. Comput. Graph. 27(2), 1160–1170 (2021). https://doi.org/10.1109/TVCG.2020.3028976

    Article  Google Scholar 

  14. Devlin, J., et al.: BERT: pre-training of deep bidirectional transformers for language understanding. In: Burstein, J., Doran, C., Solorio, T. (eds.) Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2019, Minneapolis, MN, USA, June 2–7, 2019, vol. 1 (Long and Short Papers), pp. 4171–4186. Association for Computational Linguistics (2019). ISBN: 978-1-950737-13-0. https://doi.org/10.18653/v1/n19-1423

  15. Dufter, P., Schütze, H.: Identifying necessary elements for BERT’s multilinguality (2020). arXiv: 2005.00396

  16. Ebrahimi, J., et al.: HotFlip: white-box adversarial examples for text classification. In: Gurevych, I., Miyao, Y. (eds.) Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, ACL 2018, Melbourne, Australia, July 15–20, 2018, Volume 2: Short Papers, pp. 31–36. Association for Computational Linguistics (2018). ISBN: 978-1-948087-34-6. https://doi.org/10.18653/v1/P18-2006

  17. Eger, S., Daxenberger, J., Gurevych, I.: How to probe sentence embeddings in low-resource languages: on structural design choices for probing task evaluation. In: Fernández, R., Linzen, T. (eds.) Proceedings of the 24th Conference on Computational Natural Language Learning, CoNLL 2020, Online, November 19–20, 2020, pp. 108–118. Association for Computational Linguistics (2020). https://doi.org/10.18653/v1/2020.conll-1.8

  18. Ettinger, A., Elgohary, A., Resnik, P.: Probing for semantic evidence of composition by means of simple classification tasks. In: Proceedings of the 1st Workshop on Evaluating Vector-Space Representations for NLP, RepEval@ACL 2016, Berlin, Germany, August 2016. Association for Computational Linguistics, pp. 134–139 (2016). https://doi.org/10.18653/v1/W16-2524

  19. Fan, A., et al.: ELI5: long form question answering. In: Korhonen, A., Traum, D.R., Marquez, L. (eds.) Proceedings of the 57th Conference of the Association for Computational Linguistics, ACL 2019, Florence, Italy, July 28–August 2, 2019, Volume 1: Long Papers, pp. 3558–3567. Association for Computational Linguistics (2019). ISBN: 978-1-950737-48-2. https://doi.org/10.18653/v1/p19-1346

  20. Fankhauser, P., Knappen, J., Teich, E.: Exploring and visualizing variation in language resources. In: Calzolari, N., et al. (eds.) Proceedings of the Ninth International Conference on Language Resources and Evaluation, LREC 2014, Reykjavik, Iceland, May 26–31, 2014, pp. 4125–4128. European Language Resources Association (ELRA) (2014)

    Google Scholar 

  21. Fey, M., Eric Lenssen, J.: Fast graph representation learning with PyTorch geometric (2019). CoRR arXiv: abs/1903.02428

  22. Florea, A., Andonie, R.: Weighted random search for hyper-parameter optimization. Int. J. Comput. Commun. Control 14(2), 154-169 (2019). https://doi.org/10.15837/ijccc.2019.2.3514

  23. Gan, Z., et al.: Large-scale adversarial training for vision-and-language representation learning. In: Larochelle, H., et al. (eds.) Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6–12, 2020, virtual (2020)

    Google Scholar 

  24. Gao, T., et al.: FewRel 2.0: towards more challenging few-shot relation classification. In: Inui, K., et al. (eds.) Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, EMNLP-IJCNLP 2019, Hong Kong, China, November 3–7, 2019, pp. 6249–6254. Association for Computational Linguistics (2019). ISBN: 978-1-950737-90-1. https://doi.org/10.18653/v1/D19-1649

  25. Gauthier, J., et al.: SyntaxGym: an online platform for targeted evaluation of language models. In: Çelikyilmaz, A., Wen, T.-H. (eds.) Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics: System Demonstrations, ACL 2020, Online, July 5–10, 2020, pp. 70–76. Association for Computational Linguistics (2020). ISBN: 978-1-952148-04-0. https://doi.org/10.18653/v1/2020.acl-demos.10

  26. Gonen, H., Goldberg, Y.: Lipstick on a pig: debiasing methods cover up systematic gender biases in word embeddings but do not remove them. In: Axelrod, A., et al. (eds.) Proceedings of the 2019 Workshop on Widening NLP@ACL 2019, Florence, Italy, July 28, 2019, pp. 60–63. Association for Computational Linguistics (2019). ISBN: 978-1-950737-42-0

    Google Scholar 

  27. Guyon, I., Elisseeff, A.: An introduction to variable and feature selection. J. Mach. Learn. Res. 3, 1157–1182 (2003)

    MATH  Google Scholar 

  28. Han, K., et al.: Transformer in transformer (2021). CoRR arXiv: 2103.00112

  29. Han, X., et al.: OpenNRE: an open and extensible toolkit for neural relation extraction. In: Padó, S., Huang, R. (eds.) Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, EMNLP-IJCNLP 2019, Hong Kong, China, November 3–7, 2019—System Demonstrations pp. 169–174. Association for Computational Linguistics (2019). ISBN: 978-1-950737-92-5. https://doi.org/10.18653/v1/D19-3029

  30. Hao, Y., et al.: Self-attention attribution: interpreting information interactions inside transformer (2020). CoRR arXiv: 2004.11207

  31. Heer, J.: Agency plus automation: Designing artificial intelligence into interactive systems. Proc. Natl. Acad. Sci. USA 116(6), 1844–1850 (2019). https://doi.org/10.1073/pnas.1807184115

    Article  Google Scholar 

  32. Heinrich, J., Weiskopf, D.: Parallel coordinates for multidimensional data visualization: basic concepts. Comput. Sci. Eng. 17(3), 70–76 (2015). https://doi.org/10.1109/MCSE.2015.55

    Article  Google Scholar 

  33. Hewitt, J., Manning, C.D.: A structural probe for finding syntax in word representations. In: Burstein, J., Doran, C., Solorio, T. (eds.) Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2019, Minneapolis, MN, USA, June 2–7, 2019, Volume 1 (Long and Short Papers), pp. 4129–4138. Association for Computational Linguistics (2019). ISBN: 978-1-950737-13-0. https://doi.org/10.18653/v1/n19-1419

  34. Hohman, F., et al.: Visual analytics in deep learning: an interrogative survey for the next frontiers. IEEE Trans. Vis. Comput. Graph. 25(8), 2674–2693 (2019). https://doi.org/10.1109/TVCG.2018.2843369

    Article  Google Scholar 

  35. Hoover, B., Strobelt, H., Gehrmann, S.: ex BERT: a visual analysis tool to explore learned representations in transformer models. In: Celikyilmaz, A., Wen, T.-H. (eds.) Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics: System Demonstrations, ACL 2020, Online, July 5–10, 2020, pp. 187–196. Association for Computational Linguistics (2020). https://doi.org/10.18653/v1/2020.acl-demos.22

  36. Jain, S., Wallace, B.C.: Attention is not explanation. In: Burstein, J., Doran, C., Solorio, T. (eds.) Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2019, Minneapolis, MN, USA, June 2–7, 2019, Volume 1 (Long and Short Papers), pp. 3543–3556. Association for Computational Linguistics (2019). ISBN: 978-1-950737-13-0. https://doi.org/10.18653/v1/n19-1357

  37. Jin, D., et al.: Is BERT really robust? A strong baseline for natural language attack on text classification and entailment. In: The Thirty-Fourth AAAI Conference on Artificial Intelligence, AAAI 2020, The Thirty-Second Innovative Applications of Artificial Intelligence Conference, IAAI 2020, The Tenth AAAI Symposium on Educational Advances in Artificial Intelligence, EAAI 2020, New York, NY, USA, February 7–12, 2020, pp. 8018–8025. AAAI Press (2020). ISBN: 978-1-57735-823-7

    Google Scholar 

  38. Kahng, M., et al.: ActiVis: visual exploration of industry-scale deep neural network models. IEEE Trans. Vis. Comput. Graph. 24(1), 88–97 (2018). https://doi.org/10.1109/TVCG.2017.2744718

    Article  Google Scholar 

  39. Karimi, A., Rossi, L., Prati, A.: Adversarial training for aspect-based sentiment analysis with BERT. In: 25th International Conference on Pattern Recognition, ICPR 2020, Virtual Event/Milan, Italy, January 10–15, 2021, pp. 8797–8803. IEEE (2020). ISBN: 978-1-7281-8808-9. https://doi.org/10.1109/ICPR48806.2021.9412167

  40. Karpathy, A., Johnson, J., Li, F.-F.: Visualizing and understanding recurrent networks (2015). CoRR arXiv: 1506.02078

  41. Kessler, J.S.: Scattertext: a browser-based tool for visualizing how corpora differ. In: Bansal, M., Ji, H. (eds.) Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, ACL 2017, Vancouver, Canada, July 30–August 4, System Demonstrations, pp. 85–90. Association for Computational Linguistics (2017). https://doi.org/10.18653/v1/P17-4015

  42. Kim, W., Son, B., Kim, I.: ViLT: vision-and-language transformer without convolution or region supervision. In: Meila, M., Zhang, T. (eds.) Proceedings of the 38th International Conference on Machine Learning, ICML 2021, 18–24 July 2021, Virtual Event, vol. 139, pp. 5583–5594. Proceedings of Machine Learning Research. PMLR (2021). http://proceedings.mlr.press/v139/kim21k.html

  43. Kitaev, N., Kaiser, L., Levskaya, A.: Reformer: the efficient transformer. In: 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26–30, 2020. OpenReview.net (2020)

    Google Scholar 

  44. Kobayashi, G., et al.: Attention module is not only a weight: analyzing transformers with vector norms. (2020). CoRR arXiv: 2004.10102

  45. Kohonen, T.: Self-organizing Maps. Springer Series in Information Sciences, vol. 30. Springer (1995). ISBN: 978-3-642-97612-4. https://doi.org/10.1007/978-3-642-97610-0

  46. Lakretz, Y., et al.: The emergence of number and syntax units in LSTM language models. In: Burstein, J., Doran, C., Solorio, T. (eds.) Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2019, Minneapolis, MN, USA, June 2–7, 2019, Volume 1 (Long and Short Papers), pp. 11–20. Association for Computational Linguistics (2019). ISBN: 978-1-950737-13-0. https://doi.org/10.18653/v1/n19-1002

  47. Lan, Z., et al.: ALBERT: A Lite BERT for self-supervised learning of language representations. In: 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26–30, 2020. OpenReview.net (2020)

    Google Scholar 

  48. Li, J., et al.: Visualizing and understanding neural models in NLP. In: Knight, K., Nenkova, A., Rambow, O. (eds.) NAACL HLT 2016, The 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, San Diego California, USA, June 12–17, 2016, pp. 681–691. The Association for Computational Linguistics (2016). https://doi.org/10.18653/v1/n16-1082

  49. Li, Y., Wang, H., Luo, Y.: A comparison of pre-trained vision-and-language models for multimodal representation learning across medical images and reports. In: Park, T., et al. (eds.) IEEE International Conference on Bioinformatics and Biomedicine, BIBM 2020, Virtual Event, South Korea, December 16–19, 2020, pp. 1999–2004. IEEE (2020). ISBN: 978-1-7281-6215-7. https://doi.org/10.1109/BIBM49941.2020.9313289

  50. Liaw, R., et al. (eds.): Tune: a research platform for distributed model selection and training (2018). CoRR arXiv: 1807.05118

  51. Liu, Y., et al.: RoBERTa: a Robustly Optimized BERT pretraining approach (2019). CoRR arXiv: 1907.11692

  52. Lundberg, S.M., et al.: Explainable machine-learning predictions for the prevention of hypoxaemia during surgery. Nat. Biomed. Eng. 2(10), 749–760 (2018)

    Article  Google Scholar 

  53. Lundberg, S.M., Lee, S.-I.: A unified approach to interpreting model predictions. In: Guyon, I., et al. (eds.) Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017(4–9), December 2017, pp. 4765–4774. Long Beach, CA, USA (2017)

    Google Scholar 

  54. Luo, H., et al.: Improving neural language models by segmenting, attending, and predicting the future. In: Korhonen, A., Traum, D.R., Marquez, L. (eds.) Proceedings of the 57th Conference of the Association for Computational Linguistics, ACL 2019, Florence, Italy, July 28–August 2, 2019, Volume 1: Long Papers, pp. 1483–1493. Association for Computational Linguistics (2019). ISBN: 978-1-950737-48-2. https://doi.org/10.18653/v1/p19-1144

  55. Mandelbrot, B.: Fractal Geometry of Nature. Freeman, W. H (1977)

    Google Scholar 

  56. Maudslay, R.H., et al.: A tale of a probe and a parser (2020). CoRR abs/2005.01641

    Google Scholar 

  57. Ming, Y., et al.: Understanding hidden memories of recurrent neural networks. In: Fisher, B.D., Liu, S., Schreck, T. (eds.) 12th IEEE Conference on Visual Analytics Science and Technology, IEEE VAST 2017, Phoenix, AZ, USA, October 3–6, 2017, pp. 13–24. IEEE Computer Society (2017). https://doi.org/10.1109/VAST.2017.8585721

  58. Moritz, P., et al.: Ray: a distributed framework for emerging AI applications. In: Arpaci-Dusseau, A.C., Voelker, G. (eds.) 13th USENIX Symposium on Operating Systems Design and Implementation, OSDI 2018, Carlsbad, CA, USA, October 8–10, 2018, pp. 561–577. USENIX Association (2018)

    Google Scholar 

  59. Morris, J.X., et al.: TextAttack: a framework for adversarial attacks, data augmentation, and adversarial training in NLP. In: Liu, Q., Schlangen, D. (eds.) Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, EMNLP 2020—Demos, Online, November 16–20, 2020, pp. 119–126. Association for Computational Linguistics (2020). ISBN: 978-1-952148-62-0. https://doi.org/10.18653/v1/2020.emnlp-demos.16

  60. Nguyen, A., Yosinski, J., Clune, J.: Understanding neural networks via feature visualization: a survey. In: Samek, W., et al. (eds.) Explainable AI: Interpreting, Explaining and Visualizing Deep Learning. Lecture Notes in Computer Science, vol. 11700, pp. 55–76. Springer (2019). ISBN: 978-3-030-28953-9. https://doi.org/10.1007/978-3-030-28954-6_4

  61. Park, D., et al.: ConceptVector: text visual analytics via interactive lexicon building using word embedding. IEEE Trans. Vis. Comput. Graph. 24(1), 361–370 (2018). https://doi.org/10.1109/TVCG.2017.2744478

    Article  Google Scholar 

  62. Pilault, J., Park, J., Pal, C.J.: On the impressive performance of randomly weighted encoders in summarization tasks. In: CoRR abs/2002.09084 (2020).https://arxiv.org/abs/2002.09084

  63. Ponte, J.M., Bruce Croft, W.: A language modeling approach to information retrieval. In: Bruce Croft, W., et al. (eds.) SIGIR’98: Proceedings of the 21st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, August 24–28, 1998, Melbourne, Australia, pp. 275–281. ACM (1998). ISBN: 1-58113-015-5. https://doi.org/10.1145/290941.291008

  64. Qin, Z., et al.: How convolutional neural networks see the world-a survey of convolutional neural network visualization methods. Math. Found. Comput. 1(2), 149–180 (2018). https://doi.org/10.3934/mfc.2018008

    Article  Google Scholar 

  65. Raghu, M., Schmidt, E.: A survey of deep learning for scientific discovery (2020). CoRR abs/2003.11755

  66. Reif, E., et al.: Visualizing and measuring the geometry of BERT. In: Wallach, H.M., et al. (eds.) Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, NeurIPS 2019(8–14), December 2019, pp. 8592–8600. Canada, Vancouver, BC (2019)

    Google Scholar 

  67. Ribeiro, M.T., Singh, S., Guestrin, C.: “Why Should I Trust You?”: explaining the predictions of any classifier. In: Krishnapuram, B., et al. (eds.) Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, San Francisco, CA, USA, August 13–17, 2016, pp. 1135-1144. ACM (2016). ISBN: 978-1-4503-4232-2. https://doi.org/10.1145/2939672.2939778

  68. Samek, W., et al. (eds.): Explainable AI: Interpreting, Explaining and Visualizing Deep Learning. Lecture Notes in Computer Science, vol. 11700. Springer (2019). ISBN: 978-3-030-28953-9. https://doi.org/10.1007/978-3-030-28954-6

  69. Sanh, V., et al.: DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter. (2019). CoRR arXiv: 1910.01108

  70. Satyanarayan, A., et al.: Vega-Lite: a grammar of interactive graphics. IEEE Trans. Vis. Comput. Graph. 23(1), 341–350 (2017). https://doi.org/10.1109/TVCG.2016.2599030

    Article  Google Scholar 

  71. Sawatzky, L., Bergner, S., Popowich, F.: Visualizing RNN states with predictive semantic encodings. In: 30th IEEE Visualization Conference, IEEE VIS 2019—Short Papers, Vancouver, BC, Canada, October 20–25, 2019, pp. 156–160. IEEE (2019). ISBN: 978-1-7281-4941-7. https://doi.org/10.1109/VISUAL.2019.8933744

  72. Shwartz, V., Choi, Y.: Do neural language models overcome reporting bias? In: Scott, D., Bel, N., Zong, C. (eds.) Proceedings of the 28th International Conference on Computational Linguistics, COLING 2020, Barcelona, Spain (Online), December 8–13, 2020, pp. 6863–6870. International Committee on Computational Linguistics (2020). https://doi.org/10.18653/v1/2020.coling-main.605

  73. Skrlj, B., et al.: AttViz: online exploration of self-attention for transparent neural language modeling (2020). CoRR arXiv: 2005.05716

  74. Slack, D., et al.: Fooling LIME and SHAP: adversarial attacks on post hoc explanation methods. In: Markham, A.N., et al. (eds.) AIES’20: AAAI/ACM Conference on AI, Ethics, and Society, New York, NY, USA, February 7–8, 2020, pp. 180–186. ACM (2020). ISBN: 978-1-4503-7110-0. https://doi.org/10.1145/3375627.3375830

  75. Slack, D., et al.: How can we fool LIME and SHAP? Adversarial attacks on post hoc explanation methods (2019). CoRR arXiv: 1911.02508

  76. Song, Y., et al.: Utilizing BERT intermediate layers for aspect based sentiment analysis and natural language inference (2020). CoRR arXiv: 2002.04815

  77. Strobelt, H., et al.: LSTMVis: a tool for visual analysis of hidden state dynamics in recurrent neural networks. IEEE Trans. Vis. Comput. Graph. 24(1), 667–676 (2018). https://doi.org/10.1109/TVCG.2017.2744158

    Article  Google Scholar 

  78. Strubell, E., Ganesh, A., McCallum, A.: Energy and policy considerations for deep learning in NLP. In: Korhonen, A., Traum, D.R., Marquez, L. (eds.) Proceedings of the 57th Conference of the Association for Computational Linguistics, ACL 2019, Florence, Italy, July 28–August 2, 2019, Volume 1: Long Papers, pp. 3645–3650. Association for Computational Linguistics (2019). https://doi.org/10.18653/v1/p19-1355

  79. Su, W., et al.: VL-BERT: pre-training of generic visual-linguistic representations. In: 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26–30, 2020. OpenReview.net (2020)

    Google Scholar 

  80. Sun, L., et al.: Adv-BERT: BERT is not robust on misspellings! Generating nature adversarial samples on BERT (2020). CoRR arXiv: 2003.04985

  81. Tenney, I., Das, D., Pavlick, E.: BERT rediscovers the classical NLP pipeline. In: Korhonen, A., Traum, D.R., Marquez, L. (eds.) Proceedings of the 57th Conference of the Association for Computational Linguistics, ACL 2019, Florence, Italy, July 28–August 2, 2019, Volume 1: Long Papers, pp. 4593–4601. Association for Computational Linguistics (2019). ISBN: 978-1-950737-48-2. https://doi.org/10.18653/v1/p19-1452

  82. van Aken, B., et al.: VisBERT: hidden-state visualizations for transformers. In: El Fallah Seghrouchni, A., et al. (eds.) Companion of the 2020 Web Conference 2020, Taipei, Taiwan, April 20–24, 2020, pp. 207–211. ACM (2020). ISBN: 978-1-4503-7024-0. https://doi.org/10.1145/3366424.3383542

  83. van der Heijden, N., Abnar, S., Shutova, E.: A comparison of architectures and pretraining methods for contextualized multilingual word embeddings. In: The Thirty-Fourth AAAI Conference on Artificial Intelligence, AAAI 2020, The Thirty-Second Innovative Applications of Artificial Intelligence Conference, IAAI 2020, The Tenth AAAI Symposium on Educational Advances in Artificial Intelligence, EAAI 2020, New York, NY, USA, February 7–12, 2020, pp. 9090–9097. AAAI Press (2020). ISBN: 978-1-57735-823-7

    Google Scholar 

  84. Vellido, A.: The importance of interpretability and visualization in machine learning for applications in medicine and health care. Neural Comput. Appl. 32(24), 18069–18083 (2020). https://doi.org/10.1007/s00521-019-04051-w

    Article  Google Scholar 

  85. Vaswani, A., et al.: Attention is all you need. In: Guyon, I., et al. (eds.) Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017(4–9), December 2017, pp. 5998–6008. Long Beach, CA, USA (2017)

    Google Scholar 

  86. Vig, J.: A multiscale visualization of attention in the transformer model. In: Costa-jussà, M.R., Enrique Alfonseca, M.R. (eds.) Proceedings of the 57th Conference of the Association for Computational Linguistics, ACL 2019, Florence, Italy, July 28–August 2, 2019, Volume 3: System Demonstrations, pp. 37–42. Association for Computational Linguistics (2019). ISBN: 978-1-950737-49-9. https://doi.org/10.18653/v1/p19-3007

  87. Vig, J.: Visualizing attention in transformer-based language representation models (2019). CoRR arXiv: 1904.02679

  88. Vig, J., et al.: BERTology meets biology: interpreting attention in protein language models (2020). CoRR arXiv: 2006.15222

  89. Vig, J., et al.: Causal mediation analysis for interpreting neural NLP: the case of gender bias (2020). CoRR arXiv: 2004.12265

  90. Voita, E., Sennrich, R., Titov, I.: The bottom-up evolution of representations in the transformer: a study with machine translation and language modeling objectives. In: Inui, K., et al. (eds.) Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, EMNLP-IJCNLP 2019, Hong Kong, China, November 3–7, 2019, pp. 4395–4405. Association for Computational Linguistics (2019). ISBN: 978-1-950737-90-1. https://doi.org/10.18653/v1/D19-1448

  91. Voita, E., Titov, I.: Information-theoretic probing with minimum description length. In: Webber, B. (eds.), Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, EMNLP 2020, Online, November 16–20, 2020. Association for Computational Linguistics, pp. 183–196 (2020). https://doi.org/10.18653/v1/2020.emnlp-main.14

  92. Voita, E., Titov, I.: Information-theoretic probing with minimum description length (2020). CoRR arXiv: 2003.12298

  93. Voita, E., et al.: Analyzing multi-head self-attention: specialized heads do the heavy lifting, the rest can be pruned. In: Korhonen, A., Traum, D.R., Marquez, L. (eds.) Proceedings of the 57th Conference of the Association for Computational Linguistics, ACL 2019, Florence, Italy, July 28–August 2, 2019, Volume 1: Long Papers, pp. 5797–5808 Association for Computational Linguistics (2019). ISBN: 978-1-950737-48-2. https://doi.org/10.18653/v1/p19-1580

  94. Wadden, D., et al.: Entity, relation, and event extraction with contextualized span representations. In: Inui, K., et al. (eds.) Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, EMNLP-IJCNLP 2019, Hong Kong, China, November 3–7, 2019, pp. 5783–5788. Association for Computational Linguistics (2019). ISBN: 978-1-950737-90-1. https://doi.org/10.18653/v1/D19-1585

  95. Wallace, E., et al.: AllenNLP interpret: a framework for explaining predictions of NLP models. In: Padó, S., Huang, R. (eds.) Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, EMNLP-IJCNLP 2019, Hong Kong, China, November 3–7, 2019—System Demonstrations, pp. 7–12. Association for Computational Linguistics (2019). ISBN: 978-1-950737-92-5. https://doi.org/10.18653/v1/D19-3002

  96. Wallace, E., et al.: Universal adversarial triggers for attacking and analyzing NLP. In: Inui, K., et al. (eds.) Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, EMNLP–IJCNLP 2019, Hong Kong, China, November 3–7, 2019, pp. 2153–2162. Association for Computational Linguistics (2019). ISBN: 978-1-950737-90-1. https://doi.org/10.18653/v1/D19-1221

  97. Wang, H., Leskovec, J.: Unifying graph convolutional neural networks and label propagation (2020). arXiv: 2002.06755

  98. Wang, J., et al.: Gradient-based analysis of NLP models is manipulable. In: Cohn, T., He, Y., Liu, Y. (eds.) Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: Findings, EMNLP 2020, Online Event, 16–20 November 2020, pp. 247–258. Association for Computational Linguistics (2020). ISBN: 978-1-952148-90-3. https://doi.org/10.18653/v1/2020.findings-emnlp.24

  99. Wiegreffe, S., Pinter, Y.: Attention is not explanation. In: Inui, K., et al. (eds.) Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, EMNLP–IJCNLP 2019, Hong Kong, China, November 3–7, 2019, pp. 11–20. Association for Computational Linguistics (2019). ISBN: 978-1-950737-90-1. https://doi.org/10.18653/v1/D19-1002

  100. Wilkinson, L.: The Grammar of Graphics. Statistics and Computing, 2nd edn. Springer (2005). ISBN: 978-0-387-24544-7

    Google Scholar 

  101. Wolf, T., et al.: HuggingFace transformers: state-of-the-art natural language processing (2019). CoRR arXiv: 1910.03771

  102. Wu, M., et al.: Unsupervised domain adaptive graph convolutional networks. In: Huang, Y., et al. (eds.) WWW’20: The Web Conference 2020, Taipei, Taiwan, April 20–24, 2020, pp. 1457–1467. ACM (2020). https://doi.org/10.1145/3366423.3380219

  103. Yang, Z., et al.: XLNet: generalized autoregressive pretraining for language understanding. In: Wallach, H.M., et al. (eds.) Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, NeurIPS 2019(8–14), December 2019, pp. 5754–5764. Canada, Vancouver, BC (2019)

    Google Scholar 

  104. Yun, Z., et al.: Transformer visualization via dictionary learning: contextualized embedding as a linear superposition of transformer factors (2021). CoRR arXiv: 2103.15949

  105. Zeng, G., et al.: OpenAttack: an open-source textual adversarial attack toolkit (2020). CoRR arXiv: 2009.09191

  106. Zhang, Q., Zhu, S.-C.: Visual interpretability for deep learning: a survey. Front. Inf. Technol. Electron. Eng. 19(1), 27–39 (2018). https://doi.org/10.1631/FITEE.1700808

    Article  Google Scholar 

  107. Zhang, Y., et al.: Every document owns its structure: inductive text classification via graph neural networks. In: Jurafsky, D., et al. (eds.) Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, ACL 2020, Online, July 5–10, 2020, pp. 334–339. Association for Computational Linguistics (2020). https://doi.org/10.18653/v1/2020.acl-main.31

  108. Zhong, M. et al.: A closer look at data bias in neural extractive summarization models. In: CoRR abs/1909.13705 (2019). http://arxiv.org/abs/1909.13705

Download references

Acknowledgements

The research presented in this paper has been partially conducted within the EPOCH and GENTIO projects funded by the Austrian Federal Ministry for Climate Action, Environment, Energy, Mobility and Technology (BMK) via the ICT of the Future Program (GA No. 867551 and 873992).

Author information

Authors and Affiliations

  1. Modul Technology GmbH, Am Kahlenberg 1, 1190, Vienna, Austria

    Adrian M. P. Braşoveanu

  2. Transylvania University of Brasov, Bulevardul Eroilor 29, Braşov, 500036, Romania

    Adrian M. P. Braşoveanu & Răzvan Andonie

  3. Central Washington University, 400 E University Way, Ellensburg, WA, 98926, USA

    Răzvan Andonie

Authors
  1. Adrian M. P. Braşoveanu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Răzvan Andonie
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Adrian M. P. Braşoveanu .

Editor information

Editors and Affiliations

  1. Department of Computer Science, Central Washington University, Ellensburg, WA, USA

    Boris Kovalerchuk

  2. Department of Media, Darmstadt University of Applied Sciences, Darmstadt, Hessen, Germany

    Kawa Nazemi

  3. Department of Computer Science, Central Washington University, Ellensburg, WA, USA

    Răzvan Andonie

  4. Department of Electronics, Telecommunications and Computers Engineering, Lisbon School of Engineering, Lisbon, Portugal

    Nuno Datia

  5. Department of Informatics, London South Bank University, London, UK

    Ebad Banissi

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Braşoveanu, A.M.P., Andonie, R. (2022). Visualizing and Explaining Language Models. In: Kovalerchuk, B., Nazemi, K., Andonie, R., Datia, N., Banissi, E. (eds) Integrating Artificial Intelligence and Visualization for Visual Knowledge Discovery. Studies in Computational Intelligence, vol 1014. Springer, Cham. https://doi.org/10.1007/978-3-030-93119-3_8

Download citation

Publish with us

Policies and ethics

Search

Navigation