Peiling Jiang
Jude Rayan
Haijun Xia
ABSTRACT
Large language models (LLMs) have recently soared in popularity due to their ease of access and the unprecedented ability to synthesize text responses to diverse user questions. However, LLMs like ChatGPT present significant limitations in supporting complex information tasks due to the insufficient affordances of the text-based medium and linear conversational structure. Through a formative study with ten participants, we found that LLM interfaces often present long-winded responses, making it difficult for people to quickly comprehend and interact flexibly with various pieces of information, particularly during more complex tasks. We present Graphologue, an interactive system that converts text-based responses from LLMs into graphical diagrams to facilitate information-seeking and question-answering tasks. Graphologue employs novel prompting strategies and interface designs to extract entities and relationships from LLM responses and constructs node-link diagrams in real-time. Further, users can interact with the diagrams to flexibly adjust the graphical presentation and to submit context-specific prompts to obtain more information. Utilizing diagrams, Graphologue enables graphical, non-linear dialogues between humans and LLMs, facilitating information exploration, organization, and comprehension.
Supplemental Material
Available for Download
zip
3606737.zip (108.9 MB)
Supplemental File
- Maneesh Agrawala, Wilmot Li, and Floraine Berthouzoz. 2011. Design principles for visual communication. Commun. ACM 54, 4 (2011), 60–69.Google Scholar
Digital Library
- Shaaron Ainsworth. 2006. DeFT: A conceptual framework for considering learning with multiple representations. Learning and Instruction 16, 3 (2006), 183–198. https://doi.org/10.1016/j.learninstruc.2006.03.001Google ScholarCross Ref
- Shaaron Ainsworth and Andrea Th Loizou. 2003. The effects of self-explaining when learning with text or diagrams. Cognitive science 27, 4 (2003), 669–681.Google Scholar
- Tareq Al-Moslmi, Marc Gallofré Ocaña, Andreas L. Opdahl, and Csaba Veres. 2020. Named Entity Extraction for Knowledge Graphs: A Literature Overview. IEEE Access 8 (2020), 32862–32881. https://doi.org/10.1109/ACCESS.2020.2973928Google ScholarCross Ref
- Razvan Azamfirei, Sapna R Kudchadkar, and James Fackler. 2023. Large language models and the perils of their hallucinations. Critical Care 27, 1 (2023), 1–2.Google ScholarCross Ref
- Sriram Karthik Badam, Zhicheng Liu, and Niklas Elmqvist. 2018. Elastic documents: Coupling text and tables through contextual visualizations for enhanced document reading. IEEE transactions on visualization and computer graphics 25, 1 (2018), 661–671.Google Scholar
- David Baidoo-Anu and Leticia Owusu Ansah. 2023. Education in the era of generative artificial intelligence (AI): Understanding the potential benefits of ChatGPT in promoting teaching and learning. Available at SSRN 4337484 (2023). http://dx.doi.org/10.2139/ssrn.4337484Google ScholarCross Ref
- Jeff Baker, Donald Jones, and Jim Burkman. 2009. Using visual representations of data to enhance sensemaking in data exploration tasks. Journal of the Association for Information Systems 10, 7 (2009), 2.Google ScholarCross Ref
- Benjamin B. Bederson and James D. Hollan. 1994. Pad++: A Zooming Graphical Interface for Exploring Alternate Interface Physics. In Proceedings of the 7th Annual ACM Symposium on User Interface Software and Technology (Marina del Rey, California, USA) (UIST ’94). Association for Computing Machinery, New York, NY, USA, 17–26. https://doi.org/10.1145/192426.192435Google ScholarDigital Library
- Richard A. Bolt. 1980. “Put-That-There”: Voice and Gesture at the Graphics Interface. In Proceedings of the 7th Annual Conference on Computer Graphics and Interactive Techniques (Seattle, Washington, USA) (SIGGRAPH ’80). Association for Computing Machinery, New York, NY, USA, 262–270. https://doi.org/10.1145/800250.807503Google ScholarDigital Library
- Zalán Borsos, Raphaël Marinier, Damien Vincent, Eugene Kharitonov, Olivier Pietquin, Matt Sharifi, Olivier Teboul, David Grangier, Marco Tagliasacchi, and Neil Zeghidour. 2022. AudioLM: a Language Modeling Approach to Audio Generation. arxiv:2209.03143 [cs.SD]Google Scholar
- Virginia Braun and Victoria Clarke. 2012. Thematic analysis.American Psychological Association.Google Scholar
- Peter Brusilovsky. 2001. Adaptive hypermedia. User modeling and user-adapted interaction 11 (2001), 87–110.Google Scholar
- Sébastien Bubeck, Varun Chandrasekaran, Ronen Eldan, Johannes Gehrke, Eric Horvitz, Ece Kamar, Peter Lee, Yin Tat Lee, Yuanzhi Li, Scott Lundberg, Harsha Nori, Hamid Palangi, Marco Tulio Ribeiro, and Yi Zhang. 2023. Sparks of Artificial General Intelligence: Early experiments with GPT-4. arxiv:2303.12712 [cs.CL]Google Scholar
- Alberto J. Cañas, Roger Carff, Greg Hill, Marco Carvalho, Marco Arguedas, Thomas C. Eskridge, James Lott, and Rodrigo Carvajal. 2005. Concept Maps: Integrating Knowledge and Information Visualization. Springer Berlin Heidelberg, Berlin, Heidelberg, 205–219. https://doi.org/10.1007/11510154_11Google ScholarCross Ref
- Stuart K. Card, Jock D. Mackinlay, and Ben Shneiderman (Eds.). 1999. Readings in Information Visualization: Using Vision to Think. Morgan Kaufmann Publishers Inc., San Francisco, CA, USA.Google ScholarDigital Library
- Angel Chang, Will Monroe, Manolis Savva, Christopher Potts, and Christopher D. Manning. 2015. Text to 3D Scene Generation with Rich Lexical Grounding. arxiv:1505.06289 [cs.CL]Google Scholar
- Huiwen Chang, Han Zhang, Jarred Barber, AJ Maschinot, Jose Lezama, Lu Jiang, Ming-Hsuan Yang, Kevin Murphy, William T. Freeman, Michael Rubinstein, Yuanzhen Li, and Dilip Krishnan. 2023. Muse: Text-To-Image Generation via Masked Generative Transformers. arxiv:2301.00704 [cs.CV]Google Scholar
- Kuo-En Chang, Yao-Ting Sung, and Ine-Dai Chen. 2002. The effect of concept mapping to enhance text comprehension and summarization. The Journal of Experimental Education 71, 1 (2002), 5–23.Google ScholarCross Ref
- Chaomei Chen. 2010. Information visualization. Wiley Interdisciplinary Reviews: Computational Statistics 2, 4 (2010), 387–403.Google ScholarDigital Library
- Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde de Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, Alex Ray, Raul Puri, Gretchen Krueger, Michael Petrov, Heidy Khlaaf, Girish Sastry, Pamela Mishkin, Brooke Chan, Scott Gray, Nick Ryder, Mikhail Pavlov, Alethea Power, Lukasz Kaiser, Mohammad Bavarian, Clemens Winter, Philippe Tillet, Felipe Petroski Such, Dave Cummings, Matthias Plappert, Fotios Chantzis, Elizabeth Barnes, Ariel Herbert-Voss, William Hebgen Guss, Alex Nichol, Alex Paino, Nikolas Tezak, Jie Tang, Igor Babuschkin, Suchir Balaji, Shantanu Jain, William Saunders, Christopher Hesse, Andrew N. Carr, Jan Leike, Josh Achiam, Vedant Misra, Evan Morikawa, Alec Radford, Matthew Knight, Miles Brundage, Mira Murati, Katie Mayer, Peter Welinder, Bob McGrew, Dario Amodei, Sam McCandlish, Ilya Sutskever, and Wojciech Zaremba. 2021. Evaluating Large Language Models Trained on Code. arxiv:2107.03374 [cs.LG]Google Scholar
- Zhutian Chen and Haijun Xia. 2022. CrossData: Leveraging Text-Data Connections for Authoring Data Documents. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems (New Orleans, LA, USA) (CHI ’22). Association for Computing Machinery, New York, NY, USA, Article 95, 15 pages. https://doi.org/10.1145/3491102.3517485Google ScholarDigital Library
- Peter C.-H. Cheng, Ric K. Lowe, and Mike Scaife. 2001. Cognitive Science Approaches To Understanding Diagrammatic Representations. Springer Netherlands, Dordrecht, 79–94. https://doi.org/10.1007/978-94-017-3524-7_5Google ScholarCross Ref
- Peggy Chi, Nathan Frey, Katrina Panovich, and Irfan Essa. 2021. Automatic Instructional Video Creation from a Markdown-Formatted Tutorial. In The 34th Annual ACM Symposium on User Interface Software and Technology (Virtual Event, USA) (UIST ’21). Association for Computing Machinery, New York, NY, USA, 677–690. https://doi.org/10.1145/3472749.3474778Google ScholarDigital Library
- John Joon Young Chung, Wooseok Kim, Kang Min Yoo, Hwaran Lee, Eytan Adar, and Minsuk Chang. 2022. TaleBrush: Visual Sketching of Story Generation with Pretrained Language Models. In Extended Abstracts of the 2022 CHI Conference on Human Factors in Computing Systems (New Orleans, LA, USA) (CHI EA ’22). Association for Computing Machinery, New York, NY, USA, Article 172, 4 pages. https://doi.org/10.1145/3491101.3519873Google ScholarDigital Library
- Antoine Clarinval, Isabelle Linden, Anne Wallemacq, and Bruno Dumas. 2018. Evoq: A Visualization Tool to Support Structural Analysis of Text Documents. In Proceedings of the ACM Symposium on Document Engineering 2018 (Halifax, NS, Canada) (DocEng ’18). Association for Computing Machinery, New York, NY, USA, Article 27, 10 pages. https://doi.org/10.1145/3209280.3209533Google ScholarDigital Library
- James M Clark and Allan Paivio. 1991. Dual coding theory and education. Educational psychology review 3 (1991), 149–210.Google Scholar
- Philip R. Cohen, Michael Johnston, David McGee, Sharon Oviatt, Jay Pittman, Ira Smith, Liang Chen, and Josh Clow. 1997. QuickSet: Multimodal Interaction for Distributed Applications. In Proceedings of the Fifth ACM International Conference on Multimedia (Seattle, Washington, USA) (MULTIMEDIA ’97). Association for Computing Machinery, New York, NY, USA, 31–40. https://doi.org/10.1145/266180.266328Google ScholarDigital Library
- Weiwei Cui, Xiaoyu Zhang, Yun Wang, He Huang, Bei Chen, Lei Fang, Haidong Zhang, Jian-Guan Lou, and Dongmei Zhang. 2019. Text-to-viz: Automatic generation of infographics from proportion-related natural language statements. IEEE transactions on visualization and computer graphics 26, 1 (2019), 906–916.Google Scholar
- Giulia Di Fede, Davide Rocchesso, Steven P. Dow, and Salvatore Andolina. 2022. The Idea Machine: LLM-Based Expansion, Rewriting, Combination, and Suggestion of Ideas. In Proceedings of the 14th Conference on Creativity and Cognition (Venice, Italy) (C&C ’22). Association for Computing Machinery, New York, NY, USA, 623–627. https://doi.org/10.1145/3527927.3535197Google ScholarDigital Library
- Patrick Esser, Johnathan Chiu, Parmida Atighehchian, Jonathan Granskog, and Anastasis Germanidis. 2023. Structure and Content-Guided Video Synthesis with Diffusion Models. arxiv:2302.03011 [cs.CV]Google Scholar
- Ethan Fast, Binbin Chen, Julia Mendelsohn, Jonathan Bassen, and Michael S. Bernstein. 2018. Iris: A Conversational Agent for Complex Tasks. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (Montreal QC, Canada) (CHI ’18). Association for Computing Machinery, New York, NY, USA, 1–12. https://doi.org/10.1145/3173574.3174047Google ScholarDigital Library
- Hendijanifard Fatemeh, Kardan Ahmad, and Dibay Moghadam Mohammad. 2011. ICMAP: An interactive tool for concept map generation to facilitate learning process. Procedia Computer Science 3 (2011), 524–529.Google ScholarCross Ref
- Tong Gao, Mira Dontcheva, Eytan Adar, Zhicheng Liu, and Karrie G. Karahalios. 2015. DataTone: Managing Ambiguity in Natural Language Interfaces for Data Visualization. In Proceedings of the 28th Annual ACM Symposium on User Interface Software & Technology (Charlotte, NC, USA) (UIST ’15). Association for Computing Machinery, New York, NY, USA, 489–500. https://doi.org/10.1145/2807442.2807478Google ScholarDigital Library
- Katy Ilonka Gero, Vivian Liu, and Lydia Chilton. 2022. Sparks: Inspiration for Science Writing Using Language Models. In Designing Interactive Systems Conference (Virtual Event, Australia) (DIS ’22). Association for Computing Machinery, New York, NY, USA, 1002–1019. https://doi.org/10.1145/3532106.3533533Google ScholarDigital Library
- Charles Goodwin. 2015. Professional Vision. Springer Fachmedien Wiesbaden, Wiesbaden, 387–425. https://doi.org/10.1007/978-3-531-19381-6_20Google ScholarCross Ref
- Stephen J Guastello, Mary Traut, and Gene Korienek. 1989. Verbal versus pictorial representations of objects in a human-computer interface. International journal of man-machine studies 31, 1 (1989), 99–120.Google ScholarDigital Library
- Jungpil Hahn and Jinwoo Kim. 1999. Why Are Some Diagrams Easier to Work with? Effects of Diagrammatic Representation on the Cognitive Intergration Process of Systems Analysis and Design. ACM Trans. Comput.-Hum. Interact. 6, 3 (sep 1999), 181–213. https://doi.org/10.1145/329693.329694Google ScholarDigital Library
- Tessai Hayama and Shuma Sato. 2020. Supporting Online Video e-Learning with Semi-automatic Concept-Map Generation. In Learning and Collaboration Technologies. Designing, Developing and Deploying Learning Experiences. Springer International Publishing, Cham, 64–76.Google Scholar
- Devamardeep Hayatpur, Daniel Wigdor, and Haijun Xia. 2023. CrossCode: Multi-Level Visualization of Program Execution. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems (Hamburg, Germany) (CHI ’23). Association for Computing Machinery, New York, NY, USA, Article 593, 13 pages. https://doi.org/10.1145/3544548.3581390Google ScholarDigital Library
- Marti Hearst and Melanie Tory. 2019. Would You Like A Chart With That? Incorporating Visualizations into Conversational Interfaces. In 2019 IEEE Visualization Conference (VIS). IEEE, 1–5. https://doi.org/10.1109/VISUAL.2019.8933766Google ScholarCross Ref
- Fangzhou Hong, Mingyuan Zhang, Liang Pan, Zhongang Cai, Lei Yang, and Ziwei Liu. 2022. AvatarCLIP: Zero-Shot Text-Driven Generation and Animation of 3D Avatars. arxiv:2205.08535 [cs.CV]Google Scholar
- Mengdie Hu, Krist Wongsuphasawat, and John Stasko. 2017. Visualizing Social Media Content with SentenTree. IEEE Transactions on Visualization and Computer Graphics 23, 1 (2017), 621–630. https://doi.org/10.1109/TVCG.2016.2598590Google ScholarDigital Library
- Gwo-Jen Hwang, Po-Han Wu, and Hui-Ru Ke. 2011. An interactive concept map approach to supporting mobile learning activities for natural science courses. Computers & education 57, 4 (2011), 2272–2280.Google Scholar
- Gwo-Jen Hwang, Li-Hsueh Yang, and Sheng-Yuan Wang. 2013. A concept map-embedded educational computer game for improving students’ learning performance in natural science courses. Computers & Education 69 (2013), 121–130.Google ScholarDigital Library
- Yu-Cin Jian and Chao-Jung Wu. 2015. Using eye tracking to investigate semantic and spatial representations of scientific diagrams during text-diagram integration. Journal of Science Education and Technology 24 (2015), 43–55.Google ScholarCross Ref
- Peiling Jiang, Fuling Sun, and Haijun Xia. 2023. Log-It: Supporting Programming with Interactive, Contextual, Structured, and Visual Logs. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems (Hamburg, Germany) (CHI ’23). Association for Computing Machinery, New York, NY, USA, Article 594, 16 pages. https://doi.org/10.1145/3544548.3581403Google ScholarDigital Library
- Dae Hyun Kim, Enamul Hoque, Juho Kim, and Maneesh Agrawala. 2018. Facilitating Document Reading by Linking Text and Tables. In Proceedings of the 31st Annual ACM Symposium on User Interface Software and Technology (Berlin, Germany) (UIST ’18). Association for Computing Machinery, New York, NY, USA, 423–434. https://doi.org/10.1145/3242587.3242617Google ScholarDigital Library
- Yea-Seul Kim, Mira Dontcheva, Eytan Adar, and Jessica Hullman. 2019. Vocal Shortcuts for Creative Experts. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (Glasgow, Scotland Uk) (CHI ’19). Association for Computing Machinery, New York, NY, USA, 1–14. https://doi.org/10.1145/3290605.3300562Google ScholarDigital Library
- Tiffany H Kung, Morgan Cheatham, Arielle Medenilla, Czarina Sillos, Lorie De Leon, Camille Elepaño, Maria Madriaga, Rimel Aggabao, Giezel Diaz-Candido, James Maningo, 2023. Performance of ChatGPT on USMLE: Potential for AI-assisted medical education using large language models. PLoS digital health 2, 2 (2023), e0000198.Google Scholar
- Philippe Laban, Tobias Schnabel, Paul Bennett, and Marti A. Hearst. 2021. Keep it Simple: Unsupervised Simplification of Multi-Paragraph Text. arxiv:2107.03444 [cs.CL]Google Scholar
- Jill H Larkin and Herbert A Simon. 1987. Why a diagram is (sometimes) worth ten thousand words. Cognitive science 11, 1 (1987), 65–100.Google Scholar
- Ching Liu, Juho Kim, and Hao-Chuan Wang. 2018. ConceptScape: Collaborative Concept Mapping for Video Learning. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (Montreal QC, Canada) (CHI ’18). Association for Computing Machinery, New York, NY, USA, 1–12. https://doi.org/10.1145/3173574.3173961Google ScholarDigital Library
- Michael Xieyang Liu, Andrew Kuznetsov, Yongsung Kim, Joseph Chee Chang, Aniket Kittur, and Brad A. Myers. 2022. Wigglite: Low-Cost Information Collection and Triage. In Proceedings of the 35th Annual ACM Symposium on User Interface Software and Technology (Bend, OR, USA) (UIST ’22). Association for Computing Machinery, New York, NY, USA, Article 32, 16 pages. https://doi.org/10.1145/3526113.3545661Google ScholarDigital Library
- Vivian Liu and Lydia B Chilton. 2022. Design Guidelines for Prompt Engineering Text-to-Image Generative Models. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems (New Orleans, LA, USA) (CHI ’22). Association for Computing Machinery, New York, NY, USA, Article 384, 23 pages. https://doi.org/10.1145/3491102.3501825Google ScholarDigital Library
- Stephen MacNeil, Andrew Tran, Juho Leinonen, Paul Denny, Joanne Kim, Arto Hellas, Seth Bernstein, and Sami Sarsa. 2023. Automatically Generating CS Learning Materials with Large Language Models. In Proceedings of the 54th ACM Technical Symposium on Computer Science Education V. 2 (Toronto ON, Canada) (SIGCSE 2023). Association for Computing Machinery, New York, NY, USA, 1176. https://doi.org/10.1145/3545947.3569630Google ScholarDigital Library
- J.H. McClellan, L.D. Harvel, R. Velmurugan, M. Borkar, and C. Scheibe. 2004. CNT: concept-map based navigation and discovery in a repository of learning content. In 34th Annual Frontiers in Education, 2004. FIE 2004.IEEE, F1F–13. https://doi.org/10.1109/FIE.2004.1408581Google ScholarCross Ref
- Ronald Metoyer, Qiyu Zhi, Bart Janczuk, and Walter Scheirer. 2018. Coupling Story to Visualization: Using Textual Analysis as a Bridge Between Data and Interpretation. In 23rd International Conference on Intelligent User Interfaces (Tokyo, Japan) (IUI ’18). Association for Computing Machinery, New York, NY, USA, 503–507. https://doi.org/10.1145/3172944.3173007Google ScholarDigital Library
- Priyanka More and Rashmi Phalnikar. 2012. Generating UML diagrams from natural language specifications. International Journal of Applied Information Systems 1, 8 (2012), 19–23.Google ScholarCross Ref
- Raphael Moura, Michael Beer, Edoardo Patelli, and John Lewis. 2017. Learning from major accidents: Graphical representation and analysis of multi-attribute events to enhance risk communication. Safety science 99 (2017), 58–70.Google Scholar
- Lluís Màrquez, Xavier Carreras, Kenneth C. Litkowski, and Suzanne Stevenson. 2008. Semantic Role Labeling: An Introduction to the Special Issue. Computational Linguistics 34, 2 (06 2008), 145–159. https://doi.org/10.1162/coli.2008.34.2.145 arXiv:https://direct.mit.edu/coli/article-pdf/34/2/145/1798596/coli.2008.34.2.145.pdfGoogle ScholarDigital Library
- Arpit Narechania, Arjun Srinivasan, and John Stasko. 2020. NL4DV: A toolkit for generating analytic specifications for data visualization from natural language queries. IEEE Transactions on Visualization and Computer Graphics 27, 2 (2020), 369–379.Google ScholarCross Ref
- Donald A. Norman and Stephen W. Draper. 1986. User Centered System Design; New Perspectives on Human-Computer Interaction. L. Erlbaum Associates Inc., USA.Google ScholarDigital Library
- OpenAI. 2023. GPT-4 Technical Report. arxiv:2303.08774 [cs.CL]Google Scholar
- Srishti Palani, Yingyi Zhou, Sheldon Zhu, and Steven P. Dow. 2022. InterWeave: Presenting Search Suggestions in Context Scaffolds Information Search and Synthesis. In Proceedings of the 35th Annual ACM Symposium on User Interface Software and Technology (Bend, OR, USA) (UIST ’22). Association for Computing Machinery, New York, NY, USA, Article 93, 16 pages. https://doi.org/10.1145/3526113.3545696Google ScholarDigital Library
- M. Palmer, D. Gildea, and N. Xue. 2011. Semantic Role Labeling. Morgan & Claypool Publishers. https://books.google.com/books?id=saBdAQAAQBAJGoogle Scholar
- Hammond Pearce, Benjamin Tan, Baleegh Ahmad, Ramesh Karri, and Brendan Dolan-Gavitt. 2022. Examining Zero-Shot Vulnerability Repair with Large Language Models. arxiv:2112.02125 [cs.CR]Google Scholar
- Jakub Piskorski and Roman Yangarber. 2013. Information Extraction: Past, Present and Future. Springer Berlin Heidelberg, Berlin, Heidelberg, 23–49. https://doi.org/10.1007/978-3-642-28569-1_2Google ScholarCross Ref
- Chengwei Qin, Aston Zhang, Zhuosheng Zhang, Jiaao Chen, Michihiro Yasunaga, and Diyi Yang. 2023. Is ChatGPT a General-Purpose Natural Language Processing Task Solver?arxiv:2302.06476 [cs.CL]Google Scholar
- Laria Reynolds and Kyle McDonell. 2021. Prompt Programming for Large Language Models: Beyond the Few-Shot Paradigm. arxiv:2102.07350 [cs.CL]Google Scholar
- Michail Schwab, Hendrik Strobelt, James Tompkin, Colin Fredericks, Connor Huff, Dana Higgins, Anton Strezhnev, Mayya Komisarchik, Gary King, and Hanspeter Pfister. 2017. booc.io: An Education System with Hierarchical Concept Maps and Dynamic Non-linear Learning Plans. IEEE Transactions on Visualization and Computer Graphics 23, 1 (2017), 571–580. https://doi.org/10.1109/TVCG.2016.2598518Google ScholarDigital Library
- Noah Shinn, Beck Labash, and Ashwin Gopinath. 2023. Reflexion: an autonomous agent with dynamic memory and self-reflection. arxiv:2303.11366 [cs.AI]Google Scholar
- Uriel Singer, Adam Polyak, Thomas Hayes, Xi Yin, Jie An, Songyang Zhang, Qiyuan Hu, Harry Yang, Oron Ashual, Oran Gafni, Devi Parikh, Sonal Gupta, and Yaniv Taigman. 2022. Make-A-Video: Text-to-Video Generation without Text-Video Data. arxiv:2209.14792 [cs.CV]Google Scholar
- Ishika Singh, Valts Blukis, Arsalan Mousavian, Ankit Goyal, Danfei Xu, Jonathan Tremblay, Dieter Fox, Jesse Thomason, and Animesh Garg. 2022. ProgPrompt: Generating Situated Robot Task Plans using Large Language Models. arxiv:2209.11302 [cs.RO]Google Scholar
- Anselm Spoerri. 1993. InfoCrystal: A Visual Tool for Information Retrieval & Management. In Proceedings of the Second International Conference on Information and Knowledge Management (Washington, D.C., USA) (CIKM ’93). Association for Computing Machinery, New York, NY, USA, 11–20. https://doi.org/10.1145/170088.170095Google ScholarDigital Library
- Chase Stokes and Marti Hearst. 2022. Why More Text is (Often) Better: Themes from Reader Preferences for Integration of Charts and Text. arxiv:2209.10789 [cs.HC]Google Scholar
- Chase Stokes, Vidya Setlur, Bridget Cogley, Arvind Satyanarayan, and Marti A. Hearst. 2023. Striking a Balance: Reader Takeaways and Preferences when Integrating Text and Charts. IEEE Transactions on Visualization and Computer Graphics 29, 1 (2023), 1233–1243. https://doi.org/10.1109/TVCG.2022.3209383Google ScholarCross Ref
- Tamara Sumner, Faisal Ahmad, Sonal Bhushan, Qianyi Gu, Francis Molina, Stedman Willard, Michael Wright, Lynne Davis, and Greg Janée. 2005. Linking learning goals and educational resources through interactive concept map visualizations. International Journal on Digital Libraries 5 (2005), 18–24.Google ScholarDigital Library
- Ivan E. Sutherland. 1964. Sketch Pad a Man-Machine Graphical Communication System. In Proceedings of the SHARE Design Automation Workshop(DAC ’64). Association for Computing Machinery, New York, NY, USA, 6.329–6.346. https://doi.org/10.1145/800265.810742Google ScholarDigital Library
- Chien-Lin Tang, Jingxian Liao, Hao-Chuan Wang, Ching-Ying Sung, and Wen-Chieh Lin. 2021. ConceptGuide: Supporting Online Video Learning with Concept Map-Based Recommendation of Learning Path. In Proceedings of the Web Conference 2021 (Ljubljana, Slovenia) (WWW ’21). Association for Computing Machinery, New York, NY, USA, 2757–2768. https://doi.org/10.1145/3442381.3449808Google ScholarDigital Library
- Barbara Tversky, Julie Bauer Morrison, and Mireille Betrancourt. 2002. Animation: can it facilitate?International journal of human-computer studies 57, 4 (2002), 247–262. https://doi.org/10.1006/ijhc.2002.1017Google ScholarDigital Library
- Priyan Vaithilingam, Tianyi Zhang, and Elena L. Glassman. 2022. Expectation vs. Experience: Evaluating the Usability of Code Generation Tools Powered by Large Language Models. In Extended Abstracts of the 2022 CHI Conference on Human Factors in Computing Systems (New Orleans, LA, USA) (CHI EA ’22). Association for Computing Machinery, New York, NY, USA, Article 332, 7 pages. https://doi.org/10.1145/3491101.3519665Google ScholarDigital Library
- Bret Victor. 2011. Up and Down the Ladder of Abstraction: A Systematic Approach to Interactive Visualization. http://worrydream.com/LadderOfAbstraction/Google Scholar
- Johannes Wheeldon. 2011. Is a Picture Worth a Thousand Words? Using Mind Maps to Facilitate Participant Recall in Qualitative Research.Qualitative Report 16, 2 (2011), 509–522.Google Scholar
- Terry Winograd 1972. Shrdlu: A system for dialog.Google Scholar
- Po-Han Wu, Gwo-Jen Hwang, Marcelo Milrad, Hui-Ru Ke, and Yueh-Min Huang. 2012. An innovative concept map approach for improving students’ learning performance with an instant feedback mechanism. British Journal of Educational Technology 43, 2 (2012), 217–232.Google ScholarCross Ref
- Tongshuang Wu, Ellen Jiang, Aaron Donsbach, Jeff Gray, Alejandra Molina, Michael Terry, and Carrie J Cai. 2022. PromptChainer: Chaining Large Language Model Prompts through Visual Programming. In Extended Abstracts of the 2022 CHI Conference on Human Factors in Computing Systems (New Orleans, LA, USA) (CHI EA ’22). Association for Computing Machinery, New York, NY, USA, Article 359, 10 pages. https://doi.org/10.1145/3491101.3519729Google ScholarDigital Library
- Tongshuang Wu, Michael Terry, and Carrie Jun Cai. 2022. AI Chains: Transparent and Controllable Human-AI Interaction by Chaining Large Language Model Prompts. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems (New Orleans, LA, USA) (CHI ’22). Association for Computing Machinery, New York, NY, USA, Article 385, 22 pages. https://doi.org/10.1145/3491102.3517582Google ScholarDigital Library
- Haijun Xia. 2020. Crosspower: Bridging Graphics and Linguistics. In Proceedings of the 33rd Annual ACM Symposium on User Interface Software and Technology (Virtual Event, USA) (UIST ’20). Association for Computing Machinery, New York, NY, USA, 722–734. https://doi.org/10.1145/3379337.3415845Google ScholarDigital Library
- Haijun Xia, Ken Hinckley, Michel Pahud, Xiao Tu, and Bill Buxton. 2017. WritLarge: Ink Unleashed by Unified Scope, Action, & Zoom. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (Denver, Colorado, USA) (CHI ’17). Association for Computing Machinery, New York, NY, USA, 3227–3240. https://doi.org/10.1145/3025453.3025664Google ScholarDigital Library
- Haijun Xia, Jennifer Jacobs, and Maneesh Agrawala. 2020. Crosscast: Adding Visuals to Audio Travel Podcasts. In Proceedings of the 33rd Annual ACM Symposium on User Interface Software and Technology (Virtual Event, USA) (UIST ’20). Association for Computing Machinery, New York, NY, USA, 735–746. https://doi.org/10.1145/3379337.3415882Google ScholarDigital Library
- Ann Yuan, Andy Coenen, Emily Reif, and Daphne Ippolito. 2022. Wordcraft: Story Writing With Large Language Models. In 27th International Conference on Intelligent User Interfaces (Helsinki, Finland) (IUI ’22). Association for Computing Machinery, New York, NY, USA, 841–852. https://doi.org/10.1145/3490099.3511105Google ScholarDigital Library
- Muru Zhang, Ofir Press, William Merrill, Alisa Liu, and Noah A. Smith. 2023. How Language Model Hallucinations Can Snowball. arxiv:2305.13534 [cs.CL]Google Scholar
- Kaiyang Zhou, Jingkang Yang, Chen Change Loy, and Ziwei Liu. 2022. Learning to prompt for vision-language models. International Journal of Computer Vision 130, 9 (2022), 2337–2348.Google ScholarDigital Library
Index Terms
- Graphologue: Exploring Large Language Model Responses with Interactive Diagrams
Recommendations
Trust by Interface: How Different User Interfaces Shape Human Trust in Health Information from Large Language Models
CHI EA '24: Extended Abstracts of the 2024 CHI Conference on Human Factors in Computing SystemsThe integration of Large Language Models (LLMs) with Conversational User Interfaces (CUIs) has significantly transformed health information seeking, offering interactive access to health resources. Despite the importance of trust in adopting health ...
Effective Non-visual Access to Diagrams via an Augmented Natural Language Interface
Computers Helping People with Special NeedsAbstractThis paper describes the design and validation of a number of HCI techniques that enable more effective non-visual access to diagrammatically displayed data through an adapted Natural Language Interface (NLI). These techniques have been ...
Towards seamless semantic zooming techniques for UML diagrams
SoftVis '08: Proceedings of the 4th ACM symposium on Software visualizationModels become increasingly important for software development processes. Though there is a multitude of software modeling tools available, the handling of complex UML diagrams is still difficult. In particular, the visualization of a global overview and ...
Comments