Abstract
Language models have been increasingly popular for automatic creativity assessment, generating semantic distances to objectively measure the quality of creative ideas. However, there is currently a lack of an automatic assessment system for evaluating creative ideas in the Chinese language. To address this gap, we developed TransDis, a scoring system using transformer-based language models, capable of providing valid originality (novelty) and flexibility (variety) scores for Alternative Uses Task (AUT) responses in Chinese. Study 1 demonstrated that the latent model-rated originality factor, comprised of three transformer-based models, strongly predicted human originality ratings, and the model-rated flexibility strongly correlated with human flexibility ratings as well. Criterion validity analyses indicated that model-rated originality and flexibility positively correlated to other creativity measures, demonstrating similar validity to human ratings. Study 2 and 3 showed that TransDis effectively distinguished participants instructed to provide creative vs. common uses (Study 2) and participants instructed to generate ideas in a flexible vs. persistent way (Study 3). Our findings suggest that TransDis can be a reliable and low-cost tool for measuring idea originality and flexibility in Chinese language, potentially paving the way for automatic creativity assessment in other languages. We offer an open platform to compute originality and flexibility for AUT responses in Chinese and over 50 other languages (https://osf.io/59jv2/).
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Notes
According to Beaty & Johnson (2021), raters should familiarize themselves with all facets of creativity, such as uncommonness, remoteness, and cleverness, before making judgments. It is important for them to quickly review all responses to discern commonness and uniqueness trends. The entire scale should be utilized to ensure an approximately normal distribution of scores. After completing the initial rating, revisions are encouraged to ensure accuracy.
Sung et al. (2022) constructed and validated a computerized creativity assessing system based on a figure association task, in which the originality and flexibility scores were calculated based on a Word2Vec language model. This scoring system indirectly generated the originality score by calculating the semantic distances between the noun extracted from the response and noncreative benchmark responses, which require word segmentation of the multi-word Chinese responses. Therefore, it cannot calculate the semantic distance between the response and the AUT prompt directly as most English AUT scoring systems do.
An averaged random intraclass correlation coefficient (ICC2k) using an absolute agreement definition was calculated. The two items with lower ICCs were later left out of the analysis.
Word2Vec fastText-chinese (Bojanowski et al., 2016): https://fasttext.cc/docs/en/pretrained-vectors.html.
BERT bert-base-Chinese (Devlin et al., 2018): https://huggingface.co/bert-base-chinese.
SBERT paraphrase-multilingual-mpnet-base-v2 (Reimers & Gurevych, 2020): https://huggingface.co/sentence-transformers/paraphrase-multilingual-mpnet-base-v2.
SBERT paraphrase-multilingual-MiniLM-L12-v2 (Reimers & Gurevych, 2020): https://huggingface.co/sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2.
SimCSE simcse-chinese-roberta-wwm-ext (Gao et al., 2021): https://huggingface.co/cyclone/simcse-chinese-roberta-wwm-ext.
The semantic distance calculated from the [CLS] embeddings of BERT showed nearly zero correlations with human ratings. Therefore, we only included mean-pooling strategy for BERT.
The results remained consistent when applying Top-2 scoring instead of Top-3 scoring (see Supplementary Table 2). SBERT_mpnet, SBERT_MiniLM, and SimCSE consistently exhibited optimized performance in scoring the originality of AUT bedsheet and AUT toothbrush.
A sensitivity analysis with different prior information is presented in Supplementary Table 1. The correlations between latent human-rated originality and latent model-rated originality remained positive.
As Supplementary Fig. 1 showed, the correlation between latent semantic distance originality and human-rated originality remained nearly the same when using top-2 scoring instead of top-3 scoring. r = .87, p < .001.
Word2Vec word count was determined after the removal of stop words.
References
Acar, S., Ogurlu, U., & Zorychta, A. (2022). Exploration of discriminant validity in divergent thinking tasks: A meta-analysis. Psychology of aesthetics, creativity, and the arts. Advance online publication. https://doi.org/10.1037/aca0000469
Acar, S., & Runco, M. A. (2014). Assessing associative distance among ideas elicited by tests of divergent thinking. Creativity Research Journal, 26(2), 229–238. https://doi.org/10.1080/10400419.2014.901095
Acar, S., Runco, M. A., & Park, H. (2020). What should people be told when they take a divergent thinking test? A meta-analytic review of explicit instructions for divergent thinking. Psychology of Aesthetics, Creativity, and the Arts, 14(1), 39–49. https://doi.org/10.1037/aca0000256
Amabile, T. M. (1983). The social psychology of creativity: A componential conceptualization. Journal of Personality and Social Psychology, 45(2), 357–376. https://doi.org/10.1037/0022-3514.45.2.357
Arthur, W., & Day, D. V. (1994). Development of a short form for the raven advanced progressive matrices test. Educational and Psychological Measurement, 54(2), 394–403. https://doi.org/10.1177/0013164494054002013
Barbot, B. (2018). The dynamics of creative ideation: Introducing a new assessment paradigm. Frontiers in Psychology, 9, 2529. https://doi.org/10.3389/fpsyg.2018.02529
Beaty, R. E., Johnson, D. R., Zeitlen, D. C., & Forthmann, B. (2022). Semantic distance and the alternate uses task: Recommendations for reliable automated assessment of originality. Creativity Research Journal, 34(3), 245–260. https://doi.org/10.1080/10400419.2022.2025720
Beaty, R. E., & Johnson, D. R. (2021). Automating creativity assessment with SemDis : An open platform for computing semantic distance. Behavior Research Methods, 53(2), 757–780. https://doi.org/10.3758/s13428-020-01453-w
Benedek, M., Franz, F., Heene, M., & Neubauer, A. C. (2012). Differential effects of cognitive inhibition and intelligence on creativity. Personality and Individual Differences, 53(4), 480–485. https://doi.org/10.1016/j.paid.2012.04.014
Benedek, M., Jauk, E., Sommer, M., Arendasy, M., & Neubauer, A. C. (2014). Intelligence, creativity, and cognitive control: The common and differential involvement of executive functions in intelligence and creativity. Intelligence, 46, 73–83. https://doi.org/10.1016/j.intell.2014.05.007
Benedek, M., Mühlmann, C., Jauk, E., & Neubauer, A. C. (2013). Assessment of divergent thinking by means of the subjective top-scoring method: Effects of the number of top-ideas and time-on-task on reliability and validity. Psychology of Aesthetics, Creativity, and the Arts, 7(4), 341–349. https://doi.org/10.1037/a0033644
Bessmertny, I. A., Huang, X., Platonov, A. V., Yu, C., & Koroleva, J. A. (2020). Applying the Bell’s test to Chinese texts. Entropy, 22(3), 275. https://doi.org/10.3390/e22030275
Bojanowski, P., Grave, E., Joulin, A., & Mikolov, T. (2016). Enriching word vectors with subword information. arXiv preprint. 10.48550/arXiv.1607.04606
Bowman, S., Angeli, G., Potts, C., & Manning, C. D. (2015). A large annotated corpus for learning natural language inference. In proceedings of the 2015 conference on empirical methods in natural language processing (pp. 632–642). 10.18653/v1/d15-1075.
Brown, T., Mann, B., Ryder, N., Subbiah, M., Kaplan, J. D., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., & Askell, A. (2020). Language models are few-shot learners. Advances in neural information processing systems, 33, 1877–1901. 10.48550/arXiv.2005.14165.
Chen, Y., Liang, Z., Zhao, Q., Huang, Y., Li, S., Yu, Q., & Zhou, Z. (2022). Semantic search during creative thinking: A quantitative analysis based on cumulative distribution and semantic similarity of responses. Acta Psychologica Sinica, 54(8), 881–891. https://doi.org/10.3724/SP.J.1041.2022.00881
Cseh, G. M., & Jeffries, K. K. (2019). A scattered CAT: A critical evaluation of the consensual assessment technique for creativity research. Psychology of Aesthetics, Creativity, and the Arts, 13(2), 159–166. https://doi.org/10.1037/aca0000220
Cui, Y., Che, W., Liu, T., Qin, B., & Yang, Z. (2021). Pre-training with whole word masking for Chinese BERT. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 29, 3504–3514. https://doi.org/10.1109/TASLP.2021.3124365
De Dreu, C. K. W., Nijstad, B. A., & Baas, M. (2011). Behavioral activation links to creativity because of increased cognitive flexibility. Social Psychological and Personality Science, 2(1), 72–80. https://doi.org/10.1177/1948550610381789
Devlin, J., Chang, M. W., Lee, K., & Toutanova, K. (2018). Bert: Pretraining of deep bidirectional transformers for language understanding. arXiv preprint. https://arxiv.org/abs/1810.04805
Dumas, D., & Dunbar, K. N. (2014). Understanding fluency and originality: A latent variable perspective. Thinking Skills and Creativity, 14, 56–67. https://doi.org/10.1016/j.tsc.2014.09.003
Dumas, D., Organisciak, P., & Doherty, M. (2021). Measuring divergent thinking originality with human raters and text-mining models: A psychometric comparison of methods. Psychology of Aesthetics, Creativity, and the Arts, 15(4), 645–663. https://doi.org/10.1037/aca0000319
Ethayarajh, K. (2019). How Contextual are Contextualized Word Representations? Comparing the Geometry of BERT, ELMo, and GPT-2 Embeddings. arXiv preprint. 10.48550/arXiv.1909.00512
Faul, F., Erdfelder, E., Lang, A.-G., & Buchner, A. (2007). G*power 3: A flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behavior Research Methods, 39(2), 175–191. https://doi.org/10.3758/BF03193146
Forster, E. A., & Dunbar, K. N. (2009). Creativity evaluation through latent semantic analysis. In N. A. Taatgen & H. van Rijn (Eds.), Proceedings of the 31st annual conference of the cognitive science society (pp. 602–607). Cognitive Science Society.
Forthmann, B., Bürkner, P.-C., Szardenings, C., Benedek, M., & Holling, H. (2019). A new perspective on the multidimensionality of divergent thinking tasks. Frontiers in Psychology, 10, Article 985. https://doi.org/10.3389/fpsyg.2019.00985
Forthmann, B., Holling, H., Zandi, N., Gerwig, A., Çelik, P., Storme, M., & Lubart, T. (2017). Missing creativity: The effect of cognitive workload on rater (dis-)agreement in subjective divergent-thinking scores. Thinking Skills and Creativity, 23, 129–139. https://doi.org/10.1016/j.tsc.2016.12.005
Forthmann, B., Paek, S. H., Dumas, D., Barbot, B., & Holling, H. (2020). Scrutinizing the basis of originality in divergent thinking tests: On the measurement precision of response propensity estimates. British Journal of Educational Psychology, 90(3), 683–699. https://doi.org/10.1111/bjep.12325
Frith, E., Elbich, D. B., Christensen, A. P., Rosenberg, M. D., Chen, Q., Kane, M. J., Silvia, P. J., Seli, P., & Beaty, R. E. (2021). Intelligence and creativity share a common cognitive and neural basis. Journal of Experimental Psychology: General, 150(4), 609–632. https://doi.org/10.1037/xge0000958
Gao, T., Yao, X., & Chen, D. (2021). SimCSE: Simple contrastive learning of sentence embeddings. arXiv preprint. 10.48550/arXiv.2104.08821
George, T., & Wiley, J. (2019). Fixation, flexibility, and forgetting during alternate uses tasks. Psychology of Aesthetics, Creativity, and the Arts, 13, 305–313. https://doi.org/10.1037/aca0000173
Golub, G. H., & Reinsch, C. (1971). Singular value decomposition and least squares solutions. Linear algebra, 2, 134–151.
Grajzel, K., Acar, S., Dumas, D., Organisciak, P., & Berthiaume, K. (2023a). Measuring flexibility: A text-mining approach. Frontiers in Psychology, 13, 1093343. https://doi.org/10.3389/fpsyg.2022.1093343
Grajzel, K., Acar, S., & Singer, G. (2023b). The big five and divergent thinking: A meta-analysis. Personality and Individual Differences, 214, 112338. https://doi.org/10.1016/j.paid.2023.112338
Gray, K., Anderson, S., Chen, E. E., Kelly, J. M., Christian, M. S., Patrick, J., Huang, L., Kenett, Y. N., & Lewis, K. (2019). “Forward flow”: A new measure to quantify free thought and predict creativity. American Psychologist, 74(5), 539–554. https://doi.org/10.1037/amp0000391
Guilford, J. P. (1950). Creativity. American Psychologist, 5(9), 444–454. https://doi.org/10.1037/h0063487
Guilford, J. P. (1967). The nature of human intelligence. McGraw-Hill.
Günther, F., Dudschig, C., & Kaup, B. (2015). LSAfun-an R package for computations based on latent semantic analysis. Behavior Research Methods, 47, 930–944. https://doi.org/10.3758/s13428-014-0529-0
Hao, N., Xue, H., Yuan, H., Wang, Q., & Runco, M. A. (2017). Enhancing creativity: Proper body posture meets proper emotion. Acta Psychologica, 173, 32–40. https://doi.org/10.1016/j.actpsy.2016.12.005
Harbison, J. I., & Haarmann, H. (2014). Automated scoring of originality using semantic representations. In P. Bello, M. Guarini, M. McShane, & B. Scassellati (Eds.), Proceedings of the 36th annual conference of the cognitive science society (CogSci 2014) (pp. 2327–2332). Cognitive Science Society.
Harris, Z. S. (1954). Distributional structure. Word, 10, 146–162. https://doi.org/10.1080/00437956.1954.11659520
Hass, R. W., Rivera, M., & Silvia, P. J. (2018). On the dependability and feasibility of layperson ratings of divergent thinking. Frontiers in Psychology, 9, article 1343. https://doi.org/10.3389/fpsyg.2018.01343.
Heinen, D. J. P., & Johnson, D. R. (2018). Semantic distance: An automated measure of creativity that is novel and appropriate. Psychology of Aesthetics, Creativity, and the Arts, 12(2), 144–156. https://doi.org/10.1037/aca0000125
Jauk, E., Benedek, M., & Neubauer, A. C. (2014). The road to creative achievement: A latent variable model of ability and personality predictors. European Journal of Personality, 28(1), 95–105. https://doi.org/10.1002/per.1941
John, O. P., Donahue, E. M., & Kentle, R. L. (1991). The big five inventory—Versions 4a and 54. University of California, Berkeley, Institute of Personality and Social Research.
John, O. P., Naumann, L. P., & Soto, C. J. (2008). Paradigm shift to the integrative big five trait taxonomy. In O. P. John, R. W. Robins, & L. A. Pervin (Eds.), Handbook of personality: Theory and research (pp. 114–158). Guilford Press.
Johnson, D. R., Cuthbert, A. S., & Tynan, M. E. (2021). The neglect of idea diversity in creative idea generation and evaluation. Psychology of Aesthetics, Creativity, and the Arts, 15(1), 125–135. https://doi.org/10.1037/aca0000235
Johnson, D. R., Kaufman, J. C., Baker, B. S., et al. (2022). Divergent semantic integration (DSI): Extracting creativity from narratives with distributional semantic modeling. Behavior Research Methods. https://doi.org/10.3758/s13428-022-01986-2
Kandler, C., Riemann, R., Angleitner, A., Spinath, F. M., Borkenau, P., & Penke, L. (2016). The nature of creativity: The roles of genetic factors, personality traits, cognitive abilities, and environmental sources. Journal of Personality and Social Psychology, 111(2), 230–249. https://doi.org/10.1037/pspp0000087
Karwowski, M. (2012). Did curiosity kill the cat? Relationship between trait curiosity, creative self-efficacy and creative personal identity. Europe’s Journal of Psychology, 8, 547–558. https://doi.org/10.5964/ejop.v8i4.513
Kaufman, J. C. (2012). Counting the muses: Development of the Kaufman domains of creativity scale (K-DOCS). Psychology of Aesthetics, Creativity, and the Arts, 6(4), 298–308. https://doi.org/10.1037/a0029751
Kenett, Y. N., & Faust, M. (2019). A semantic network cartography of the creative mind. Trends in Cognitive Sciences, 23(4), 271–274. https://doi.org/10.1016/j.tics.2019.01.007
Kenett, Y. N., Anaki, D., & Faust, M. (2014). Investigating the structure of semantic networks in low and high creative persons. Frontiers in Human Neuroscience, 8, 407. https://doi.org/10.3389/fnhum.2014.00407
Landauer, T. K., & Dumais, S. T. (1997). A solution to Plato's problem: The latent semantic analysis theory of acquisition, induction, and representation of knowledge. Psychological Review, 104(2), 211–240. https://doi.org/10.1037/0033-295X.104.2.211
Landauer, T. K., Foltz, P. W., & Laham, D. (1998). An introduction to latent semantic analysis. Discourse Processes, 25(2–3), 259–284. https://doi.org/10.1080/01638539809545028
MacQueen, J. (1967). Some methods for classification and analysis of multivariate observations. In L. M. Le Cam & J. Neyman (Eds.), Proceedings of the fifth Berkeley symposium on mathematical statistics and probability (pp. 281–297). University of California Press.
Mednick, S. (1962). The associative basis of the creative process. Psychological Review, 69(3), 220–232. https://doi.org/10.1037/h0048850
Mikolov, T., Chen, K., Corrado, G., & Dean, J. (2013). Efficient estimation of word representations in vector space. arXiv preprint. https://doi.org/10.48550/arXiv.1301.3781
Muthén, B., & Asparouhov, T. (2012). Bayesian structural equation modeling: A more flexible representation of substantive theory. Psychological Methods, 17(3), 313–335. https://doi.org/10.1037/a0026802
Neelakantan, A., Xu, T., Puri, R., Radford, A., Han, J. M., Tworek, J., Yuan, Q., Tezak, N., Kim, J. W., & Hallacy, C (2022). Text and code embeddings by contrastive pre-training. arXiv. https://doi.org/10.48550/arXiv.2201.10005
Nijstad, B. A., De Dreu, C. K. W., Rietzschel, E. F., & Baas, M. (2010). The dual pathway to creativity model: Creative ideation as a function of flexibility and persistence. European Review of Social Psychology, 21(1), 34–77. https://doi.org/10.1080/10463281003765323
Nijstad, B. A., & Stroebe, W. (2006). How the group affects the mind: A cognitive model of idea generation in groups. Personality and Social Psychology Review, 10(3), 186–213. https://doi.org/10.1207/s15327957pspr1003_1
Nusbaum, E. C., & Silvia, P. J. (2011). Are intelligence and creativity really so different? Fluid intelligence, executive processes, and strategy use in divergent thinking. Intelligence, 39(1), 36–45. https://doi.org/10.1016/j.intell.2010.11.002
Nusbaum, E. C., Silvia, P. J., & Beaty, R. E. (2014). Ready, set, create: What instructing people to “be creative” reveals about the meaning and mechanisms of divergent thinking. Psychology of Aesthetics, Creativity, and the Arts, 8(4), 423–432. https://doi.org/10.1037/a0036549
Olsona, J. A., Nahas, J., Chmoulevitch, D., Cropper, S. J., & Webb, M. E. (2021). Naming unrelated words predicts creativity. PNAS proceedings of the National Academy of Sciences of the United States of America, 118(25), article e2022340118.
Organisciak, P., Acar, S., Dumas, D., & Berthiaume, K. (2023). Beyond semantic distance: Automated scoring of divergent thinking greatly improves with large language models. Thinking Skills and Creativity, 101356. https://doi.org/10.1016/j.tsc.2023.101356
Pennington, J., Socher, R., & Manning, C. (2014). Glove: Global vectors for word representation. In A. Moscitti, A. Pang, & B. Daelemans (Eds.), Proceedings of the 2014 conference on empirical methods in natural language processing (pp. 1532–1543). Association for Computational Linguistics.
Plucker, J. A., & Makel, M. C. (2010). Assessment of creativity. In J. C. Kaufman & R. J. Sternberg (Eds.), The Cambridge handbook of creativity (pp. 48–73). Cambridge University Press.
Plucker, J. A., Qian, M., & Schmalensee, S. L. (2014). Is what you see what you really get? Comparison of scoring techniques in the assessment of real-world divergent thinking. Creativity Research Journal, 26(2), 135–143. https://doi.org/10.1080/10400419.2014.901023
Plucker, J. A., Qian, M., & Wang, S. (2011). Is originality in the eye of the beholder? Comparison of scoring techniques in the assessment of divergent thinking. The Journal of Creative Behavior, 45(1), 1–22. https://doi.org/10.1002/j.2162-6057.2011.tb01081.x
Portney, L. & Watkins, M. (1993). Foundations of clinical research: Applications to practice. Appleton and Lange, Connecticut, U.S.A.
Prabhakaran, R., Green, A. E., & Gray, J. R. (2014). Thin slices of creativity: Using single-word utterances to assess creative cognition. Behavior Research Methods, 46(3), 641–659. https://doi.org/10.3758/s13428-013-0401-7
Preiss, D. D. (2022). Metacognition, mind wandering, and cognitive flexibility: Understanding creativity. Journal of Intelligence, 10(3), 69. https://doi.org/10.3390/jintelligence10030069
Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., ... & Sutskever, I. (2021). Learning transferable visual models from natural language supervision. arXiv preprint. https://doi.org/10.48550/arXiv.2103.00020
Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., & Liu, P. J. (2020). Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of Machine Learning Research, 21(140), 1–67. https://doi.org/10.5555/3455716.3455856
Reimers, N., & Gurevych, I. (2019). Sentence-BERT: Sentence embeddings using siamese BERT-networks. arXiv preprint. https://doi.org/10.48550/arXiv.1908.10084
Reimers, N., & Gurevych, I. (2020). Making monolingual sentence embeddings multilingual using knowledge distillation. arXiv preprint. https://doi.org/10.48550/arXiv.2004.09813
Reiter-Palmon, R., Forthmann, B., & Barbot, B. (2019). Scoring divergent thinking tests: A review and systematic framework. Psychology of Aesthetics, Creativity, and the Arts, 13(2), 144–152. https://doi.org/10.1037/aca0000227
Rossmann, E., & Fink, A. (2010). Do creative people use shorter associative pathways? Personality and Individual Differences, 49(8), 891–895. https://doi.org/10.1016/j.paid.2010.07.025
Runco, M. A., & Okuda, S. M. (1991). The instructional enhancement of the flexibility and originality scores of divergent thinking tests. Applied Cognitive Psychology, 5(5), 435–441. https://doi.org/10.1002/acp.2350050505
Schroff, F., Kalenichenko, D., & Philbin, J. (2015). Facenet: A unified embedding for face recognition and clustering. arXiv preprint. https://doi.org/10.48550/arXiv.1503.03832.
Shaw, A. (2021). It works… but can we make it easier? A comparison of three subjective scoring indexes in the assessment of divergent thinking. Thinking Skills and Creativity, 40, 100789. https://doi.org/10.1016/j.tsc.2021.100789
Silvia, P. J., Martin, C., & Nusbaum, E. C. (2009). A snapshot of creativity: Evaluating a quick and simple method for assessing divergent thinking. Thinking Skills and Creativity, 4(2), 79–85. https://doi.org/10.1016/j.tsc.2009.06.005
Silvia, P. J., Winterstein, B. P., Willse, J. T., Barona, C. M., Cram, J. T., Hess, K. I., Martinez, J. L., & Richard, C. A. (2008). Assessing creativity with divergent thinking tasks: Exploring the reliability and validity of new subjective scoring methods. Psychology of Aesthetics, Creativity, and the Arts, 2(2), 68–85. https://doi.org/10.1037/1931-3896.2.2.68
Smid, S. C., McNeish, D., Miočević, M., & van de Schoot, R. (2020). Bayesian versus frequentist estimation for structural equation models in small sample contexts: A systematic review. Structural Equation Modeling, 27(1), 131–161. https://doi.org/10.1080/10705511.2019.1577140
Sung, Y.-T., Cheng, H.-H., Tseng, H.-C., Chang, K.-E., & Lin, S.-Y. (2022). Construction and validation of a computerized creativity assessment tool with automated scoring based on deep-learning techniques. Psychology of Aesthetics, Creativity, and the Arts. Advance online publication. https://doi.org/10.1037/aca0000450.
Torrance, E. P. (1972). Predictive validity of the Torrance tests of creative thinking. The Journal of Creative Behavior, 6(4), 236–252. https://doi.org/10.1002/j.2162-6057.1972.tb00936.x
Torrance, P. E. (1998). Torrance tests of creative thinking. Bensenville, IL: Scholastic Testing Services, Inc.
Traxler, M. (2011). Introduction to psycholinguistics: Understanding language science (p. 1934). New Jersey: Wiley-Blackwell.
Wang, H. (2009). Ciyi, cichang, cipin -- Xiandai Hanyu Cidian (diwuban) duoyici jiliang fenxi [Polysemous Words: Meaning, Length and Frequency]. Studies of the Chinese Language, 329(2), 120–130.
Williams, A., Nangia, N., & Bowman, S. (2018). A broad-coverage challenge corpus for sentence understanding through inference. In proceedings of the 2018 conference of the north American chapter of the Association for Computational Linguistics: Human language technologies, volume 1 (long papers) (pp. 1112–1122). https://doi.org/10.18653/v1/N18-1101.
Williams, S. D. (2004). Personality, attitude, and leader influences on divergent thinking and creativity in organizations. European Journal of Innovation Management, 7(3), 187–204. https://doi.org/10.1108/14601060410549883
Wilson, R. C., Guilford, J. P., & Christensen, P. R. (1953). The measurement of individual differences in originality. Psychological Bulletin, 50(5), 362–370. https://doi.org/10.1037/h0060857
Yang, T., & Wu, G. (2022). Spontaneous or deliberate: The dual influence of mind wandering on creative incubation. The Journal of Creative Behavior. https://doi.org/10.1002/jocb.553
Yu, C., Beckmann, J. F., & Birney, D. P. (2019). Cognitive flexibility as a meta-competency. Studies in Psychology, 40(3), 563–584. https://doi.org/10.1080/02109395.2019.1656463
Yu, Y., Beaty, R. E., Forthmann, B., Beeman, M., Cruz, J. H., & Johnson, D. (2023). A MAD method to assess idea novelty: Improving validity of automatic scoring using maximum associative distance (MAD). Psychology of aesthetics, creativity, and the arts. Advance online publication. https://doi.org/10.1037/aca0000573.
Zedelius, C. M., & Schooler, J. W. (2015). Mind wandering “Ahas” versus mindful reasoning: Alternative routes to creative solutions. Frontiers in Psychology, 6, Article 834. https://doi.org/10.3389/fpsyg.2015.00834
Author note
This research was supported by a grant from the Chinese National Natural Science Foundation (32271125) awarded to Yubo Hou. We have no conflicts of interest to disclose.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Supplementary information
ESM 1
(DOCX 152 kb)
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Yang, T., Zhang, Q., Sun, Z. et al. Automatic assessment of divergent thinking in Chinese language with TransDis: A transformer-based language model approach. Behav Res (2023). https://doi.org/10.3758/s13428-023-02313-z
Accepted:
Published:
DOI: https://doi.org/10.3758/s13428-023-02313-z