Skip to main content
SpringerLink
Log in

Across the disciplines: our gained knowledge in assessing a first-year integrated experience

  • Published:
International Journal of Technology and Design Education Aims and scope Submit manuscript

Abstract

This study discusses a writing assessment for State University’s first year integrated experience of Technology 120: Design Thinking in Technology and English 106: First-Year Writing and the implications it provides for curriculum design. The study used Adaptive Comparative Judgment to determine whether integrated research essays that utilized design thinking were more rhetorically effective (i.e., better attuned to addressing a purpose, audience, and context) than non-integrated English composition research essays. Results showed the non-integrated research essays had a statistically significant better rating than the integrated courses; this finding was counterintuitive to our expectations, therefore we utilized NVivo to conduct a word frequency analysis on judges’ holistic comments to determine strengths and weaknesses of the essays. The word frequency analysis demonstrated writing concepts were much stronger and more effective in a standard composition course. These findings led to the development of three literature-based curriculum and pedagogical changes to ensure that integrated students achieve general education writing outcomes in concert with design thinking principles—something our results showed as lacking significantly in these courses.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2

Similar content being viewed by others

Notes

  1. See Elliot (2005) for an extensive account of writing assessment history.

  2. Harvard University was one of the first universities to require an assessment of students’ writing abilities focusing largely on students’ spelling, punctuation, grammar, and expression correctness (Brereton 1995).

  3. Composition in response to open enrollment adopted Students’ Right to their Own Language in 1974.

  4. See Diederich et al. (1961) and Godshalk et al. (1981) in Elliot (2005).

  5. Bartholomew and Yoshikawa-Ruesch present an overview of current ACJ literature.

  6. Open-ended problems, or what Wardle (2013) calls ill-structured problems, are crucial to writing development and transfer.

References

  • Adler-Kassner, L., Majewski, J., & Koshnick, D. (2012). The value of troublesome knowledge: Transfer and threshold concepts in writing and history. In Composition forum, 26.

  • Adler-Kassner, L., & Wardle, E. A. (Eds.). (2015). Naming what we know: Threshold concepts of writing studies. Logan: Utah State University Press.

    Google Scholar 

  • Baniya, S., Mentzer, N., Bartholomew, S. R., Chesley, A., Moon, C., & Sherman, D. (2019). Using adaptive comparative judgment in writing assessment: An investigation of reliability among interdisciplinary evaluators. Journal of Technology Studies. https://doi.org/10.21061/jots.v45i1.a.3.

    Article  Google Scholar 

  • Bannerot, R., Kastor, R., & Ruchhoeft, P. (2010). Multidisciplinary capstone design at the University of Houston. Advances in Engineering Education, 2(1), 1–33.

    Google Scholar 

  • Bartholomae, D. (1985). Inventing the university. In M. Rose (Ed.), When a writer can’t write: Studies in writer’s block and other composing-process problem (pp. 134–165). New York: Guilford.

    Google Scholar 

  • Bartholomew, S. (2017). Assessing open-ended design problems. Technology and Engineering Teacher, 76(6), 13–17.

    Google Scholar 

  • Bartholomew, S., Strimel, G., & Jackson, A. (2018). A comparison of traditional and adaptive comparative judgment assessment techniques for freshman engineering design projects. International Journal of Engineering Education, 34(1), 20–33.

    Google Scholar 

  • Bartholomew, S., & Yoshikawa-Ruesch, E. (2018). A systematic review of research around adaptive comparative judgement (ACJ) in K-16 education. CTETE - Research Monograph Series, 1(1), 6–28. https://doi.org/10.21061/ctete-rms.v1.c.1.

    Article  Google Scholar 

  • Bazerman, C. (1992). Linguistic and rhetorical studies of writing in the disciplines. In B. R. Clark & G. R. Neave (Eds.), Encyclopedia of higher education (Vol. 3, pp. 1847–1852). New York: Pergamon Press.

    Google Scholar 

  • Bergmann, L. S., & Zepernick, J. (2007). Disciplinarity and transfer: Students’ perceptions of learning to write. Writing Program Administration, 31(1–2), 124–149.

    Google Scholar 

  • Berlin, J. (1988). Rhetoric and ideology in the writing class. College English, 50(5), 477. https://doi.org/10.2307/377477.

    Article  Google Scholar 

  • Bizzell, P. (1994). “Contact zones” and English studies. College English, 56(2), 163. https://doi.org/10.2307/378727.

    Article  Google Scholar 

  • Blythe, S., & Gonzales, L. (2016). Coordination and transfer across the metagenre of secondary research. College Composition and Communication, 67(4), 607–633.

    Google Scholar 

  • Brent, D. (2005). Reinventing WAC (again): The first-year seminar and academic literacy. College Composition and Communication, 57(2), 253–276.

    Google Scholar 

  • Brereton, J. C. (Ed.). (1995). The origins of composition studies in the American college, 1875–1925: A documentary history. Pittsburgh: University of Pittsburgh Press.

    Google Scholar 

  • Brizee, A., & Langmead, J. (2014). Cross-disciplinary collaboration: Fostering professional communication skills in a graduate accounting certificate program. Across the Disciplines, 11(1), 1–26.

    Article  Google Scholar 

  • Bruffee, K. A. (1984). Collaborative learning and the “conversation of mankind”. College English, 46(7), 635. https://doi.org/10.2307/376924.

    Article  Google Scholar 

  • Cottell, P. G., Hansen, S., & Ronald, K. (2000). From transparency toward expertise: Writing-Across-the-Curriculum as a site for new collaborations in organizational, faculty, and instructional development. To Improve the Academy, 18(1), 164–180.

    Article  Google Scholar 

  • Council of Writing Program Administrators, National Council of Teachers of English, & National Writing Project (2011). Framework for success in postsecondary writing. Retrieved from https://files.eric.ed.gov/fulltext/ED516360.pdf.

  • Cripps, M. J., & Robinson, H. M. (2016). “A way to talk about the institution as opposed to just my field”: WAC fellowships and graduate student professional development. Across the Disciplines, 13(3), 1–37.

    Article  Google Scholar 

  • Dearing, B. M., & Daugherty, M. K. (2004). Delivering engineering content in technology education. The Technology Teacher, 64(3), 8.

    Google Scholar 

  • Diederich, P. B., French, J. W., & Carlton, S. T. (1961). Factors in judgments of writing ability. ETS Research Bulletin Series, 1961(2), i–93. https://doi.org/10.1002/j.2333-8504.1961.tb00286.x.

    Article  Google Scholar 

  • Diefes-Dux, H. A., Moore, T., Zawojewski, J., Imbrie, P. K., & Follman, D. (2004). A framework for posing open-ended engineering problems: model-eliciting activities. In 34th annual frontiers in education, 2004. FIE 2004 (pp. 455–460). https://doi.org/10.1109/FIE.2004.1408556.

  • Downs, D., & Wardle, E. (2007). Teaching about writing, righting misconceptions: (Re)envisioning “First-Year Composition” as “Introduction to Writing Studies”. College Composition and Communication, 58(4), 552–584.

    Google Scholar 

  • Elliot, N. (2005). On a scale: A social history of writing assessment in America. New York: Peter Lang.

    Google Scholar 

  • Elon Research Seminar. (2015). Elon statement on writing transfer. Retrieved April 21, 2019, from http://cel.wpengine.com/wp-content/uploads/2014/09/Elon-Statement-Writing-Transfer-2015.pdf.

  • Emig, J. (1977). Writing as a mode of learning. College Composition and Communication, 28(2), 122. https://doi.org/10.2307/356095.

    Article  Google Scholar 

  • Engle, R. A., Lam, D. P., Meyer, X. S., & Nix, S. E. (2012). How does expansive framing promote transfer? Several proposed explanations and a research agenda for investigating them. Educational Psychologist, 47(3), 215–231. https://doi.org/10.1080/00461520.2012.695678.

    Article  Google Scholar 

  • Freeman, S., Eddy, S. L., McDonough, M., Smith, M. K., Okoroafor, N., Jordt, H., et al. (2014). Active learning increases student performance in science, engineering, and mathematics. Proceedings of the National Academy of Sciences, 111(23), 8410–8415.

    Article  Google Scholar 

  • Godshalk, F. I., Swineford, F., & Coffman, W. E. (1981). The measurement of writing ability.

  • Geisler, C. (1994). Academic literacy and the nature of expertise: Reading, writing, and knowing in academic philosophy. Lawrence Erlbaum Associates, Inc.

  • Gwet, K. L. (2014). Handbook of inter-rater reliability: The definitive guide to measuring the extent of agreement among raters; [a handbook for researchers, practitioners, teachers & students] (4th ed.). Gaithersburg: Advanced Analytics LLC.

    Google Scholar 

  • Halasz, J., Brickner, M., Gambs, D., Geraci, D., Queeley, A., & Soloyova, S. (2006). Making it your own: Writing fellows re-evaluate faculty “resistance”. Across the Disciplines, 3, 1–12.

    Article  Google Scholar 

  • Honey, M., Pearson, G., & Schweingruber, H. (Eds.). (2014). STEM integration in K-12 education: Status, prospects, and an agenda for research. Washington, DC: The National Academies Press.

    Google Scholar 

  • Huddleston, E. M. (1948). Recent studies of the English composition test. College Board Review, 1(4), 45+.

    Google Scholar 

  • Hughes, B., & Miller, E. L. (2018). WAC seminar participants as surrogate WAC consultants: Disciplinary faculty developing and deploying WAC expertise. The WAC Journal, 29, 7–41.

    Article  Google Scholar 

  • Journet, D. (1993). Interdisciplinary discourse and “boundary rhetoric”: The case of S. E. Jelliffe. Written Communication, 10(4), 510–541. https://doi.org/10.1177/0741088393010004002.

    Article  Google Scholar 

  • Kimbell, R. (2012). Evolving project e-scape for national assessment. International Journal of Technology and Design Education, 22(2), 135–155. https://doi.org/10.1007/s10798-011-9190-4.

    Article  Google Scholar 

  • Klein, J. T. (1993). Blurring, cracking, and crossing: Permeation and the fracturing of discipline. In E. Messer-Davidow, D. R. Shumway, & D. Sylvan (Eds.), Knowledges: Historical and critical studies in disciplinarity. Charlottesville: University Press of Virginia.

    Google Scholar 

  • Leverenz, C. (2014). Design thinking and the wicked problem of teaching writing. Computers and Composition, 33(2014), 1–12.

    Article  Google Scholar 

  • Liu, O. L., Frankel, L., & Roohr, K. C. (2014). Assessing critical thinking in higher education: Current state and directions for next-generation assessment. ETS Research Report Series, 2014(1), 1–23. https://doi.org/10.1002/ets2.12009.

    Article  Google Scholar 

  • Maimon, E. P. (1982). Writing across the curriculum: Past, present, and future. In New directions for teaching and learning: Teaching writing in all disciplines (pp. 67–73). San Francisco: Jossey-Bass.

  • Marback, R. (2009). Embracing wicked problem: The turn to design in composition studies. College Composition and Communication, 61(2), W397–W419.

    Google Scholar 

  • McLeod, S. (1975). The pedagogy of writing across the curriculum. In T. M. Zawacki & P. M. Rogers (Eds.), Writing across the curriculum: A critical sourcebook (pp. 53–68). Boston: Bedford/St. Martin’s.

    Google Scholar 

  • McLeod, S. H., & Maimon, E. P. (2000). Clearing the air: WAC myths and realities. College English, 62(5), 573–583.

    Article  Google Scholar 

  • Melzer, D. (2014). Assignments across the curriculum: A national study of college writing. Logan: Utah State University Press.

    Google Scholar 

  • Merriam-Webster. (n.d.). Genre. In Merriam-Webster.com dictionary. Retrieved April 15, 2020, from https://www.merriam-webster.com/dictionary/genre.

  • Meyer, J., & Land, R. (Eds.). (2006). Overcoming barriers to student understanding: Threshold concepts and troublesome knowledge. London: Routledge.

    Google Scholar 

  • Meyer, J., Land, R., & Baillie, C. (2010). Threshold concepts and transformational learning. Rotterdam: Sense Publishers.

    Book  Google Scholar 

  • Middendorf, J., & Pace, D. (2004). Decoding the disciplines: A model for helping students learn disciplinary ways of thinking. New Directions for Teaching and Learning, 2004(98), 1–12. https://doi.org/10.1002/tl.142.

    Article  Google Scholar 

  • Mullin, J. A. (2008). Interdisciplinary work as professional development: Changing the culture of teaching. Pedagogy, 8(3), 495–508.

    Article  Google Scholar 

  • Murray, D. M. (1972). Teach writing as a process not product. In V. Villanueva & K. L. Arola (Eds.), Cross-talk in comp theory: A reader (3rd ed., pp. 3–6). Urbana: National Council of Teachers of English.

    Google Scholar 

  • Myers, G. (1986). Reality, consensus, and reform in the rhetoric of composition teaching. College English, 48(2), 154. https://doi.org/10.2307/377298.

    Article  Google Scholar 

  • National Academy of Engineering, (NAE), & National Research Council, (NRC). (2014). STEM integration in K-12 education: Status, prospects, and an agenda for research. Washington, D.C.: National Academies Press.

    Google Scholar 

  • National Research Council, (NRC). (2009). Engineering in K-12 education: Understanding the status and improving the prospects. Washington, D.C.: National Academies Press.

    Google Scholar 

  • Neely, M. E. (2017). Faculty beliefs in successful writing fellow partnerships: How do faculty understand teaching, learning, and writing? Across the Disciplines, 14, 1–22.

    Article  Google Scholar 

  • Nowacek, R. S. (2011). Agents of integration: Understanding transfer as a rhetorical act. Carbondale: Southern Illinois University Press.

    Google Scholar 

  • Outcomes (n.d.). Retrieved April 21, 2019, from https://icap.rhetorike.org/outcomes/.

  • Paretti, M. C. (2008). Teaching communication in capstone design: The role of the instructor in situated learning. Journal of Engineering Education, 97(4), 491–503. https://doi.org/10.1002/j.2168-9830.2008.tb00995.x.

    Article  Google Scholar 

  • Parrish, J., Hesse, D., & Bateman, G. (2016). Assessing a writing intensive general education capstone: Research as faculty development. Across the Disciplines, 13(4), 1–22.

    Article  Google Scholar 

  • Perl, S. (1979). The composing processes of unskilled college writers. Research in the Teaching of English, 13(4), 317–336.

    Google Scholar 

  • Pollitt, A. (2012). The method of adaptive comparative judgement. Assessment in Education: Principles, Policy & Practice, 19(3), 281–300. https://doi.org/10.1080/0969594X.2012.665354.

    Article  Google Scholar 

  • Pollitt, A., & Murray, N. L. (1996). What raters really pay attention to. Studies in Language Testing, 3, 74–91.

    Google Scholar 

  • Potter, T., Englund, L., Charbonneau, J., MacLean, M. T., Newell, J., & Roll, I. (2017). ComPAIR: A new online tool using adaptive comparative judgement to support learning with peer feedback. Teaching & Learning Inquiry, 5(2), 89. https://doi.org/10.20343/teachlearninqu.5.2.8.

    Article  Google Scholar 

  • Purdy, J. (2014). What can design thinking offer writing studies? College Composition & Communication, 65(4), 612–641.

    Google Scholar 

  • Radermacher, A., Walia, G., & Knudson, D. (2014). Investigating the skill gap between graduating students and industry expectations. In Companion proceedings of the 36th international conference on software engineering—ICSE companion 2014, 291–300. https://doi.org/10.1145/2591062.2591159.

  • Rasch, G. (1993). Probabilistic models for some intelligence and attainment tests. Chicago: MESA Press.

    Google Scholar 

  • Report of the committee on evaluation of engineering education. (1994). Journal of Engineering Education, 83(1), 74–93.

  • Rounsaville, A. (2012). Selecting genres for transfer: The role of uptake in students’ antecedent genre knowledge. Composition Forum, 26.

  • Russell, D. R. (2002). Writing in the academic disciplines: A curricular history (2nd ed.). Carbondale: Southern Illinois University Press.

    Google Scholar 

  • Russell, D. R., & Yañez, A. (2003). “Big picture people rarely become historians”: Genre systems and the contradictions of general education. Writing Selves/Writing Societies, 331–362.

  • Rutz, C., & Grawe, N. D. (2009). Pairing WAC and quantitative reasoning through portfolio assessment and faculty development. Across the Disciplines, 6, 1–12.

    Article  Google Scholar 

  • Rutz, C., & Lauer-Glebov, J. (2005). Assessment and innovation: One darn thing leads to another. Assessing Writing, 10(2), 80–99. https://doi.org/10.1016/j.asw.2005.03.001.

    Article  Google Scholar 

  • Salem, L., & Jones, P. (2010). Undaunted, self-critical, and resentful: Investigating faculty attitude towards teaching writing in a large university writing-intensive course program. Writing Program Administration, 34(1), 60–83.

    Google Scholar 

  • Sanders, M. (2009). STEM, STEM education, STEM mania. The Technology Teacher, 68(4), 20–26.

    Google Scholar 

  • Sheils, M. (1975). Why Johnny can’t write. Newsweek, p. 58.

  • Sommers, N. (1980). Revision strategies of student writers and experienced adult writers. College Composition and Communication, 31(4), 378. https://doi.org/10.2307/356588.

    Article  Google Scholar 

  • Stroumbakis, K., Moh, N., & Kokkinos, D. (2016). Community college STEM faculty views on the value of writing assignments. WAC Journal, 27, 142–154.

    Article  Google Scholar 

  • The Common Ground Project: Articulating cross-disciplinary concepts and language for writing and speaking (n.d.). Retrieved November 23, 2018, from https://cwsp.ncsu.edu/resources/commongroundproject/.

  • Thurstone, L. L. (1927). A law of comparative judgment. Psychological Review, 34(4), 273–286. https://doi.org/10.1037/h0070288.

    Article  Google Scholar 

  • Trimbur, J. (1989). Consensus and difference in collaborative learning. College English, 51(6), 602. https://doi.org/10.2307/377955.

    Article  Google Scholar 

  • van Daal, T., Lesterhuis, M., Coertjens, L., Donche, V., & De Maeyer, S. (2019). Validity of comparative judgement to assess academic writing: Examining implications of its holistic character and building on a shared consensus. Assessment in Education: Principles, Policy & Practice, 26(1), 59–74. https://doi.org/10.1080/0969594X.2016.1253542.

    Article  Google Scholar 

  • WAC glossary of terms (n.d.). Retrieved April 21, 2019, from https://wac.appstate.edu/writing-disciplines/wac-glossary-terms.

  • Wardle, E. (2009). “Mutt genres” and the goal of FYC: Can we help students write the genres of the university? College Composition and Communication, 60(4), 765–789.

    Google Scholar 

  • Wardle, E. (2013). What is transfer? In R. Malenczyk (Ed.), A rhetoric for writing program administrators (pp. 143–155). Anderson: Parlor Press.

    Google Scholar 

  • Weiser, I. (1999). Portfolios in the teaching and assessing of writing. In L. Q. Troyka (Ed.), The Simon and Schuster Handbook for Writers (5th ed., pp. AIE 14–21). Prentice Hall.

  • Werner, C. L. (2013). Constructing student learning through faculty development: Writing experts, writing centers, and faculty resources. The CEA Forum, 79–92.

  • What is design thinking (n.d.). Retrieved April 21, 2019, from https://www.ideou.com/blogs/inspiration/what-is-design-thinking.

  • White, E. M. (1984). Holisticism. College Composition and Communication, 35(4), 400. https://doi.org/10.2307/357792.

    Article  Google Scholar 

  • Wicklein, R. C. (2006). Five good reasons for engineering as the focus for technology education. The Technology Teacher, 65(7), 25.

    Google Scholar 

  • Wolfe, J., Olson, B., & Wilder, L. (2014). Knowing what we know about writing in the disciplines: A new approach for teaching for transfer in FYC. WAC Journal, 25, 42–77.

    Article  Google Scholar 

  • Yancey, K. B. (1999). Looking back as we look forward: Historicizing writing assessment. College Composition and Communication, 50(3), 483. https://doi.org/10.2307/358862.

    Article  Google Scholar 

  • Yancey, K. B. (2012). Writing assessment in the early twenty-first century: A primer. In K. Ritter & P. K. Matsuda (Eds.), Exploring composition studies: Sites, issues, perspectives (pp. 167–187). Boulder: University Press of Colorado.

    Chapter  Google Scholar 

  • Yancey, K. B., Robertson, L., & Taczak, K. (2014). Writing across contexts: Transfer, composition, and sites of writing. Logan: Utah State University Press.

    Book  Google Scholar 

Download references

Funding

The research was not funded or supported.

Author information

Authors and Affiliations

  1. English Department, Purdue University, West Lafayette, IN, USA

    Derek Sherman

  2. Department of Technology Leadership and Innovation, Purdue University, West Lafayette, IN, USA

    Nathan Mentzer & Scott Bartholomew

  3. English Department, Embry-Riddle Aeronautical University, Prescott, AZ, USA

    Amelia Chesley

  4. Department of Computer and Information Technology, Purdue University, West Lafayette, IN, USA

    Dawn Laux

  5. English Department, Virginia Polytechnic Institute and State University, Blacksburg, VA, USA

    Sweta Baniya

Authors
  1. Derek Sherman
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Nathan Mentzer
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Scott Bartholomew
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Amelia Chesley
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Sweta Baniya
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Dawn Laux
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Derek Sherman.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Sherman, D., Mentzer, N., Bartholomew, S. et al. Across the disciplines: our gained knowledge in assessing a first-year integrated experience. Int J Technol Des Educ 32, 1369–1391 (2022). https://doi.org/10.1007/s10798-020-09650-6

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10798-020-09650-6

Keywords

Search

Navigation