Translation Difficulty: How to Measure and What to Measure

Reference

Anckaert, Ph., Eyckmans, J., Justens, D. and Segers, W. (2013): Bon sens, faux sens, contresens et non-sens sens dessus dessous: pour une évaluation fidèle et valide de la compétence de traduction. In J. Le Disez, W. Segers (Eds.). Le bon sens en traduction (pp. 79–93). Rennes: Presses Universitaires de Rennes.Search in Google Scholar

Ary, D., Jackobs, L. C., Sorenson, C. and Razavieh, A. (2010): Introduction to Research in Education (8^th edition). California: Wadsworth. Search in Google Scholar

Biber, D. (1989): A Typology of English Texts. Linguistics, 27 (1), 3–44.10.1515/ling.1989.27.1.3Search in Google Scholar

Blascovich, J. (2004): Psychophysiological Measures. In: Lewis-Beck, M. S. Bryman, A. and Liao, T.F. (eds) The Sage Encyclopedia of Social Science Research Methods. Thousand Oaks, Calif.: Sage. Vol. 1. 881–883.Search in Google Scholar

Bormuth, J. R. (1966): Readability: A New Approach. Reading Research Quarterly, 1, 79–132.10.2307/747021Search in Google Scholar

Bowker, L. (2001): Toward a Methodology for a Corpus Based Approach to Translation Evaluation. Meta, 46, 2.10.7202/002135arSearch in Google Scholar

Brenner, M., Doherty, E. T., and Shipp, T. (1994): Speech Measures Indicating Workload Demand. Aviation Space and Environmental Medicine, 65(1), 21–26.Search in Google Scholar

Bühler, K. (1984): Karl Bühler’s Theory of Language. Proceeding of the Conference Held at Kirchberg, Aug 26, John Benjamins. Search in Google Scholar

Campbell, S. and Hale, S. (2003): Translation and Interpreting Assessment in the Context of Educational Measurement. In: Anderman, G. M. and Rogers, M. (eds) Translation Today: Trends and Perspectives. Clevedon; Buffalo, N.Y.: Multilingual Matters, pp. 205–224.Search in Google Scholar

Ceschin, A. (2004): Memória de Tradução: Auxílio ou Empecilho? PhD Thesis. Pontifica Universidade Católica do rio de janeiro. 10.17771/PUCRio.acad.4974Search in Google Scholar

Coleman, M. and Liau, T. L. (1975): A Computer Readability Formula Designed for Machine Scoring. Journal of Applied Psychology, 60, 283–284.10.1037/h0076540Search in Google Scholar

Conde, T. (2011): Translation Evaluation on the Surface of Text: A Preliminary Analysis. The Journal of Specialized Translation, 15.Search in Google Scholar

Cruces, S. (2001): El Origen de los Errores en Traduccíon. Domingo Pujante Gonzalez et al. (eds.) (2001) Écrire, Traduire et Représenter la Fête. Valencia: Universitat de Valencia, 813–822. Search in Google Scholar

Dahl, Ö. (2004): The Growth and Maintenance of Linguistic Complexity. Amsterdam; Philadelphia: John Benjamins.10.1075/slcs.71Search in Google Scholar

Dale, E and Chall, J. (1948): A Formula for Predicting Readability. Educational Research Bulletin, 27, 11–28.Search in Google Scholar

Darwish, A. (2001): Transmetrics: A Formative Approach to Translator Competence Assessment and Translation Quality Evaluation for the New Millenium.Search in Google Scholar

De Waard, D. (1996): The Measurement of Drivers’ Mental Workload. University of Groningen: Groningen.Search in Google Scholar

Dragsted, B. (2004): Segmentation in Translation and Translation Memory Systems: An Empirical Investigation of Cognitive Segmentation and Effects of Integrating a TM-System into the Translation Process. Unpublished PhD dissertation, Copenhagen Business School.Search in Google Scholar

Ebel, R. (1979): Essentials of Educational Measurement. Englewood Cliffs, NJ : Prentice Hall. Search in Google Scholar

EMT Exert Group (2009): Competences for Professional Translators, Experts in Multilingual and Multimedia Communication, Retrieved October 1. 2013, from: http://ec.europa.eu/dgs/translation/programmes/emt/key_documents/emt_competences_translators_en.pdf.Search in Google Scholar

Ervin, S. and Bower, R. (1953): Translation Problems in International Surveys. Public Opinion Quarterly, 16 (4), 1952–53.10.1086/266421Search in Google Scholar

Eyckmans, J., Anckaert, Ph. and Segers, W. (2009): The Perks of Norm-referenced Translation Evaluation. In C. Angelelli, H. Jacobson (Eds.). Testing and assessment in translation and interpreting studies (pp. 73–93). Amsterdam: John Benjamins.10.1075/ata.xiv.06eycSearch in Google Scholar

Eyckmans, J., Anckaert, Ph. and Segers, W. (2013): Assessing Translation Competence. Actualizaciones en Comunicación Social Centro de Lingüística Applicada.Search in Google Scholar

Ferrando, J. P. (2010): Assessing the Discriminating Power of Item and Test Scores in the Linear Factor Analysis Model. Psicologica, 33, 111–134.Search in Google Scholar

Flesch, R. (1948): A New Readability Yardstick. Journal of Applied Psychology, 32 (3), 221–233.10.1037/h0057532Search in Google Scholar

Fry, E. B. (1989): Reading Formulas: Maligned but Valid. Journal of Reading, 32 (4). 292–297.Search in Google Scholar

Galley, N. (1993): The Evaluation of the Electrooculogram as a Psychophysiological Measuring Instrument in the Driver Study of Driver Behavior. Ergonomics, 36 (9), 1063–1070.10.1080/00140139308967978Search in Google Scholar

Garant, M. (2009): A Case for Holistic Translation Assessment. A Finlande Soveltavan Kielitieteen Tutkimuksia, 1, 5–17.Search in Google Scholar

Gopher, D. and Donchin, E. (1986): Workload: An Examination of the Concept. In: Boff, K. R., Kaufman, L. and Thomas, J. P. (eds) Handbook of Perception and Human Performance, Vol. II: Cognitive Processes and Performance. New York: Wiley.Search in Google Scholar

Gray, W. S. and Leary, B. E. (1935): What Makes a Book Readable. Chicago, Ill.,: The University of Chicago Press.Search in Google Scholar

Hajdú, P. (2002): “The New Hungarian Translation of Aristotle’s Poetics: When Translation and Commentary Disagree.” Across Languages and Cultures, 3 (2), 239–250.10.1556/Acr.3.2002.2.7Search in Google Scholar

Hankins, T. C., and Wilson, G. F. (1998): A Comparison of Heart Rate, Eye Activity, EEG and Subjective Measures of Pilot Mental Workload during Flight. Aviation Space and Environmental Medicine, 69 (4), 360–367.Search in Google Scholar

Hart, S. G. (2006): NASA-Task Load Index (NASA-TLX); 20 Years Later. Proceedings of the Human Factors and Ergonomics Society 50th Annual Meeting. Santa Monica: Human Factors and Ergonomics Society. 904–908.10.1037/e577632012-009Search in Google Scholar

Hart, S. G. and Staveland, L. E. (1988): Development of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research. In: Hancock, P. A. and Meshkati, N. (eds) Human Mental Workload. Amsterdam: New York: North-Holland. 139–183.10.1016/S0166-4115(08)62386-9Search in Google Scholar

Hatim, B. and Mason, I. (1997): The Translator as Communicator. London: Routledge.Search in Google Scholar

Hendy, K. C., Hamilton, K. M., and Landry, L. N. (1993): Measuring Subjective Workload – When Is One Scale Better Than Many. Human Factors, 35 (4), 579–601.10.1177/001872089303500401Search in Google Scholar

Hill, S. G., Lavecchia, H. P., Byers, J. C., Bittner, A. C., Zaklad, A. L., and Christ, R. E. (1992): Comparison of 4 Subjective Workload Rating-Scales. Human Factors, 34 (4), 429–439.10.1177/001872089203400405Search in Google Scholar

House, J. (2001): How do We Know when a Translation is Good? Erich Steiner and Colin Yallop (Eds) (2001). Exploring Translation and Multilingual Text Production: Beyond Content. Berlin/New York: Mouton de Gruyter, pp. 127–160.10.1515/9783110866193.127Search in Google Scholar

Hurtado, A. (2007): Comptence-based Curriculum Design for Training Translators, The Interpreter and Translator Trainer, 1, 163–195.10.1080/1750399X.2007.10798757Search in Google Scholar

Jensen, K. T. (2009): Indicators of Text Complexity. Copenhagen Studies in Language, 37, 61–80.Search in Google Scholar

Kade, O. (1968): Zufall und Gesetzmässigkeit in der Übersetzung, Leipzig. Search in Google Scholar

Kalyuga, S. (2009): Managing Cognitive Load in Adaptive Multimedia Learning. Hershey, PA:Information Science Reference.10.4018/978-1-60566-048-6Search in Google Scholar

Kamil, M. L., Pearson, P. D., Moje, E. B. and Afflerbach, P. P. (eds) (2011): Handbook of Reading Research, Volume IV. New York: Routledge.10.4324/9780203840412Search in Google Scholar

Karwowski, W. (ed.) (2006): International Encyclopedia of Ergonomics and Human Factors (2^nd ed.). Boca Raton, FL: CRC/Taylor and Francis.Search in Google Scholar

Kincaid, JP, Braby, R, and Mears, J. (1988): Electronic Authoring and Delivery of Technical Information. Journal of Instructional Development, 11, 8–13. doi:10.1007/bf02904998.10.1007/BF02904998Search in Google Scholar

Kockaert, H.J. and Segers, W. (2012): L’assurance Qualité des Traductions : Items Sélectionnés et Évaluation Assistée par Ordinateur. Meta : journal des traducteurs / Meta: Translators’ Journal, 57 (1), 159–176.10.7202/1012747arSearch in Google Scholar

Kockaert, H.J. and Segers, W. (2014): Evaluation de la Traduction : la Méthode PIE (Preselected Items Evaluation). Turjuman, 23 (2), 232–250.Search in Google Scholar

Kockaert, H.J. and Segers, W. (2016): Evaluation of Legal Translation: PIE Method (Preselected Items Evaluation). Journal of Specialized Translation, (forthcoming). Search in Google Scholar

Kockaert, H. J., Segers, W., Wylin, B., and Verbeke, D. (2016): TranslationQ: Automated Translation and Evaluation Process with Real-time Feedback. KU Leuven, Televic Education. Search in Google Scholar

Koo, S. L and Kinds, H. (2000): A Quality-Assurance Model for Language Projects. Robert C. Sprung (Ed.) (2000). Translation into Success. Cutting-edge strategies for going multilingual in a global age. Amsterdam: John Benjamins, pp. 147–157.10.1075/ata.xi.16kooSearch in Google Scholar

Koskinen, K. (2008): Translating Institution: An Ethnographic Study of EU Translation, Manchester: St Jerome Publication. Search in Google Scholar

Krings, H. P. (2001): Repairing Texts: Empirical Investigations of Machine Translation Postediting Processes (Koby, G., Shreve, G., Mischerikow, K. and Litzer, S. Trans.). Kent, Ohio: Kent State University Press.Search in Google Scholar

Larose, R. (1998): Méthodologie de L’évaluation des Traductions. Meta, 43 (2), 163–186.10.7202/003410arSearch in Google Scholar

Lasnier, F. (2000): Réussir la Formation par Compétences, Montreal: Guérin. Search in Google Scholar

Levine, R. and Lord, F.M. (1959): An Index of the Discriminating Power of a Test at Different Parts of the Score Range. Educational and Psychological Measurement, 19, 497–503.10.1177/001316445901900402Search in Google Scholar

Lord, F.M. (1980): Applications of Item Response Theory to Practical Testing Problems. Hillsdale: LEA.Search in Google Scholar

Mariana, V., Cox, T., and Melby, A. (2015): The Multidimensional Quality Metrics (MQM) Framework: A New Framework for Translation Quality Assessment. The Journal of Specialized Translation, 23. Search in Google Scholar

Martínez, M. N. and Hurtado, A. (2001): Assessment in Translation Studies: Research Needs, Meta, 46 (2).10.7202/003624arSearch in Google Scholar

McAlester, G. (2000): The Evaluation of Translation into a Foreign Language. In C. Schaeffner and B. Adab (eds.), Developing Translation Competence (pp. 229–242). Benjamins Translation Library, 38. Amsterdam: Benjamins. 10.1075/btl.38.21mcaSearch in Google Scholar

McDonald, R. P. (1999): Test theory: A unified treatment. Mahwah (NJ): LEA.Search in Google Scholar

McLaughlin, G. H. (2007): SMOG Grading: A New Readability Formula. Journal of Reading. http://www.harrymclaughlin.com/SMOG_Readibility_Formula_G._Harry_McLaughlin_(1996).pdfSearch in Google Scholar

Melis, N. M. and Albir, A. H. (2001): Assessment in Translation Studies: Research Needs. Meta, 46 (2), 272–287. Search in Google Scholar

Meshkati, N. (1988): Toward Development of a Cohesive Model of Workload. In: Hancock, P. A. and Meshkati, N. (eds) Human Mental Workload. Amsterdam; New York: North-Holland, pp. 305–314.10.1016/S0166-4115(08)62394-8Search in Google Scholar

Miller, S. (2001): Workload Measures. Iowa: The University of Iowa Press. Search in Google Scholar

Mobaraki, M. and Aminzadeh, S. (2012): A Study on Different Translation Evaluation Strategies to Introduce an Eclectic Method. International Journal of English Linguistics, 2 (6). 10.5539/ijel.v2n6p63Search in Google Scholar

Moray, N. (1979): Mental Workload: Its Theory and Measurement. New York: Plenum Press.10.1007/978-1-4757-0884-4Search in Google Scholar

Moessner, L. (2001): Genre, Text Type, Style, Register: A Terminological Maze? European Journal of English Studies Vol. 5. No. 2. 131–138.10.1076/ejes.5.2.131.7312Search in Google Scholar

Muñoz Martín, R. (2010): On Paradigms and Cognitive Translatology, in G. Schreve and E. Angelone (eds.) Translation and Cognition, Amsterdam and Philadelphia: John Benhamins, pp. 169–187. 10.1075/ata.xv.10munSearch in Google Scholar

Mulder, G. (1979): Mental Load, Mental Effort and Attention. In N. Moray (Ed.), Mental Workload: Its Theory and Measurement. New York and London: Plenum Press.10.1007/978-1-4757-0884-4_18Search in Google Scholar

Nord, C. (2005): Text Analysis in Translation: Theory, Methodology, and Didactic Application of a Model for Translation-oriented Text Analysis (2^nd Eds.). Amsterdam: Rodopi.10.1163/9789004500914Search in Google Scholar

O’Donnell, R. D. and Eggemeier, F. T. (1986): Workload Assessment Methodology. In: Boff, K. R., Kaufman, L. and Thomas, J. P. (eds) Handbook of Perception and Human Performance, Vol. II: Cognitive Processes and Performance. New York: Wiley. 42/41–42/49.Search in Google Scholar

PACTE. (2000): Acquiring Translation Competence: Hypotheses and MethodologicalProblems in a Research Project. In Investigating Translation, Beeby, A., Esinger, D., Presas, M. (eds). 99–106. Amsterdam: John Benjamins. 10.1075/btl.32.13pacSearch in Google Scholar

PACTE. (2003): Building a Translation Competence Model. In: Alves, F. (ed.) Triangulating Translation: Perspectives in Process Oriented Research. Amsterdam: John Benjamins Pub, pp. 43–66.10.1075/btl.45.06pacSearch in Google Scholar

Plass, J. L., Moreno, R. and Brünken, R. (2010): Introduction. In: Plass, J. L., Moreno, R. and Brünken, R. (eds) Cognitive Load Theory. Cambridge; New York: Cambridge University Press, pp. 1–6.10.1017/CBO9780511844744.002Search in Google Scholar

Pym, A. (2010): Exploring Translation Theories. London/New York: Routledge.10.4324/9780203869291Search in Google Scholar

RAND. (2002): Reading for understanding: toward a research and development program in reading comprehension Available from http://www.rand.org/pubs/monograph_reports/2005/ MR1465.pdf.Search in Google Scholar

Rasinger, S. (2008): Quantitative Research in Linguistics: An Introduction, London: Continuum. Search in Google Scholar

Rehmann, A. J. (1995): Handbook of Human Performance Measures and Crew Requirements for Flightdeck Research (DOT/FAA/CT-TN95/49).10.1037/e664922007-001Search in Google Scholar

Robinson, P. (ed.) (2011): Second Language Task Complexity: Researching the Cognition Hypothesis of Language Learning and Performance. Amsterdam; Philadelphia: John Benjamins.10.1075/tblt.2Search in Google Scholar

Rosenmund, A. (2001): Konstruktive Evaluation: Versuch Eines Evaluationskonzepts für den Unterricht. Meta, 46 (2), 301–310.10.7202/003987arSearch in Google Scholar

Sabri, S. (2013): Item Analysis of Student Comprehensive Test for Research in Teaching Beginners Strings Ensemble Using Model Based Teaching Among Music Students in Public Universities, International Journal of Education and Research, 1 (12). Search in Google Scholar

Saldanha, G. and O’Brien, S. (2013): Research Methodologies in Translation Studies, Manchester, New York: St Jerome Publication. Search in Google Scholar

Sammer, G. (2006): Workload and Electro-encephalography Dynamics. In: Karwowski, W. (ed.) International Encyclopedia of Ergonomics and Human Factors. Boca Raton, FL: CRC/Taylor and Francis.10.1201/9780849375477.ch118Search in Google Scholar

Shreve, G. M., Danks, J. H. and Lacruz, I. (2004): Cognitive Processes in Translation: Research Summary for the Center for the Advanced Study of Language, University of Maryland.Search in Google Scholar

Shrock, S. A. and Coscarelli, W. C. C. (2007): Criterion-referenced Test Development: Technical and Legal Guidelines for Corporate Training. San Francisco: Pfeiffer.Search in Google Scholar

Spache, G. (1953): A New Readability Formula for Primary-Grade Reading Materials. The Elementary School Journal, 53 (7): 410–13. doi:10.1086/458513.10.1086/458513Search in Google Scholar

Sun, S. (2015): Measuring Translation Difficulty: Theoretical and Methodological Considerations. Across Language and Cultures, 16 (1), 29–45.10.1556/084.2015.16.1.2Search in Google Scholar

Taras, M. (2005): Assessment-Summative and Formative- Some Theoretical Reflections. British Journal of Educational Studies, 53 (4), 466–478.10.1111/j.1467-8527.2005.00307.xSearch in Google Scholar

Vidulich, M. A. (1988): The Cognitive Psychology of Subjective Mental Workload. In: Hancock, P. A. and Meshkati, N. (eds) Human Mental Workload. Amsterdam/New York: North-Holland. 219–229.10.1016/S0166-4115(08)62388-2Search in Google Scholar

Vollmar, G. (2001): Mantaining Quality in the Flood of Translation Projects: A Model for Practical Quality Assurance. The ATA Chronicle, 30 (9), 24–27.Search in Google Scholar

Waddington, C. (2001): Different Methods of Evaluating Student Translations: The Question of Validity. Meta : journal des traducteurs / Meta: Translators’ Journal, 46 (2), 311–325.10.7202/004583arSearch in Google Scholar

Waddington, C. (2004): Should Student Translations be Assessed Holistically or through Error Analysis? Lebende Sprachen, 49 (1), 28–35.10.1515/LES.2004.28Search in Google Scholar

Wierwille, W. W., Eggemeier, F.T. (1993): Recommendations for Mental Workload Measurement in a Test and Evaluation Environment. Human Factors, 35 (2), 263–281.10.1177/001872089303500205Search in Google Scholar

Wiersma, W. and Jurs, S. G. (1990): Educational Measurement and Testing. London: Allyn and Bacon.Search in Google Scholar

Williams, M. (2001): The Application of Argumentation Theory to Translation Quality Assessment. Meta, 46 (2), 326–344.10.7202/004605arSearch in Google Scholar

Wilson, G. F. and Eggemeier, F. T. (2006): Mental Workload Measurement. In: Karwowski, W. (ed.) International Encyclopedia of Ergonomics and Human Factors. Boca Raton, FL: CRC/Taylor and Francis. 2nd ed., Vol. 1. 814–817.Search in Google Scholar

Wilss, W. (1982): The Science of Translation: Problems and Methods. Tübingen: G. Narr. Search in Google Scholar

Yeh, Y. Y., and Wickens, C. D. (1988): Dissociation of Performance and Subjective Measures of Workload. Human Factors, 30 (1), 111–120.10.1177/001872088803000110Search in Google Scholar