Cover not available
Article published In:
ITL - International Journal of Applied Linguistics
Vol. 168:2 (2017) ► pp.143–163
Critical position paper

Just testing

Applying theories of justice to high-stakes language tests

| KU Leuven

Justice has been the topic of comparatively few papers in the fields of applied linguistics or language assessment. This may be due to the lack of a clear and agreed-upon definition on the one hand, or to the difficulty of operationalizing justice for test development on the other. This paper aims to remedy both problems by discussing prior conceptualizations of justice and by introducing six justice principles, which are based on theories of distributive justice that focus on human rights, fairness, equal opportunity, and dignity. The overarching aim of this paper is to advance the debate on justice, and to provide a consistent way of considering ethical and moral dilemmas that language testers face today.

Keywords: language testing, high-stakes test, justice, justice, language testing policy
Article outline
  • 1.Introduction
  • 2.Approaches to justice
    • 2.1Taking position
    • 2.2 Rawls, Rawls, Rawls
    • 2.3Justice in language testing
  • 3.Six principles of just testing
    • 3.1Even a coercive testing policy should be just
    • 3.2 A just testing policy treats test takers with dignity
    • 3.3A just testing policy relies on empirical evidence
    • 3.4All primary stakeholders should be consulted
    • 3.5Agency implies accountability
    • 3.6Test takers have a right to privacy
  • 4.Conclusion
  • Acknowledgments
  • References
Published online: 30 December 2017
DOI logo
https://doi.org/10.1075/itl.00001.dey
References

References

ALTE
(2010) Minimum standards for establishing quality profiles in ALTE examinations. ALTE. Retrieved from [URL]> (10 March 2013).
Altman, A., & Wellman, C. H.
(2009) A liberal theory of international justice. Oxford: Oxford University Press. DOI logoGoogle Scholar
Bachman, L., & Palmer, A.
(2010) Language assessment in practice. Oxford: Oxford University Pres.Google Scholar
Bentham, J., & Mill, J. S.
(1987) Utilitarianism and other essays. (A. Ryan, Ed.). Harmondsworth: Penguin Classics.Google Scholar
Boylan, M.
(2004) A just society. Lanham, MD: Rowman & Littlefield Publishers.Google Scholar
Connolly, T., Arkes, H. R., & Hammond, K. R.
(1999) Judgment and decision making: An interdisciplinary reader. Cambridge: Cambridge University Press.Google Scholar
Davies, A.
(2010) Test fairness: A response. Language Testing, 27(2), 171–176. DOI logoGoogle Scholar
Deygers, B., Van den Branden, K., & Van Gorp, K.
(2017) University entrance language tests: A matter of justice. Language Testing. (Published online, awaiting print) DOI logoGoogle Scholar
Dworkin, R.
(2002) Sovereign virtue: The theory and practice of equality. Cambridge, MA: Harvard University Press.Google Scholar
(2013a) Justice for hedgehogs. Cambridge, MA: Belknap Press.Google Scholar
(2013b) Taking rights seriously. London: Bloomsbury Academic.Google Scholar
EALTA
(2000) EALTA Guidelines for good practice in language testing and assessment. Retrieved from [URL]> (10 March 2013).
ETS
(2016) Privacy and security. Retrieved from [URL]> (29 September 2016).
Foucault, M.
(1977) Discipline and punish. The birth of the prison. London: Penguin.Google Scholar
Fulcher, G.
(2015) Re-examining Language Testing: A Philosophical and Social Inquiry. London : Routledge. DOI logoGoogle Scholar
Hobbes, T.
(1982) Leviathan. London: Penguin Classics. (Original work published 1651).Google Scholar
IELTS
(2016) IELTS candidate identity verification. Retrieved from [URL]> (29 September 2016).
ILTA
(2000) Code of ethics. Retrieved from [URL]> (10 March 2013).
(2007) Guidelines for practice. International Language Testing Association. Retrieved from [URL]
Kane, M. T.
(2010) Validity and fairness. Language Testing, 27(2), 177–182. DOI logoGoogle Scholar
(2013) Validating the interpretations and uses of test scores. Journal of Educational Measurement, 50(1), 1–73. DOI logoGoogle Scholar
Kant, I.
(2016) Fundamental principles of the metaphysic of morals (T. K. Abbott, Trans.). Create Space Independent Publishing Platform. (Original work published 1785).Google Scholar
Kaufmann, P., Kuch, H., Neuhaeuser, C., & Webster, E.
(2010) Humiliation, degradation, dehumanization: human dignity violated. Berlin: Springer Science & Business Media.Google Scholar
Khan, K., & McNamara, T.
(2017) Citizenship, immigration laws, and language. In S. Canagarajah (Ed.), The Routledge handbook of migration and language (pp. 451–467). London: Routledge. DOI logoGoogle Scholar
Kunnan, A. J.
(2000) Fairness and justice for all. In A. J. Kunnan (Ed.), Fairness and validation in language assessment. (pp. 1–14). Cambridge: Cambridge University Press.Google Scholar
(2004) Test fairness. In M. Milanovic & C. Weir (Eds.), European language testing in a global context (pp. 27–48). Cambridge: Cambridge University Press.Google Scholar
(2010) Test fairness and Toulmin’s argument structure. Language Testing, 27(2), 183–189.Google Scholar
(2014) Fairness and justice in language assessment. In A. J. Kunnan (Ed.), The companion to language assessment. New York, NY: John Wiley & Sons, Inc.Google Scholar
Locke, J.
(1980) Second treatise of government. (C. B. Macpherson, Ed.). Indianapolis, IN: Hackett Publishing Company, Inc. (Original work published 1689).Google Scholar
Long, M. H.
(2005) Methodological issues in learner needs analysis. In M. H. Long (Ed.), Second language needs analysis (pp. 19–79). Cambridge: Cambridge University Press. DOI logoGoogle Scholar
McNamara, T.
(2009) Australia: The dictation test redux? Language Assessment Quarterly, 6(1), 106–111. DOI logoGoogle Scholar
(2012) Language assessments as shibboleths: A poststructuralist perspective. Applied Linguistics, 33(5), 564–581. DOI logoGoogle Scholar
McNamara, T., & Ryan, K.
(2011) Fairness versus justice in language testing: The place of english literacy in the australian citizenship test. Language Assessment Quarterly, 8(2), 161–178. DOI logoGoogle Scholar
Messick, S.
(1989) Validity. In Educational Measurement (pp. 13–103). Washington, DC: American Council on Education / Macmillan.Google Scholar
Mill, J. S.
(2015) On liberty, utilitarianism and other essays (M. Philp & F. Rosen, Eds.). Oxford: Oxford University Press. (Original work published 1861).Google Scholar
Nagel, T.
(1989) The view from nowhere. Oxford: Oxford University Press.Google Scholar
Nelson, L. S.
(2006) Constructing policy: The unsettled question of biometric technology and privacy. In K. J. Strandburg & D. S. Raicu (Eds.), Privacy and technologies of identity (pp. 151–173). New York, NY: Springer. DOI logoGoogle Scholar
Norris, J.
(2008) Validity evaluation in language assessment. New York, NY: Peter Lang. DOI logoGoogle Scholar
Nozick, R.
(1974) Anarchy, state, and utopia. New York, NY: Basic Books.Google Scholar
Nussbaum, M.
(2002) Capabilities and social justice. International Studies Review, 4(2), 123–135. DOI logoGoogle Scholar
(2004) Beyond the social contract: Capabilities and global justice. Oxford Development Studies, 32(1), 3–18.Google Scholar
(2006) Frontiers of justice. Disability, nationality, species membership. Cambridge, MA: Belknap Press.Google Scholar
(2011) Creating capabilities. The human development approach. Cambridge, MA: Belknap Press. DOI logoGoogle Scholar
Phillips, D. C.
(2007) Adding complexity: Philosophical perspectives on the relationship between evidence and policy. Yearbook of the National Society for the Study of Education, 106(1), 376–402. DOI logoGoogle Scholar
Pierce, C. P.
(2010) Idiot America: How stupidity became a virtue in the land of the free. New York, NY: Anchor Books.Google Scholar
Pogge, T.
(2010) A critique of the capability approach. In H. Brighouse & I. Robeyns (Eds.), Measuring justice: Primary goods and capabilities (pp. 17–61). Cambridge: Cambridge University Press. DOI logoGoogle Scholar
Prometric
(2016) ID Management. Retrieved from [URL]> (12 October 2016).
Rawls, J.
(1958) Justice as fairness. The Philosophical Review, 67(2), 164–194. DOI logoGoogle Scholar
(1971) A Theory of Justice. Cambridge, MA: Harvard University Press.Google Scholar
(2001) Justice as Fairness: A Restatement. Cambridge, MA: Belknap Press.Google Scholar
Reisch, M.
(2014) Social justice and liberalism. In M. Reisch (Ed.), The Routledge international handbook of social justice (pp. 132–147). London: Routledge. DOI logoGoogle Scholar
Rousseau, J. -J.
(2016) The social contract (G. Cole, Trans.). CreateSpace Independent Publishing Platform. (Original work published 1762).Google Scholar
Sandel, M. J.
(2010) Justice: What’s the right thing to do? New York, NY: Farrar, Straus and Giroux.Google Scholar
Saville, N.
(2010) Developing a model for investigating the impact of language assessment. Cambridge ESOL Research Notes, 421, 2–8.Google Scholar
Scanlon, T. M.
(1998) What we owe to each other. Cambridge, MA: Belknap Press.Google Scholar
Scientific American’s Board of Editors
(2014) Beware the eye spies. Scientific American, 310(1), 10–10.Google Scholar
Sen, A.
(1980) Equality of what? In S. M. McMurrin (Ed.), The Tanner lectures on human values (pp. 196–220). Salt Lake City, UT: University of Utah Press and Cambridge University Press.Google Scholar
(2010) The idea of justice. London: Penguin.Google Scholar
Sethi, I. K.
(2006) Biometrics: Overview and applications. In K. J. Strandburg & D. S. Raicu (Eds.), Privacy and technologies of identity (pp. 117–135). New York, NY: Springer. DOI logoGoogle Scholar
Shohamy, E.
(1997) Testing methods, testing consequences: Are they ethical? Are they fair? Language Testing, 14(3), 340–349. DOI logoGoogle Scholar
(2001) The power of tests. New York, NY: Pearson.Google Scholar
Strik, T., Böcker, A., Luiten, M., & van Oers, R.
(2010) The INTEC project: Synthesis report. Nijmegen: Radboud University.Google Scholar
Tistarelli, M., Li, S. Z., & Chellappa, R.
(Eds.) (2009) Ethics and policy of biometrics. In Handbook of remote biometrics for surveillance and security (pp. 293–312). New York, NY: Springer. DOI logoGoogle Scholar
Toulmin, S.
(2001) Return to reason. Cambridge, MA: Harvard University Press.Google Scholar
(2003) The uses of argument (Updated edition). Cambridge: Cambridge University Press. DOI logoGoogle Scholar
UN General Assembly
(1948, December 10). Universal declaration of human rights. UN General Assembly. Retrieved from [URL]> (19 September 2016)
Valentini, L.
(2009) Coercion and (global) justice: Towards a unified framework. CSSJ Working Papers Series, SJ010, 1–23.Google Scholar
(2012) Justice in a globalized world: A normative framework. Oxford: Oxford University Press.Google Scholar
Van Avermaet, P.
(2009) Fortress Europe? Language policy regimes for immigration and citizenship. In G. Hogan-Brun, C. Mar-Molinero, & P. Stevenson (Eds.), Discourses on language and integration (pp. 15–44). Amsterdam: John Benjamins. DOI logoGoogle Scholar
Van Oers, R.
(2013) Citizenship tests in Germany, the Netherlands and the UK. Unpublished PhD dissertation. Radboud University Nijmegen.Google Scholar
Walters, S. F.
(2012) Fairness. In G. Fulcher & F. Davidson (Eds.), The Routledge handbook of language testing (pp. 469–479). London: Routledge.Google Scholar
Walzer, M.
(1983) Spheres of justice. New York, NY: Basic Books.Google Scholar
Weir, C.
(2005) Language testing and validation. New York, NY: Palgrave Macmillan. DOI logoGoogle Scholar
Witkin, S. L., & Irving, A.
(2014) Postmodern perspectives on social justice. In M. Reisch (Ed.), The Routledge international handbook of social justice (pp. 188–202). London: Routledge.Google Scholar
Cited by

Cited by 6 other publications

Annick De Houwer & Lourdes Ortega
2018. The Cambridge Handbook of Bilingualism, DOI logo
Deygers, Bart
2019. Fairness and Social Justice in English Language Assessment. In Second Handbook of English Language Teaching [Springer International Handbooks of Education, ], ► pp. 541 ff. DOI logo
Deygers, Bart
2019. Fairness and Social Justice in English Language Assessment. In Second Handbook of Information Technology in Primary and Secondary Education [Springer International Handbooks of Education, ], ► pp. 1 ff. DOI logo
Hamid, M. Obaidul, Ian Hardy & Vicente Reyes
2019. Test-takers’ perspectives on a global test of English: questions of fairness, justice and validity. Language Testing in Asia 9:1 DOI logo
Hamid, M. Obaidul, Ngoc T. H. Hoang & Andy Kirkpatrick
2019. Language tests, linguistic gatekeeping and global mobility. Current Issues in Language Planning 20:3 ► pp. 226 ff. DOI logo
Kamasak, Rifat, Mustafa Ozbilgin & Ali Rıza Esmen
2020. The High Stakes Use of Language Proficiency Tests as Illusio and Pyramid Scheme. In Optimizing Higher Education Learning Through Activities and Assessments [Advances in Higher Education and Professional Development, ], ► pp. 188 ff. DOI logo

This list is based on CrossRef data as of 15 april 2022. Please note that it may not be complete. Sources presented here have been supplied by the respective publishers. Any errors therein should be reported to them.