Validation of an instrument to measure the feedback conceptions scale

Nor Hasnida Che Md Ghazali

Abstract

It is very important to check for validity and reliability of self-report scales or measures. This study aimed to determine the validity and reliability of an instrument which is adapted from Harris and Brown (2008) known as ‘Teachers' Conceptions of Feedback (TCoF) inventory’. The instrument consists of 37 items from 9 constructs namely irrelevance, improvement, accountability, encouragement and self-type, task, process, self-regulation, peer and self-assessment and timeliness. One hundred and eighty-five (N=185) undergraduate students who are also the student teachers had completed the questionnaire. Respondents used a six-point agreement rating scale coded as strongly disagree=1, mostly disagree=2, slightly agree=3, moderately agree=4, mostly agree=5 and strongly agree=6. The content was validated by three experts in the field of measurement and evaluation. Data analysis was completed using the Rasch measurement model. The findings revealed that most items fit the model as their MNSQ values are between 0.50 and 1.50 except for the three items. Only two items showed a negative point measure correlation indicated that overall, the item discrimination is good. Item reliability and item separation is 0.98 and 7.29 respectively, while person reliability and person separation is 0.89 and 2.81 respectively. In total, four items were deleted altogether. The statistical analysis provides strong evidence to support the validity and reliability of the scale. Therefore, this instrument could be adapted or adopted by other researchers in the Malaysian educational context. Implications of the study is also discussed. Keywords: Feedback, Validity, Reliability, Self-Assessment, Peer-Assessment

References

1. Arsaythambi, V. & Rosidah, R. (2016). Assessment Practises among English teachers in Malaysian Secondary Schools, International Journal of Infonomics (IJI), 9(4), 1220-1227.

2. Aziz, A. A. (2010). Rasch Model Fundamentals: Scale Construct and Measurement Structure. Kuala Lumpur: Integrated Advanced Planning Sdn.Bhd

3. Aziz, A. A. (2010). Rasch Model Fundamentals: Scale Construct and Measurement Structure. Kuala Lumpur: Integrated Advanced Planning Sdn Bhd.

4. Black, P. (2007) Formative Assessment: Promises or problems? Available at: http://www.mantleoftheexpert.com/studying/articles/Paul%20Black2007.pdf/ (Accessed: 6 July 2013).

5. Bond, T. G. & Fox, C. M. (2007). Applying the Rasch model: Fundamental measurement in the human sciences. Mahwah: Lawrence.

6. Bond, T. G. & Fox, C. M. (2001) Applying the Rasch model: Fundamental measurement in the human sciences. Mahwah, N.J.: Erlbaum.

7. Brown, G. T. L. (2008). Conceptions of assessment: Understanding what assessment means to teachers and students. New York: Nova Science Publishers

8. Fisher, W. P. (2007). Rasch Measurement Transaction. Transaction of the Rasch Measurement SIG American Educational Research Association, 21(1), 1095.

9. Fisher, W.P. Jr (2007). Rasch Measurement Transaction. Transaction of the Rasch Measurement SIG American Educational Research Association.Vol. 21 No.1, p. 1095

10. Fisher, W.P. Jr (2007). Rasch Measurement Transaction. Transaction of the Rasch Measurement SIG American Educational Research Association.Vol. 21 No.1, p. 1095

11. Fisher, W.P. Jr (2007). Rasch Measurement Transaction. Transaction of the Rasch Measurement SIG American Educational Research Association.Vol. 21 No.1, p. 1095

12. Harris, L. R., & Brown, G. T. (2008). Teachers' Conceptions of Feedback (TCoF) inventory. Unpublished test. Auckland, NZ: University of Auckland, Measuring Teachers’ Assessment Practices (MTAP) Project.

13. Hattie, J. & Timperly, H. (2007). The Power of Feedback, Review of Educational Research, 77(1), 81-112.

14. Hattie, J. (2009). Visible learning: A synthesis of 800+ meta-analyses on achievement. Abingdon: Routledge.

15. Hattie, J. A. C. (1999). Influences on student learning. Inaugural Professorial Address, University of Auckland.

16. Hattie, J. A. C. (2003). Teachers make a difference:What is the research evidence? Keynote presentation at the Building Teacher Quality:The ACER Annual Conference, Melbourne, Australia.

17. Heritage, M. (2007) Formative Assessment: What Do Teachers Need to Know and Do?, Phi Delta Kappan, 89(2), pp. 140-145.

18. Kluger, A. N., & DeNisi, A. (1996). The effects of feedback interventions on performance: A historical review, a meta-analysis, and a preliminary feedback intervention theory. Psychological Bulletin, 119(2), 254-284.

19. Kluger, A. N., & DeNisi, A. (1998). Feedback interventions: Towards the understanding of a doubleedge sword. Current Directions in Psychological Science, 7, 67-72

20. Leahy, S., Lyon, C., Thompson, M. and Wiliam, D. (2005) ‘Classroom Assessment: Minute by minute, Day by day’, Educational Leadership, 63(3), pp.1-7.

21. Linacre, J. M. (2011). A user’s guide to Winsteps & Ministep Rasch-Model computer programs. Program Manual 3.72.1 http://www. Winsteps.com/a/winsteps.pdf

22. Nicol, D. J. and Macfarlene-Dick, D. (2006) Formative assessment and self-regulated learning: A model and seven principles of good feedback practice, Studies in Higher Education, 31(2), pp. 199-218.

23. Pallant, J. (2007) SPSS Survival Manual: A step by step guide to data analysis using SPSS for Windows, 3rd ed. New South Wales: Allen and Unwin.

24. Rayment, T. (2006) 101 Essential Lists on Assessment. Kuala Lumpur: Institut Terjemahan Negara Malaysia Bhd.

25. Suriani M, Arasinah, K. & Norhayati, A. (2016). Gauging the assessment literacy of Malaysia’s home economic teachers: An empirical study, Malaysian Journal of Society and Space, 12(3), 130-138.

26. Wiliam, D. (2011) ‘What is assessment for learning?’ Studies in educational evaluation,37, pp. 3-14.

27. Wright, B. D. & Stone, M. H. (1979). Best Test Design: Rasch Measurement. Chicago, IL: Mesa Press.