Estimating teacher effect using hierarchical linear modelling

• Authors: Maciej Koniewski
• Languages of publication: EN

Abstracts

EN

This paper reports a study addressing how teacher effectiveness influences student outcomes in Polish Lower Secondary schools. Data from a Polish nationwide lower secondary school study were analysed. Data included 3883 pupils in 246 classes, in 137 schools with 202 maths teachers and 4119 pupils in 260 classes, in 143 schools with 215 language teachers. Variance of exam scores explained by teacher effect was 12% (maths) and 8% (language skills). Controlling for prior achievement, intelligence and student family background, teacher effects were 5% (maths) and 4% (language skills). Until now there has been no consensus about which teacher characteristics could explain variance in their effectiveness. The scale of “teacher authority/classroom management” explains 91% of exam scores in maths and 81% in language skills, when controlling for prior student achievement, intelligence, student family background and school location.

Keywords

EN

sociology of education; teacher effect; school determinants of student achievement; hierarchical models; HLM

• Publisher: Instytut Badań Edukacyjnych
• Journal: Edukacja
• Year: 2014
• Issue: 5(130)
• Pages: 70–91

Dates

published

2014-02-06

Contributors

author

Maciej Koniewski

• Educational Research Institute

References

• Akiba, M., LeTendre, G. K. and Scribner, J. P. (2007). Teacher quality, opportunity gap, and national achievement in 46 countries. Educational Researcher, 36(7), 369–387.
• Anderson, C. S. (1982). The search for school climate – a review of the research. Review of Educational Research, 52(3), 368–420.
• Armour, D. T. (1976). Analysis of the school preferred reading program in selected Los Angeles minority schools. Santa Monica, CA: Rand Corporation.
• Ashton, P. and Crocker, L. (1987). Systematic study of planned variations – the essential focus of teacher – education reform. Journal of Teacher Education, 38(3), 2–8.
• Blau, P. M. and Duncan, O. D. (1967). The American occupational structure. New York: John Wiley and Sons.
• Bourdieu, P. (1986). The forms of social capital. In J. Richardson (ed.), Handbook of theory and research for the sociology of education (pp. 241–258). New York, NY: Greenwood Press.
• Bourdieu, P. and Passeron, J. C. (1990). Reprodukcja: elementy teorii systemu nauczania [Reproduction: elements of the theory of the educational system]. Warszawa: PWN.
• Bowles, S. S. and Levin, H. M. (1968). More on multi-collinearity and the effectiveness of schools. The Journal of Human Resources, 3(3), 393–400.
• Brookover, W. B. (1982). Creating effective schools: an in-service program for enhancing school learning climate and achievement. Holmes Beach, FL: Learning Publications.
• Brown, C., Smith, M. and Stein. M. (1996). Linking teacher support to enhanced classroom instruction. Paper presented at the conference of tThe American Educational Research Association, New York, NY.
• Brown, D. F. (2001). The value of advisory sessions for urban young adolescents. Middle School Journal, 32(4), 14–22.
• Bryk, A. S., and Raudenbush S. (1992). Hierarchical linear models in social and behavioral research: applications and data analysis methods. Newbury Park, CA: Sage Publications.
• Byrne, B., Coventry, W. L., Olson, R. K., Wadsworth, S. J., Samuelsson, S., Petrill, S. A., Willcutt, E. G. and Corley, R. (2010). „Teacher effects” in early literacy development: evidence from a study of twins. Journal of Educational Psychology, 102(1), 32–42.
• Byrne, C. (1983). Teacher knowledge and teacher effectiveness: a literature review, theoretical analysis and discussion of research strategy. Paper presented at the conference Northwestern Educational Research Association, Ellenville, NY.
• Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: L. Erlbaum.
• Coleman, J. S., Campbell, E. Q., Hobson, C. J., McPartland, J., Mood, A. B., Weinfeld, F. D. and York, R. L. (1966). Equality of educational opportunity. Washington, D.C.: Government Printing Office.
• Darling-Hammond, L. (2000). Teacher quality and student achievement: a review of state policy evidence. Education Policy Analysis Archives, 1(8), 1–44.
• Dolata, R. (2008). Szkoła – segregacje – nierówności [School – segregation – inequality]. Warszawa: Wydawnictwa Uniwersytetu Warszawskiego.
• Domański, H. and Pokropek, A. (2011). Podziały terytorialne, globalizacja a nierówności społeczne. Wprowadzenie do modeli wielopoziomowych [The territorial divisions, globalization and social inequality. An introduction to multilevel models]. Warszawa: Wydawnictwo IFiS PAN.
• Domański, H., Karpiński, Z., Pokropek, A., Przybysz, D., Sawiński, Z., Słomczyński, K. M. and Trzciński, R. (2012). Metodologia badań nad stratyfikacją społeczną [The methodology of social stratification research]. Warszawa: Scholar.
• Edmonds, R. (1979). Effective schools for urban poor. Educational Leadership, 37(1), 15–24.
• Ehrenberg, R. G. and Brewer, D. J. (1995). Did teachers’ verbal ability and race matter in the 1960’s? Coleman revisted. Economics of Education Review, 14(1), 1–21.
• Evertson, C. M., Hawley, W. D. and Zlotnik, M. (1985). Making A difference in educational quality through teacher – education. Journal of Teacher Education, 36(3), 2–12.
• Ferguson, R. F. and Ladd, H. L. (1996). How and why money matters: an analysis of Alabama schools. In H. F. Ladd (ed.), Holdine schools accountable. Washington, D.C.: The Brookings Institution.
• Forero, C. G., Maydeu-Olivares, A. and Gallardo-Pujol, D. (2009). Factor analysis with ordinal indicators: a Monte Carlo study comparing DWLS and ULS estimation. Structural Equation Modeling A Multidisciplinary Journal, 16(4), 625–641.
• Freiberg, H. J. (ed.). (1999). School climate: measuring, improving and sustaining healthy learning environments. Philadelphia, PA: Falmer Press.
• Goldhaber, D. D. and Brewer, D. J. (1997). Why don’t schools and teachers seem to matter? Assessing the impact of unobservables on educational productivity. Journal of Human Resources, 32(3), 505–523.
• Goldhaber, D. D. and Brewer, D. J. (2000). Does teacher certification matter? High school teacher certification status and student achievement. Educational Evaluation and Policy Analysis, 22(2), 129–145.
• Hanushek, E. A. (1971). Teacher characteristics and gains in student achievement – estimation using micro data. American Economic Review, 61(2), 280–288.
• Hanushek, E. A. (1986). The economics of schooling production and efficiency in public schools. Journal of Economic Literature, 24(3), 1141–1177.
• Hanushek, E. A. (1989). The impact of differential expenditures on school performance. Educational Researcher, 18(4), 45–62.
• Hanushek, E. A. (1992). The trade-off between child quantity and quality. Journal of Political Economy, 100(1), 84–117.
• Hanushek, E. A. (1996). A more complete picture of school resource policies. Review of Educational Research, 66(3), 397–409.
• Hanushek, E. A. (1997). Assessing the effects of school resources on student performance: an update. Educational Evaluation and Policy Analysis, 19(2), 141–164.
• Hart, S. A., Petrill, S. A. and Thompson, L. A. (2009). A factorial analysis of timed and untimed measures of maths and reading abilities in school aged twins. Learning and Individual Differences, 20(2), 63–69.
• Hattie, J. A. C. (2009). Visible learning. A synthesis of over 800 meta-analyses relating to achievement. London, New York: Routledge.
• Hedges, L. V., Laine, R. D. and Greenwald, R. (1994a). Does money matter? A meta-analysis of studies of the effects of differential school inputs on student outcomes. Educational Researcher, 23(3), 5–14.
• Hedges, L. V., Laine, R. D. and Greenwald, R. (1994b). Money does matter somewhere: a response to Hanushek. Educational Researcher, 23(4), 9–10.
• Hedges, L. V., Laine, R. D. and Greenwald, R. (1996). The effect of school resources on student achievement. Review of Educational Research, 66(3), 361–396.
• Jakubowski, M., Pokropek, A. (2009). Badając egzaminy. Podejście ilościowe w badaniach edukacyjnych [Analysing the exams. The quantitative approach in educational research]. Warszawa: Centralna Komisja Egzaminacyjna.
• Jordan, H. R., Mendro, R. and Weerasinghe, D. (1997). Teacher effects on longitudinal student achievement: a preliminary report on research on teacher effectiveness. Paper presented in The National Evaluation Institute, Indianapolis, IN.
• Klitgaard, R. E. and Hall, G. R. (1975). Are there unusually effective schools? Journal of Human Resources, 10(1), 90–106.
• Koniewski, M. (2012). Szacowanie wpływu liczebności klasy na osiągnięcia edukacyjne uczniów z wykorzystaniem eksperymentu ex post facto [Estimating the effect of class size on academic achievement by ex post facto experiment]. Edukacja, 117(1), 23–43.
• Kovas, Y., Haworth, C. M. A., Petrill, S. A. and Plomin, R. (2007). Mathematical ability of 10-year-old boys and girls: genetic and environmental etiology of typical and low performance. Journal of Learning Disabilities, 40(6), 554–568.
• Kreft, I. and Leeuw, J. de (1998). Introducing multilevel modeling. Thousand Oaks, CA: Sage Publications.
• Lee, V. E. and Bryk, A. S. (1989). A multilevel model of the social distribution of high school achievement. Sociology of Education, 62(3), 172–192.
• Marzano, R. J., Marzano, J. S. and Pickering, D. J. (2003). Classroom management that works. Alexandria, VA: ASCD.
• McDonald, R. P., Ho, M-H. (2002). Principles and Practice in reporting structural equation analyses. Psychological Methods, 7(1), 64–82.
• McGue, M., Keyes, M., Sharma, A., Elkins, I., Legrand, L., Johnson, W. and Iacono, W. G. (2007). The environments of adopted and non-adopted youth: evidence on range restriction from the Sibling Interaction and Behavior Study (SIBS). Behavior Genetics, 37(3), 449–462.
• Mosteller, F. and Moynihan, D. P. (1972). On equality of educational opportunity. New York: Random House.
• Mullen, B. and Copper, C. (1994). The Relation between group cohesiveness and performance – an integration. Psychological Bulletin, 115(2), 210–227.
• Murnane, R. J. (1985). Do effective teachers have common characteristics: interpreting the quantitative research evidence. Paper presented at The National Research Council Conference on Teacher Quality in Science and Maths, Washington, D.C.
• Murnane, R. J. and Phillips, B. R. (1981). What Do effective teachers of inner-city children have in common. Social Science Research, 10(1), 83–100.
• Nestler, S. (2012). A Monte Carlo study comparing PIV, ULS and DWLS in the estimation of dichotomous confirmatory factor analysis. British Journal of Mathematical and Statistical Psychology, 66(1), 127–143.
• Nye, B., Konstantopoulos, S. and Hedges, L. V. (2004). How large are teacher effects? Educational Evaluation and Policy Analysis, 26(3), 237–257.
• Oliver, B., Harlaar, N., Hayiou, T., Hayiou, M. E., Kovas, Y., Walker, S. O., Petrill, S. A., Spinath, F. M., Dale, P. S. and Plomin, R. (2004). A twin study of teacher – reported maths performance and low performance in 7-year-olds. Journal of Educational Psychology, 96(3), 504–517.
• Plowden, B. (1967). Children and their primary schools. London: HMSO.
• Pokropek, A. (2009). Metody statystyczne wykorzystywane w szacowaniu trzyletnich wskaźników egzaminacyjnych [The statistical methods used in estimating the three-year indicators of examination]. In B. Niemierko and M. K. Szmigel (eds.), Badania międzynarodowe and wzory zagraniczne w diagnostyce edukacyjnej [International studies and foreign models in educational diagnosis] (pp. 137–153). Kraków: Polskie Towarzystwo Diagnostyki Edukacyjnej.
• Pokropek, A. (2013). Efekt rówieśników w nauczaniu szkolnym [The effect of peers in school education]. Warszawa: Instytut Badań Edukacyjnych.
• Pokropek, A. and Żółtak, T. (2012). Nowe modele jednorocznej EWD [New models of one-year educational value added]. In B. Niemierko and M. K. Szmigel (eds.), Regionalne i lokalne diagnozy edukacyjne [Regional and local educational diagnoses] (pp.178–187). Kraków: Polskie Towarzystwo Diagnostyki Edukacyjnej.
• Raudenbush, S. W. and Bryk, A. S. (2002). Hierarchical linear models (2nd ed.). Thousand Oaks: Sage Publications.
• Rice, J. M. (1902), Educational research: a test in arithmetic. The Forum, October, 281 –297.
• Rivkin, S. G., Hanushek, E. A. and Kain, J. F. (2005). Teachers, schools and academic achievement. Econometrica, 73(2), 417–458.
• Robinson, G. E. (1990). Synthesis of research on the effects of class size. Educational Leadership, 47(7), 80–90.
• Rosenholtz, S. J. (1986). Organizational conditions of teacher learning. Teaching and Teacher Education, 2(2), 91–104.
• Rowan, B., Chiang, F. S. and Miller, R. J. (1997). Using research on employees’ performance to study the effects of teachers on students’ achievement. Sociology of Education, 70(4), 256–284.
• Rowan, B., Correnti, R. and Miller, R. J. (2002). What large-scale, survey research tells us about teacher effects on student achievement: insights from the prospects study of elementary schools. Teachers College Record, 104(8), 1525–1567.
• Samejima, F. (1969). Estimation of latent ability using a response pattern of graded scores. Psychometric Monograph No. 17. Richmond, VA: Psychometric Society.
• Sanders, W. L. and Rivers J. (1996). Cumulative and residual effects of teachers on future student academic achievement. Research progress report. Knoxville, TN: University of Tennessee Value Added Research and Assessment Center.
• Scheerens, J. and Bosker, R. (1997). The foundations of educational effectiveness. New York: Pergamon.
• Taylor, J., Roehrig, A. D., Hensler, B. S., Connor, C. M. and Schatschneider, C. (2010). Teacher quality moderates the genetic effects on early reading. Science, 328(5977), 512–514.
• Wade, R. K. (1984). What makes a difference in inservice teacher education? A meta-analysis of research. Educational Leadership, 42(4), 48–54.
• Wayne, A. J. and Youngs, P. (2003). Teacher characteristics and student achievement gains: a review. Review of Educational Research, 73(1), 89–122.
• Wiley, D. E. and Yoon, B. (1995). Teacher reports on opportunity to learn-analyses of the 1993 California Learning Assessment System (CLAS). Educational Evaluation and Policy Analysis, 17(3), 355–370.
• Wilkinson, I. A. G. (2002). Introduction: peer influences on learning: where are they? International Journal of Educational Research, 37(5), 395–401.
• Woods, C. M. (2002). Factor analysis of scales composed of binary items: illustration with the Maudsley Obsessional Compulsive Inventory. Journal of Psychopathology and Behavioral Assessment, 24(4), 215–223.
• Wright, P. S., Horn. S. P. and Sanders. W. L. (1997). Teacher and classroom context effects on student achievement: implications for teacher evaluation. Journal of Personnel Evaluation in Education, 11(1), 57–67.

Notes

EN

http://www.edukacja.ibe.edu.pl/images/numery/2014/5-5-koniewski-estimating-teacher-effect.pdf

Identifiers

ISSN

0239-6858