Abeywickrama, P. S. (2007). Measuring the knowledge of textual cohesion and coherence in

learners of English as a second language (ESL). (Unpublished PhD dissertation).

University of California at Los Angeles.

 

Alharby, E. R. (2006). A comparison between two scoring methods, holistic vs analytic, using

two measurement models, the generalizability theory and many-facet Rasch measurement,

within the context of performance assessment. (Unpublished PhD dissertation).

Pennsylvania State University, State College, PA.

 

Bachman, L. F. (1997). Generalizability theory. In C. Clapham & D. Corson (Eds.),

Encyclopedia of languages and education Volume 7: Language testing and assessment (pp.

255 ‒ 262). Dordrecht, Netherlands: Kluwer Academic.

 

Bachman, L. F. 2004: Statistical analyses for language assessment. Cambridge: Cambridge

University Press.

 

Bachman, L. F., Lynch, B. K., & Mason, M. (1995). Investigating variability in tasks and rater

judgments in a performance test of foreign language speaking. Language Testing, 12(2),

239 ‒ 257.

 

Banno, E. (2008). Investigating an oral placement test for learners of Japanese as a second

language. (Unpublished PhD dissertation). Temple University, Philadelphia, PA.

 

Blok, H. (1999). Reading to young children in educational settings: A meta-analysis of recent

research. Language Learning, 49(2), 343 ‒ 371.

 

Bolus, R. E., Hinofotis, F. B., & Bailey, K. M. (1982). An introduction to generalizability theory

in second language research. Language Learning, 32, 245 ‒ 258.

 

Brennan, R. L. (1983). Elements of generalizability theory. Iowa City, IA: American College

Testing Program.

 

Brennan, R. L. (2001). Generalizability theory. New York: Springer.

 

Brown, J. D. (1982). Testing EFL reading comprehension in engineering English. (Unpublished

PhD dissertation). University of California at Los Angeles.

 

Brown, J. D. (1984). A norm-referenced engineering reading test. In A.K. Pugh & J.M. Ulijn

(Eds.), Reading for professional purposes: studies and practices in native and foreign

languages. London: Heinemann Educational Books.

 

Brown, J. D. (1988). 1987 Manoa Writing Placement Examination: Technical Report #1.

Honolulu, HI: Manoa Writing Program, University of Hawai‘i at Manoa.

 

Brown, J. D. (1989). 1988 Manoa Writing Placement Examination: Technical Report #2.

Honolulu, HI: Manoa Writing Program, University of Hawai‘i at Manoa.

 

Brown, J. D. (1990a). 1989 Manoa Writing Placement Examination: Technical Report #5.

Honolulu, HI: Manoa Writing Program, University of Hawai‘i at Manoa.

 

Brown, J. D. (1990b). Short-cut estimates of criterion-referenced test consistency. Language

Testing, 7(1), 77 ‒ 97.

 

Brown, J. D. (1991). 1990 Manoa Writing Placement Examination: Technical Report #11.

Honolulu, HI: Manoa Writing Program, University of Hawai‘i at Manoa.

 

Brown, J. D. (1993). A comprehensive criterion-referenced language testing project. In D.

Douglas & C. Chapelle (Eds.), A New Decade of Language Testing Research (pp. 163 ‒

184). Washington, DC: TESOL.

 

Brown, J. D. (1999). Relative importance of persons, items, subtests and languages to TOEFL

test variance. Language Testing, 16(2), 216 ‒ 237.

 

Brown, J. D. (2005a). Testing in language programs: A comprehensive guide to English

language assessment (New edition). New York: McGraw-Hill.

 

Brown, J. D. (2005b). Statistics corner ‒ Questions and answers about language testing

statistics: Generalizability and decision studies. Shiken: JALT Testing & Evaluation SIG

Newsletter, 9(1), 12 – 16. Retrieved from http://jalt.org/test/bro_21.htm. [accessed Dec. 10,

2006].

 

Brown, J. D. (2007). Multiple views of L1 writing score reliability. Second Language Studies

(Working Papers), 25(2), 1-31.

 

Brown, J. D. (2008). Raters, functions, item types, and the dependability of L2 pragmatic tests.

In E. Alcón Soler & A. Martínez-Flor (Eds.), Investigating pragmatics in foreign language

learning, teaching and testing (pp. 224 ‒ 248). Clevedon, UK: Multilingual Matters.

 

Brown, J. D., & Bailey, K. M. (1984). A categorical instrument for scoring second language

writing skills. Language Learning, 34, 21 ‒ 42.

 

Brown, J. D., & Hudson, T. (2002). Criterion-referenced language testing. Cambridge:

Cambridge University.

 

Brown, J. D., & Ross, J. A. (1996). Decision dependability of item types, sections, tests, and

the overall TOEFL test battery. In M. Milanovic & N. Saville (Eds.), Performance testing,

cognition and assessment (pp. 231 ‒ 265). Cambridge: Cambridge University.

 

Chiu, C. W.T. (2001). Scoring performance assessments based on judgments: Generalizability

theory. Boston: Kluwer Academic.

 

Cronbach, L. J., Gleser, G. C., Nanda, H., & Rajaratnam, N. (1972). The dependability of

behavioral measurements: Theory of generalizability for scores and profiles. New York:

Wiley.

 

Cronbach, L. J., Rajaratnam, N., & Gleser, G. C. (1963). Theory of generalizability: A

liberalization of reliability theory. British Journal of Statistical Psychology, 16, 137 ‒ 163.

 

Gao, L., & Rodgers, T. (2007). Cognitive-psychometric modeling of the MELAB reading items.

Paper presented at the National Council of Measurement in Education Conference,

Chicago,IL.

 

Gerbil, A. (2009). Score generalizability of academic writing tasks: Does one test method fit

all? Language Testing, 26, 507 ‒ 531.

 

Gerbil, A. (2010). Bringing reading-to-writing and writing-only assessment tasks together: A

generalizability analysis. Assessing Writing, 15, 100 ‒ 117.

 

Glass, G. V. (1976). Primary, secondary, and meta-analysis. Educational Researcher, 5, 3 ‒ 8.

Goldschneider, J., & DeKeyser, R. M. (2001). Explaining the “natural order of L2 morpheme

acquisition” in English: A meta-analysis of multiple determinants. Language Learning, 51,

1–50.

 

Jeon, E., & Kaya, T. (2006). Effects of L2 instruction on interlanguage pragmatic development:

A meta-analysis. In J. Norris & L. Ortega (Eds.), Synthesizing Research on Language

Learning and Teaching (pp. 165 ‒ 211). Philadelphia: John Benjamins.

 

Kim, Y.H. (2009). A G-theory analysis of rater effect in ELS speaking assessment. Applied

Linguistics, 30(3), 435 ‒ 440.

 

Kirk, R. E. (1968). Experimental design: Procedures for the behavioral sciences. Belmont, CA:

Brooks/Cole.

 

Kozaki, Y. (2004). Using GENOVA and FACETS to set multiple standards on performance

assessment for certification in medical translation of Japanese into English. Language

Testing, 21(1), 1 ‒ 27.

 

Kunnan, A. J. (1992). An investigation of a criterion-referenced test using G-theory, and factor

and cluster analysis. Language Testing, 9(1), 30-49.

 

Lane, S., & Sabers, D. (1989). Use of generalizability theory for estimating the dependability

of a scoring system for sample essays. Applied measurement in education, 2(3), 195 ‒ 205.

 

Lee, Y.-W. (2005) Dependability of scores for a new ESL speaking test: Evaluating prototype

tasks. TOEFL Monograph MS-28. Princeton, NJ: ETS.

 

Lee, Y.-W. (2006). Dependability of scores for a new ESL speaking assessment consisting of

integrated and independent tasks. Language Testing, 23(2), 131 ‒ 166.

 

Lee, Y.-W, Gentile, C., & Kantor, R. (2008). Analytic scoring of TOEFL CBT essays: Scores

from humans and e-rater. TOEFL Research Report RR-81. Princeton, NJ: ETS.

 

Lee, Y.-W, & Kantor, R. (2005). Dependability of ESL writing test scores: Evaluating prototype

tasks and alternative rating schemes. TOEFL Monograph MS-31. Princeton, NJ: ETS.

 

Lee, Y.-W, & Kantor, R. (2007). Evaluating prototype tasks and alternative rating schemes for

a new ESL writing test through G-theory. International Journal of Testing, 7(4), 353 – 385

 

Lynch, B. K., & McNamara, T. F. (1998). Using G-theory and many-facet Rasch measurement

in the development of performance assessments of the ESL speaking skills of immigrants.

Language Testing, 15, 158 ‒ 180.

 

Mackey, A., & Goo, J. (2007). Interaction research in SLA: A meta-analysis and research

synthesis. In A. Mackey (Ed.), Conversational interaction in second language acquisition:

A series of empirical studies (pp. 407 – 452). Oxford: Oxford University.

 

Masgoret, A.-M., & Gardner, R. C. (2003). Attitudes, motivation, and second language learning:

A meta-analysis of studies conducted by Gardner and associates. Language Learning, 53,

123 – 163.

 

McNamara, T. F. (1996). Measuring second language performance. New York: Longman.

 

Molloy, H., & Shimura, M. (2005). An examination of situational sensitivity in medium-scale

interlanguage pragmatics research. In T Newfields, Y. Ishida, M. Chapman, & M. Fujioka

(Eds.), Proceedings of the May 22 ‒ 23, 2004 JALT Pan-SIG Conference Tokyo: JALT

 

Pan SIG Committee (pp. 16-32). Available online at www.jalt.org/pansig/2004/HTML/

ShimMoll.htm. [accessed Dec. 10, 2006].

 

Norris, J. M., & Ortega, L. (2000). Effectiveness of L2 instruction: A research synthesis and

quantitative meta-analysis. Language Learning, 50, 417 – 528.

 

Norris, J. M., & Ortega, L. (2006). The value and practice of research synthesis for language

learning and teaching. In J. M. Norris & L. Ortega (Eds.), Synthesizing research on

language learning and teaching (pp. 3 – 50). Philadelphia: John Benjamins

 

Norris, J. M., & Ortega, L. (2007). The future of research synthesis in applied linguistics:

Beyond art or science. TESOL Quarterly, 41(4), 805 ‒ 815.

 

Oswald, F. L., & Plonsky, L. (2010). Meta-analysis in second language research: Choices and

challenges. Annual Review of Applied Linguistics, 30, 85 ‒ 110.

 

Park, T. (2007). Investigating the construct validity of the Community Language Program

(CLP) English Writing Test. (Unpublished PhD dissertation). Teachers College, Columbia

University, New York, NY.

 

Rolstad, K., Mahoney, K., & Glass, G. (2005). Weighing the evidence: A meta-analysis of

bilingual education in Arizona. Bilingual Research Journal, 29, 43 ‒ 67.

 

Ross, S. (1998). Self-assessment in second language testing: A meta-analysis and analysis of

experiential factors. Language Testing, 15(1), 1 ‒ 20.

 

Russell, J., & Spada, N. (2006). The effectiveness of corrective feedback for the acquisition of

L2 grammar: A meta-analysis of the research. In J. M. Norris & L. Ortega (Eds.),

Synthesizing research on language learning and teaching (pp. 133 ‒ 164). Philadelphia:

John Benjamins.

 

Sahari, M. (1997). Elaboration as a text-processing strategy: A meta-analytic review. RELC

Journal, 28(1), 15 ‒ 27.

 

Sawaki, Y. (2003). A comparison of summarization and free recall as reading comprehension

tasks in web-based assessment of Japanese as a foreign language. (Unpublished PhD

dissertation). University of California at Los Angeles.

 

Sawaki, Y. (2007). Construct validation of analytic rating scales in a speaking assessment:

Reporting a score profile and a composite. Language Testing, 24(3), 355-390.

 

Schoonen, R. (2005). Generalizability of writing scores: An application of structural equation

modeling. Language Testing, 22(1), 1-30.

 

Shavelson, R. J., & Webb, N. M. (1981). Generalizability theory: 1973-1980. British Journal

of Mathematical and Statistical Psychology, 34, 133-166.

 

Shavelson, R. J., & Webb, N. M. (1991). Generalizability theory: A primer. Newbury Park, CA:

Sage.

 

Shin, S. (2002). Effects of subskills and text types on Korean EFL reading scores. Second

Language Studies (Working Papers), 20(2), 107-130. Retrieved from http://www.hawaii.

edu/sls/uhwpesl/on-line_cat.html. [accessed Dec. 10, 2006].

 

Solano-Flores, G., & Li, M. (2006). The use of generalizability (G) theory in testing of linguistic

minorities. Educational Measurement: Issues and Practice, Spring, 13-22.

 

Stansfield, C. W., & Kenyon, D. M. (1992). Research of the comparability of the oral

proficiency interview and the simulated oral proficiency interview. System, 20, 347-364.

 

Sudweeks, R. R., Reeve, S., & Bradshaw, W. S. (2005). A comparison of generalizability theory

and many-facet Rasch measurement in an analysis of college sophomore writing. Assessing

Writing, 9, 239–261

 

Suen, H. K. (1990). Principles of test theories. Hillsdale, NJ: Lawrence Erlbaum.

 

Tang, X. (2006). Investigating the score reliability of the English as a Foreign Language

Performance Test. (Unpublished PhD dissertation). Queen’s University, Kingston, Ontario,

Canada.

 

Taylor, A., Stevens, J., & Asher, W. (2006). The effects of explicit reading strategy training on

L2 reading comprehension: A meta-analysis. In J. M. Norris & L. Ortega (Eds.),

Synthesizing research on second language learning and teaching (pp. 3-50). Philadelphia:

John Benjamins.

 

Van Moere, A. (2006). Validity evidence in a university group oral test. Language Testing,

23(4), 411 ‒ 440.

 

Van Weeren, J., & Theunissen, T. J. J. M. (1987). Testing pronunciation: An Application of

generalizability theory. Language Learning, 37(1), 109 – 122.

 

Xi, X. (2003). Investigating language performance on the graph description task in a semidirect

oral test. (Unpublished PhD dissertation). University of California at Los Angeles.

 

Xi, X. (2007). Evaluating analytic scoring for the TOEFL® Academic Speaking Test (TAST)

for operational use. Language Testing, 24(2) 251 ‒ 286.

 

Xi, X., & Mollaun, P. (2006). Investigating the utility of analytic scoring for the TOEFL

Academic Speaking Test (TAST). TOEFL iBT Research Report, TOEFLiBT-01. Princeton,

NJ: ETS.

 

Yamamori, K. (2003). Evaluation of students’ interest, willingness, and attitude toward English

lessons: Multivariate generalizability theory. The Japanese Journal of Educational

Psychology, 51(2), 195 ‒ 204.

 

Yamanaka, H. (2005). Using generalizability theory in the evaluation of L2 writing. JALT

Journal, 27(2), 169-185.

 

Yoshida, H. (2004). An analytic instrument for assessing EFL pronunciation. (Unpublished

Ed.D. PhD dissertation). Philadelphia, PA: Temple University.

 

Yoshida, H. (2006). Using generalizability theory to evaluate reliability of a performance-based

pronunciation measurement. (Unpublished ms). Osaka Jogakuin College.

 

Zhang, S. (2004). Investigating the relative effects of persons, items, sections, and languages on

TOEIC score dependability. (Unpublished MA thesis). Ontario Institute for Studies in

Education of the University of Toronto.

 

Zhang, S. (2006). Investigating the relative effects of persons, items, sections, and languages on

TOEIC score dependability. Language Testing, 23(3), 351 – 369.

 

Zhang, Y. (2003). Effects of persons, items, and subtests on UH ELIPT reading test scores.

Second Language Studies, 21(2), 107-128. Retrieved from http://www.hawaii.edu/sls/

uhwpesl/on-line_cat.html. [accessed Dec. 10, 2006].