Pilot Testing as a Strategy to Develop Interview and Questionnaire Skills for Scholar Practitioners

A Selection of Education Doctorate Students’ Reflective Vignettes





research methods, pilot testing, critical self-reflections, Education Doctorate, course design, learning during COVID-19


This essay presents the reflections of four Education Doctorate (EdD) students on the pilot testing strategies used during an online research methods course. Rigorous questionnaire and interview development skills are challenging to acquire. Pilot testing is an under-researched stage of instrument design, yet it is crucial to ensure validity and reliability, reduce bias, and psychologically prepare researchers for data collection. A structured, multi-step pilot testing process led to the collective development of stronger scholar-practitioner identities, the use of innovative synchronous/asynchronous methods during COVID-19 and increased academic rigor. These reflections demonstrate how several types of pilot testing can support the development of rigorous data collection instruments and prepare post-graduate students for the psychological and technical challenges they may encounter in future research.


Acton, K. (2022). LHA3005: Introduction to Research Methods for the EdD. [Syllabus]. Toronto: Ontario Institute for Studies in Education.

Buss, R., Zambo, R., Zambo, D., & Williams, T. (2014). Developing research professionals in an EdD program: From learners and leaders to scholarly and influential practitioners. Higher Education, Skills and Work-Based Learning, 4(2), 137–160. https://doi.org/10.1108/HESWBL-11-2013-0022

Carnegie Project on the Education Doctorate. (2021). The CPED framework. https://www.cpedinitiative.org/the-framework

Castillo-Montoya, M. (2016). Preparing for interview research: The interview protocol refinement framework. The Qualitative Report, 21(5), 811–831. https://doi.org/10.46743/2160-3715/2016.2337

Cohen, L., Manion, L., & Morrison, K. (2018). Research methods in education (8th ed.). Routledge.

Collins, D. (2003). Pretesting survey instruments: An overview of cognitive methods. Quality of Life Research, 12(3), 229–238. https://www.jstor.org/stable/4038871?seq=1&cid=pdfreference#references_tab_contents

Coronel Llamas, J. M., & Boza, Á. (2011). Teaching research methods for doctoral students in education: learning to enquire in the university. International Journal of Social Research Methodology, 14(1), 77–90. https://doi.org/10.1080/13645579.2010.492136

Creswell, J.W. & Guetterman, T. C. (2019). Educational research: Planning, conducting, and evaluating quantitative and qualitative research (6th ed.). Pearson.

De Vaus, D. (2014). Surveys in social research (6th ed.). Routledge.

Kelley, K., Clark, B., Brown, V., & Sitzia, J. (2003). Good practice in the conduct and reporting of survey research. International Journal for Quality in Health Care, 15(3), 261–266. https://doi.org/10.1093/intqhc/mzg031

Labaree, D. F. (2003). The peculiar problems of preparing educational researchers. Educational Researcher, 32(4), 13–22. https://doi.org/10.3102/0013189X032004013

Majid, M. A. A., Othman, M., Mohamad, S. F., Lim, S. A. H., & Yusof, A. (2017). Piloting for interviews in qualitative research: Operationalization and lessons learnt. International Journal of Academic Research in Business and Social Sciences, 7(4). https://doi.org/10.6007/IJARBSS/v7-i4/2916

Merriam, S. B., & Tisdell, E. J. (2016). Qualitative research: A guide to design and implementation. John Wiley & Sons.

Ontario Institute for Studies in Education. (n.d.). Doctor of Education in Educational Leadership & Policy. Retrieved October 31, 2022, from https://www.oise.utoronto.ca/lhae/doctoral-degrees/doctor-education-educational-leadership-policy

Patton, M. Q. (2015). Qualitative research & evaluation methods: Integrating theory and practice (4th ed.). SAGE Publications, Inc.

Privitera, G. J., & Ahlgrim-Delzell, L. (2018). Research methods for education. Sage Publications.

Roulston, K., deMarrais, K., & Lewis, J. B. (2003). Learning to interview in the social sciences. Qualitative Inquiry, 9(4), 643–668. https://doi.org/10.1177/1077800403252736

Saeed, M. A., & Al Qunayeer, H. S. (2021). Can we engage postgraduates in active research methodology learning? Challenges, strategies and evaluation of learning. International Journal of Research & Method in Education, 44(1), 3-19. https://doi.org/10.1080/1743727X.2020.1728526

Truong, T., (2021). Critical reflections on using interviews - A research method. Academia Letters. (Article 2335), 1–5. https://doi.org/10.20935/al2335

Willis, G. B., (1999). Cognitive interviewing: A “how to” guide. [Short course] Research Triangle Park, NC: Research Triangle Institute. Meeting of the American Statistical Association.




How to Cite

Tate, R., Beauregard, F., Peter, C., & Marotta, L. (2023). Pilot Testing as a Strategy to Develop Interview and Questionnaire Skills for Scholar Practitioners: A Selection of Education Doctorate Students’ Reflective Vignettes . Impacting Education: Journal on Transforming Professional Practice, 8(4), 20–25. https://doi.org/10.5195/ie.2023.333



Themed-Reimagining Research Methods Coursework for the Preparation of Scholar-Practitioners