Development of a Rubric for Deep and Interactive Learning in the Pharmacy Virtual Simulation Environment for College Students
Main Article Content
Abstract
Pharmacy virtual simulation experiments have become a technical means in higher education to simulate the real world and help students acquire skills or experience. They help students understand complex concepts more deeply through visualisation and interactive operations and enhance skills in a safe, controlled setting. However, limited realism, limited scenarios and sensory experiences, the lack of face-to-face human interaction, a focus on technique or procedure rather than soft skills, and virtual simulation cannot automatically trigger deep, interactive learning. This study aimed to develop a rubric to measure students’ deep, interactive learning in the pharmacy virtual simulation environment, aligned with the requirements of future pharmacists. It involved a multi-phase process to ensure the rubric’s content and construct validity, integrating the literature review, expert review, students’ feedback and statistical analysis using the Rasch measurement model. The participants were second-year pharmacy students from a Chinese medical university. The rubric had 10 dimensions, each with 4 levels. It provided a clear set of task performance criteria and had good usability and administrability for college students’ self-assessments. The Rasch analysis showed that the rubric was unidimensional, indicating that the rubric items functioned together to measure a single underlying construct, integrating deep and interactive learning in a pharmacy virtual simulation environment. This rubric empowered students to objectively self-assess complex learning performance and encouraged them to engage deeply with the learning material. It addressed current assessment needs and laid the groundwork for future innovations in educational assessment.
Article Details
This work is licensed under a Creative Commons Attribution 4.0 International License.
References
Potkonjak V, Gardner M, Callaghan V, Mattila P, Guetl C, Petrović VM, et al. Virtual laboratories for education in science, technology, and engineering: a review. Comput Educ. 2016;95:309–27. https://doi.org/10.1016/j.compedu.2016.02.002
Lateef F. Simulation-based learning: just like the real thing. J Emerg Trauma Shock. 2010;3(4):348–52. https://doi.org/10.4103/0974-2700.70743
Qorbani S, Dalili S, Arya A, Joslin C. Assessing learning in an immersive virtual reality: a curriculum-based experiment in chemistry education. Educ Sci. 2024;14(5):476. https://doi.org/10.3390/educsci14050476
Zheng K, Shen Z, Chen Z, Che C, Zhu H. Application of AI-empowered scenario-based simulation teaching mode in cardiovascular disease education. BMC Med Educ. 2024;24(1):1003. https://doi.org/10.1186/s12909-024-05977-z
Boone WJ, Noltemeyer A. Rasch analysis: a primer for school psychology researchers and practitioners. Cogent Educ. 2017;4(1):1416898. https://doi.org/10.1080/2331186X.2017.1416898.
Kim J, Kim K, Kim W. Impact of immersive virtual reality content using 360-degree videos in undergraduate education. IEEE Trans Learn Technol. 2022;15(1):137–49. https://doi.org/10.1109/tlt.2022.3157250
Casselman MD, Eichler JF, Atit K. Advancing multimedia learning for science: comparing the effect of virtual versus physical models on student learning about stereochemistry. Sci Educ. 2021;105(6):1285–314. https://doi.org/10.1002/sce.21675
Serrano-Perez JJ, González-García L, Flacco N, Taberner-Cortés A, García-Arnandis I, Pérez-López G, et al. Traditional vs. virtual laboratories in health sciences education. J Biol Educ. 2023;57(1):36–50. https://doi.org/10.1080/00219266.2021.1877776
Bridge P, Trapp JV, Kastanis L, Pack D, Parker JC. A virtual environment for medical radiation collaborative learning. Australas Phys Eng Sci Med. 2015;38(2):369–74. https://doi.org/10.1007/s13246-015-0348-3
Zou Y, Yuan M, Mo L, Mustakim SSB. Enhancing teaching and learning through assessment strategies: a practical guide. IIJARBSS. 2024;14(7):1024–36. https://doi.org/10.6007/ijarbss/v14-i7/21928
Bolt DM, Adams DJ. Exploring rubric-related multidimensionality in polytomously scored test items. Appl Psychol Meas. 2017;41(3):163–77. https://doi.org/10.1177/0146621616677715
Nayahangan LJ, Konge L, Park C, Dubrowski A, Yudkowsky R. Development of a rubric to evaluate implementation quality of simulation-based courses: a consensus study. Simul Healthc. 2023;18(6):351–8. https://doi.org/10.1097/sih.0000000000000684
Luctkar-Flude M, Tregunno D, Sears K, Pulling C, Lee K, Egan R. Reliability and validity of scenario-specific versus generic simulation assessment rubrics. J Nurs Educ Pract. 2020;10(8):74–8. https://doi.org/10.5430/jnep.v10n8p74
Barnett SG, Porter AL, Allen SM, Bastianelli KMS, Chen JS, Kachlic MD, et al. Expert consensus to finalize a universal evaluator rubric to assess pharmacy students’ patient communication skills. Am J Pharm Educ. 2020;84(12):848016. https://doi.org/10.5688/ajpe848016
Baumgartner L, Ip EJ, Sasaki-Hill D, Wong T, Israel H, Barnett MJ. Implementation of mock acute care advance pharmacy practice experience simulations and an assessment rubric. Am J Pharm Educ. 2019;83(9):7331. https://doi.org/10.5688/ajpe7331
Schroeder MN, Lengel AJ. Evaluation of a rubric to assess pharmacy student performance in a telehealth simulation exercise. Am J Pharm Educ. 2022;86(8):ajpe8778. https://doi.org/10.5688/ajpe8778
Garling KA, Wong B. An initial reliability analysis of a patient counseling rubric to objectively measure student pharmacist performance. Heliyon. 2023;9(5):e15768. https://doi.org/10.1016/j.heliyon.2023.e15768
Marton F, Säljö R. On qualitative differences in learning: I—Outcome and process. Br J Educ Psychol. 1976;46(1):4–11. https://doi.org/10.1111/j.2044-8279.1976.tb02980.x
Bloom BS, Engelhart MD, Furst EJ, Hill WH, Krathwohl DR. Taxonomy of educational objectives: the classification of educational goals. Handbook 1: Cognitive domain. New York: David Mckay Company, Inc.; 1956.
Anderson LW, Krathwohl DR, editors. A taxonomy for learning, teaching, and assessing: a revision of Bloom’s taxonomy of educational objectives. Complete edition. London: Longman, Inc.; 2001.
Thompson AR, Lake LPO. Relationship between learning approach, Bloom’s taxonomy, and student performance in an undergraduate human anatomy course. Adv Health Sci Educ Theory Pract. 2023;28(4):1115-30. https://doi.org/10.1007/s10459-023-10208-z
Sawyer RK. The new science of learning. The Cambridge handbook of the learning sciences. Cambridge: Cambridge University Press; 2006. https://doi.org/10.1017/cbo9780511816833.002
National Research Council. Education for life and work: developing transferable knowledge and skills in the 21st century. Washington, DC: The National Academies Press; 2012.
Alliance for Excellent Education. Demography as destiny: how America can build a better future [Internet]. Washington, DC: Alliance for Excellent Education; 2006 [cited 2026 Sep 20]. Available from: https://www.adlit.org/topics/policy-legislation-initiatives/demography-destiny-howamerica-can-build-better-future
Hopfenbeck TN, Lenkeit J, El Masri Y, Cantrell K, Ryan J, Baird J-A. Lessons learned from PISA: a systematic review of peer-reviewed articles on the programme for international student assessment. Scand J Educ Res. 2018;62(3):333–53. https://doi.org/10.1080/00313831.2016.1258726
Panadero E, Alonso-Tapia J, García-Pérez D, Fraile J, Galán JMS, Pardo R. Deep learning self regulation strategies: validation of a situational model and its questionnaire. Rev Psicodidáct. 2021;26(1):10–19. https://doi.org/10.1016/j.psicoe.2020.11.003
Means B, Toyama Y, Murphy R, Bakia M, Jones K. Evaluation of evidence-based practices in online learning: a meta-analysis and review of online learning studies. Washington DC: U.S. Department of Education, Office of Planning, Evaluation, and Policy Development Policy and Program Studies Service; 2009.
Laird TFN, Shoup R, Kuh GD. Measuring deep approaches to learning using the National Survey of Student Engagement. Annual Meeting of the Association for Institutional Research; 2005 May 14–18; Chicago, IL: Association for Institutional Research; 2006.
Panadero E, Romero M. To rubric or not to rubric? The effects of self-assessment on self regulation, performance and self-efficacy. Assess Educ Princ Policy Pract. 2014;21(2):133–48. https://doi.org/10.1080/0969594x.2013.877872
Li Z, Li S, Wang Y, Zhang C. Design and verification of college students’ deep learning evaluation scale under embodied cognition environment. E-Educ Res. 2020;41(12):92–8. https://doi.org/10.13811/j.cnki.eer.2020.12.013
Wang MK, Wang Z, Chen ZZ. Deep learning evaluation: theoretical models, related technologies and practical cases. J Guangxi Norm Univ. 2023;59(06):66–75.
Zhu LM. Construction of deep learning evaluation indicator system under the concept of STEM education. J Sichuan Normal Univ. 2022;49(4):125–33.
Masuku MM, Jili NN, Sabela PT. Assessment as a pedagogy and measuring tool in promoting deep learning in institutions of higher learning. Int J High Educ. 2021;10(2):274–83. https://doi.org/10.5430/ijhe.v10n2p274
Stupans I, March G, Owen SM. Enhancing learning in clinical placements: reflective practice, self-assessment, rubrics and scaffolding. Assess Eval High Educ. 2013;38(5):507–19. https://doi.org/10.1080/02602938.2012.658017
Er HM, Ming MKJ, Keng PS, Nadarajah VD. Pharmacy students’ perceptions of reflective portfolios and their effect on students’ deep information-processing skills. Am J Pharm Educ. 2019;83(6):6851. https://doi.org/10.5688/ajpe6851
Vygotsky LS, Cole M. Mind in society: development of higher psychological processes. Harvard University Press; 1978.
Simpson RJ, Galbo JJ. Interaction and learning: theorizing on the art of teaching. Interchange. 1986;17(4):37–51. https://doi.org/10.1007/bf01807015
Wagner ED. In support of a functional definition of interaction. Am J Distance Educ. 1994;8(2):6–29. https://doi.org/10.1080/08923649409526852
Moore MG. Three types of interaction. Am J Distance Educ. 1989;3(2):1–7. https://doi.org/10.1080/08923648909526659
Hillman DCA, Willis DJ, Gunawardena CN. Learner‐interface interaction in distance education: an extension of contemporary models and strategies for practitioners. Am J Distance Educ. 1994;8(2):30–42. https://doi.org/10.1080/08923649409526853
Sabry K, Baldwin L. Web‐based learning interaction and learning styles. Br J Educ Technol. 2003;34(4):443–54. https://doi.org/10.1111/1467-8535.00341
Holden JT, Westfall PJL. Instructional media selection for distance learning: a learning environment approach. Distance Learn. 2006;3(2):1–12. https://doi.org/10.1108/DL-06-2006-0001
Ashwin P. Conceptualising teaching and learning interactions in researching higher education. In: Tight M, Mok KH, Huisman J, Morphew C, editors. The Routledge international handbook of higher education. New York: Routledge; 2009. p. 37–46. https://doi.org/10.4324/9780203882221-4
Nunn R. Designing rating scales for small-group interaction. ELT J. 2000;54(2):169–78. https://doi.org/10.1093/elt/54.2.169
Pearce G, Lee G. Viva voce (oral examination) as an assessment method: insights from marketing students. J Mark Educ. 2009;31(2):120–30. https://doi.org/10.1177/0273475309334050
Tuma F. The use of educational technology for interactive teaching in lectures. Ann Med Surg. 2021;62:231–5. https://doi.org/10.1016/j.amsu.2021.01.051
Koster AS, Vermunt JD. Longitudinal changes of deep and surface learning in a constructivist pharmacy curriculum. Pharmacy. 2020;8(4):200. https://doi.org/10.3390/pharmacy8040200
Andrade HL, Brookhart SM, Yu EC. Classroom assessment as co-regulated learning: a systematic review. Front Educ. 2021;6:751168. https://doi.org/10.3389/feduc.2021.751168
Chang C-L. Effect of immersive virtual reality on post-baccalaureate nursing students’ indwelling urinary catheter skill and learning satisfaction. Healthcare. 2022;10(8):1473. https://doi.org/10.3390/healthcare10081473
Issleib M, Kromer A, Pinnschmidt HO, Süss-Havemann C, Kubitz JC. Virtual reality as a teaching method for resuscitation training in undergraduate first-year medical students: a randomized controlled trial. Scand J Trauma Resusc Emerg Med. 2021;29:27. https://doi.org/10.1186/s13049-021-00836-y
Lo K-W, Yang B-H. Development and learning efficacy of a simulation rubric in childhood pneumonia for nursing students: a mixed methods study. Nurse Educ Today. 2022;119:105544. https://doi.org/10.1016/j.nedt.2022.105544
Rouleau G, Pelletier J, Côté J, Gagnon M-P, Martel-Laferrière V, Lévesque R, et al. Codeveloping a virtual patient simulation to foster nurses’ relational skills consistent with motivational interviewing: a situation of antiretroviral therapy nonadherence. J Med Internet Res. 2020;22(7):e18225. https://doi.org/10.2196/18225
Lee E, Mabry JL, Roberts T, Davis KM. Development of the simulation-positive learning evaluation (SIM-PLE) rubric for use in medical-surgical nursing high fidelity simulations. Nurse Educ Pract. 2021;51:102992. https://doi.org/10.1016/j.nepr.2021.102992
Ma C-W, Cheng P-S, Chan Y-S, Tipoe GL. Virtual reality: a technology to promote active learning of physiology for students across multiple disciplines. Adv Physiol Educ. 2023;47(3):594–603. https://doi.org/10.1152/advan.00172.2022
Karaduman GS, Basak T. Is virtual patient simulation superior to human patient simulation: a randomized controlled study. Comput Inform Nurs. 2023;41(6):467–76. https://doi.org/10.1097/CIN.0000000000000957
Helle N, Vikman MD, Dahl-Michelsen T, Lie SS. Health care and social work students’ experiences with a virtual reality simulation learning activity: qualitative study. JMIR Med Educ. 2023;9:e49372. https://doi.org/10.2196/49372
Wong W-K, Chen K-P, Chang H-M. A comparison of a virtual lab and a microcomputer-based lab for scientific modelling by college students. J Balt Sci Educ. 2020;19(1):157–73. https://doi.org/10.33225/jbse/20.19.157
Makarova I, Mustafina J, Boyko A, Fatikhova L, Parsin G, Buyvol P, et al. A virtual reality lab for automotive service specialists: a knowledge transfer system in the digital age. Information. 2023;14(3):163. https://doi.org/10.3390/info14030163
De Lorenzis F, Visconti A, Restivo S, Mazzini F, Esposito S, Garofalo SF, et al. Combining virtual reality with asymmetric collaborative learning: a case study in chemistry education. Smart Learn Environ. 2024;11(1):43. https://doi.org/10.1186/s40561-024-00331-8
Jallad ST, Natsheh I, Helo LA, Ibdah DM, Salah A, Muhsen R, et al. Nursing student’s perceptions, satisfaction, and knowledge toward utilizing immersive virtual reality application in human anatomy course: quasi-experimental. BMC Nurs. 2024;23(1):601. https://doi.org/10.1186/s12912-024-02254-8
Tokatlidis C, Rapti S, Tselegkaridis S, Sapounidis T, Papakostas D. Virtual environment in engineering education: the role of guidance, knowledge and skills development in electronic circuits teaching. Educ Sci. 2024;14(12):1336. https://doi.org/10.3390/educsci14121336
Chircop A, Cobbett S, Schofield RE, Boudreau C, Egert A, Filice S, Harvey A, et al. Multijurisdictional evaluation of Sentinel City virtual simulation for community health nursing clinical education. QANE-AFI. 2022;8(4):2. https://doi.org/10.17483/2368-6669.1352
Kolil VK, Achuthan K. Virtual labs in chemistry education: a novel approach for increasing student’s laboratory educational consciousness and skills. Educ Inf Technol. 2024;29(18):25307–31. https://doi.org/10.1007/s10639-024-12858-x
Barnett SG, Porter AL, Allen SM, Bastianelli KMS, Chen JS, Kachlic MD, et al. Expert consensus to finalize a universal evaluator rubric to assess pharmacy students’ patient communication skills. Am J Pharm Educ. 2020;84(12):848016. https://doi.org/10.5688/ajpe848016
Garling KA, Wong B. An initial reliability analysis of a patient counselling rubric to objectively measure student pharmacist performance. Heliyon. 2023;9(5):e15768. https://doi.org/10.1016/j.heliyon.2023.e15768
Brookhart SM. How to create and use rubrics for formative assessment and grading. ASCD; 2013.
Fernández-Gómez E, Martín-Salvador A, Luque-Vara T, Sánchez-Ojeda MA, Navarro-Prado S, Enrique-Mirón C. Content validation through expert judgement of an instrument on the nutritional knowledge, beliefs, and habits of pregnant women. Nutrients. 2020;12(4):1136. https://doi.org/10.3390/nu12041136
Zamanzadeh V, Ghahramanian A, Rassouli M, Abbaszadeh A, Alavi-Majd H, Nikanfar A-R. Design and implementation content validity study: development of an instrument for measuring patientcentered communication. J Caring Sci. 2015;4(2):165–78. https://doi.org/10.15171/jcs.2015.017
In J. Introduction of a pilot study. Korean J Anesthesiol. 2017;70(6):601–5. https://doi.org/10.4097/kjae.2017.70.6.601
Prasad GNR. Evaluating student performance based on bloom’s taxonomy levels. J Phys: Conf Ser. 2021;1797(1):012063. https://doi.org/10.1088/1742-6596/1797/1/012063
Brentani E, Golia S. Unidimensionality in the Rasch model: how to detect and interpret. Statistica. 2007;67(3):253–61. https://doi.org/10.6092/issn.1973-2201/3508
Ishak AH, Osman MR, Mahaiyadin MH, Tumiran MA, Anas N. Examining unidimensionality of psychometric properties via Rasch model. IJCIET. 2018;9(9):1462–7.
Yeoh TT, Foo KM, Koh SK, Quek K, Yew KW, Fenwick E, et al. Development of a pharmacy educational environment questionnaire. Curr Pharm Teach Learn. 2023;15(12):993–1005. https://doi.org/10.1016/j.cptl.2023.10.010
Smith AB, Rush R, Fallowfield LJ, Velikova G, Sharpe M. Rasch fit statistics and sample size considerations for polytomous data. BMC Med Res Methodol. 2008;8:33. https://doi.org/10.1186/1471-2288-8-33
Ravand H, Firoozi T. Examining construct validity of the master’s UEE using the Rasch model and the six aspects of the Messick’s framework. Int J Lang Test. 2016;6(1):1–23.
Arku D, Almatruk Z, Warholak T, Axon DR. Evaluating the reliability and validity of a questionnaire used to measure experiences of teamwork among student pharmacists in a quality improvement course. Curr Pharm Teach Learn. 2022;14(5):552–60. https://doi.org/10.1016/j.cptl.2022.04.015
Fisher WP. Rating scale instrument quality criteria. RMT. 2007;21(1):1095.
Newsom LC, Augustine J, Momary K. Development of a script concordance test to assess clinical reasoning in a pharmacy curriculum. Curr Pharm Teach Learn. 2022;14(9):1135–42. https://doi.org/10.1016/j.cptl.2022.07.028
Boone WJ. Rasch analysis for instrument development: why, when, and how? CBE—Life Sci Educ. 2016;15(4):1–7. https://doi.org/10.1187/cbe.16-04-0148
Kean J, Brodke DS, Biber J, Gross P. An introduction to item response theory and Rasch analysis: application using the eating assessment tool (EAT-10). Brain Impair. 2018;19(Spec Iss 1):91–102. https://doi.org/10.1017/brimp.2017.31
Yang Y, Peng Y, Li W, Lu S, Wang C, Chen S, et al. Psychometric evaluation of the academic involution scale for college students in China: an application of Rasch analysis. Front Psychol. 2023;14:1135658. https://doi.org/10.3389/fpsyg.2023.1135658
Goswami S, Barnard M, Bynum LA, Thompson S, Kang M. Psychometric evaluation of the Health Care Provider HIV/AIDS Stigma Scale (HPASS) using Rasch analysis. Curr Pharm Teach Learn. 2022;14(9):1143–53. https://doi.org/10.1016/j.cptl.2022.07.035
Saseen JJ, Ripley TL, Bondi D, Burke JM, Cohen LJ, McBane S, et al. ACCP clinical pharmacist competencies. Pharmacotherapy. 2017;37(5):630–6. https://doi.org/10.1002/phar.1923
Kumar S, Bhoi P, Gharat MS, Mohanta GP. Developing a competency assessment framework for pharmacists in primary health care settings in India. PLoS One. 2025;20(3):e0316646. https://doi.org/10.1371/journal.pone.0316646
Potgieter ML, Filmalter C, Maree C. Teaching, learning and assessment of the affective domain of undergraduate students: a scoping review. Nurse Educ Pract. 2025;86:104417. https://doi.org/10.1016/j.nepr.2025.104417
Galdonez DP. Exploring the experiences of STEM students in writing research: insights, challenges, and strategies. J Basic Educ Res. 2025;6(1):9–16. https://doi.org/10.37251/jber.v6i1.1150