Document Type : Original Article
Authors
- DANIEL ARDIAN SOESELO 1, 2
- RENNIE YOLANDA 3
- PATRICIA WINONA 3
- YUNISA ASTIARANI 4
- TOAR JEAN MAURICE LALISANG 5
- RETNO ASTI WERDHANI 6
- AGUS PURWADIANTO 7
- WIRSMA ARIF HARAHAP 8
- THEDDEUS OCTAVIANUS HARI PRASETYONO 9, 10
- DIANTHA SOEMANTRI 11
- ARDI FINDYARTINI 11
1 Department of Surgery and Medical Education Unit, School of Medicine and Health Sciences, Atma Jaya Catholic University of Indonesia, Jakarta, Indonesia
2 Doctoral Programme in Medical Sciences, Faculty of Medicine, Universitas Indonesia, Jakarta, Indonesia
3 Department of Surgery, School of Medicine and Health Sciences, Atma Jaya Catholic University of Indonesia, Jakarta, Indonesia
4 Department of Public Health and Nutrition, School of Medicine and Health Sciences, Atma Jaya Catholic University of Indonesia, Jakarta, Indonesia
5 Department of Surgery, Cipto Mangunkusumo Hospital, Faculty of Medicine, Universitas Indonesia, Jakarta, Indonesia
6 Department of Community Medicine, Faculty of Medicine, Universitas Indonesia, Jakarta, Indonesia
7 Department of Forensic Medicine and Medicolegal Studies, Cipto Mangunkusumo Hospital, Faculty of Medicine, Universitas Indonesia, Jakarta, Indonesia
8 Department of Surgery, Faculty of Medicine, Universitas Andalas, Padang, Indonesia
9 Division of Plastic Surgery, Department of Surgery, Cipto Mangunkusumo Hospital, Faculty of Medicine, Universitas Indonesia, Jakarta, Indonesia
10 Indonesian Clinical Training and Education Center (ICTEC), Cipto Mangunkusumo Hospital, Jakarta, Indonesia
11 Department of Medical Education and Medical Education Center, Indonesian Medical Education and Research Institute (IMERI), Faculty of Medicine, Universitas Indonesia, Jakarta, Indonesia
Abstract
Introduction: Assessing professionalism in a valid, objective, and costeffective manner remains a persistent challenge in clinical education. In Indonesia, this issue is particularly relevant in surgical training, where residents must integrate technical competence with professional behaviour in high-stakes, resource-limited settings. This study aimed to develop and evaluate the educational suitability of a Situational Judgement Test (SJT) combined with guided written reflection to assess
and promote professionalism among general surgery residents.
Methods: A mixed-methods sequential design was used. An SJT was developed through blueprinting, item writing, and expert validation, resulting in 26 scenarios across six professionalism domains. Residents (n=64 at baseline; n=44 completers) completed a proctored pre-test SJT, participated in an 8-week guided reflection program (four submissions), and completed an asynchronous post-test SJT. Educational suitability evidence was examined through content validity (CVI) and internal consistency (Kendall’s W). No construct or criterion validity testing was performed. Reflections were scored using Kember’s four-level reflective framework by three independent assessors with Fleiss’ Kappa for inter-rater agreement. Changes in SJT and reflection depth were analysed using paired tests and correlation analysis. Data were analysed
using SPSS version 23.
Results: The mean CVI across items was 0.97 (range 0.90–1.00). The mean SJT score increased significantly from 386.59±19.31 to 420.45±12.56 among completers, with a mean difference of 33.86±16.03 (p<0.001). Reliability improved from Kendall’s W=0.38 to 0.52. Sensitivity analysis comparing baseline pre-test scores between completers (386.6±19.3) and dropouts (391.0±15.5) showed no significant difference (Mann–Whitney U, p=0.258), indicating limited attrition bias. Reflective depth progressed from median level 1 to 4 across four submissions (median Δ=+3, p<0.001), with substantial inter-rater agreement (κ=0.68). There was no significant correlation between Δ reflection and Δ SJT (r=0.059, p=0.703).
Conclusions: Integrating SJT with guided reflection demonstrated strong content validity, internal consistency, and educational benefit, supporting its suitability for formative educational use in surgical professionalism training.
Highlights
Keywords
- Meara JG, Leather AJM, Hagander L, Alkire BC, Alonso N, Ameh EA, et al. Global Surgery 2030: Evidence and solutions for achieving health, welfare, and economic development. The Lancet. 2015;386(9993):569–624.
- Accreditation Council for Graduate Medical Education (ACGME). Surgery Milestones: The Accreditation Council for Graduate Medical Education [Internet]. 2019 [Cited 4 Apr 2019]. Available from: https://www.acgme.org/globalassets/pdfs/milestones/surgerymilestones.pdf.
- Konsil Kedokteran Indonesia. Peraturan Konsil Kedokteran Indonesia No. 73 Tahun 2020 tentang Standar Pendidikan Profesi Dokter Spesialis Bedah [Internet]. 2020 [Cited 2 Jan 2020]. Available from: https://kki.go.id/uploads/cms_file/Peraturan_KKI_No._73_Tahun_2020_tentang_Std_Pendk_Profesi_Dokter_Spesialis_Bedah-_.pdf.
- Soeselo DA, Werdhani RA, Lalisang TJM, Purwadianto A, Harahap WA, Yolanda R, et al. Understanding surgeon professionalism in Indonesia: A qualitative study in a multicultural and resource-limited context. J Adv Med Educ Prof. 2025;13(4):294–302.
- ABIM Foundation, ACP-ASIM Foundation, European Federation of Internal Medicine. Medical professionalism in the new millennium: a physician charter. Ann Intern Med. 2002;136(3):243–6.
- ACS Task Force on Professionalism. Code of professional conduct. J Am Coll Surg. 2004;199(5):734–5.
- Marshall S, Nataraja R. Chapter 29: Patient safety and surgical education. In: Advancing Surgical Education: Theory, Evidence, and Practice. Singapore: Springer; 2019. p. 327–37.
- Alexis DA, Kearney MD, Williams JC, Xu C, Higginbotham EJ, Aysola J. Assessment of perceptions of professionalism among faculty, trainees, staff, and students in a large university-based health system. JAMA Netw Open. 2020;3(11):e2021452.
- Davis CH, DiLalla GA, Perati SR, Lee JS, Oropallo AR, Reyna CR, et al. A review of professionalism in surgery. InBaylor University Medical Center Proceedings. Taylor & Francis. 2024;37(6):993-7.
- Al Ansari A, Al Khalifa K, Al Azzawi M, Al Amer R, Al Sharqi D, Al-Mansoor A, et al. Cross-cultural challenges for assessing medical professionalism among clerkship physicians in a Middle Eastern country (Bahrain): Feasibility and psychometric properties of multisource feedback. Adv Med Educ Pract. 2015;6:509–15.
- Cullen MJ, Zhang C, Sackett PR, Thakker K, Young JQ. Can a situational judgment test identify trainees at risk of professionalism issues? A multi-institutional, prospective cohort study. Acad Med J Assoc Am Med Coll. 2022;97(10):1494–503.
- Al Hashmi W, Klassen RM. Developing a situational judgement test for admission into initial teacher education in Oman: An exploratory study. Int J Sch Educ Psychol. 2020;8(sup1):187–98.
- Goss BD, Ryan AT, Waring J, Judd T, Chiavaroli NG, O’Brien RC, et al. Beyond selection: The use of situational judgement tests in the teaching and assessment of professionalism. Acad Med J Assoc Am Med Coll. 2017;92(6):780–4.
- Sandars J. The use of reflection in medical education: AMEE Guide No. 44. Med Teach. 2009;31(8):685–95.
- Boutet I, Vandette MP, Valiquette-Tessier SC. Evaluating the implementation and effectiveness of reflection writing. Can J Scholarsh Teach Learn. 2017;8(1):1-18.
- Soleimani-Nouri P, Entwistle OH, Fehervari M, Spalding D. A novel framework for surgical reflection. Annals of Laparoscopic and Endoscopic Surgery. 2023;8:1.
- Poola VP, Suh B, Parr T, Boehler M, Han H, Mellinger J. Medical students’ reflections on surgical educators’ professionalism: Contextual nuances in the hidden curriculum. Am J Surg. 2021;221(2):270–6.
- Purwadianto A, SpS S, Gunawan S, Budiningsih Y, Prawiroharjo P, Firmansyah A. Kode etik kedokteran Indonesia. Jakarta: Pengurus Besar Ikatan Dokter Indonesia; 2012.
- Sorrel MA, Olea J, Abad FJ, de la Torre J, Aguado D, Lievens F. Validity and reliability of situational judgement test scores: A new approach based on cognitive diagnosis models. Organ Res Methods. 2016;19(3):506–32.
- Whetzel DL, Sullivan TS, McCloy RA. Situational judgment tests: An overview of development practices and psychometric characteristics. Personnel Assessment and Decisions. 2020;6(1):1.
- Kember D, McKay J, Sinclair K, Wong FKY. A four‐category scheme for coding and assessing the level of reflection in written work. Assess Eval High Educ. 2008;33(4):369–79.
- Gardner AK, Dunkin BJ. Evaluation of validity evidence for personality, emotional intelligence, and situational judgment tests to identify successful residents. JAMA Surg. 2018;153(5):409–16.
- Smith KJ, Neely S, Dennis VC, Miller MM, Medina MS. Use of situational judgment tests to teach empathy, assertiveness, communication, and ethics. Am J Pharm Educ. 2022;86(6):8761.
- Patterson F, Zibarras L, Ashworth V. Situational judgement tests in medical education and training: Research, theory and practice: AMEE Guide No. 100. Med Teach. 2016;38(1):3–17.
- Peshkepija AN, Basson MD, Davis AT, Ali M, Haan PS, Gupta RN, et al. Perioperative self-reflection among surgical residents. Am J Surg. 2017;214(3):564–70.
- Ganni S, Botden SMBI, Schaap DP, Verhoeven BH, Goossens RHM, Jakimowicz JJ. “Reflection-Before-Practice” improves self-assessment and end-performance in laparoscopic surgical skills training. J Surg Educ. 2018;75(2):527–33.
- Aziz A, Mahboob U, Saleem T. Benefits of reflective writing in health care through the vivid lens of house officers. MedEdPublish. 2020;9:60.
)