Introduction

The evolution of medical education demands that medical students acquire and demonstrate competence in procedural skills—a vital step toward delivering quality patient care and preparing future healthcare professionals for clinical excellence ( 1 ). Studies have shown a consensus among clinical teachers that, upon graduation, medical students should possess proficiency in basic procedural skills ( 2 - 5 ). Globally, governing bodies of medical education, including the National Medical Commission of India (NMC), have adopted a competency-based approach to teaching procedural skills ( 6 - 8 ).

However, a notable gap exists in documenting which skills are taught, the level of competency achieved, and the performance of these skills, as indicated by Gregory et al. in their survey study ( 2 ). LeAnn Coberly et al. discovered that fourth year students generally do not follow fundamental procedures during internship rotations ( 9 ), and variability in intern competency in procedural skills is a common issue ( 10 ).

Given the importance of procedural skills, comprehensive training ( 11 ), regular practice ( 12 ), skill assessment, and certification ( 13 - 16 ) are essential prerequisites for students. This preparation is vital, as the competency of clinicians in performing procedural skills significantly impacts patient outcomes ( 6 - 8 ). Challenges in providing hands-on experience have arisen due to shorter hospitalizations and the delegation of certain skills to other healthcare professionals ( 14 , 15 ). Additionally, certifying competency criteria pose difficulties, and traditional teaching methods have shown limited effectiveness ( 9 , 17 ).

Presently, to establish uniform standards and capabilities among medical students, simulation-based teaching has been implemented and found to be effective ( 11 , 18 ). Gregory et al.'s survey established a consensus that clerkship directors believe third-year medical students should be adequately prepared in desired procedural skills during their clinical postings to ensure readiness for fourth-year internships ( 2 ).

While both the national and international medical education governing bodies has listed the procedural skills to be taught, selecting these essential skills from a vast pool of recommended procedures remains a complex and multifaceted task, particularly when teaching third-year medical students during their surgical clinical postings ( 7 - 9 , 10 , 18 , 19 ).

Therefore, this Delphi study aims to address this challenge by connecting the collective understanding and consensus of medical educators and practitioners. It seeks to compile a list of skills that third-year students should learn and practice during their clinical postings using the consensus-building approach of the Delphi method ( 20 - 22 ). The Delphi method, a well-established research technique ( 23 - 28 ), enables the systematic generation of expert consensus through iterative rounds of feedback and evaluation. Importantly, this study utilizes a modified Delphi technique by incorporating a pre-recommended item list ( 26 , 27 ).

Methods

The present study utilized the modified Delphi technique, an established iterative survey method, to gather anonymous consensus from education academics and healthcare clinicians in the field of medical education. This mixed research approach combines both qualitative and quantitative methods to collect opinions and survey data ( 22 , 24 ). The primary objective of the Delphi study was to identify essential procedural skills from the NMC curriculum 2019 that should be taught and assessed during the clinical postings of third-year medical students. This study is considered modified as it diverges from the traditional Delphi process in that the initial list shared with the experts was the pre-prepared NMC prescribed list. Contrary to the standard Delphi method where experts generate items themselves, in this modified approach, experts were asked to assess and prioritize skills from the existing NMC curriculum list ( 25 , 27 ). They were requested to rank these skills in order of importance, selecting those that received at least 80% of votes. After three consecutive rounds of assessment, the final list of skills was determined and agreed upon by the experts, thereby shaping the study's methodology through this modified approach.

For the Delphi expert panel, a purposive sampling approach was utilized to select participants capable of providing valuable insights into identifying and teaching essential skills to medical students during their clerkship posting. The study invited clinical supervisors and senior faculty members from departments such as medicine, surgery, emergency medicine, obstetrics and gynaecology, and dermatology. The panel consisted of 30 experts, aligning with the recommended panel size in the literature, which suggests a range between 10 and 50 experts ( 24 , 28 ). The inclusion criterion was a minimum of ten years of experience in teaching undergraduate medical students at the researcher's institute. Participants were contacted via email, provided with information about the Modified Delphi study, and the inclusion criteria. The email contained a Google form with 2 Sessions: Session 1 collected demographic data, while Session 2 included an electronic informed consent form for participation in the study.

Data Collection

The survey was facilitated using the QuestionPro application free version, a web-based survey tool. Each expert in the panel received a link to access the survey rounds. The survey included the research question and aim of the study on the first web page, followed by the survey for ranking the skill list and a comparative ranking survey report from the previous round. A free text field allowed for general comments and suggestions. In case of non-responsive experts after two weeks, a follow-up email reminder was sent.

Round 1: Experts were presented with the Competency-based undergraduate curriculum 2019 for the Indian Medical Graduate, comprising 54 prescribed skills. They were asked to rank the skills based on their importance, specifically focusing on the most essential procedural skill to be taught during the clerkship posting. A comprehensive outline of the modified Delphi process is depicted in (Figure 1).

Figure 1. Flow chart of the Delphi process

Round 2: In the second round, experts were allowed to clarify and re-rank the skill list, considering the report from the first Delphi round. The same rating and analysis procedure was followed.

Round 3: For the third round, experts were invited to review a detailed report on the ranking of skills from the second round. They were informed that the expert panel attempted to produce the final skill list and were asked for their agreement. If there were any disagreements, an additional survey round would be conducted.

Data Analysis

The data analysis involved using descriptive statistics to describe the demographic characteristics of the experts and their responses in each round. Frequency data was calculated for each item throughout the rounds.

To determine the indicators for selection in the next round and provide quantitative feedback, descriptive statistics such as mean was calculated. SPSS statistical software (version 19, Chicago, IL) was used to analyse the percentage of agreement, range of ratings (interquartile ratings) and mean.

In line with previous studies, a consensus threshold of 80% agreement among the experts was set to define consensus. This approach ensured a robust level of agreement in determining consensus among the expert panel.

Ethical Consideration

Prior to the study, all participating faculties were provided with informed consent and the option to withdraw at any point. Survey responses were treated with confidentiality and anonymity, exclusively used for research. Faculties were informed in advance that the anticipated survey completion time would be 15 minutes, with some uncertainties regarding the specific details of rounds and time frame. Ethical clearance (JIP/IEC/2021/054) was obtained from the institute on December 15, 2021, and participant recruitment commenced in January 2022.

Results

Participation in the expert’s panel

The Delphi round of the study took place from December 2021 to May 2022, and a total of 22 experts out of the 30 who were approached agreed to participate. The expert panel was diverse, comprising 10 pre-clinical educators and 12 clinicians who were experts in their respective fields. Notably, 19 of the experts had significant teaching experience, averaging 9 years or more, with some having over 10 years of experience.

Importantly, the 22 experts who provided their consent and actively participated in the first round continued their involvement in subsequent rounds, ensuring consistency and reliability throughout the study. Their consistent engagement allowed for valuable insights and opinions to be gathered throughout the iterative process.

Overall, the composition of the expert panel, their extensive teaching experience, practical expertise, and continued participation throughout the study greatly contributed to the depth and credibility of the findings. The demographic characteristics of the expert panel who participated in the study are presented in Table 1.

Round 1

During the first round of the study, all 22 experts received a questionnaire containing a list of 54 procedural skills outlined in the NMC 2019 curriculum.

In round 1, two procedural skills achieved 100% agreement among the experts, three skills achieved 90-99% agreement, and four skills achieved 81-89% agreement. Only four skills reached a minimum agreement of 80%. Based on scoring stability, these 13 skills with 80% agreement or higher were included in round 2.

Round 2

In the second round of our study, a panel of experts was presented with a set of 13 procedural skills for assessment. From this selection, experts collectively recognized six procedural skills as prime for medical students in achieving competence. These skills achieved agreement levels of 80% or higher among the panel. However, the remaining seven procedural skills failed to attain the requisite level of consensus among the experts. This discrepancy indicated a lack of agreement among the experts regarding the relative importance of these skills.

Among the seven skills that lacked consensus, six demonstrated comparatively high agreement, with consensus levels ranging from 91% to 100%. It is worth noting that the skill of bandaging achieved a moderate agreement level of 59%, while the application of splints and slings exhibited notably lower consensus at 27%, suggesting discord among the experts regarding their significance. The consensus levels for the remaining seven skills that did not attain consensus were consistently below 80%.

Notably, it was determined that an additional list of procedural skills would not be submitted for further iteration. Instead, a summary of these findings was forwarded to the experts for their insights and suggestions in the forthcoming round 3 of the study.

Round 3

In round 3, each expert received an individual summary of the findings and was asked for additional feedback and suggestions. Encouragingly, full agreement was reached among the experts for all six procedures that were identified as essential for teaching (Table 2).

Based on this consensus, the six procedural skills was selected for the development of the training module. This marks a significant step forward in the study, as the identified skills will now be integrated into the curriculum to enhance the education and training of medical skills which is one of the objective of the research protocol developed by the authors. Additionally, interclass correlation analysis among the experts revealed a substantial level of agreement (ICC=0.767), indicating high reliability in the assessments or variables considered.

Discussion

The field of medical education is constantly evolving, and it is crucial to align skill requirements with the needs of medical graduates. This ensures that medical students are adequately prepared for their future careers and enhances patient care and safety. In India, the National Medical Council ( 6 ) introduced a competency-based medical education curriculum in 2019, which listed 54 skills that Indian medical graduates need to be taught and certified. However, the curriculum does not specify the essential procedural skills to be taught during the clerkship posting.

The study aimed to determine the essential procedural skills for third-year medical students during their clerkship posting. To achieve this, a modified Delphi method was used, involving three rounds of expert consensus. The involvement of clinician experts was important as it brought together their teaching experience and direct engagement in clerkship activities as supervisors. This combination of teaching and practical expertise enhanced the credibility and applicability of their perspectives. The experts' extensive teaching experience played a vital role in shaping the research outcomes, ensuring the identified procedural skills were relevant and credible. Their expertise also provided valuable recommendations for improving medical education and training.

Through this process, six procedural skills were identified as the most essential for medical students. These skills guide clinical supervisors in teaching and certifying student skill competency.

The six identified skills represent the most basic therapeutic procedural skills that Indian medical graduates are expected to competently perform. They are considered must-learn procedures during the clerkship, as they are crucial in surgical and emergency medicine postings. By focusing on specific skills, the study addresses the limited time available for supervised practice in these areas.

The research findings align with the recommended skill lists of the UK General Medical Council and the US and Canadian clerkship Directors of Internal Medicine (CDIM) ( 7 , 28 ). There is consistency in the inclusion of four procedural skills and one interpretative skill in all three recommendations. This underscores the importance of these skills in medical education and highlights their relevance for medical students during their clerkship.

However, there are also differences in the findings. This study found Basic Life Support (BLS) as an essential procedural skill with 100% agreement among the experts. This aligns with the recommendations of the US and Canadian clerkship Directors of Internal Medicine and the Australian Core Procedural Skill List, ( 6 - 8 ) which recognized the importance of BLS training. In contrast, the UK GMC graduate outcome list of practical skills not include BLS as a specified procedural skill.

Another distinction is the absence of interpretative skills in the study findings. While interpretative skills are emphasized in the recommended skill lists of the UK GMC and CDIM, the study focused specifically on identifying essential procedural skills. Interpretative skills may be covered in other components of the medical curriculum or developed through other educational activities.

The findings contribute to understanding the specific procedural skills that should be prioritized and emphasized during the clerkship period. This allows for a more targeted approach to teaching and assessing procedural competencies, ensuring that students develop the necessary skills to provide effective patient care ( 30 - 32 ).

It is important to consider the unique context of medical education in India. The NMC has introduced a competency-based curriculum that specifies 54 skills to be taught and certified for Indian medical graduates. Comparing the identified procedural skills in this study with the requirements of the Indian medical education system would provide insights into the essential skills in the Indian context and ensure that medical graduates are well-prepared for their future practice ( 6 ).

By examining the similarities and differences between the findings of studies conducted in different countries, a comprehensive understanding of the evolving skill requirements in medical education can be gained. This can help tailor educational approaches and ensure that medical students receive the necessary training to meet the demands of their future careers ( 24 ).

The identified skill list from this study can serve as a valuable resource for medical science teachers and institutes planning to incorporate simulation-based skill training into their curriculum. By including procedural skills in addition to the certifiable skills recommended by the NMC, teachers and institutes can enhance the focus on practical skills development.

Structured training modules based on the essential skill list can be developed, providing faculty with a uniform curriculum for teaching clinical skills. These modules can be designed to be self-contained, comprehensive, and flexible, enabling effective simulation ( 18 - 19 , 30 ).

Limitations

Although the modified Delphi technique used in this study was conducted in a tertiary teaching hospital, its findings are only applicable to similar institutions and may not be generalizable to other settings.

The list of 13 skills identified in the first round of this study only includes those that can be performed independently on a manikin, which may have limited the inclusion of additional skills that are important for medical students to learn.

This study focused specifically on third-year medical students, and while the identified skills are appropriate for pre-clinical and clinical phases of medical education, future research may need to address the skill needs of other medical specialties.

Conclusion

This study provides an important first step in identifying essential skills for medical students and demonstrates the feasibility of using a modified Delphi technique to establish consensus for developing specific skill training modules. Further research is needed to validate and generalize the identified skills for simulation-based training across a wider range of institutions and medical specialties. Medical schools must identify essential skills and develop structured simulation-based skill modules to ensure the competency of all graduating medical students. This study provides a starting point for this important process.

Acknowledgment

The authors would like to thank the JIPMER administration for funding the project JIP/Res/Intramural/Phs-3/2022-23 and the Department of Medical Education, JIPMER for its support.

Authors’ Contributions

S.M.K and Z.Z conceived and designed the study. All authors contributed to the discussion, read and approved the manuscript.

Conflict of Interest

The authors declare no conflicts of interest.

References

1. Hans K. Global standards in medical education for better health care. Med Educ. 2002; 36(12):1116.
2. Magarian GJ, Mazur DJ. The procedural and interpretive skills that third-year medicine clerks should master. J Gen Intern Med. 1991; 6(5):469-71.
3. Elnicki DM, van Londen J, Hemmer PA, Fagan M, Wong R. US and Canadian internal medicine clerkship directors’ opinions about teaching procedural and interpretive skills to medical students. Acad Med. 2004; 79(11):1108-13.
4. Coberly L, Goldenhar LM. Ready or Not, Here They Come: Acting Interns’ Experience and Perceived Competency Performing Basic Medical Procedures. J Gen Intern Med. 2007; 22:491-4.
5. Yang RL, Reinke CE, Mittal MK, Kean CR, Diaz E, Fishman NO, et al. The surgery clerkship: an opportunity for preclinical credentialing in urinary catheterization. Am J Surg. 2012; 204(4):535-9.
6. Medical Council of India. Assessment Module for Undergraduate Medical Education Training Program. NMC: New Delhi; 2019.
7. General Medical Council. Practical skills and procedures. GMC: UK; 2019.
8. Medical Deans of Australia and New Zealand. Clinical practice core competencies for graduating medical students. Medical Deans Australia and New Zealand: Sydney; 2020.
9. Ragsdale JW, Seelbach EB, Vick S, Schadler A, Hall AM. Practice Doesn't Make Perfect: Clinical Experience With Procedures Does Not Correlate Well With Competence in Third-Year Medical Students. Journal of Surgical Education. 2022; 79(6):1441-6.
10. Okusanya OT, Kornfield ZN, Reinke CE, Morris JB, Sarani B, Williams NN, et al. The effect and durability of a pre-graduation boot camp on the confidence of senior medical students entering surgical residencies. J Surg Educ. 2012; 69:536-43.
11. Chacko TV. Simulation-based Medical Education: Using Best Practices and Curriculum Mapping to Maximize Educational Benefits in the Context of Shift toward Competency-based Medical Education. Archives of Medicine and Health Sciences. 2017; 5(1):9-15.
12. Ericsson KA. Acquisition and maintenance of medical expertise: a perspective from the expert-performance approach with deliberate practice. Acad Med. 2015; 90(11):1471-86.
13. Bashook PG, Parboosingh J. Recertification and the maintenance of competence. BMJ. 1998; 316(7130):545-8.
14. Dauphinee D. The content of certifying examinations. Cambridge University Press: Cambridge; 1994.
15. Hamdy H. Blueprinting for the assessment of health care professionals. The Clinical Teacher. 2006; 3(3):175-9.
16. Williams RG, George BC, Bohnen JD, Dunnington GL, Fryer JP, Klamen DL, et al. A Proposed Blueprint for Operative Performance Training, Assessment, and Certification. Ann Surg. 2021; 273(4):701-8.
17. Williams RG, Swanson DB, Fryer JP, Meyerson ShL, Bohnen JD, Dunnington GL, et al. How many observations are needed to assess a surgical trainee's state of operative competency?. Ann Surg. 2019; 269:377-82.
18. Urbina J, Monks SM. Validating Assessment Tools in Simulation [Internet].. StatPearls Publishing: Treasure Island (FL); 2023.
19. Sakakushev BE, Marinov BI, Stefanova PP, Kostianev SS, Georgiou EK. Striving for Better Medical Education: the Simulation Approach. Folia Med (Plovdiv).. 2017; 59(2):123-31.
20. Ault GT, Sullivan M, Chalabian J, Skinner KA. A focused breast skills workshop improves the clinical skills of medical students. J Surg Res. 2002; 106(2):303-7.
21. Sawyer T, White M, Zaveri P, Chang T, Ades A, French H, et al. Learn, see, practice, prove, do, maintain: an evidence-based pedagogical framework for procedural skill training in medicine. Acad Med. 2015; 90(8):1025-33.
22. Meyers MO, Meyer AA, Stewart RD, Dreesen EB, Barrick J, Lange PA, et al. Teaching technical skills to medical students during a surgery clerkship: results of a small group curriculum. J Surg Res. 2011; 166(2):171-5.
23. Masud T, Blundell A, Gordon AL, Mulpeter K, Roller R, Singler K, et al. European undergraduate curriculum in geriatric medicine developed using an international modified Delphi technique. Age Ageing. 2014; 43(5):695-702.
24. Green P, Edwards EJ, Tower M. Core procedural skills competencies and the maintenance of procedural skills for medical students: A Delphi study. BMC Med Educ. 2022; 22(1):259.
25. Custer RL, Scarcella JA, Stewart BR. The modified Delphi technique – a rotational modification. CTE J. 1999; 15(2):50-8.
26. Broder MS, Gibbs SN, Yermilov I. An Adaptation of the RAND/UCLA Modified Delphi Panel Method in the Time of COVID-19. J Healthc Leadersh. 2022; 14:63-70.
27. Urushibara-Miyachi Y, Kikukawa M, Ikusaka M, Otaki J, Nishigori H. Lists of potential diagnoses that final-year medical students need to consider: a modified Delphi study. BMC Med Educ. 2021; 21(1):234.
28. Howard KJ. Emergence of a new method: The Grounded Delphi method. Libr Inf Res. 2018; 42(126):5-31.
29. Zayapragassarazan Z, Sumathy MK, Wyawahare M, Pai D. A protocol to evaluate the effectiveness of competency-based simulation training modules on the educational outcomes among MBBS students-A mixed method study. Journal of Education and Health Promotion. 2023; 12(1):375.
30. Frank JR. The CanMEDS 2005 Physician Competency Framework: Better Standards, Better Physicians, Better Care. The Royal College of Physicians and Surgeons of Canada: Ottawa; 2005.
31. Cohen ER, Barsuk JH, Moazed F, Caprio T, Didwania A, McGaghie WC, et al. Making July safer: simulation-based mastery learning during intern boot camp. Acad Med. 2013; 88(2):233-9.
32. McGaghie WC, Issenberg SB, Cohen ER, Barsuk JH, Wayne DB. Does simulation-based medical education with deliberate practice yield better results than traditional clinical education? A meta-analytic comparative review of the evidence. Acad Med. 2011; 86(6):706-11.