Document Type : Original Article
Authors
- HEKMAT ALLAH MORADI 1
- MASOUME RAMBOD 2
- RITA REZAEE 3
- MILAD AHMADI MARZALEH 1
- MAHMOUDREZA PEYRAVI 1
- MITRA AMINI 3
1 Department of Health in Disasters and Emergencies, School of Health Management and Information Sciences, Shiraz University of Medical Sciences, Shiraz, Iran
2 Community-Based Psychiatric Care Research Center, Department of Nursing, School of Nursing and Midwifery, Shiraz University of Medical Sciences, Shiraz, Iran
3 Clinical Education Research Center, Shiraz University of Medical Sciences, Shiraz, Iran
Abstract
Introduction: This study is novel in that it is one of the first efforts in Iran to develop and validate a culturally relevant assessment tool tailored to evaluate the professional competencies of emergency managers. The study provides a foundation for evidence-based training of managers, competency development, and improved disaster preparedness and response in Iran by addressing this unmet need.
Methods: This methodological exploratory study which aimed to design and validate a competency assessment tool for the education of emergency managers was conducted in 2022. First, an item pool was collected from the literature review (previously designed questionnaires and existing models of reflection) and experts’ and researchers’ perspectives. After that, 40 experts were shown the first draft of the questionnaire to assess its face and content validity. To assess the construct validity, exploratory factor analysis (EFA) and confirmatory factor analysis (CFA) were done. 288 participants consciously completed the questionnaire. The data were analyzed using IBM SPSS version 26.0 (Statistical Package for the Social Sciences) Statistics and AMOS 26.0 software.
Results: The study yielded a structured questionnaire with 36 items across eight competency areas, including Critical thinking, Strategic thinking, Knowledge and experience, Skill management, Effective Communication, Ethical, Resiliency, and Personality. The content validity index (CVI), content validity ratio (CVR), and impact score (IS) for the questionnaire were 0.97, 0.62, and 4.61, respectively. This tool demonstrated strong reliability (Cronbach’s alpha=0.74) and robust convergent and discriminant validity. EFA identified eight key factors that, taken together, explained 51.9 percent of the overall variation. CFA revealed that there was a positive and significant correlation.
Conclusion: The findings confirmed that the assessment tool is a valid and reliable instrument for evaluating professional competencies in Iranian emergency managers. Beyond assessment, it offers a foundation for developing competency-based training strategies that enhance the effectiveness and resilience of emergency response teams. This tool was designed with cultural adaptation to address current limitations and can potentially significantly influence Iran’s emergency management procedures in the future.
Highlights
Keywords
Introduction
Natural and other disasters are among the crisis-causing events that harm people, property, the economy, and the environment while upsetting the routine processes of societies worldwide ( 1 ). Due mainly to the regularity of events and disasters, Iran is among the top ten countries most vulnerable to disasters; Iranian rankings are fourth in Asia and sixth globally, according to United Nations data ( 2 ). Only 3% of all cities in the country are situated in low-risk regions. The geography of the country has made it susceptible to various natural hazards, resulting in significant loss of life and economic damage over the past century ( 3 ). More than 140,000 people have lost their lives due to natural disasters in Iran ( 4 ). Accounting for 6% of the world's natural disaster casualties, Iran has only 1% of the global population. In the 2009 Global Disaster Risk Reduction Report published by the International Strategy for Disaster Reduction (ISDR), the Islamic Republic of Iran was assigned a risk level of 8 out of 10 ( 5 ). The infrastructure and functional and administrative societal vulnerabilities of the country are deteriorating ( 6 ).
Effective crisis management relies on core principles, including a constant belief in the probability of an event, organizational strength, convergence and synergy among responsible authorities, decentralization, implementation of risk reduction measures, availability of finance, and the integration of post-disaster lessons ( 7 ). Natural disasters can deplete resources and hard-won accomplishments that take decades to achieve if they are not adequately planned to minimize the resulting damages and cascading consequences. Effective policymaking, providing necessary facilities and resources, creating systematic forecasting, creating workable scenarios, and operationalizing planned measures during the pre-event, during-event and post-event phases are critical governmental responsibilities. However, the methods used to carry out these duties differ considerably in different countries ( 6 ).
Given the growing importance of disasters and the critical role managers play in addressing them, ensuring managers possess the necessary professional competencies is essential. Additionally, fierce competition and quick technological advancements drive organizations to maximize employee productivity. Workers are now supposed to play a bigger part in determining the company's goods and services ( 8 ). This has resulted in competence having a significant impact on the competitive environment. The human resources environment in Iran is inevitably influenced by these changes and developments, necessitating a shift in its goals to align with trends ( 9 ). T he widespread adoption of competency-based approaches in human resource development programs is primarily attributed to their numerous benefits. Competency-based reward systems recognize and reward employees based on their skills and abilities, leading to enhanced employee performance, increased organizational effectiveness, and a stronger emphasis on employee skills. This approach fosters a positive organizational culture, enhances teamwork, and ultimately provides a competitive advantage ( 10 ). The findings above highlight the factors that necessitate the development of highly developed management competencies, as these competencies are linked to improved performance and results. Despite the development of the Emergency Medical Services (EMS) training program, comprehensive studies on the professional proficiency of emergency managers and coordinators of disaster preparedness have not been carried out in Iran. In some studies on the challenges of EMS personnel in Iran, the lack of knowledge and skills has been reported ( 11 - 13 ).
In a qualitative investigation conducted by Ahangarzadeh Rezaei, et al. (2020), entitled “Identifying the professional capabilities required by nurses to provide care for victims of earthquakes,” it was demonstrated that four distinct categories of competence exist, which encompass clinical competence (including professional knowledge and clinical skills), individual competence (including communication skills, resilience, creativity, and innovation in the provision of care), moral competence (entailing a commitment to ethical standards and professional responsibility), and fundamental skills necessary for the treatment of the injured ( 14 ).
A mixed-methods study was carried out by Siddiqi, et al. The report entitled “Model of Professional Competence for Emergency Medical Technicians in Iran” (2025), found that the final model contained five main topics (physical health, mental health, professional competence, organizational competence, clinical competence, and attitude), 11 sub- topics (management skills, competences, cognitive ability, professional competence, and attitude), and 47 categories. Test of content validity has shown that the model and its elements are acceptable and valid (Content Validity Index (CVI)=0.82, Content Validity Ratio (CVR)=0.73) ( 15 ).
Saxena and colleagues conducted a study in Delhi in 2022, focusing on identifying competencies of disaster management professionals. This study highlighted the skills necessary for disaster management professionals to operate effectively and efficiently, aiming to minimize human casualties and adverse effects during disasters. It also found that organizational, technical, and interpersonal skills significantly impacted the performance of Delhi disaster management professionals. The study supports the idea that implementing a competency-based approach can improve the employees’ performance and morale. Additionally, it shows that having a suitable mix of a positive organizational culture, current knowledge, sufficient technical skills, strong communication abilities, teamwork, and practical interpersonal skills will undoubtedly enhance the performance of disaster management professionals in Delhi's disaster management field ( 16 ).
Thirteen key competencies for managers were identified in a study by Shirley Feldmann-Jensen (2019) entitled "Next Generation Core Competencies for Emergency Management." Some of these identified thirteen essential competencies for managers were critical thinking, preventive approaches, professional ethics, lifelong learning, sociocultural literacy, technological literacy, scientific literacy, geographic literacy, community participation, governance, civic engagement, efficiency, and leadership ( 17 ).
In the study conducted by Meri et al. entitled “Health care managers: Health care leadership and the alliance model,” it has been demonstrated that five distinct categories of competence exist, which encompass 1) the management of communication and interpersonal relationships, 2) professionalism, 3) leadership, 4) knowledge of the health care system, and 5) business skills and knowledge ( 18 ).
Proficiency and skills-focused education in incident and emergency management should receive increased emphasis as the professional landscape becomes intricate, unexpected incidents occur frequently, and knowledge continues to expand. These elements have brought about changes in society ( 19 ). Managing and reducing disaster risk and the incident and emergency managers’ expertise have always been the key challenges for the healthcare system in Iran ( 20 ). Iran continues to face significant challenges despite previous efforts and international frameworks, such as the Hyogo Resolution, to improve disaster management and risk reduction. Although the country has developed various strategies and plans, including those focused on social, economic, and cultural development, the ongoing impact of disasters remains significant. The success of these measures depends on the effective use of professional competencies in incident and emergency managers in the healthcare system ( 21 ).
Identifying the competencies required for disaster management personnel is critically important, especially in Iran, which experiences approximately 250 natural disasters yearly ( 22 ). Despite this high frequency, the country's current disaster management system remains inadequate to meet the growing demands of incident response. Compounding the issue is a lack of research specifically focused on the competencies and skillsets essential for effective disaster management in the Iranian context. Very few studies have explored the dimensions and characteristics of these competencies, leaving a significant gap in both academic literature and practical tools. This study is novel in that it is one of the first efforts in Iran to develop and validate a culturally relevant assessment tool tailored to evaluate the professional competencies of emergency managers and disaster preparedness coordinators. The study provides a foundation for evidence-based training, competency development, and improved disaster preparedness and response in Iran by addressing this unmet need. This competency assessment tool supports the training and education of accident and emergency managers by identifying the strengths and areas for improvement and providing a structured framework to develop critical thinking, strategic decision-making, and practical knowledge. The tool guides evidence-based curriculum design, promotes higher professional standards, and is equally valuable in the recruitment and employment of new emergency managers by offering an objective measure of competencies. Overall, the tool ensures workforce readiness to meet the complex demands of accident and emergency management.
Methods
Design and study participants
This study (instrument development and validation) was conducted in 2022. Since the original process was primarily theoretical, it needed to be modified for data collection and analysis. The process was classified into four phases (Figure 1). In the first study, the professional competencies of incident and emergency managers were characterized using a qualitative approach. A content analysis method was employed to evaluate the data. The ability of a qualitative approach to offer deep insights into people's perceptions and experiences served as justification for its use. Purposive sampling was used to select 20 incident and emergency managers from five provinces (Fars, Khuzestan, YasUj, Tehran, and Isfahan). After data collection reached saturation, five major categories and eighteen subcategories were identified through data analysis. Thinking skills (strategic, critical), professional disaster management skills (knowledge and experience, planning, organizing, decision-making, leadership, research power, risk management), effective communication (collaboration, interpersonal, intersectoral communication), personal qualities (personality, physical attributes, and demographics), and morality and legal knowledge (organizational, professional, legal, and policy) were the five primary categories.
Figure 1. The process of study
The methodological approach is a quantitative study with a mixed methods design and multiple stages. In the first stage, the study was conducted through various surveys and interviews with participants' emergency managers to identify the key concepts, dimensions, and indicators of the phenomenon under study in Iran. In the second stage, interviews were conducted with 24 experts in the field of health in disasters and emergencies, emergency medicine, and medicine. In this stage, the qualitative findings derived from interviews were integrated with the theoretical framework and prior literature to refine the key dimensions and indicators. These indicators were then translated into measurable items, and the research questionnaire was developed. Finally, the preliminary validity of the instrument was assessed through expert review, and necessary revisions were made. According to the previous 2 steps, the questionnaire was designed. Then, the reliability of the questionnaire sub-scales was also evaluated; the questionnaire was completed by 288 incident and emergency managers. It was evaluated through face and construct validity, and the reliability of the questionnaire was estimated using Cronbach's alpha and Intraclass Correlation Coefficient (ICC) methods.
Face and content validity assessment
To check the formal and qualitative validity, the designed and adapted items as well as written explanations of the study objectives, how the research instrument was used, and the topics examined in, we gave the questionnaire to 10 experts in the fields of emergency medicine, nuclear medicine, physicians, health in disasters and emergency experts, passive defense and crisis management, as well as 10 beneficiaries in the emergency room who had some knowledge of the subject under investigation or had professional experience in this area. The inclusion criteria for participants were a minimum bachelor's degree for crisis managers, at least three years of executive work experience in disaster management, participation in managing one of the major disasters in the past 10 years, and willingness to participate in the research. Specifically, emergency medicine and health in disasters and emergencies specialists were expected to have experience in disaster management, while other specialists were expected to possess relevant publications and acceptable knowledge. Participants who were not interested in participating in the study were excluded. Also, the items were given to 3 literature experts to be reviewed from a literary point of view (This assessment considered factors such as rationality, appropriateness, attractiveness, logical sequence, eloquence, comprehensibility, brevity, and comprehensiveness). Face-to-face interviews with participants and experts were used to assess qualitative face validity.
The quantitative method of statement impact was used after determining the qualitative formal validity and revising the inappropriate items to reduce and eliminate inappropriate statements and determine the importance of each statement ( 23 ). For each statement in the instrument, a 5-point Likert scale was used, ranging from "completely important" with a score of 5 to "not important at all" with a score of 1. The impact score for each instrument statement was calculated as follows: The Impact Score (IS) was calculated by rating each item on a 5-point Likert scale, with five being completely important and one being never important. The following formula was used for this calculation. The number of subjects who gave a score of 4 or 5 to each item, denoted by f, and the mean importance of the item according to the Likert scale, denoted by i, were used to determine the IS. If the IS was greater than 1.5, indicating that the item was suitable for further analysis.
The content validity was assessed in terms of quality and quantity. To determine the quality aspect, 20 experts in various fields evaluated the questionnaire, made modifications to overlapping items, and adjusted the format of certain items. The researchers also decided to select one item from each pair of overlapping items. For the quantitative aspect, Fe's techniques, specifically the CVR and CVI were utilized. These methods determine the importance and necessity of each item in the questionnaire based on expert judgment. 20 experts with relevant experience rated the importance of the questionnaire items according to a specific classification.
The value deemed acceptable for each item varied based on the number of experts determining the content validity. In Lawshe's methods of quantitative content validity, the CVI was determined. Items that scored below the minimum acceptable CVR values were removed, and the remaining items were used to calculate the final CVI for the questionnaire. In the CVI, experts rated each item on a 4-point scale for relevance, clarity, and simplicity ( 24 - 26 ). Its clarity and simplicity were assessed if the item scored over 75 percent for relevance. An item was only considered as valid if it had a CVI higher than 0.8.
Reliability assessment
Then, the items were analyzed using a sample size of 30 subjects to determine Cronbach's alpha for primary reliability and identify the reliability of the items. The correlation coefficient of all the items was also sought in the item analysis. In case an item did not correlate with at least one other item by 0.3-0.7, it was eliminated. Furthermore, the items were removed or merged if their correlation coefficients were >0.7. The items with a total correlation coefficient <0.3 were excluded as well ( 27 ).
exploratory factor analysis
Construct validity was ensured using exploratory factor analysis. To this end, the Kaiser-Meyer-Olkin (KMO) measure of sampling adequacy was used in the first stage. Bartlett's test of sphericity was used in the next stage to determine whether the observed matrix significantly differed from zero, based on which factor analysis was justified. Once the functionality of the items was verified, Eigenvalue (Kaiser's criterion) and Scree Plot were used to identify the principal components of the questionnaire. Eigenvalue refers to the sum of squared cumulative factor loadings.
Construct validity assessment
Confirmatory Factor Analysis (CFA) was conducted to confirm the factor structure of the questionnaire, which had been initially developed through exploratory factor analysis utilizing AMOS 26.0 software. Exploratory Factor Analysis (EFA) was executed in SPSS Statistics 26. The confirmatory factor analysis used covariance matrices and the maximum likelihood estimation. The following criteria determined the acceptable model fit: χ2/df≤5, Root Mean Square Error of Approximation (RMSEA)<0.08, Adjusted Goodness of Fit Index (AGFI)>0.9, Goodness of Fit Index (GFI)>0.9, and Comparative Fit Index (CFI)>0.9 ( 17 ). If necessary, the reliability of the questionnaire sub-scales was also evaluated. To assess Cronbach's alpha, we asked 288 incident and emergency managers to complete the developed questionnaire. The participants were selected from different provinces so that the sample distribution roughly reflected the number of active incident and emergency managers in each province. The largest samples were collected from more populous provinces, while smaller samples came from less populated provinces. Although the exact number of participants per province was not recorded, the participants were selected from all regions under coverage."
Ethical considerations
The study was approved by the Ethics Committee (IR.SUMS.NUMIMG.REC.1400.41) in biomedical research. Participants had the option to withdraw from the study and were assured of the confidentiality of their information, with no disclosure of their names or hospitals.
Results
The questionnaire items were developed considering article reviews and expert opinions. The questionnaire comprised 36 items and was divided into eight themes, with eight main categories: critical thinking, strategic thinking, professional management skills, resiliency, effective communication, knowledge and experience, personal quality, and ethical considerations. Validity (including face, content, and construct validity) and reliability (measured through Cronbach's alpha and test-retest) were estimated to assess the questionnaire. All 50 items passed the critical Item Impact Score (IS) threshold of 1.5 during the face validity evaluation, which means they will be retained in the following analytical phases. In the content validity assessment phase, 14 items were removed because of poor validity metrics. After the items and correlation coefficient analysis, no items were deemed necessary to eliminate. The study displayed the characteristics of the participants at each stage, with each item being correlated to another item by at least 0.3. The results of the formal and content validity and item analysis are shown in Table 1.
| Face and content validity of the intended questionnaire | ||||||
|---|---|---|---|---|---|---|
| No. | Items | IS | CVR | CVI | Item correlation | Final results |
| 1 | I employ innovative approaches to solving organizational problems. | 4.6 | 0.47 | 0.8 | 0.547 | Accepted |
| 2 | I have a comprehensive understanding of internal and external factors affecting the organization. | 4.5 | 0.68 | 0.9 | 0.746 | Accepted |
| 3 | I freely express my suggestions and opinions without fear of superiors. | 3.1 | 0.47 | 1 | 0.774 | Accepted |
| 4 | I effectively resolve workplace conflicts where I work. | 2.7 | 0.47 | 1 | 0.788 | Accepted |
| 5 | With flexibility, I manage complex issues and incidents. | 2.9 | 0.78 | 0.8 | 0.810 | Accepted |
| 6 | I recognize the importance of having a receptive and constructive mindset towards criticism and feedback as a key attribute. | 4.17 | 0.78 | 1 | 0.838 | Accepted |
| 7 | I look for new opportunities for organizational and personal learning and growth. | 3.5 | 0.57 | 1 | -0.48 | Accepted |
| 8 | I am an active and good listener. | 2.5 | 0.47 | 1 | 0.358 | Accepted |
| 9 | I am fully aware of the Incident Command System (ICS). | 4.6 | 0.68 | 1 | 0.568 | Accepted |
| 10 | I actively involve my employees in formulating organizational policies and decision-making processes. | 2.6 | 0.68 | 1 | 0.536 | Accepted |
| 11 | I promote ethical behavior within the organization. | 3.1 | 0.47 | 1 | 0.645 | Accepted |
| 12 | I possess knowledge of crisis management principles and processes. | 3.9 | 1 | 1 | 0.447 | Accepted |
| 13 | I make informed decisions in crises. | 4.2 | 1 | 1 | 0.438 | Accepted |
| 14 | I respect the dignity of individuals. | 4.5 | 0.89 | 1 | 0.589 | Accepted |
| 15 | I adopt a proactive approach rather than a reactive approach in managing incidents. | 2.9 | 0.68 | 1 | 0.427 | Accepted |
| 16 | I am not easily discouraged if I fail. | 3.3 | 0.57 | 0.8 | 0.521 | Accepted |
| 17 | I am familiar with the principles of risk management. | 4.2 | 0.57 | 1 | 0.367 | Accepted |
| 18 | It is necessary for me to have experience in incident management. | 4.5 | 0.57 | 1 | 0.761 | Accepted |
| 19 | I utilize lessons learned from past events to anticipate future issues. | 3.7 | 1 | 1 | 0.513 | Accepted |
| 20 | I have relevant academic qualifications in the field. | 2.9 | 0.47 | 1 | 0.650 | Accepted |
| 21 | I have a keen interest in gathering comprehensive perspectives from colleagues and utilizing expert opinions. | 3.3 | 0.68 | 1 | 0.600 | Accepted |
| 22 | I have effective and productive working relationships with employees. | 3.3 | 0.78 | 1 | 0.573 | Accepted |
| 23 | I am fully familiar with research methods. | 2.3 | 0.5 | 1 | 0.541 | Accepted |
| 24 | I am a resilient person. | 3.6 | 0.68 | 1 | 0.756 | Accepted |
| 25 | I know the right way to communicate with the media, the public, and politicians. | 3.3 | 0.47 | 1 | 0.755 | Accepted |
| 26 | I accept the responsibility of my performance. | 4.1 | 0.57 | 1 | 0.578 | Accepted |
| 27 | I delegate orders and operations well. | 3.6 | 0.89 | 1 | 0.424 | Accepted |
| 28 | I accept constructive criticism from employees with confidence. | 4.01 | 0.68 | 1 | 0.385 | Accepted |
| 29 | My personal goals are in line with the individual goals of the organization. | 2.4 | 0.05 | 1 | 0.524 | Accepted |
| 30 | I have administrative discipline. | 3.1 | 0.68 | 0.8 | 0.522 | Accepted |
| 31 | I must be in good physical and physiological condition. | 2.08 | 0.47 | 1 | 0.703 | Accepted |
| 32 | I emphasize lifelong learning and knowledge sharing in the organization. | 3.1 | 0.47 | 1 | 0.801 | Accepted |
| 33 | I have high formal external communication skills to solve problems. | 3.3 | 0.47 | 1 | 0.394 | Accepted |
| 34 | I can identify and prioritize strategic issues and topics. | 4.04 | 0.47 | 1 | 0.606 | Accepted |
| 35 | I respect different beliefs and cultures. | 4.1 | 0.78 | 1 | 0.472 | Accepted |
| 36 | I can interpret and apply the rules related to crisis management. | 3.1 | 0.57 | 1 | 0.549 | Accepted |
Table 2 presents the demographic characteristics of the study participants (N=288). Age, work experience, and educational level were reported in years, while gender was reported as the number and percentage of participants in each category (Table 2).The final questionnaire consisted of 36 items with 5-point Likert scales, ranging from "completely" (score 5) to "not at all" (score 1) and "always" (score 5) to "never" (score 1). To assess the construct validity of the questionnaire using exploratory factor analysis (EFA), we first examined the Kaiser-Meyer-Olkin (KMO) measure of sampling adequacy. The KMO value was 0.61, which is considered acceptable for EFA. Additionally, Bartlett's test of sphericity was conducted, and the result was significant (p<0.001), with a value of 2793.44. These results indicate that the correlation matrix obtained from the sample is suitable for performing EFA. The two mentioned criteria (KMO and Bartlett's test) are essential indicators for confirming the adequacy of the sample size and the factorability of the items. KMO and the results of Bartlett's test of Sphericity confirmed the suitability of the data and adequacy of variable sampling for EFA. The results of the KMO and Bartlett's test are shown in Table 3.
| Demographic Characteristics | N0 (%) | |
|---|---|---|
| Age(y) | <30 | 7 (2.4) |
| 31-40 | 168 (58.3) | |
| >41 | 113 (39) | |
| Gender | Female | 70 (24) |
| Male | 218 (76) | |
| Work experience(y) | <5 | 4 (1.4) |
| 5-10 | 68 (23.8) | |
| 11-15 | 160 (55.6) | |
| 16-20 | 46 (16) | |
| >20 | 10 (3.1) | |
| Educational level | Bachelor | 120 (42) |
| MS | 81 (28.1) | |
| PhD | 87 (29.9) | |
| KMO and Bartlett's Test | ||
|---|---|---|
| Kaiser-Meyer-Olkin Measure of Sampling Adequacy | 0.619 | |
| Bartlett's Test of Sphericity | Approx. Chi-Square | 2793.446 |
| df | 630 | |
| P | <0.001 | |
The overall Cronbach's alpha for the 288 emergency managers was 0.74 after completing the questionnaires. Additionally, the overall ICC was 0.74. Both Cronbach's alpha and ICC for the eight main components are provided. The ranges of Spearman correlation in convergent and discriminant validity were 0.3 to 0.83 (Table 4).
| Cronbach's alpha, Mean, and Standard Deviation of the subscales | ||||||
|---|---|---|---|---|---|---|
| No. | Subscale | No of items | Cronbach's alpha | Convergent validity | Discriminant validity (percent) | |
| Range of correlation | Scaling success (percent) | |||||
| 1 | Critical thinking | 4(1,2,3,4) | 0. 73 | 0.6-0.83 | 4/4 (100) | 28/28 (100) |
| 2 | Strategic thinking | 5 | 0.69 | 0.55-0.70 | 5/5 (100) | 35/35 (100) |
| 3 | Skill managmeent | 4 | 0.68 | 0.5-0.7 | 4/4 (100) | 28/28 (100) |
| 4 | Resieliency | 3 | 0.69 | 0.55-0.74 | 3/3 (100) | 21/21 (100) |
| 5 | Ethical | 5 | 0. 73 | 0.3-0.74 | 4/5 (80) | 32/35 (91) |
| 6 | Knowledge and experience | 5 | 0.75 | 0.43-0.78 | 5/5 (100) | 35/35 (100) |
| 7 | Effective Communication | 4 | 0.69 | 0.4-0.70 | 4/4 (100) | 28/28 (100) |
| 8 | Personality | 3 | 0.73 | 0.6-0.81 | 3/3 (100) | 21/21 (100) |
| Total | 33 | 0.74 | ||||
The EFA resulted in the extraction of eight factors, and the theoretical structure of the questionnaire was explained by these factors. A Scree plot of the factors defining the participant's reflection is presented in Figure 2. After Varimax rotation, each item was assigned more specifically to one of eight factors. Items 15 and 30 were excluded from the scale due to insufficient factor loadings (<0.40) across all components. Additionally, Item 7 was flagged for potential deletion during reliability analysis (reported subsequently) owing to its significant negative correlation with Dimension 8 (r=-0.48). This anomalous response pattern suggests possible sensitivity/reverse scoring effects. Nevertheless, item 7 underwent further evaluation in the confirmatory factor analysis (CFA) phase.
Figure 2. Scree plot of the factors defining participants’ reflection skills
The 36-item factor structure's goodness-of-fit was assessed using the chi-square goodness-of-fit test within a confirmatory factor analysis. Additional indices were evaluated to further assess the model fit. All indices confirmed the appropriate fit of the final model (Table 5).
| NO | SCALE | Goodness-of-fit indices for confirmatory factor analysis (CFA) with acceptable | P-value | Result | ||||
|---|---|---|---|---|---|---|---|---|
| χ2 (less than 5) | (GFI) (more than 9).0 | AGFI (More than 9).0 | CFI (More than 0.9)) | RMSEA.Less than 0.08 | ||||
| 1 | Critical thinking | 0.91 | 0.99 | 0.98 | 1 | 0.00 | 0.400 | Accepted |
| 2 | Strategic thinking | 59.2 | 98.0 | 94.0 | 97.0 | 07.0 | 0.023 | Accepted |
| 3 | Skill management | 86.0 | 99.0 | 98.0 | 1 | 0.00 | 0.421 | Accepted |
| 4 | Resiliency | 020.0 | 1 | 1 | 1 | 0.00 | 0.980 | Accepted |
| 5 | Ethical | 85.2 | 98.0 | 93.0 | 89.0 | 08.0 | 0.022 | Accepted |
| 6 | Knowledge and experience | 87.1 | 98.0 | 95.0 | 91.0 | 057.0 | 0.095 | Accepted |
| 7 | Effective communication | 69.2 | 99.0 | 95.0 | 91.0 | 079.0 | 0.068 | Accepted |
| 8 | Personality | 88.1 | 99.0 | 96.0 | 95.0 | 057.0 | 0.151 | Accepted |
Discussion
Competent emergency managers play a vital role in creating safe and effective learning environments for medical students and residents. Systematic evaluation of these competencies not only contributes to enhancing the quality of management in critical situations but also improves educational processes, strengthens clinical learning, and fosters the development of physicians who are capable, responsible, and effective in responding to emergency conditions. The content validity ratio of the questionnaire (78%) further indicated that the instrument was clear, comprehensive, and relevant, supporting the advancement of competency-based approaches in medical education. The construct validity of the questionnaire was supported by EFA, which revealed eight distinct factors corresponding to the hypothesized dimensions of the questionnaire. The reliability of the questionnaire was assessed using Cronbach's alpha coefficient, which yielded a value of 0.74 for the overall questionnaire, demonstrating acceptable internal consistency. Furthermore, this study is among the first to address the development of a valid and reliable tool for assessing emergency and disaster management practices. In this study, the Likert scale was also used due to the diversity of responses. Huijuan Ma and his colleagues designed a questionnaire with 76 items in his study. The content validity index was 0.83, and the internal consistency reliability was 0.8, which was consistent with the present research ( 28 ). The RMSEA value indicates the suitability of the 8-factor questionnaire for the population covariance matrix. This fit index is particularly valuable due to its sensitivity to the number of parameters estimated in the questionnaire. In our research, the CFI, commonly used to assess the goodness-of-fit of a statistical model, exceeds the threshold of 0.9, suggesting that the 8-factor model fits better than an independent or null model. The GFI, similar to R2 in multiple regression, reflects the amount of variance in the sample that can be estimated population covariance. Cronbach's alphacoefficients of the subscales Critical thinking, Strategic thinking, Skill management, Resiliency, Ethical, Knowledge and experience, Effective Communication, and Personality were 0.73, 0.69, 0.68, 0.69, 0.73, 0.75, 0.69, and 0.73. Cronbach's alpha of the whole questionnaire was 0.74, which shows that this instrument is reliable ( 29 - 31 ).
The eight key dimensions of critical thinking, strategic thinking, skill management, resiliency, ethics, knowledge and experience, effective communication, and personality are aligned with the global approach of Competency-Based Medical Education (CBME), which can be used to improve patient safety and disaster preparedness in times of crisis by teaching these competencies to medical students ( 32 ).
Clinical supervision is crucial for effective service delivery. International standards emphasizing professional competency have heightened the significance of supervision in improving client outcomes. Consequently, supervisory competence tools are acknowledged as vital elements of effective supervision ( 33 ). Various studies have validated professional competency assessment tools for managers in diverse fields. Amani (2022), for instance, highlighted the importance and strong psychometric properties (validity and reliability, with a Cronbach's alpha of 0.98) of their questionnaire for assessing the professional competence of preschool managers. However, in the present study, Cronbach's alpha for internal consistency was 0.74, potentially due to different study populations ( 34 ). The results of this study supported what Nilsson et al. found in their research about a competency scale for ambulance nurses. The ambulance nurse competency scale, with its 43 items, assessed eight key areas: general nursing care, ethical considerations, medical procedures, working within the community, handling emergencies, leadership, professional conduct, and continue advancements in the field. Notably, all areas except leadership showed strong reliability (Cronbach's alpha>0.71) and explained a significant portion of the overall effectiveness (59.62%). This alignment with a previous study by Nilsson et al. may be due to similar participant groups in both studies. International studies on emergency nurses and educational managers have shown that reliable and valid tools play a significant role in improving the quality of education and services. The alignment of the present findings with research such as the study by Nilsson et al. (competency scale for ambulance nurses) and the research related to educational managers indicates that psychometric-based education can also be a foundation for enhancing the quality of medical education in Iran ( 35 ).
An Australian study was the first to validate an instrument for assessing managerial competencies in health service leaders, confirming its effectiveness through data from 117 senior and middle managers across healthcare facilities in Victoria. These findings are highly relevant to medical education, where competent managers are essential for fostering safe and effective learning environments for students and residents. The current study expands upon this framework. These resources not only aid in the education of responsible and qualified medical professionals, but they also train future emergency and disaster managers by giving them the tools they need to be vital to the provision of healthcare and crisis leadership ( 36 ).
A mixed-methods study by Sadeghi, et al. (2024) developed a model of professional competence for emergency medical technicians in Iran, encompassing five main domains, 11 sub-domains, and 47 categories. Content validity testing confirmed the acceptability and relevance of the model (CVI=0.82, CVR=0.73). These findings underscore the importance of developing competency assessment tools in medical education, particularly for training emergency medical technicians and physicians. The results of the present study were generally consistent with those of Sadeghi et al.; however, higher CVI and CVR values in our study indicate stronger expert agreement regarding the relevance and essentiality of the assessment items. Such tools can be effectively integrated into competency-based medical education programs, enhancing professional health standards, clinical skills, and organizational competencies, and thereby playing a critical role in preparing the next generation of emergency physicians and technicians for effective performance in high-pressure and critical situations ( 15 ).
A study in Iran explored the development and validation of a self-assessment tool for preschool managers' professional competence. This descriptive study involved 380 preschool managers. The researchers assessed the internal consistency of the tool (Cronbach's alpha) and various aspects of validity, including face validity, content validity (relevance, clarity, and necessity), and structural validity. The results indicated good face validity and appropriate content validity indicators. Notably, Cronbach's alpha reached a high value of 0.98, demonstrating strong internal consistency. Additionally, a weak correlation (0.08) with no significant association (p=0.58) was found between the managers' self-reported competence and their self-esteem, suggesting that the tool effectively measures professional competence independent of self-esteem. The questionnaire used for assessing preschool managers’ competence demonstrated strong validity and reliability, effectively measuring their professional skills. This alignment between the current research and others might be attributed to the way these instruments are designed and structured, ensuring consistent and accurate evaluation of managerial competence ( 34 ).
In 2025, Mohammadi and colleagues conducted a study which aimed to develop and evaluate a valid instrument for assessing safe care practices provided by pre-hospital emergency medical services personnel at incident scenes. The present study developed and psychometrically evaluated the Emergency Medical Services Safe Care Scale (EMSSCS) using a mixed-methods sequential exploratory design in Iran. Safe care was conceptualized across three dimensions: incident scene management, efficient clinical skills, and effective interaction. The final 30-item scale demonstrated strong validity and reliability (Cronbach’s alpha=0.95), confirming its suitability for assessing safe care practices. The findings of this study, while overlapping with the present research, underscore the significant role of structured, evidence-based assessment tools in medical education and clinical management, enabling EMS managers to identify the gaps, implement targeted training, and strengthen the competence of pre-hospital personnel in delivering safe, principled care under high-pressure conditions ( 37 ).
Given that competency assessment is a key step in developing a potent and skillful team of nurses, a study was carried out to expand a professional competency scale for military nurses and test its psychometric properties. The study was carried out by nurses from military hospitals in the Eastern area. The study included three main phases: scale development (item analysis and exploratory factor analysis), scale validation (confirmatory factor analysis and reliability test), and item development (pilot testing). Four dimensions including clinical nursing knowledge and skills, military nursing knowledge and skills, professional capacity, and comprehensive quality were considered for the 65-item military nursing assessment. The reliability and validity of the instrument was satisfactory, and the four determinants mentioned above accounted for 66.9% of the total variance. Therefore, the scale of professional competency for military nurses is consequently a good instrument for assessing the competence of military nurses in military hospitals. The results of the current research showed that the eight investigated dimensions justified 50.56% of the total variance. These variations in the proportion of justified variance may be due to the different samples and their culture in different regions ( 28 ).
In the same way, Hamilton developed the Generalized Supervisory Assessment Tool (GSAT) to assess the skills of supervisors, confirming the reliability of psychometrics and its use in various areas. As a non-punitive supervisory tool, it provides a reliable and transparent method to compare the skills of clinical supervisors in different areas, thereby improving supervisory evaluation procedures and increasing supervisory efficiency. This study is consistent with the goals of the current research ( 33 ).
Castaño validated a management competency model for public administration, demonstrating meaningful relationships between competencies such as leadership and job performance in the public sector. The study also developed a management competency model in Spain, including competencies such as leadership, communication, and ethics. These findings are consistent with the present study, particularly regarding the dimensions of effective communication and professional ethics. The relevance of such models in medical education is clear, as physicians and clinical managers require strong managerial competencies to lead healthcare teams, make ethical decisions, and effectively organize clinical processes. Assessing and strengthening skills such as leadership and communication not only enhances the quality of clinical services but also fosters critical thinking and problem-solving among physicians, preparing them to handle challenging circumstances and react appropriately in demanding clinical settings ( 38 ).
Paz and Odelius previously identified the key skills for public administration managers through a survey which focused on strategy, service delivery, and team management. This study builds on that concept. The goal here was to develop a validated survey measuring managerial competence in public administration. Following a similar approach to Paz and Odelius, researchers created a questionnaire based on a literature review and expert feedback. Over 447 public sector professionals (managers and non-managers) working in state-level technical assistance and rural development programs completed the electronic survey. Exploratory factor analysis confirmed the validity and reliability of the survey, identifying eight key dimensions for public service managers. Therefore, in line with this research, the present study used data collection methods based on literature analysis, and through the results of exploratory factor analysis, eight dimensions were identified, and the construct validity was confirmed ( 39 ).
Conclusion
A professional competency assessment instrument for emergency managers was developed and validated in this study. The reliability and validity of the scale were also confirmed. The professional competency assessment tool for emergency managers can be used to measure the professional competency levels of emergency managers and disaster preparedness coordinators and the adoption of competency-based management.Also, this research clearly demonstrates that the design and validation of professional competency assessment tools are essential not only for evaluating disaster and emergency managers but also for holding a special place in medical education. Using these tools can help train future doctors and managers who will not only have clinical expertise but also have the ability to manage, make strategic choices, and lead successfully under pressure. Future studies should be conducted on the application of the professional competency assessment tool for emergency managers in other regions to validate and adapt the scale for use worldwide.
Strengths and limitations of the study
This study is distinguished by several strengths that enhance its value. It is one of the first attempts in Iran for designing a culturally adapted tool for assessing the professional competencies of emergency managers, thereby addressing a significant gap in disaster management. The use of a mixed-methods approach, combining qualitative insights with quantitative validation, added rigor and credibility to the findings. Moreover, the tool has strong practical utility, offering a structured framework for training, assessment, and professional development to strengthen emergency management systems.
This study has several strengths, including its comprehensive approach to assessing the validity and reliability of the professional competency tool for emergency managers. It is one of the first efforts in Iran to create and validate such an assessment, filling a significant research gap. The development of the tool was based on both a literature review and the perspectives of experts and researchers, which enhances its relevance and practicality. The study also highlights the importance of the tool for evidence-based training and competency development, offering a structured framework to enhance critical thinking and strategic decision-making among emergency managers. This is especially pertinent for medical professionals, such as physicians who may act as crisis managers, as the tool can help identify their strengths and areas for improvement, guiding the creation of effective, competency-based training programs.
A notable weakness of the study is the self-reported nature of the data, which may affect the accuracy of the findings. The study results, being based on samples from Iran, cannot be easily generalized to a global context, suggesting a need for similar studies in other regions. While the sampling adequacy for factor analysis was deemed acceptable, it was on the lower end, which could potentially limit the robustness of the results. However, despite these limitations, the tool is a valuable asset for enhancing the readiness of emergency response teams by providing a reliable method for evaluating professional competencies and supporting targeted educational strategies for medical and other professionals in crisis management roles.
Acknowledgment
This article was extracted from a PhD dissertation written by Hekmat Allahe Moradi on Health in Disasters and Emergencies, which was approved and financially supported by Shiraz University of Medical Sciences, Shiraz, Iran. We appreciate the collaboration of Shiraz University of Medical Sciences, the Center for Research in Health and Human Resources Management, and all the experts and researchers who helped us with this investigation.
Authors' Contribution
All authors contributed to the discussion, read and approved the manuscript, and agreed to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.
Conflict of Interest
No conflicts of interest need to be disclosed by the authors.
Declaration on the use of AI
No artificial intelligence (AI) tools were used in the preparation, writing, data analysis, or editing of this manuscript.
References
- Azizi A, Eskandari Sedgh H, Ghazi A. Smart Disaster Management Solutions for Risk Mitigation and Preparedness in Tehran, Iran. Disaster Prevention and Management Knowledge (quarterly). 2023; 13(2):252-77.
- Habibi Arejan R, Shabany M, Ghodsi Z, Ghashghaei S, Abbaszadeh M, Rahimi-Movaghar V. Lessons from Flood Disaster Management of Khuzestan Province, Iran, Based on the Experiences of Policymakers: A Qualitative Study. Iranian Red Crescent Medical Journal (IRCMJ). 2023; 25(10):e2199.
- Saberian P, Kolivand PH, Hasani-Sharamin P, Dadashi F, Farhoud AR. Iranian emergency medical service response in disaster; report of three earthquakes. Advanced journal of emergency medicine. 2019; 3(2):e13.
- Jafari H, Jafari AJ, Nekoei-Moghadam M, Goharinezhad S. Morbidity and mortality from technological disasters in Iran: A narrative review. J Educ Health Promot. 2019; 8(1):147.
- Fekete A, Asadzadeh A, Ghafory-Ashtiany M, Amini-Hosseini K, Hetkämper C, Moghadas M, et al. Pathways for advancing integrative disaster risk and resilience management in Iran: Needs, challenges and opportunities. International Journal of Disaster Risk Reduction. 2020; 49:101635.
- Blaikie P, Cannon T, Davis I, Wisner B. At risk: natural hazards, people’s vulnerability and disasters. UK: Routledge; 2014.
- Zarghami SA, Dumrak J. A system dynamics model for social vulnerability to natural disasters: Disaster risk assessment of an Australian city. International journal of disaster risk reduction. 2021; 60:102258.
- Naser WN, Saleem HB. Emergency and disaster management training; knowledge and attitude of Yemeni health professionals- a cross-sectional study. BMC emergency medicine. 2018; 18(1):23.
- Bordbar G, Konjkav Monfared A, Sabokro M, Dehghani N, Hosseini E. Human resources competencies scale development and validation: An Iranian measure. Industrial and Commercial Training. 2021; 53(3):250-67.
- Feufel MA, Antes G, Gigerenzer G. [Competence in dealing with uncertainty: lessons to learn from the influenza pandemic (H1N1) 2009]. Bundesgesundheitsblatt, Gesundheitsforschung, Gesundheitsschutz. 2010; 53(12):1283-9.
- Hadian M, Jabbari A, Abdollahi M, Hosseini E, Sheikhbardsiri H. Explore pre-hospital emergency challenges in the face of the COVID-19 pandemic: a quality content analysis in the Iranian context. Frontiers in Public Health. 2022; 10:864019.
- Mohammadi M, Firouzkouhi M, Abdollahimohammad A, Shivanpour M. The challenges of pre-hospital emergency personnel in Sistan area: a qualitative study. Journal of Qualitative Research in Health Sciences. 2019; 8(3):221-32.
- Poornazari M, Roshanzadeh M, Mohammadi S, Taj A, Shirvani M, Naghneh MHK, et al. Explaining the challenges of pre-hospital emergency healthcare workers in providing care at the scene. Journal of Emergency Medicine, Trauma & Acute Care. 2023; 2023(2):21.
- Rezaei SA, Abdi A, Akbari F, Moradi K. Nurses' professional competences in providing care to the injured in earthquake: A qualitative study. J Educ Health Promot. 2020; 9:188.
- Siddiqi A, Beikzad J, Nejadhaji Ali Irani F, Vedadi A. Designing a Professional Competence Model for Emergency Medical Technicians of the National Emergency Management Organization. Health in Emergencies and Disasters Quarterly. 2025; 10(2):143-52.
- Saxena S, Gujral HK, Pathak S. A study of competency-based approach to identify high performers among disaster management professionals. Studies of Applied Economics. 2022; 40(S1):1-12.
- Feldmann-Jensen S, Jensen SJ, Smith SM, Vigneaux G. The next generation core competencies for emergency management. Journal of Emergency Management. 2019; 17(1):17-25.
- Stefl ME. Common competencies for all healthcare managers: the healthcare leadership alliance model. Journal of healthcare management. 2008; 53(6):360-73.
- Cripe. Competency Development Guide: Resource Guide for Developing Competencies. USA: Workitect, Incorporated; 2012.
- Shojaei F, Sohrabizadeh S, Qaraeian P, Khoshsabeghe HY, Jandaghi J, Farivar F. Comparative analysis of two disaster risk assessment tools for primary health care centers: The case of Iran. International Journal of Disaster Risk Reduction. 2024; 107:104466.
- Mohammadinia L, Khorasani-Zavareh D, Ardalan A. Children capacity in disaster risk reduction: a call for action. Journal of injury and violence research. 2017; 9(1):69-70.
- Farzanegan MR, Fischer S, Noack P. Natural disaster literacy in Iran: Survey-based evidence from Tehran. International Journal of Disaster Risk Reduction. 2024; 100:104204.
- DeVon HA, Block ME, Moyle‐Wright P, Ernst DM, Hayden SJ, Lazzara DJ, et al. A psychometric toolbox for testing validity and reliability. Journal of Nursing scholarship. 2007; 39(2):155-64.
- Hu Lt, Bentler PM. Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural equation modeling: a multidisciplinary journal. 1999; 6(1):1-55.
- Lawshe CH. A quantitative approach to content validity. Personnel psychology. 1975; 28(4):563-75.
- Mokkink LB, Terwee CB, Patrick DL, Alonso J, Stratford PW, Knol DL, et al. The COSMIN checklist for assessing the methodological quality of studies on measurement properties of health status measurement instruments: an international Delphi study. Quality of Life Research. 2010; 19(4):539-49.
- Pollard B, Dixon D, Dieppe P, Johnston M. Measuring the ICF components of impairment, activity limitation and participation restriction: an item analysis using classical test theory and item response theory. Health and Quality of Life Outcomes. 2009; 7(1):41.
- Ma H, Zhu X, Zhang S, Huang J, Lu Y, Luo Y. Development and validation of professional competency scale for military nurses: an instrument design study. BMC Nursing. 2022; 21(1):90.
- Marin SM, Hutton A, Witt RR. Development and psychometric testing of a tool measuring nurses’ competence for disaster response. Journal of emergency nursing. 2020; 46(5):623-32.
- Li B, Akintoye A, Holt G. Empirical study of project governance among Chinese project management professionals. International Journal of Architecture, Engineering and Construction. 2017; 6(2):29-39.
- Lleo A, Ruiz-Palomino P, Viles E, Muñoz-Villamizar AF. A valid and reliable scale for measuring middle managers’ trustworthiness in continuous improvement. International Journal of Production Economics. 2021; 242:108280.
- Lockyer J, Carraccio C, Chan MK, Hart D, Smee S, Touchie C, et al. Core principles of assessment in competency-based medical education. Med Teach. 2017; 39(6):609-16.
- Hamilton SJ, Briggs L, Peterson EE, Slattery M, O’Donovan A. Supporting conscious competency: Validation of the generic supervision assessment tool (GSAT). Psychology and Psychotherapy: Theory, Research and Practice. 2022; 95(1):113-36.
- Amani S, Dolatabadi ZA, Bardsiri TI, Nouri M, Saraji JN, Poursadeqiyan M, et al. Designing and validation of self-assessment tool of professional competency and its psychometrics in the Administrators of Preschool Educational Centers. Iranian Journal of Public Health. 2022; 51(11):2582.
- Nilsson J, Johansson S, Nordström G, Wilde-Larsson B. Development and validation of the ambulance nurse competence scale. Journal of emergency nursing. 2020; 46(1):34-43.
- Howard PF, Liang Z, Leggat S, Karimi L. Validation of a management competency assessment tool for health service managers. Journal of health organization and management. 2018; 32(1):113-34.
- Mohammadi F, Salimi R, Shokouhi M, Khazaei S, Bijani M. Design and psychometric evaluation of the Emergency Medical Services Safe Care Scale (EMSSCS): a mixed-method study. BMC emergency medicine. 2025; 25(1):67.
- Castaño AM, Zuazua M, García-Izquierdo AL. Development, measurement, and validation of a managerial competency model in Spain: The pumac questionnaire. European Journal of Psychological Assessment. 2023; 40(4):298-310.
- Paz LMCÁ, Odelius CC. Managerial competencies scale in a public management context: development and validation evidences. Organizações & Sociedade. 2021; 28:370-97.