Introduction

"Clinical competency assessment" is significantly important ( 1 ). Assessment makes managers and nurses aware of cognitive competencies, skills, and problems. Therefore, they attempt to eliminate the defects and increase the nurses' competency ( 2 ). "Clinical competency" is closely related to the quality of care. It reduces medical errors, provides safe and cost-effective services ( 3 ), and provides opportunities for developing and promoting the nursing profession ( 3 , 4 ). The lack of clinical competency endangers the safety of the patients and society ( 5 ).

Clinical competence is anticipated to be achieved by academic training ( 6 ). Ineffective mental nursing education may have a permanent negative impact on the health in society since the quality of medical science education is crucial to the health in society ( 7 , 8 ). A substantial part of the nursing curriculum includes clinical instruction. It provides the opportunities for students to acquire necessary competency in caring for the patients ( 9 ). Similarity, evaluation is the heart of learning and training process ( 10 ). It recognizes the strengths and weaknesses to improve the situation. In general, they are the most important tasks of universities ( 7 , 8 ).

However, so far, there is no agreement on the best way to teach or evaluate clinical skills. This can be a challenge for both professionals and educators ( 10 ). Given the complexity of life, increasing chronic diseases, and needs to competent health cares ( 11 , 12 ), it is necessary that the competency concept gets updated and continuously evaluated, especially among nursing students preparing for their future role as a nurse ( 13 ). However, there are many concerns about effective evaluation and differentiation among the students' performance levels ( 14 ). Students' competency assessment according to clear professional standards is the core of ensuring that students will provide safe nursing care ( 15 ). To the best of our knowledge, postgraduate psychiatric nursing clinical competence has no established definition ( 16 ). Clinical competence evaluation is a difficult procedure as a result of the many definitions and the expanding nature of competency ( 17 ). Various tools have been developed to objectively assess clinical competency. They assess the student's clinical competency in nursing care based on the safe and quality standards ( 15 ). Most of them were designed to evaluate the clinical performance of undergraduate students and graduates in nursing. Also, they focus on medical surgical field, such as studies by Axley’ (2016), Mwaniki, and Abuya (2013) ( 18 , 19 ). Few articles have been found on the field of psychiatric nursing competency, including the study of McKnight (2013), Jormfieldet, et al. (2018), and Moskoei, et al. (2017) ( 20 - 22 ). However, they do not evaluate postgraduate psychiatric nursing students‘clinical competency. On the other hand, there is no agreement in the nursing profession about how nursing care quality can be measured ( 4 ). Consequently, clinical competency evaluation remained a major challenge in the nursing profession ( 23 ).

Broad and sophisticated techniques are needed to develop competence in evaluating strategies that are based on a holistic viewpoint ( 24 ). The use of valid and reliable instruments ( 25 ) to appropriately verify the desired goals ( 4 ) is, therefore, one of the ways used to overcome the complexity of the clinical skills assessment process, as well as to overcome subjective and inaccurate evaluation techniques ( 26 , 27 ). "Clinical Competence of Mental Health Nurse (CCMHN)" tool with 45 items was designed and psychometrically evaluated by Moskoei, et al. (2017) in Iran ( 22 ). It assesses the competence of graduate nurses working in psychiatric settings. In Iran, a graduate nurse has passed a 4-year undergraduate nursing education course, which is full-time with four theoretical units, and two practical units on mental health and mental disorders ( 7 ). The master of mental nursing program is full-time, similar to other programs. However, it comprises of 12 theoretical units and 14.5 units of apprenticeship on mental diseases and health, and psychiatric nursing treatments. It takes 2-3 years ( 8 ). Definitely, the last one later consists of more specific tasks. Therefore, it seems ineffective to evaluate the specific level of competence required for MSc psychiatric nurses by CCMHN. Furthermore, the CCMHN tool does not meet all criteria of consensus-based standards for the selection of health status measurement instruments (COSMIN (such as SEM, responsiveness, hypothesis test, and criterion validity.

Another questionnaire designed by Feng, et al. (2018) ( 28 ) is used for graduate nurses working in the psychiatric setting. It has 17 items in five domains of Responsibility, Recognition of description of duties, Acceptability, Capacity of cooperation, and Accuracy, with Cronbach's alpha above 0.7. However, according to COSMIN criteria, it lacks quantitative content validity such as CVI, CVR, and modified Kappa. Besides, there is no specific psychiatric nursing items. Overall, the instruments assessing psychiatric nurses' competencies measure the expected competencies for graduate nurses working in the psychiatric ward. They have not examined the clinical competencies expected for postgraduate psychiatric nurses, which are devoted to the curriculum and nursing interventions classification. This study aimed to develop and evaluate psychometric properties of an instrument assessing postgraduate psychiatric nursing clinical competency.

Methods

Study design

This is a sequential exploratory mixed-method study conducted in a qualitative and quantitative phase, from 2019 to 2022, in Iran.

The qualitative phase

This phase explored the postgraduate psychiatric nursing clinical competency concept and its dimensions to generate the primary item pool.

Sampling and data collection

Sampling was purposively performed with maximum variation in participants’ work experience and education. Data were collected via semi-structured interviews in a private and calm place, according to the participants' preferences. An example of interview questions was, "in your opinion, what competencies are required for postgraduate psychiatric nursing students?" Moreover, participants were asked some probing questions, such as "could you explain more about this?" Sampling and data collection continued until data saturation, when no new data are obtained from the interviews ( 29 ).

Data were analyzed using the conventional content analysis method suggested by Granheim and Lundman (2004) ( 30 ), and the dimensions were determined. Data trustworthiness was ensured using four criteria proposed by Guba and Lincoln, called credibility, dependability, confirmability, and transferability ( 31 ).

Inclusion criteria

Postgraduate psychiatric nurses and students, specialists, and psychiatric team members working at universities and settings offering a postgraduate psychiatric nursing courses.

The quantitative phase

In this phase, an item pool was generated based on the findings of qualitative section, and a systematic review of the literature, documents, guidelines, and instruments on clinical competency assessment by the research team members. We used the keywords as "Postgraduate" OR "master's degree" AND "Psychiatric nursing", "Psychiatry", "Clinical competency OR competence" AND "Tools" OR "Instruments" OR "Questionnaires" AND, "Interventions", "Curriculum", in Persian and English, separately and combined, in the information database, including Scientific Information Database (SID), Magiran, information bank of medical science articles (IranMedex), ProQuest, PubMed, Google Scholar and PsychInfo. The purpose was to find the items that were probably not extracted from the qualitative phase. Psychometric properties were evaluated based on the COSMIN criteria, including face, content, construct (convergent, known group, and exploratory factor analysis) validity, and its reliability. Reliability was evaluated using the internal consistency, stability, and absolute reliability assessments. Then, responsiveness (through minimally detectable changes) and interpretability (through minimal important changes) were calculated.

Validity

Face validity

Face validity was evaluated qualitatively (in person) with regards to readability. Ten randomly selected professors from mental nursing internship programs were questioned on the items' difficulty and vagueness. The research team’s feedback resulted in changes to the products.

Content validity

Simultaneously, qualitative and quantitative content validity were performed. For qualitative content validity assessment, ten experts (in psychiatric nursing) purposively sampled were invited to read and comment on grammar, wording, item allocation, and scoring. The items were revised based on their comments. Quantitative content validity assessment was performed by calculating content validity ratio (CVR), content validity index (CVI), and chance agreement among experts according to Polit &Yang’s (2016) proposal to decrease the chance agreement of experts ( 32 ). For CVR calculation, ten experts in psychiatric nursing sampled purposively were asked to rate the essentiality of each item. Items with a CVR of 0.62 or more were considered acceptable according to Lawshe (1975) ( 33 ). For CVI and the modified Kappa method calculation, ten experts, purposively sampled, were asked to rate the relevance of each item.

Item content validity (I-CVI) would be≥0.78, the scale content validity index/average (SCVI/Ave) >0.8, and Kappa designating agreement on relevance (K*) ≥0.74 were considered acceptable (K* was calculated by using formula of K*= (I_CVI-PC)/(1-PC). In this formula, PC is the chance agreement between evaluators.

Primary reliability

A pilot research was carried out to evaluate the fundamental dependability. As a result, the Loop technique was used to perform the item analysis procedure. Accordingly, 30 postgraduate psychiatric nursing students answered the questionnaire for this research. The reliability coefficient for the whole questionnaire and each item was calculated. If the reliability decreases by eliminating each item, it indicates that the item plays an effective role in coordination with other items. Therefore, it is an appropriate item ( 35 ). A correlation coefficient of ≥0.3 between at least two statements was acceptable. The correlation of two statements more than 0.8 with semantic similarity got merged ( 25 ).

Construct validity

Exploratory Factor Analysis (EFA)

Construct validity shows how much the concept formed is based on theoretical foundations measured by a tool ( 32 ). It was assessed via exploratory factor analysis (EFA), convergent, and known group validity ( 36 - 39 ). Due to the coronavirus pandemic, most of universities did not offer internship for MSc psychiatric nursing students, so the sample size was decided according to Munro (2005). In this method, a sample size of 3-10 subjects per item is adequate for most objectives ( 40 ). Moreover, the Postgraduate Psychiatric Nursing Clinical Competency Questionnaire (PPNCCQ) was completed for 199 postgraduate students sampled conveniently for EFA. The sample size adequacy is indicated in the anti-image correlation matrix diagonal. A Kaiser-Myer-Olkin (KMO) value of ≥0.80 was used to indicate an adequate sample. Significant Bartlett’s sphericity test was used to indicate appropriateness of EFA. KMO and Bartlett test evaluate all available data together.  A KMO value over 0.5 and a significance level for the Bartlett's test below 0.05 suggest that there is substantial correlation in the data. Variable collinearity indicates how strongly a single variable is correlated with other variables ( 32 ). The minimum acceptable factor loading was 0.4 ( 25 ).

The normality of univariate data was evaluated by assessing skewness (less than±3) and kurtosis (less than±7). For identifying outliers and missing data in univariate data, a frequency table and box plot were used. Any data point that deviates from the mean by a distance of 3 standard deviations is considered an outlier ( 41 ).

In EFA, latent factors were extracted through maximum likelihood estimation with Promax rotation, Eigen value >1, use of scree plot, and variance >%5 for each factor. They were interpreted and labeled by the research team based on factor loading and meaning.

Convergent validity

It is used to study the correlation among the measurements obtained; using this new scale and another scale demonstrated its validity by previous studies, and assesses similar construct ( 32 ). As a result, in this research, both the final PPNCCQ and the "self-assessment clinical competency questionnaire" were used to evaluate the clinical competency of 30 postgraduate psychiatric nursing students who were selected using convenient sampling. Convergent validity between the two instruments was calculated. In this study, a correlation between 0.4 and 0.7 is expected.

The tool of "self-assessment clinical competency questionnaire", as a golden standard, was a 46-item questionnaire designed by Liou and Cheng (2014) to assess the competency of undergraduate nursing students ( 42 ). This was translated into Persian, and psychometric properties were evaluated by Nehrir, et al. (2018), yielding a reliability coefficient of 0.96 and ICC=0.98 ( 43 ). The questionnaire is divided into five areas: Practical competency (9items), Advanced competency (9items), Ethical competency (12items), Care management competency (9 items), and Safety qualification (7items). The psychometric properties of tool selected as golden standard, should be valid and reliable ( 36 ).

Known Groups Validity

This type of construct validity measures the ability of the tool to distinguish between two specific groups and determine the differences in the groups ( 32 ). This is the best method for assessing the construct validity of checklists, and observational and formative tools ( 44 ).

In order to compare the groups that were predicted to vary in a certain feature, the researchers employed the PPNCCQ. An independent T-test was used to compare the differences between postgraduate students with and without experience in working in psychiatric settings for more than five years. Criterion validity was not conducted in the current study because, according to Polit and Yang (2016) ( 32 ), it requires an objective and external variable that was not available in this study.

Reliability

Reliability is evaluated by three characteristics of internal consistency, stability (i.e. intra rater), and standard error of measurement (SEM) ( 36 ).

Internal consistency

After completing the questionnaire developed in this study for 33 postgraduate students, internal consistency was determined via obtaining Cronbach’s alpha coefficient for the whole tool and subscales. Satisfactory internal consistency is established when Cronbach’s alpha value is between 0.70-0.95 ( 45 ).

Stability

For the stability of the observers (intra-rater), the researcher (as an observer) completed the questionnaire twice, separated by one-month intervals, for 33 MSc mental nursing students. The intra-class correlation coefficient (ICC) was then computed using the absolute agreement technique and the two-way mixed effects model ( 46 ).

Absolute reliability

Absolute reliability was calculated using SEM and the following formula, $\mathrm{SEM}=\mathrm{SD}\times \sqrt{(1-{\mathrm{ICC}}_{\mathrm{agreement}})}$ ( 29 ). SD is the standard deviation of the observed set of scores, and the ICC is an intra-class continuity ( 32 ).

Scoring

A linear transformation formula ($\frac{\mathrm{Raw\; score-Min\; Score}}{\mathrm{Max\; Score-Min\; Score}}\times 100$) was used. Raw score = a score of each person from the whole questionnaire, Min score and Max score are the minimum and maximum score of questionnaire, respectively ( 32 ).

Feasibility

Feasibility was defined by determining the relative frequency of the items not responded. Moreover, it was assessed based on the average time needed for filling up ( 32 ).

Responsiveness

The ability of the questionnaire to detect changes over time shows responsiveness, which is studied by comparing the minimal detectable change (MDC) ( 32 ). In this study, changes over time were calculated through minimally detectable changes with formula: $\mathrm{SEM}\times Z\times \sqrt{2}$ .

Interpretability

Interpretability means the amount of the significance of the changes in "minimum important changes (MIC)" in the instrument score ( 35 ). The difference among clinical competency scores (total instruments) in different groups as to demographic characteristics of students was studied.

Data analysis

Statistical data analysis was performed using SPSS16. The correlation coefficient for convergent validity, independent T _test for known group validity, Cronbach's alpha for item analysis and internal consistency, and ICC for reliability were used.

Ethical consideration

This research was a subset of a bigger study and was taken from the doctoral dissertation that was approved by the Islamic Azad University of Medical Sciences Ethics Committee in Tehran, Iran (IR. IAU. TMU. REC. 1399.171). All participants provided their informed permission after being made aware of the confidentiality of the study data management.

Results

Qualitative: Concept definition

21 participants were interviewed, including postgraduate psychiatric nursing students and instructors, psychiatric nurses, psychologists, and psychiatrists from universities offering courses with at least two years of experience (in Tehran and Mashhad, Iran) (Table 1). Interviews lasted 25-60 min.

The competency of postgraduate students studying psychiatric nursing was described as having the ability to “dynamistically learn specialized knowledge and skills, to implement creatively in clinical practice, and to do critically clinical reasoning; as well as having individual and family properties, physical and mental competence, and commitment to ethical principles. By managing nursing care, provides integrated care for the patient.” (Table 2).

Item generation

73-item preliminary item pool was extracted after defining postgraduate psychiatric nursing clinical competency and its dimensions, in the qualitative phase. Then, it was completed via literature review. It consisted of 73 items, 84.3% of which were derived from the qualitative section and 15.7% were from the literature reviews. The instrument was designed in three fields (Personal characteristics, Technical competency, and Meta competency), based on a ranking scale and in two kinds of Likert’s five-point scales (never=0 to always=4, and none=0 to very much=4).

Validity

Face validity

To make the items easier to understand, several statements were changed at the face validity stage. The number of statements was reduced to 67 after similar-meaning statements were combined to minimize redundancy and certain assertions were clarified.

Content validity

In quantitative content validity assessment, 4 items at CVR (<0.62) and 8 items at CVI (<0.78), and k* (<0.74) were deleted. At the end of this stage, 55 items remained in the questionnaire.

Item analysis

Cronbach’s alpha was 0.984. Nine items with no correlation with any items more than 0.3 were removed. The number of statements reached 46. By removing none of the items, it has an effect on increasing the alpha coefficient. In other words, there were no unrelated questions.

Construct validity

The data distribution was normal (skewness =0.57, Kurtoses =0.34). Missing data and outliers were zero. Based on the KMO value of 0.962, the adequacy of sampling volume was obtained (KMO>0.8). Considering the significance of Bartlett's sphericity test (p<0.001), factor analysis based on the correlation matrix can be justified. Three items were removed for loadings (p≤0.4) and excluded from the analysis. Two factors were extracted in EFA, which explained 67.5% of the total variance. The first factor labeled "Education, and nursing care" (factor loading=0.98) with 23 items was: Professional ethics: items 13,19,1; Educating patients and their family on medication and self-care: items 4,8,9,11,14; Nursing practical skills: items 2,3,5,6,7,10,13,17,18,22,23; Nursing reports and documents: items 20,21; Communication and cooperation: item 16; the second factor labeled "Evidence-based psychiatric nursing interventions" (factor loading=0.97) with 20 items was: Evidenced-based psychiatric nursing cares: items 24,25, 26; Psychiatric knowledge: items 27,29,33; Psychiatric nursing interventions: items 31,7, 35; Implement of knowledge in practice: items 34,39,40,43; Clinical reasoning: items 32,38; and Lifelong learning: item 41 (Table 3) (Figure 1).

Figure 1. Scree plot

Consequently, the final version of 43-item PPNCCQ which consisted of two domains; it was scored on a five-point Likert scale, ranging from never [1] to always [4] for 32 items and never [1] to very much [4] for 11 items.

Convergent validity

Convergent validity was obtained by a correlation coefficient of 0.488. Given the correlation obtained is between 0.4 and 0.7, hypothesis of this study is confirmed. Because correlation observed in convergent validity should not be too high to indicate that the new instrument does not provide new information compared to previous one, or it should not be too low to show that the current instrument does not assess the clinical competency construct ( 25 ).

Known groups validity

The test showed that the difference between the mean scores of the two groups was significant (P<0.001). The competency score of experienced students (n=54, Mean=178.9±35.16) was higher than that of the novice group (n=63, Mean=156.3±32.03). This demonstrated the importance of clinical skills and meta competencies, which were gradually obtained.

Reliability

Internal consistency

The Cronbach’s alpha of the total scale, factor 1, and factor 2 was 0.947, 0.897, and 0.891, respectively.

Stability (Relative reliability)

ICC _agreement was 0.956(CI: 0.907- 0.980).

SEM (Absolut reliability)

SEM was 3.14. This index tells us about the precision of a given measurement. The lower the SEM value, the greater the degree of precision ( 32 ).

Scoring

Positive ratings were given to each item. A maximum score of 115 and a minimum score of 23 may be assigned to the first factor. A maximum score of 100 and a minimum score of 20 might be assigned to the second element. Students who had a score of 0–33 had a low clinical competence rate, scores 33–66 had an intermediate rate, and scores 66–115 had an optimal rate.

Responsiveness

MDC in interval confidence %95 is equal to 8.7. It shows the minimal real changes (not the change in measurement error).

Interpretability

Results showed that the interpretability of the clinical competency score was not related to demographic variables (Table 4).

Feasibility

Given the relative frequency of missing data for each subject is zero, the feasibility is high. The average time to complete the instrument of this study was 14 minutes.

Discussion

Findings of this study show that, based on COSMIN criteria, the Postgraduate Psychiatric Nursing Clinical Competency Questionnaire (PPNCCQ) is a valid and reliable instrument to evaluate postgraduate psychiatric nursing students' competencies. Based on the COSMIN checklist, four domains of validity (face, content and construct validity), reliability (internal consistency, stability, SEM), responsiveness, and interpretability were evaluated for "PPNCCQ". This instrument includes 2 domains: The first dimension with 23 items labeled according to EFA, and total content including the description of the nursing duties such as clinical examination of the patient, nursing care, educating the client and the family about the rules of the ward, disease and drugs, oral and written nursing reports, and following professional ethics. The second dimension consisted of 22 items, focusing on the specialized duties of a psychiatric nurse in both areas of meta-competency and technical competency, such as the implementation of interventions based on psychiatric nursing evidence, conducting research, teaching colleagues and students, management and inter-professional participation in the psychiatric team, critical thinking, clinical reasoning, and lifelong learning. Therefore, it can solve the challenges of managers and instructors for objectively evaluating postgraduate psychiatric nursing students' competencies. Initially, the postgraduate students in psychiatric nursing were considered to have the following competencies for designing such an instrument: "to dynamically learn specialized knowledge and skills, to implement creatively in clinical practice, and to do critically clinical reasoning; in addition to having individual and family properties, physical and mental competence, and commitment to ethical principles.”, Integrated care is provided for the patients by managing nursing care and using interpersonal communication skills." This definition consists of psychiatric nurses' tasks based on nursing interventions classification (NIC), and curriculum more than previous ones. This instrument includes two fields: "Education and nursing cares", and "Evidence-based psychiatric nursing interventions". Prior to this study, no instrument was found that specifically measures postgraduate psychiatric nursing students' competency.

To the best of the researchers’ knowledge, there are few instruments in the psychiatric nursing domain, such as the CCMHN tool, which has two factors: Specific care competencies and Emotional/moral competency ( 22 ). Specific care competencies of the CCMHN tool, which is similar to "Education and nursing care" domain of the current study, _ are: "using different sources to collect information about the patient, using screening methods to detect hazardous behavior of the patient, recording and reporting changes in patient’s condition in a proper way, detecting defensive mechanisms in the patients, and having general practical skills (i.e. injection, administering medicines)". In the CCMHN tool, general nursing care and specific psychiatric nursing care are integrated, but, in the current study, a wider range of specialized psychiatric nursing interventions, with a holistic perspective, are emphasized separately. Also, this study meets all COSMIN criteria. This inconsistency can be explained by the fact that the CCMHN tool was designed for graduate nurses working in psychiatric settings. Hence, it focuses on general nursing cares skills more.

The 17-item questionnaire developed by Feng, et al. (2018) also serves as a useful tool ( 28 ). It comprised five factors: a sense of responsibility, vocational identity, agreeableness, cooperation ability, and carefulness. This instrument is also related to the fundamental abilities of nurses. No statements were found about specialized psychiatric nursing interventions by researchers of this study. One item is for inter-professional cooperation, one item for helping the family, and two items for assessing and determining the patient's needs by receiving adequate information from the patient. Similar to the CCMHN, Feng, et al. tool was designed for graduate nurses working in psychiatric settings, too. Therefore, it is not included in specialized psychiatric nursing interventions. However, these items are similar to "Education and nursing care" domain of the current study which is presented in general skills.

Finally, the competency-focused psychiatric nursing clinical evaluation tool by Bondy, et al. (1997) has six categories including "Self-assessment, Nursing process, Knowledge and critical thinking, Nursing interventions, Communication skills, also Professional socialization behaviors" ( 47 ). Except for critical thinking and communication skills, it seems that neither category is focused on the psychiatric nursing area and does not adhere to COSMIN standards. The intended audience for Bondy, et al.’s tool, who are students of psychiatric nurses, may be blamed for the lack of uniformity. This tool also needs to be updated in light of recent advances in healthcare, mental diseases, and treatment. The current study includes not only all of six categories, but also postgraduate psychiatric nursing skills devoted to two different domains of general and specialized skills.

The Postgraduate Mental Nursing Clinical Competency Questionnaire (PPNCCQ), which was designed in the present research, stands apart from the existing instruments on nurses' abilities since it covers a wider range of mental nursing treatments. These interventions meet the mentioned tasks in the NIC and postgraduate psychiatric nursing curriculum. Furthermore, this is emphasized on the holistic perspective. Similarly, in addition to educating the patient and his/her family, as mentioned in previous studies, training colleagues and student were also mentioned. Besides, the current study emphasizes the students' skill in sustaining their own mental health;other no study has pointed to it. However, studies considered psychiatric nursing profession as one of the most stressful professions in nursing field ( 48 - 50 ). Therefore, psychiatric nurses should be able to keep their mental health.

Limitations and strengths

There was a limited access to postgraduate psychiatric nursing professors and students during the Coronavirus pandemic, which led to closure of universities. Failure to offer internships for postgraduate psychiatric nursing students during the Coronavirus pandemic not only inhibited the sampling with high numbers, but also took a long time to sample. Therefore, access to valid Embase and ISI sites was impossible. Also, the systematic literature review was limited to the Persian and English languages.

This study has strengths that outweigh the limitations of the study. First, this is one of the first studies to develop a psychometric test, an instrument to objectively measure postgraduate psychiatric nurses' competency to improve their services, especially in this decade when mental disorders have increased.

Conclusion

The results demonstrate that the COSMIN checklist has approved the psychometric qualities. Both psychiatric nurses and postgraduate students in psychiatric nursing will be able to use the questionnaire. There are certainly some flaws in this. Therefore, additional research and practical implementation of this questionnaire might reveal its flaws. As a result, the flaw may be fixed.

Acknowledgement

The authors would like to thank and appreciate the participants of the research.

Authors’ contributions

The first author: Concept and design of study -acquisition of data -analysis and interpretation of data- Drafting the manuscript-

The second and third authors: Analysis and interpretation of data, revision and final approval of the version.

Conflict of interest:

None declared.

References

1. Istomina N, Suominen T, Razbadauskas A, Martinkėnas A, Meretoja R, Leino-Kilpi H. Competence of nurses and factors associated with it. Medicina. 2011; 47(4):230-7.
2. Drennan J. Masters in nursing degrees: an evaluation of management and leadership outcomes using a retrospective pre‐test design. Journal of Nursing Management. 2012; 20(1):102-12.
3. Tajvidi M, Moghimi Hanjani S. The relation between critical thinking and clinical competence in nurses. Strides in Development of Medical Education. 2019; 16(1):e80152.
4. Sim J, Crookes P, Walsh K, Halcomb E. Measuring the outcomes of nursing practice: A Delphi study. Journal of clinical nursing. 2018; 27(1-2):e368-78.
5. Chen SC, Lee SK, Rong JR, Wu CC, Liu WI. The development and psychometric testing on psychiatric nurses of a nurse case management competence scale in Taiwan. Journal of Nursing Research. 2018; 26(2):72-9.
6. Lovrić R, Prlić N, Milutinović D, Marjanac I, Žvanut B. Changes in nursing students&#039; expectations of nursing clinical faculties&#039; competences: A longitudinal, mixed methods study. Nurse education today. 2017; 59:38-44.
7. Jalali R, Rigi F, Parizad N, Amirian Z. Comparison of undergraduate nursing education in Iran and George Washington University. Educational Research in Medical Sciences. 2016; 5(2):64-73.
8. Alaee Karahroudy F, Sheikhbahaeddinzadeh E. A comparison of the psychiatric nursing masters&#039; curriculum in Iran and Canada. Strides in Development of Medical Education. 2020; 17(1):1-6.
9. Helminen K, Johnson M, Isoaho H, Turunen H, Tossavainen K. Final assessment of nursing students in clinical practice: Perspectives of nursing teachers, students and mentors. Journal of clinical nursing. 2017; 26(23-24):4795-803.
10. Bloomfield J, Fordham-Clarke C, Pegram A, Cunningham B. The development and evaluation of a computer-based resource to assist pre-registration nursing students with their preparation for objective structured clinical examinations (OSCEs). Nurse Education Today. 2010; 30(2):113-7.
11. Adams S. Psychiatric mental health nursing: “A seat at the table. ” J Am Psychiatr Nurs Assoc. 2015; 21:34-7.
12. Fukada M. Nursing competency: Definition, structure and development. Yonago Acta Med. 2018; 61:1-7.
13. Rebueno MC, Tiongco DD, Macindo JR. A structural equation model on the attributes of a skills enhancement program affecting clinical competence of pre-graduate nursing students. Nurse Education Today. 2017; 49:180-6.
14. Regehr C, Bogo M, Donovan K, Lim A, Regehr G. Evaluating a scale to measure student competencies in macro social work practice. Journal of Social Service Research. 2012; 38(1):100-9.
15. Immonen K, Oikarainen A, Tomietto M, Kääriäinen M, Tuomikoski AM, Kaučič BM, et al. Assessment of nursing students&#039; competence in clinical practice: a systematic review of reviews. International journal of nursing studies. 2019; 100:103414.
16. Mohtashami J, Salsali M, Pazargadi M, Manoochehri H, Majd HA. Competency-based curriculum education in mental health nursing. Open Journal of Nursing. 2013; 3(8):545-51.
17. American Psychiatric Nurses Association. Psychiatric-mental health nurse essential competencies for assessment and management of individuals at risk for suicide [Internet]. 2015 [Cited 22 March 2015]. Available from: http//www.apna.org/i4a/pages/index Cfm.
18. Axley L. Competency: A concept analysis. Nursing forum. 2008; 43(4):214-22.
19. Mwaniki MC, Abuya T. Evaluation of a Tool for Assessing Clinical Competence of Msc Nurse Students. Evaluation. 2013; 3(13):53-61.
20. McKnight SE. Mental health learning needs assessment: Competency-based instrument for best practice. Issues in mental health nursing. 2013; 34(6):459-71.
21. Jormfeldt H, Doyle L, Ellilä H, Lahti M, Higgins A, Keogh B, et al. Master’s level mental health nursing competencies, a prerequisite for equal health among service users in mental health care. International journal of qualitative studies on health and well-being. 2018; 13(sup1):1502013.
22. Moskoei S, Mohtashami J, Ghalenoeei M, Nasiri M, Tafreshi MZ. Development and psychometric properties rating scale of “clinical competency evaluation in mental health nurses”: Exploratory factor analysis. Electronic physician. 2017; 9(4):4155-61.
23. Karimi-Moonaghi H, Gazerani A, Vaghee S, Gholami H, Salehmoghaddam AR, Gharibnavaz R. Relation between spiritual intelligence and clinical competency of nurses in Iran. Iran J Nurs Midwifery Res. 2015; 20(6):665-9.
24. McMullan M, Endacott R, Gray MA, Jasper M, Miller CM, Scholes J, et al. Portfolios and assessment of competence: a review of the literature. Journal of advanced nursing. 2003; 41(3):283-94.
25. Fahy A, Tuohy D, McNamara MC, Butler MP, Cassidy I, Bradshaw C. Evaluating clinical competence assessment. Nursing Standard. 2011; 25(50):42-9.
26. Nazari shadekam F, Ghorbani A, Karimi F, Sadeghi T. Identifying Effective Factors on Quality of Nursing Curriculum and Their Competence. RME. 2015; 7(1):45-53. Persian.
27. Ebadi A, Zarshenas L, Rakhshan M, Zareyan A, Sharif Nia H, Mojahedi M. Principles of scale development in health science. Jame-e-Negar. Persian: Tehran; 2018.
28. Feng D, Li H, Meng L, Zhong G. Development of a Questionnaire to Assess Nursing Competencies for the Care of People with Psychiatric Disabilities in a Hospital Environment. Psychiatr Q. 2018; 89(3):699-706.
29. Creswell JW. Research design qualitative, quantitative and mixed methods approaches. SAGE: Los Angeluses; 2018.
30. Graneheim UH, Lundman B. Qualitative content analysis in nursing research: concepts, procedures and measures to achieve trustworthiness. Nurse education today. 2004; 24(2):105-12.
31. Tobin GA, Begley CM. Methodological rigor within a qualitative framework. Journal of advanced nursing. 2004; 48(4):388-96.
32. Polit DF, Yang F. Measurement and the measurement of change: a primer for the health professions. Wolters Kluwer: Netherlands; 2016.
33. Lawshe CH. A quantitative approach to content validity. Personnel psychology. 1975; 28(4):563-75.
34. Hyrkäs K, Appelqvist-Schmidlechner K, Oksa L. Validating an instrument for clinical supervision using an expert panel. International Journal of nursing studies. 2003; 40(6):619-25.
35. Gnaldi M, Matteucci M, Mignani S, Falocci N. Methods of item analysis in standardized student assessment: An application to an Italian case study. The International Journal of Educational and Psychological Assessment. 2013; 12:78-92.
36. Mokkink LB, Boers M, Van Der Vleuten CP, Bouter LM, Alonso J, Patrick DL, et al. COSMIN Risk of Bias tool to assess the quality of studies on reliability or measurement error of outcome measurement instruments: a Delphi study. BMC medical research methodology. 2020; 20:1-3.
37. Mokkink LB, Terwee CB, Patrick DL, Alonso J, Stratford PW, Knol DL, et al. The COSMIN study reached international consensus on taxonomy, terminology, and definitions of measurement properties for health-related patient-reported outcomes. Journal of clinical epidemiology. 2010; 63(7):737-45.
38. Aminizadeh M, Farrokhi M, Ebadi A, Masoumi GR, Beyrami-Jam M, Khankeh HR. COSMIN Checklist for Systematic Reviews of the Hospital Preparedness Instruments in Biological Events. Journal of Nursing Measurement. 2021; 29(3):1-22.
39. Mokkink LB, Terwee CB, Knol DL, Stratford PW, Alonso J, Patrick DL, et al. The COSMIN checklist for evaluating the methodological quality of studies on measurement properties: a clarification of its content. BMC medical research methodology. 2010; 10(1):1-8.
40. Munro B. Statistical methods for health care research. Lippincott Williams&amp;Wilkins: Philadelphia; 2005.
41. Nulty DD. The adequacy of response rates to online and paper surveys: what can be done?. Assessment & evaluation in higher education. 2008; 33(3):301-14.
42. Liou SR, Cheng CY. Developing and validating the Clinical Competence Questionnaire: A self-assessment instrument for upcoming baccalaureate nursing graduates. Journal of Nursing Education and Practice. 2014; 4(2):56.
43. Nehrir B, Ebadi A, Vanaki Z, Nouri JM, Khademolhosseini SM. Psychometric properties of self-assessment clinical competency questionnaire in baccalaureate nursing students. Electron Physician. 2018; 10(3):6494-9.
44. Taghizadeh Z, Ebadi A, Montazeri A, Shahvari Z, Tavousi M, Bagherzadeh R. Psychometric properties of health related measures. Part 1: Translation, development, and content and face validity. Payesh (Health Monitor).. 2017; 16(3):343-57. Persian.
45. Cordier R, Chen YW, Speyer R, Totino R, Doma K, Leicht A, et al. Child-report measures of occupational performance: a systematic review. PLoS One. 2016; 11(1): e0147751.
46. Schreiber JB, Nora A, Stage FK, Barlow EA, King J. Reporting structural equation modeling and confirmatory factor analysis results: A review. The Journal of educational research. 2006; 99(6):323-38.
47. Bondy KN, Jenkins K, Seymour L, Lancaster R, Ishee J. The development and testing of a competency-focused psychiatric nursing clinical evaluation instrument. Archives of Psychiatric Nursing. 1997; 11(2):66-73.
48. Ward L. Mental health nursing and stress: Maintaining balance. International Journal of Mental Health Nursing. 2011; 20(2):77-85.
49. Abdalrahim A. Stress and coping among psychiatric nurses. Middle East Journal of Nursing & Health Sciences. 2013; 7(4):30-7.
50. Zarea K, Fereidooni-Moghadam M, Baraz Sh, Tahery N. Challenges encountered by nurses working in acute psychiatric wards: A qualitative study in Iran. Issues in Mental Health Nursing. 2018; 39(3):244-50.