In an effort to ensure continuous professional development of healthcare professionals, national public health systems create various institutions to stimulate or motivate medical personnel to upgrade their skills [1–3]. The domestic public health system is no exception. Currently, the institutions of certification, categorization, continuing medical education, independent qualification evaluation, internal categorization, are more or less active in our country [4–7]. Functioning of all the above institutions is regulated by the statutory acts of various levels. In particular, categorization of physicians is regulated by the Order of the Ministry of Health [4] updated in 2023 to bring the procedure of assigning qualification grades in line with the newly created categorization and continuing medical education systems.

It is no secret that the domestic institution for categorization of healthcare professionals and pharmacists has been the subject of criticism: overlapping of tasks with the new qualification evaluation institutions was reported, and the gaps in the normative regulation oriented towards the outdated ideas about the qualification and formal characteristics of professional development were disclosed [8–10]. It is clear that the use of formal professional development characteristics as evaluation criteria has led to the categorization weakness in terms of qualification measurement, which is confirmed by the data of expert assessment of the quality of medical care provided by individuals, who have passed categorization [11]. Furthermore, healthcare professionals don't want to give up the categorization institution, but see the need to change it in accordance with the current understanding of physician’s professional development [12].

It should be noted that creating incentives for passing categorization and objectivization of measuring professional competence of evaluated individuals have been leading among proposals to improve the categorization system for healthcare professionals for many years [13]. It is likely that these proposals are reflected in the current procedure of assigning qualification grades in the form of the requirements for knowledge, abilities, skills established for qualification grades of certain levels.

In this regard, the study was aimed to test the qualification requirements of physicians for the association with their qualification and the possibility of accurate full-fledged measurement of those in the evaluated physicians.

METHODS

To achieve the goal, we performed expert assessment of the requirements for assigning various qualification grades established by the Order of the Ministry of Health of the Russian Federation dated 31 August 2023 № 458n “On Approval of the Procedure and Timing for Healthcare and Pharmaceutical Professionals to Undergo Categorization to be Assigned a Qualification Grade” (hereinafter, the Order). The expert assessment was performed based on four items:

• association with the physician’s qualification, i.e. determining whether the required knowledge, abilities, and skills are direct (not formal) signs of professional qualification;
• usability, i.e. feasibility of measuring the required knowledge, abilities, and skills;
• objectivity, i.e. determining whether the measurement results depend on the instruments used for assessment, competence or personal preferences of the expert panel members;
• grade, i.e. determining whether it is possible to accurately determine the extent of the knowledge, abilities, and skills required for each qualification grade.

Expert assessment of all items was performed using the conventional Stapel rating scale (“‒5” to “+5”) without a nullvalue, where “‒5” meant that the requirement did not ensure the association with the physician’s qualification, usability, objectivity, grade, while “+5” meant that the requirement guaranteed the association with the physician’s qualification, usability, objectivity, grade.

All the experts were provided the guidelines on understanding the items and scales. They were also given an evaluation sheet with the described above Stapel rating scale containing the list of 22 requirements for knowledge, abilities, and skills specified in the Order. Expert assessment was moderated by the authors. The experts did not communicate with each other.

The group of experts consisted of seven people. Criteria for selection of experts: graduate in medicine, academic degree of Doctor of Medical Sciences, medical experience exceeding 10 years, experience of scientific and pedagogical activity of at least 7 years, experience in categorization commissions, experience in expert panels. Exclusion criteria: age over 60 years, formally ended medical, scientific and pedagogical professional activity. The experts’ average medical experience was 24.0 ± 2.0 years, and their average experience of scientific and pedagogical activity was 13.4 ± 2.6 years.

Statistical processing of the expert assessment results was performed using the IBM SPSS software. We calculated the sums of scores for four items and 22 requirements. Given the fact that the majority of variables had non-normal distributions after applying the Kolmogorov–Smirnov test (р ≤ 0.0001), we applied the Mann–Whitney U test to confirm the differences in expert assessment (pairwise for four items and distinct requirements) and the Kruskal–Wallis test  to prove the differences in expert assessment (for all requirements).

RESULTS

The sums of expert assessment scores by items were as follows: association with qualification — 477, usability — 316, objectivity — ‒662, grade — ‒699. The Mann–Whitney U test calculation revealed significant differences between variables for pairs of all items (р ≤ 0.0001), except the objectivity–grade pair (р = 0.103).

The table provides the sums of the experts’ scores for each requirement (by item) and the total scores. The Kruskal–Wallis test calculation showed that the experts assessed the required knowledge, abilities, and skills by four items differently (р ≤ 0.001).

In experts’ opinion, the requirement for the ability to estimate the data of specific assessment methods to make the diagnosis, for theoretical knowledge and practical skills in the field of professional activity, for applying the prevention, diagnosis, treatment, and rehabilitation methods used in the world’s and domestic medicine showed the equally strong associations with qualification (0.073 ≤ р ≤ 1.0). Furthermore, these associations were significantly stronger than that of other requirements (0.001 ≤ р ≤ 0.026).

The experts rated higher usability of the requirement for the ability to estimate the data of specific assessment methods to make the diagnosis, than that of other requirements (0.001≤ р ≤ 0.004). Objectivity of the requirement for using medical products was rated higher, than that of other requirements (р = 0.001). In experts’ opinion, objectivity of other requirements was the same (0.383 ≤ р ≤ 1.1). All the requirements had an equally low grade (0.710 ≤ р ≤ 1.0), except the requirements for professional experience.

Three requirements had positive total scores (in descending order): requirements for the experience, for using medical products, and for the ability to estimate the data of specific assessment methods to make the diagnosis. The requirement for the experience was assigned the highest usability, objectivity, and grade scores, but the lowest “association with qualification” scores. The requirement for the ability to estimate the data of specific assessment methods to make the diagnosis, on the contrary, was assigned the highest association with qualification and usability scores, but the lowest objectivity and grade scores. The requirement for using medical products was assigned positive association with qualification, usability, and objectivity scores, but the lowest grade scores. The total expert assessment score of the requirement for using medical products (for all items) is significantly higher, than that of other requirements (0.0001≤ р ≤ 0.002), except the requirements for professional experience.

DISCUSSION

The study has shown that the experts consider more than a half of the required knowledge, abilities, and skills as the most strongly associated with professional qualification and measurable. However, the experts believe that it is impossible to ensure objectivity (including independence from the expert panel member’s competence) for the largest part of the required knowledge, abilities, and skills. Furthermore, the experts see no possibility to accurately determine the extent of the knowledge, abilities, and skills required for each qualification grade for the vast majority of qualification requirements.

Two requirements in the “association with qualification” item were assigned negative scores: experience and completing the work report. It is important to note that experience has long been considered as a formal characteristic of professional development [14]. The consensus among experts confirms that the length of service does not guarantee the physician’s high qualification.

Poor expert assessment of the association of the requirements for completing the work report, requirements for the skills of assessing quantitative and qualitative performance indicators with qualification is interesting amidst the data on poor physicians’ competence in completing the work reports and dealing with statistical data [11].

As for usability of the requirements, i.e. feasibility of measuring the required knowledge, abilities, and skills, the experts consider only five requirements out of 22 as not measurable. It is worth mentioning that the experts have noted weak association of certain requirements with qualification. These are requirements for completing the work report, using scientific and technical information, part in resolving tactical issues, using scientific and technical information to resolve tactical issues, using scientific and technical information to resolve tactical and strategic issues.

The authors see two limitations affecting the study results: the experts’ personal attitude towards the categorization system, from hostility to sympathy, and the shortcomings of the Stapel rating scale represented by ambiguity in understanding the scale divisions by the respondents.

CONCLUSIONS

None of the qualification requirements approved by the Order enable accurate full-fledged determination of the evaluated individual’s qualification. The main shortcomings of the established requirements are as follows: absolute dependence of assessment on the expert panel member’s competence and impossibility to determine the extent of knowledge, abilities, and skills required for specific qualification grade. The experts believe that the requirement for experience, that has been assigned the highest positive objectivity, grade, and usability scores by the experts, does not reflect the physician’s professional qualification.