|
 
 |
| ORIGINAL ARTICLE |
|
| Year : 2019 | Volume
: 8
| Issue : 1 | Page : 47-53 |
|
Procedural experience of canadian neonatal-perinatal medicine fellows
Jonathan Wong1, Deepak Manhas1, Douglas M Campbell2, Ahmed Moussa3, Taylor L Sawyer4
1 Department of Pediatrics, Division of Neonatology, University of British Columbia, Vancouver, British Columbia, Canada
2 Department of Pediatrics, Allan Waters Family Simulation Center and Neonatal Intensive Care Unit, St. Michael's Hospital, University of Toronto, Toronto, Ontario, Canada
3 Department of Pediatrics, Division of Neonatology, University of Montreal, Montreal, Quebec, Canada
4 Department of Pediatrics, Division of Neonatology, University of Washington, Seattle, Washington, USA
| Date of Web Publication | 29-Jan-2019 |
Correspondence Address:
Dr. Jonathan Wong
1R34 – 4500 Oak Street, Vancouver, British Columbia, V6H 3N1
Canada
 Source of Support: None, Conflict of Interest: None  | Check |
DOI: 10.4103/jcn.JCN_110_18
Context: Neonatal-Perinatal Medicine (NPM) is a high-acuity specialty where trainees need to be proficient in complex procedural skills. The ability to perform these skills independently is an important milestone toward becoming a competent neonatologist. Aims: The aim of the study was to characterize the current procedural experience of Canadian NPM trainees, the methods for recording their experience, and how competency in procedural skills is being assessed by training programs. Subjects and Methods: A 60-item online survey was sent to all NPM trainees enrolled in an accredited training program across Canada. Statistical Analysis Used: Descriptive statistics (means, medians, standard deviations, range, and interquartile ranges) and the Mann–Whitney U-test were used. Results: Twenty fellows (27%), including ten 1st-year and ten 2nd-year fellows responded. Procedural experience increased with length of training. Most fellows are required to maintain a procedure log and track specific procedures. About 50% of fellows reported that logs are not required to be signed off and there is often no standardized process to determine when a fellow can independently perform a procedure. About 55% of fellows felt that they had sufficient experience to be competent upon graduation. Respondents indicated that both fellows and programs are actively working to improve the procedural experience for trainees. Conclusions: This study provides normative data regarding the procedural experience of Canadian NPM trainees. Experience increases with years of training, but there is a need to more clearly define what procedures are most critical to attain competency. Currently, procedural experience tracking is highly variable between programs.
Keywords: Competency, logs, medical education, neonatology, procedures
How to cite this article:
Wong J, Manhas D, Campbell DM, Moussa A, Sawyer TL. Procedural experience of canadian neonatal-perinatal medicine fellows. J Clin Neonatol 2019;8:47-53 |
How to cite this URL:
Wong J, Manhas D, Campbell DM, Moussa A, Sawyer TL. Procedural experience of canadian neonatal-perinatal medicine fellows. J Clin Neonatol [serial online] 2019 [cited 2023 Mar 21];8:47-53. Available from: https://www.jcnonweb.com/text.asp?2019/8/1/47/250977 |
| Introduction | |  |
Neonatal-perinatal medicine (NPM) is a high-acuity specialty, requiring the need for timely clinical decisions and proficiency in complex procedural skills. This can make training challenging and stressful. Furthermore, there are decreasing opportunities for trainees to gain competency in important neonatal skills.[1],[2] For trainees, the ability to perform key procedures unsupervised is an important milestone toward becoming a competent neonatologist.[3]
The Royal College of Physicians and Surgeons of Canada (RCPSC) outlines a specific set of procedural skills that trainees are expected to become competent in before certification.[4] However, the process to obtain competency and the methods for assessing competency are not specified. Each individual training program is responsible for ensuring that their fellows are competent in the prescribed set of procedural skills. Competency determination methods are likely variable given the different curriculums, resources, and training opportunities that are unique to each individual program. It is not surprising then that there is ongoing concern about the ability for trainees to attain competency in neonatal procedural skills.[1],[2],[5] The ability to validate trainee experience and proficiency will become even more important as training programs move toward competency-based medication education curriculums as mandated by the RCPSC.[6]
Only one prior study has specifically examined procedural experience in NPM training in which a national survey was conducted of trainees in the United States.[5] Sawyer et al. found that cumulative experience increased with training year; however, there was considerable variation between programs. The majority of trainees used online procedure logs. Competency was most often assessed through direct observation until an arbitrary number of procedures were completed. Overall, most respondents (85%) wanted more procedural experience. The procedural experience reported by Sawyer et al. could be used to establish normative data on current procedural experience of NPM fellows in the United States.[5] Such information is not available for Canadian training programs.
The aim of this study was to characterize the current procedural experience of Canadian NPM trainees, investigate the methods used to record procedural experience, and determine how competency in procedural skills is assessed by training programs. We hypothesize that procedural experience increases from the 1st to 2nd year of training, that variation exists in the way experience is tracked, and that challenges exist in assessing procedural competency. We hope this information will lead to opportunities to better support trainees and assist programs as they transition toward competency-based curriculums.
| Subjects and Methods | | |
Study overview
A descriptive online cross-sectional survey was conducted with the target population of all NPM fellows enrolled in a RCPSC accredited Canadian training program as of June 2017. Consent was implied based on the completion of the survey. This study was approved by the British Columbia Children's and Women's Behavioral Research Ethics Board.
Study population
The survey was sent electronically to all 1st- and 2nd-year NPM fellows enrolled in a RCPSC accredited training program as of June 2017. In total, there were 13 training programs, with an estimated total of 76 eligible fellows. The estimated total number of Canadian NPM fellows was based on the number of fellows eligible for the Canadian national Neonatal Observed Standardized Clinical Examination in the fall of 2016.
Survey tool
A 60-item survey was specifically designed for this study. This tool was modeled on a previously validated survey tool used by Sawyer et al.,[5] which was used to assess trainee procedural experience in neonatal training programs in the United States.
The survey contained information regarding demographics, the use of procedure logs, program-level determination of competency, and details on the number of procedures each fellow performed for a variety of neonatal procedures. The survey tool was evaluated for the face and content validity by the investigators and two Canadian NPM fellows.
Survey administration
All 13 Canadian NPM program directors and NPM program administrators (as identified by the RCPSC program director directory and the Canadian Residency Matching Service) were asked to forward the survey invitation to their trainees. This was done as there is no up-to-date national e-mail list of NPM trainees available. All participants received an introductory letter along with a link to the survey through e-mail. The electronic survey was administered online through Catalyst (University of Washington, 2016). In addition to the initial e-mail, reminders were sent weekly for 3 weeks. An additional round of survey invitations was sent 3 weeks later and the survey closed 1 week after that. The survey was sent at the end of the academic year in order to optimally capture the cumulative procedural experience of both 1st- and 2nd-year fellows.
Statistical analysis
Descriptive statistics, including means, medians, standard deviations, range, and interquartile ranges, were used in data analysis and interpretation. Comparisons of nonparametrically distributed procedural experience between fellowship year groups were conducted using the Mann–Whitney U-test. Data analysis was carried out using IBM SPSS version 24 (IBM Inc., Armonk, New York, USA). Statistical significance was defined as a P < 0.05.
| Results | | |
A total of 20 (27%) NPM fellows responded to the survey, including 10 (26%) 1st-year and ten (29%) 2nd-year fellows. Three out of the 13 eligible programs did not have any fellows enrolled during this study period. Responses were received from at least one fellow in nine out of the ten programs (90%) with enrolled trainees. The mean number of responses from each program was 2.4 (Standard deviation – 1.5), with a range of one to five responses per program.
Procedural experience
In general, the cumulative procedural experience of fellows increases by their year of training [Table 1]. Significant differences were noted for attendance at deliveries of premature infants <28 weeks gestational age (GA) and 33–36 + 6 weeks GA, cardioversion (medical), peripheral arterial line placement, umbilical arterial line placement, and lumbar puncture.
Most fellows (60%) rated their procedural experience as high before starting fellowship, with 20% rating their experience as low and 20% as none. About 85% of fellows indicated that they learn procedures mostly during their regular clinical rotations and were most likely to be taught by their attending physician (45%) or another fellow (35%).
Of respondents, 55% felt that they had performed a sufficient number of procedures for their level of training. However, 80% indicated that they would like more training, with the most commonly noted procedures to be percutaneously inserted central catheters (PICCs), chest tubes, and peripheral arterial lines. In total, 55% of respondents felt that trainees in their program graduate with sufficient experience to be “competent” in neonatal procedures. For the same question, a quarter of the respondents did not feel competent in procedures despite their training and 20% were unsure of their competency level.
About 55% of fellows indicated that they were proactive in improving their procedural skills, with reported methods being to seek increased clinical exposure including additional call, simulation, and reviewing textbooks. Similarly, 70% of fellows indicated that their training program was working to improve their procedural experience, mostly through simulation and procedural “boot camps,” which are dedicated sessions, often a couple of days in length, where skills can be learned and practiced.
Procedure logs
Of the fellows who responded, 90% indicated that they were required to maintain a procedure log, with 61% using an online platform or resident management software, 33% using a private log, and 6% using a common log. The majority of fellows (94%) have a standardized list of procedures to track [Table 2]. Most logs (89%) are used to track both successful and unsuccessful procedural attempts.
Half of respondents indicated that their logs needed to be reviewed and “signed off,” most commonly by the attending physician (22%) followed by the program director (17%). Fellows felt that their logs were “accurate” (33%) or at least “somewhat accurate” (44%).
In answering the survey, 40% of fellows reported that they used information obtained from their procedure logs and memory, with 55% indicating that they relied on memory alone.
Verification of procedural competency
The majority of fellows (83%) indicated that there was no standardized process to determine when a fellow could independently perform a procedure. Of those who indicated that there was a process, it involved the use of simulation and directly supervised practice or was limited to certain procedures such as intubation of infants <1000 g and PICC line placement.
| Discussion | | |
We conducted a national survey to characterize the procedural experience of the current Canadian NPM trainees, determined the methods for recording their experience, and examined how competency in procedural skills is being assessed by training programs. Although previously reported for programs in the United States,[5] this is the first description of procedural experience across Canadian NPM training programs. This is timely as there is an ongoing shift in Canada and worldwide toward competency-based medical education.[3],[7] The results obtained from this study can help guide curriculum design and also provide normative data for future comparison after the implementation of new competency-based curriculums, being applicable to neonatology and other disciplines.
Although the RCPSC defines a list of procedures that fellows are required to be competent in performing [Supplementary Information [Additional file 1]],[4] only a few of these procedures have been reported to be required across all training programs by fellows [Table 2]. Furthermore, [Table 2] shows minimal to no experience for a third of the RCPSC mandated procedures. This brings up the question of whether all the listed procedures are still relevant for a trainee to become a competent practicing neonatologist. If the answer is yes, especially for procedures which are rarely done but critical when required, then new strategies may be required to improve trainee experience, such as through simulation. This highlights the importance of continually reevaluating the procedures required of our trainees and provides an opportunity to define new standards for procedural tracking and evaluation.
We found that while 90% of the fellows who responded were required to keep a procedure log, there was variability in the platform used and differences in what procedures were required to be reported. One strategy to decrease variability may be to introduce a nationally accepted electronic procedure log. Such standardization would allow for comparison and collaboration between programs. In addition to having a standardized tool, having consistent processes to evaluate the logs and support learners would also be important.[8] Only 50% of responding fellows indicated that their logs needed to be signed off and most (83%) reported that there was no standardized process for determining when they could independently perform a specific procedure. This highlights the importance of expert feedback which can lead to improved and earlier acquisition of procedural skills.[9],[10] Procedure logs can serve as a starting point for such feedback and guidance on acquiring further experience.
We found that 60% of fellows rated their procedural experience as “high” before starting fellowship, with the rest split as “low” or “none.” While this was not explored further in our survey, we believe that it is a reflection of the heterogeneous composition of learners among Canadian training programs, which includes a large proportion of international medical graduates, many of whom have already completed a fellowship or have been a practicing neonatologist in their home country. This difference in baseline experience is important to consider when determining how to best support learners in attaining the necessary skills. Assuming that competency can be assessed for those who indicate a high level of previous experience, they may serve as valuable resources for less experienced trainees. In our study, 45% of respondents reported learning procedures from the attending, while 35% learned from other fellows, indicating that learning from senior and more experienced fellows is occurring already. While not a surprise, it may be useful to further characterize the fellows who are providing such training and how this may contribute to the procedural experience of junior fellows.
Overall, procedural experience increased with year of fellowship. This trend was most evident for code leadership and delivery attendance at various GAs. In general, this was similar to the study by Sawyer et al.[5] However, in the study by Sawyer et al., 3rd-year fellows were included, as the US training programs are 3 years in length. In Canada, the mandatory training period is 2 years, and while some programs offer a voluntary 3rd year, this is not consistent and the number of enrolled 3rd year is low, so they were not included in our survey. Although the amount of experience increased, it in itself, does not guarantee competency. The relationship between experience and competency is difficult to define.[11],[12],[13] With each attempt at a procedure, there are multiple variables that will shape the learner's experience. Some of which may include the previous experience, the degree of supervision and guidance, patient factors, and whether the attempt was successful or not.[14],[15],[16] Of the respondents who kept procedure logs, the majority (89%) tracked both successful and unsuccessful attempts. Certainly, this makes it difficult to link this variable to competency as successful attempts are what is most commonly measured in studies of competency.[17],[18] Therefore, while trainees do learn with each attempt at a procedure,[15] evaluators need to carefully consider the proportion that is successful. The required experience will also be variable based on the procedure that is being mastered. For example, Doglioni described her experience as recorded through a logbook of completing 152 neonatal intubations and calculated that an adequate level of proficiency would only be obtained after performing 100 intubations.[18] However, this may not be so simple or universally translatable. Completing a certain number of procedures but having a limited exposure to rare cases may not necessarily translate to competency, especially for a skill such as neonatal intubation where there can be significant variability between patients. Therefore, it is difficult to assess competency based on the amount of experience alone. Other assessment tools are available, including cumulative sum analysis, formal direct observation, and objective structured assessment of technical skills.[19],[20],[21] The use of such tools in NPM training programs is an area for further research.
While 75% of 1st-year fellows reported feeling that they would graduate with sufficient experience to be competent in neonatal procedures, only 40% of 2nd-year fellows felt the same. Although this question was not explored in detail in this survey, studies have shown variable degrees of confidence in procedural skills for both graduating learners and teachers.[22],[23] This may be due to a number of reasons, including lack of exposure, or lack of assessment to determine competency. It is likely that this feeling is not reflective of every procedure mandated by the RCPSC but again highlights a need to better define which key procedures are required of trainees to practice independently. Similar to the study by Sawyer et al.,[5] 80% of fellows reported that they would like more procedural experience. One method to supplement clinical experience is to use simulation,[24] which has been reported to be widely used in various forms in Canadian training programs.[25] All Canadian training programs currently use simulation in their training programs, but there is variability in fidelity, the degree of integration into existing curriculums, and faculty involvement.[25] Many common neonatal procedures (intubation, chest tubes, venous, and arterial access) can easily be practiced using simulation.
This study does have limitations. Only 27% (20/74) of fellows responded to the survey. At present, there is no contact list that includes all current NPM trainees in Canada. Therefore, we asked program directors and administrators to forward our survey to their trainees, which can result in bias. Strategies to contact trainees directly, such as at national exams, exit surveys, or creating a national contact list, may be considered to ensure that trainees are aware of such studies which could potentially lead to increased response rates. Certainly, with a low response rate, it is difficult to know whether we have obtained a representative sample of trainees across multiple programs. It is also difficult to estimate the cumulative experience of fellows in specific programs, as some programs only had one or two fellows respond. A higher response rate with the strategies outlined above would help to resolve this issue.
This study has provided data regarding the current procedural experience of Canadian NPM trainees. With the introduction of a competency-based curriculum, there will likely be effects on the levels of training, required competencies, and new assessment techniques. Therefore, it would be important to repeat this study postimplementation to assess how procedural training and experience has changed. Similar studies could also be conducted for other disciplines where procedural experience and competency are required for safe independent practice. It may also be useful to conduct a survey of recent NPM graduates to see how their procedural skill training has prepared them for independent practice and correlate this with our findings.
| Conclusion | | |
This study provides normative data regarding the procedural experience of Canadian NPM trainees. While experience does appear to increase with years of training, there is a need to more clearly define what procedures are most critical to attain competency. Although most fellows do keep track of their procedural experience, the requirements and content are highly variable. With the drive toward competency by design across Canadian training programs and this data, there is now an opportunity to reshape how procedures are taught, the experience is tracked, and competency will be assessed for the next generation of learners to ensure safe practice.
Financial support and sponsorship
Nil.
Conflicts of interest
There are no conflicts of interest.
| Supplementary Information | | |
Required procedures as per the RCPSC – Objectives of Training in the Subspecialty of Neonatal-Perinatal Medicine[4]
- Blood sampling: Capillary, venous, arterial, and from indwelling catheter
- Lumbar puncture
- Suprapubic aspiration of the bladder
- Thoracentesis
- Paracentesis
- Neonatal resuscitation and postresuscitative care
- Maintain Neonatal Resuscitation Program provider status
- Peripheral intravenous (IV) catheter placement
- Peripheral arterial catheter placement
- Umbilical arterial and venous catheter placement
- Peripherally inserted central catheter (PICC) placement
- Endotracheal intubation
- Laryngeal mask airway insertion
- Thoracentesis/chest tube placement
- Exchange transfusion
- Pericardiocentesis
- Paracentesis
- Oro/nasogastric tube placement
- Urinary catheter placement
- Surfactant administration.
| References | | |
| 1. | Falck AJ, Escobedo MB, Baillargeon JG, Villard LG, Gunkel JH. Proficiency of pediatric residents in performing neonatal endotracheal intubation. Pediatrics 2003;112:1242-7.  |
| 2. | Gaies MG, Landrigan CP, Hafler JP, Sandora TJ. Assessing procedural skills training in pediatric residency programs. Pediatrics 2007;120:715-22. |
| 3. | Frank JR, Snell LS, Cate OT, Holmboe ES, Carraccio C, Swing SR, et al. Competency-based medical education: Theory to practice. Med Teach 2010;32:638-45. |
| 4. | |
| 5. | Sawyer T, French H, Ades A, Johnston L. Neonatal-perinatal medicine fellow procedural experience and competency determination: Results of a national survey. J Perinatol 2016;36:570-4. |
| 6. | |
| 7. | Ten Cate O. Competency-based postgraduate medical education: Past, present and future. GMS J Med Educ 2017;34:Doc69. |
| 8. | Kasten SJ, Prince ME, Lypson ML. Residents make their lists and program directors check them twice: Reviewing case logs. J Grad Med Educ 2012;4:257-60. |
| 9. | Touchie C, Humphrey-Murto S, Varpio L. Teaching and assessing procedural skills: A qualitative study. BMC Med Educ 2013;13:69. |
| 10. | Bosse HM, Mohr J, Buss B, Krautter M, Weyrich P, Herzog W, et al. The benefit of repetitive skills training and frequency of expert feedback in the early acquisition of procedural skills. BMC Med Educ 2015;15:22. |
| 11. | Thomas WE. Teaching and assessing surgical competence. Ann R Coll Surg Engl 2006;88:429-32. |
| 12. | Lenchus JD, Carvalho CM, Ferreri K, Sanko JS, Arheart KL, Fitzpatrick M, et al. Filling the void: Defining invasive bedside procedural competency for internal medicine residents. J Grad Med Educ 2013;5:605-12. |
| 13. | Tariq M, Bhulani N, Jafferani A, Naeem Q, Ahsan S, Motiwala A, et al. Optimum number of procedures required to achieve procedural skills competency in internal medicine residents. BMC Med Educ 2015;15:179. |
| 14. | Boots RJ, Egerton W, McKeering H, Winter H. They just don't get enough! Variable intern experience in bedside procedural skills. Intern Med J 2009;39:222-7. |
| 15. | Kolb DA. Experiential Learning: Experience as the Source of Learning and Development. 2 nd ed. Upper Saddle River, New Jersey: Pearson Education, Inc.; 2015. |
| 16. | Martin AN, Hu Y, Le IA, Brooks KD, Mahmutovic A, Choi J, et al. Predicting surgical skill acquisition in preclinical medical students. Am J Surg 2016;212:596-601. |
| 17. | Konrad C, Schüpfer G, Wietlisbach M, Gerber H. Learning manual skills in anesthesiology: Is there a recommended number of cases for anesthetic procedures? Anesth Analg 1998;86:635-9. |
| 18. | Doglioni N, Cavallin F, Zanardo V, Trevisanuto D. Intubation training in neonatal patients: A review of one trainee's first 150 procedures. J Matern Fetal Neonatal Med 2012;25:1302-4. |
| 19. | Delfino AE, Chandratilake M, Altermatt FR, Echevarria G. Validation and piloting of direct observation of practical skills tool to assess intubation in the chilean context. Med Teach 2013;35:231-6. |
| 20. | Martin JA, Regehr G, Reznick R, MacRae H, Murnaghan J, Hutchison C, et al. Objective structured assessment of technical skill (OSATS) for surgical residents. Br J Surg 1997;84:273-8. |
| 21. | Connolly TM, Watters DA. Monitoring performance in thyroidectomy: Cumulative sum analysis of outcomes. Thyroid 2010;20:407-12. |
| 22. | Wickstrom GC, Kelley DK, Keyserling TC, Kolar MM, Dixon JG, Xie SX, et al. Confidence of academic general internists and family physicians to teach ambulatory procedures. J Gen Intern Med 2000;15:353-60. |
| 23. | Wickstrom GC, Kolar MM, Keyserling TC, Kelley DK, Xie SX, Bognar BA, et al. Confidence of graduating internal medicine residents to perform ambulatory procedures. J Gen Intern Med 2000;15:361-5. |
| 24. | Lopreiato JO, Sawyer T. Simulation-based medical education in pediatrics. Acad Pediatr 2015;15:134-42. |
| 25. | Wong J, Finan E, Campbell D. Use of simulation in Canadian neonatal-perinatal medicine training programs. Cureus 2017;9:e1448. |
[Table 1], [Table 2]
|
Search Pubmed for