Retrospective validation of the Alexandria retinopathy of prematurity model in preterm infants in Saudi Arabia

Lina H Raffa; Omar M Akeely; Saleh A Alariefy; Faisal A Alharbi; Moussa A Alkhateeb; Mohammad A Khan

doi:10.4103/jcn.jcn_87_22

ORIGINAL ARTICLE

Year : 2023 | Volume : 12 | Issue : 1 | Page : 7-12

Retrospective validation of the Alexandria retinopathy of prematurity model in preterm infants in Saudi Arabia

Lina H Raffa¹, Omar M Akeely², Saleh A Alariefy², Faisal A Alharbi², Moussa A Alkhateeb², Mohammad A Khan²
¹ Department of Ophthalmology, King Abdulaziz University Hospital; Faculty of Medicine, King Abdulaziz University, Jeddah, Saudi Arabia
² Faculty of Medicine, King Abdulaziz University, Jeddah, Saudi Arabia

Date of Submission	11-Sep-2022
Date of Decision	19-Oct-2022
Date of Acceptance	08-Nov-2022
Date of Web Publication	03-Jan-2023

Correspondence Address:
Lina H Raffa
Department of Ophthalmology, King Abdulaziz University Hospital, Prince Majid Road, Al Sulaymaniyah, P.O. Box 80215, Jeddah 21589
Saudi Arabia

Source of Support: None, Conflict of Interest: None

Check

DOI: 10.4103/jcn.jcn_87_22

Abstract

Background: In developing countries, data on the applicability of existing models to predict retinopathy of prematurity (ROP) are scarce. The study aimed to validate the Alexandria ROP (Alex-ROP) and high-grade Alex-ROP (Hg Alex-ROP) models retrospectively to identify treatable ROP in a cohort of preterm infants in Saudi Arabia. Materials and Methods: We reviewed and included the records of 281 infants born prematurely in 2015–2021. We recorded the infants' demographics, gestational age at birth (GA), birth weight (BW), and serial weight measurements (day 7, 14, 21, and 28). We determined whether the included met the Alex-ROP and Hg Alex-ROP detection criteria for treatable or any-stage ROP and calculated the specificity, sensitivity, negative and positive predictive values, and accuracy. Results: The median BW and GA was 1095 g (range: 426–1920 g) and 29 weeks (range: 23–36 weeks), respectively. ROP developed in 112 infants, of which 30 cases were treatable. The Alex-ROP sensitivity for correctly predicting any-stage ROP and treatable ROP was 77.7% and 80.0%, respectively, and its specificity for predicting any-stage ROP and treatable ROP was 49.7% and 41%, respectively. The Hg Alex-ROP had 36.6% and 50.0% sensitivity for detecting any-stage ROP and treatable ROP, respectively, and its specificity for detecting any-stage ROP and treatable ROP was 83.4% and 78.5%, respectively. Conclusion: Previously published accuracy parameters were not reproducible in this cohort and a significant number of children requiring treatment would have been missed if the Alex-ROP or Hg Alex-ROP were applied.

Keywords: Algorithm, infants, postnatal weight gain, prematurity, screening, validation

How to cite this article:
Raffa LH, Akeely OM, Alariefy SA, Alharbi FA, Alkhateeb MA, Khan MA. Retrospective validation of the Alexandria retinopathy of prematurity model in preterm infants in Saudi Arabia. J Clin Neonatol 2023;12:7-12

How to cite this URL:
Raffa LH, Akeely OM, Alariefy SA, Alharbi FA, Alkhateeb MA, Khan MA. Retrospective validation of the Alexandria retinopathy of prematurity model in preterm infants in Saudi Arabia. J Clin Neonatol [serial online] 2023 [cited 2023 Mar 21];12:7-12. Available from: https://www.jcnonweb.com/text.asp?2023/12/1/7/366894

Introduction

The proliferative retinal vascular disease of retinopathy of prematurity (ROP) mainly affects infants born prematurely.^[1],[2] A principal cause of blindness in childhood, ROP, can be avoided with early detection and intervention.^[3] According to recent studies, ROP prevalence in Saudi Arabia ranges between 33.3% and 38.6%.^[4],[5] Infants at risk for ROP face serial diagnostic retinal examinations to detect severe disease characteristics (Type 1 ROP), where treatment is suggested to lessen the risk of progression to retinal detachment.^[6] The identification of at-risk infants for ROP uses the gestational age at birth (GA) and conventional birth weight (BW) as cutoffs. Several ROP algorithms are being developed to reduce the serial examinations that such infants undergo to detect treatable ROP early. The ROP screening fundus examination is traumatic for the infant, burdens health services, and exhausts doctors' energy and time.^[7] Therefore, it would be an advantage to avoid unwarranted fundus examination without conceding the disease detection ability.^[8] A web-based tool developed by researchers in Sweden was validated by a study that involved newborns born in August 2013–October 2018 at a tertiary university hospital's neonatal intensive care unit (NICU) (Jeddah, Saudi Arabia). The authors reported 100% sensitivity by the Weight, Insulin-like Growth Factor 1 [IGF-1], Neonatal, ROP (WINROP) algorithm for identifying Type 1 ROP, which was similar to the sensitivity reported in developed countries. Nevertheless, the algorithm exhibited low specificity (31.5%).^[9] Moreover, the algorithm had lower accuracy in developing countries (Turkey, Taiwan, and Korea)^{[10],[11],[12]} compared to more developed countries (the United States and Sweden).^[13],[14] The postnatal growth and ROP model demonstrated variability when validated across countries, where Switzerland, Italy, the United Kingdom, Japan, Egypt, and Taiwan^{[15],[16],[17],[18],[19]} reported 100% sensitivity for the detection of treatment-requiring ROP, while lower sensitivity rates (91.2%) were reported in Turkey.^[20] Due to the similarity in our countries' geographic and socioeconomic backgrounds, these results piqued our interest in an algorithm created in Alexandria (Alex), Egypt. The Alex-ROP model demonstrated 100% sensitivity for low-grade and worse-grade ROP and reduced the number of infants who required screening by almost ~20%, while high-grade Alex-ROP (Hg Alex-ROP) use for detecting worse-grade ROP exhibited 100% sensitivity for detecting Type 1 and 2 prethreshold ROP, which decreased the total number of infants who required screening by >75%.^[8] The aim of the present study was to validate the Alex-ROP in Saudi preterm infants at King Abdulaziz University Hospital (KAUH), Jeddah, Saudi Arabia, to reduce unnecessary fundus examinations.

Materials and Methods

Study design and participants

This retrospective study was conducted at KAUH. All preterm infants (n = 533) indicated for ROP screening, GA <31 weeks or BW <1500 g, or those with an unstable clinical course as per the treating neonatologists and who were admitted to KAUH between 2015 and 2021 were reviewed. We excluded preterm infants who died, had no final ROP outcome, or had incomplete datasets/documentation (n = 224).

Information obtained from the infants' medical records included gender, date of birth, and NICU discharge date. The infants were weighed daily until discharge. The infants' BW, GA, serial weight measurements (7, 14, 21, and 28 days after birth), and demographics were recorded. We recorded the infant's ROP detection date, worst ROP stage in either eye, presence or absence of plus disease, ROP zone, and ROP treatment. KAUH vitreoretinal surgeons or pediatric ophthalmologists at our center assessed infants who qualified for ROP examination as per the guidelines of the American Academy of Pediatrics^[21] and categorized them with the International Classification of ROP.^[22] All treatments were administered in accordance with the Early Treatment for ROP guideline recommendations.^[6] The ROP screening was followed until therapy was necessary or the retina had fully vascularized. The postnatal weight gain ratio (PWGR) was analyzed to validate the accuracy of both the Alex-ROP and Hg Alex-ROP. Each infant was analyzed to check if they fulfilled the Alex-ROP criteria to detect ROP: GA ≤33 weeks and/or BW ≤1500 g and PWGR at 28 days <0.3. The Hg Alex-ROP recommends a cutoff of PWGR <0.15 at 28 days.

Statistical methodology

We analyzed the data with SPSS 23 (IBM, Armonk, NY, USA). The variable characteristics were reported as counts and percentages for categorical and nominal variables; continuous variables were reported as the mean and standard deviation (SD). Categorical variable relationships were established with the Chi-square test. Group means >2 were compared using one-way analysis of variance with the post hoc test of least significant difference. These tests were performed based on the supposition of normal distribution. Otherwise, the alternative post hoc test was the Games–Howell procedure.

Disease prevalence, specificity, sensitivity, negative and positive predictive values, and accuracy were reported as percentages. The specificity, sensitivity, and accuracy confidence intervals (CIs) were exact Clopper–Pearson CIs. The null hypothesis was rejected if P < 0.05. Continuous variables were reported as the mean (SD) or median (interquartile range) and qualitative variables were reported as numbers and percentages. Intergroup differences were compared using the Fisher's exact test and Mann–Whitney U-test. The Alex-ROP criteria sensitivity and specificity for predicting treatable ROP were calculated based on the actual ROP outcome. The positive and negative predictive values of the Alex-ROP were calculated based on treatable ROP prevalence in the study cohort. The 95% CIs were calculated. The gold standard was the ophthalmologic examination results. In all analyses, the differences were deemed significant when P < 0.05.

Ethical approval

Ethical approval for the study was granted by the KAUH Research Ethics Committee (ref.no. 41-20). The research was conducted according to the principles of the Declaration of Helsinki.

Results

The cohort included 309 eligible infants who underwent ROP examinations, among which 28 were excluded for incomplete weight entry at day 28. [Table 1] presents the demographic information of the remaining 281 infants based on ROP status. The mean duration of NICU stay was 61.14 ± 39.4 days. The median BW and GA was 1095 g (range: 426–1920 g) and 29 weeks (range: 23–36 weeks), respectively. Up to 57.7% of the infants were Saudi, while the rest were from different ethnic groups. ROP developed in 112 infants, among whom 70, 25, and 14 had Stage 1, Stage 2, and Stage 3 ROP, respectively, in the right eyes. Thirty patients were diagnosed as treatable ROP, among which five underwent intravitreal injections, 24 underwent laser therapy, and one underwent combined treatment. One hundred and seventy-two and 69 infants fulfilled the Alex-ROP and Hg Alex-ROP criteria, respectively. The Alex-ROP correctly predicted any-stage ROP in 87 of 172 infants (sensitivity: 77.7%, 95% CI: 68.84–85.00) and treatable ROP in 24 of 172 infants (sensitivity: 80.0%, 95% CI: 61.43–92.29) [Table 2]. The Hg Alex-ROP correctly predicted any-stage ROP in 41 of 69 infants (sensitivity: 36.6%, 95% CI: 27.71–46.24) and treatable ROP in 15 of 69 infants (sensitivity: 50.0%, 95% CI: 31.30–68.70) [Table 3]. Of the infants who did not fulfill the Alex-ROP or Hg Alex-ROP criteria, 6 and 15 cases, respectively, were missed and developed treatable ROP, while 25 and 71 missed infants, respectively, developed any-stage ROP. Using these models, unnecessary retinal examinations would have been reduced by 38.8% (109/281) using the Alex-ROP and by 75.4% (212/281) using the Hg Alex-ROP. The weight gain percentage did not differ across the ROP groups at day 7 and 14 (P = 0.475, P = 0.560). By day 28, the patients with no ROP had a significantly higher weight gain (308 ± 208 g) among the three categories (treatable ROP: 176 ± 138 g; any-stage ROP: 223 ± 140 g) (P < 0.001) followed by day 21 [P < 0.003, [Figure 1]]. The postnatal weight gains of the six treatable ROP cases missed by the Alex-ROP were higher than the 300 g cutoff at day 28 (median: 404 g, range: 313–477 g) and that of the 15 cases missed by the Hg Alex-ROP were higher than the 150 g cutoff (median: 250 g, range: 170–477 g).

Figure 1: Linear graph demonstrating the comparison in mean weight gain percentage between the ROP groups. ROP - Retinopathy of prematurity

Click here to view

Table 1: Demographics of study cohort based on retinopathy of prematurity group

Click here to view

Table 2: Validity of Alexandria retinopathy of prematurity and high-grade Alexandria retinopathy of prematurity for detecting any-stage retinopathy of prematurity

Click here to view

Table 3: Validity of Alexandria retinopathy of prematurity and high-grade Alexandria retinopathy of prematurity for detecting treatable retinopathy of prematurity

Click here to view

Discussion

Until recently, the conventional ROP screening method to prevent sight-threatening retinal detachment relied on overscreening children using robust GA and BW cutoffs. A tool that could aid clinicians with screening at-risk patients to prevent ROP with the highest sensitivity and specificity would be helpful and cost-effective. Neonatal care standard, demographic, and screening practice variability between institutes and countries should all be considered when designing such a tool. Due to variability in the reported postnatal weight gain in infants across countries, a screening tool based on day-28 postnatal weight gain was developed in Egypt to detect ROP.^[8] In the present study, we validated the Alex-ROP and Hg Alex-ROP with a cohort of infants in Saudi Arabia. The Alex-ROP demonstrated 77.7% and 80% sensitivity for detecting any-stage ROP and treatable ROP, respectively, while the Hg Alex-ROP demonstrated 36.6% and 50% sensitivity for detecting any-stage ROP and treatable ROP, respectively.

The mean GA and BW of the 281 infants included in our study were lower (GA: 26.27 weeks, BW: 800 g) than that in the study by Ahmed and Badeeb for the treatable ROP group (GA: 28 weeks, BW: 965.11 g).^[8] Patients with treatable ROP typically have lower weight gain percentages than those with no ROP. Predicting ROP development based on day-28 weight gain can be beneficial as the weight gain by the end of the first neonatal month was the most predictive factor for developing ROP.^[8],[23] This agrees with our findings, where the weight gain was significantly lower in the ROP groups as compared to the no ROP group at days 21 and 28, but this difference was not statistically significant for days 7 and 14.

In our study, unnecessary retinal examination was reduced by almost double at 38.8% when using the Alex-ROP as compared to the 19.82% of the original study conducted in Egypt,^[6] while it was the same when using the Hg Alex-ROP at 75%–76%. Unfortunately, both algorithms missed a significant number of treatable ROP cases. The subanalysis of the missed treatable cases revealed that the infants had higher weight gain than the proposed cutoffs used in both algorithms.^[6] The observed weight gain in our cohort would be better encompassed by more lenient cutoffs such as the cutoff predetermined by the Colorado ROP algorithm that suggested <650 g at 1 month.^[23] One explanation could be the differences in parenteral nutrition and neonatal protocols across countries. Another possible explanation is the different ethnicities included in both studies. The Egypt cohort only^[8] included patients with middle Eastern ethnicity, whereas our study included 50% mixed ethnicities, which is a true representation of the widespread cosmopolitan population typically observed in Saudi Arabia.

Several models use IGF-1 levels as an indirect measure to detect ROP, such as the WINROP^[13],[24] and the Children's Hospital of Philadelphia ROP model.^[25] In comparison to algorithms from developed countries, the Alex-ROP performed suboptimally and missed treatable ROP. This was not the case with the WINROP, which was validated in Saudi Arabia and had 100% sensitivity, where it identified all 13 treatable ROP cases.^[9] This reflects the importance of developing a model tailored to the local population based on our results to detect all at-risk patients without missing any treatable ROP cases but with higher specificity than detected by models designed in developed countries.

Our study has several limitations, the first of which is that this was a retrospective study, as the aforementioned algorithms were established for prospective use. The data value can be increased via multicenter prospective studies that involve more premature newborns. Due to the retrospective nature of our study, the documentation of the ophthalmoscopic observations also contains a significant subjective component. Another limitation was missing data, as many infants were excluded due to incomplete datasets.

Conclusion

This is the first study to validate the Alex-ROP in a cohort of preterm infants in Saudi Arabia. The Alex-ROP had 80% sensitivity and 24.4% specificity for detecting treatable ROP. As this tool uses a measurement that is taken routinely in our daily practice, it is a useful adjunctive tool that can aid clinicians in identifying patients at risk of developing sight-threatening treatable ROP. However, the Alex-ROP performed suboptimally and missed treatable cases; therefore, clinicians should interpret the results with caution. Accordingly, it is important to customize algorithms to better serve the local population and reflect regional and demographic differences.

Acknowledgment

The authors would like to thank Nada Bugshan, Renad Aljuhani, Reema Alghoribi, Ahd Alharbi, Wasayf Aljohani, Badr Al Harbi, and Mohammed Awadh for their aid in data collection.

Financial support and sponsorship

Nil.

Conflicts of interest

There are no conflicts of interest.

References

1.	Brown AC, Nwanyanwu K. Retinopathy of Prematurity. In: StatPearls. Treasure Island (FL): StatPearls Publishing Copyright, StatPearls Publishing LLC; 2022.
2.	Dogra MR, Katoch D, Dogra M. An update on Retinopathy of Prematurity (ROP). Indian J Pediatr 2017;84:930-6.
3.	Kim SJ, Port AD, Swan R, Campbell JP, Chan RV, Chiang MF. Retinopathy of prematurity: A review of risk factors and their clinical significance. Surv Ophthalmol 2018;63:618-37.
4.	Al-Qahtani B, Al-Otaibi M, Alabdulajabbar K, Selayem NB, Alshehri W, Omair A, et al. Retinopathy of prematurity incidence and risk factors in a tertiary hospital in Riyadh, Saudi Arabia. Middle East Afr J Ophthalmol 2019;26:235-9. [PUBMED] [Full text]
5.	Danish E, Hadrawi M, Tayyib A, Babgi R. Effects of early treatment of retinopathy of prematurity at a tertiary care hospital in Saudi Arabia: A retrospective study. J Pediatr Ophthalmol Strabismus 2021;58:240-5.
6.	Early Treatment For Retinopathy Of Prematurity Cooperative Group. Revised indications for the treatment of retinopathy of prematurity: Results of the early treatment for retinopathy of prematurity randomized trial. Arch Ophthalmol 2003;121:1684-94.
7.	Bulut O, Tarak Bozkurt O, Arslanoglu S. Oral ibuprofen versus oral paracetamol in pain management during screening for retinopathy of prematurity: A prospective observational study. J Perinat Neonatal Nurs 2022;36:305-11.
8.	Ahmed IS, Badeeb AA. The Alexandria retinopathy of prematurity model (Alex-ROP): Postnatal weight gain screening algorithm application in a developing country. Int J Ophthalmol 2019;12:296-301.
9.	Raffa LH, Alessa SK, Alamri AS, Malaikah RH. Prediction of retinopathy of prematurity using the screening algorithm WINROP in a Saudi cohort of preterm infants. Saudi Med J 2020;41:622-7.
10.	Choi JH, Löfqvist C, Hellström A, Heo H. Efficacy of the screening algorithm WINROP in a Korean population of preterm infants. JAMA Ophthalmol 2013;131:62-6.
11.	Ko CH, Kuo HK, Chen CC, Chen FS, Chen YH, Huang HC, et al. Using WINROP as an adjuvant screening tool for retinopathy of prematurity in Southern Taiwan. Am J Perinatol 2015;30:149-54.
12.	Koçak N, Niyaz L, Ariturk N. Prediction of severe retinopathy of prematurity using the screening algorithm WINROP in preterm infants. J AAPOS 2016;20:486-9.
13.	Hellström A, Hård AL, Engström E, Niklasson A, Andersson E, Smith L, et al. Early weight gain predicts retinopathy in preterm infants: New, simple, efficient approach to screening. Pediatrics 2009;123:e638-45.
14.	Wu C, Vanderveen DK, Hellström A, Löfqvist C, Smith LE. Longitudinal postnatal weight measurements for the prediction of retinopathy of prematurity. Arch Ophthalmol 2010;128:443-7.
15.	Ahmed IS, Aclimandos W, Azad N, Zaheer N, Barry JS, Ambulkar H, et al. The postnatal growth and retinopathy of prematurity model: A multi-institutional validation study. Ophthalmic Epidemiol 2022;29:296-301.
16.	Caruggi S, Scaramuzzi M, Calevo MG, Priolo E, Sposetti L, Camicione P, et al. Validation of the postnatal growth and retinopathy of prematurity screening criteria: A retrospective Italian analysis. Eur J Ophthalmol 2021;32:11206721211011362.
17.	Huang CW, Yeh PT, Tsao PN, Chou HC, Chen CY, Yen TA, et al. Validation of the postnatal growth and retinopathy of prematurity screening criteria in a Taiwanese cohort. Am J Ophthalmol 2022;237:22-31.
18.	Shiraki A, Fukushima Y, Kawasaki R, Sakaguchi H, Mitsuhashi M, Ineyama H, et al. Retrospective validation of the postnatal growth and Retinopathy of Prematurity (G-ROP) criteria in a Japanese cohort. Am J Ophthalmol 2019;205:50-3.
19.	Vinayahalingam N, McDougall J, Ahrens O, Ebneter A. Retrospective validation of the postnatal Growth and Retinopathy of Prematurity (G-ROP) criteria in a Swiss cohort. BMC Ophthalmol 2022;22:19.
20.	Yabas Kiziloglu O, Coskun Y, Akman I. Assessment of the G-ROP study criteria for predicting retinopathy of prematurity: Results from a tertiary centre in Turkey. Int Ophthalmol 2020;40:1647-52.
21.	Fierson WM, American Academy of Pediatrics Section on Ophthalmology, American Academy of Ophthalmology, American Association for Pediatric Ophthalmology and Strabismus, American Association of Certified Orthoptists. Screening examination of premature infants for retinopathy of prematurity. Pediatrics 2013;131:189-95.
22.	International Committee for the Classification of Retinopathy of Prematurity. The international classification of retinopathy of prematurity revisited. Arch Ophthalmol 2005;123:991-9.
23.	Cao JH, Wagner BD, McCourt EA, Cerda A, Sillau S, Palestine A, et al. The Colorado-Retinopathy of Prematurity model (CO-ROP): Postnatal weight gain screening algorithm. J AAPOS 2016;20:19-24.
24.	Löfqvist C, Andersson E, Sigurdsson J, Engström E, Hård AL, Niklasson A, et al. Longitudinal postnatal weight and insulin-like growth factor I measurements in the prediction of retinopathy of prematurity. Arch Ophthalmol 2006;124:1711-8.
25.	Binenbaum G, Ying GS, Tomlinson LA, Postnatal Growth and Retinopathy of Prematurity (G-ROP) Study Group. Validation of the Children's Hospital of Philadelphia Retinopathy of Prematurity (CHOP ROP) model. JAMA Ophthalmol 2017;135:871-7.