Inter-reader agreement of high-resolution computed tomography findings in patients with COVID-19 pneumonia: A multi-reader study

Purpose: To investigate the inter-reader agreement in assessing high-resolution computed tomography (HRCT) features of coronavirus disease 2019 (COVID-19) pneumonia. Method: Seventy-seven consecutive patients (mean age, 64 ± 15 years) with mild COVID-19 pneumonia that underwent HRCT were retrospectively included. Three radiologists [two devoted to thoracic imaging (R1, R2), and one generalist (R3)] on a per-examination basis independently assessed ground-glass opacity (GGO), consolidation, and crazy-paving pattern. The extent of each feature (total feature score, TFS) was semi-quantitatively assessed, and each TFS summed up to obtain total lung score (TLS). Presence of organizing pneumonia (OP) pattern was also recorded. The inter-reader agreement was calculated with Cohen’s Kappa (k) and Free-Marginal Multirater k. Multivariable analysis was run to determine whether imaging features were predictive of short-term evolution to severe disease (need for ventilation). Results: Most features showed substantial inter-reader agreement, including TLS > 6 (k = 0.69), which was an independent predictor of short-term occurrence of severe disease, regardless of the reader (OR 9–53.19). Consolidation TFS > 2 and OP pattern showed substantial and moderate agreement, respectively, only when comparing R1 and R2. Consolidation TFS > 2 and OP pattern were independent predictors of severe disease for R2 (OR 4.87) and R1 (OR 6), respectively. Conclusions: The inter-reader agreement for most HRCT features of COVID-19 pneumonia ranges moderate-to-substantial, though it depends on readers’ experience in the case of consolidation and OP pattern.