Evaluation of thyroid ultrasound reports’ quality in the Community of Madrid, Spain

Introduction

Up to 5% of the general population has palpable thyroid nodules (1). However, the prevalence of thyroid nodules is significantly higher when evaluated with ultrasound (1, 2). The increasing use of imaging techniques is uncovering a large reservoir of asymptomatic thyroid nodules that has acquired an epidemic magnitude (3, 4, 5, 6, 7). Consequently, thyroid nodules constitute nowadays one of the most frequent reasons for referral to endocrine outpatient clinics.

Although most thyroid nodules are benign, most thyroid cancers present as thyroid nodules (8). Therefore, an adequate differential diagnosis is essential. Ultrasound characterization is instrumental to select which nodules require fine-needle aspiration biopsy (9). Several classification systems have been developed in recent years to homogenize ultrasound characterization. These systems classify nodules into different categories that are associated with an estimated probability of malignancy and with a recommended threshold for biopsy (9, 10, 11). All of them consider the composition, echogenicity, shape, presence of calcifications and margins of the nodule. Therefore, knowing all these characteristics is necessary to determine the need to perform a cytological evaluation. When faced with incomplete ultrasound reports, the treating physician must decide whether to repeat the ultrasound, often prematurely hoping to obtain the missing information in the following report, or to request a fine-needle aspiration biopsy without knowing if it is really indicated (12, 13, 14). Both scenarios result in health care system inefficiency.

Furthermore, knowing the maximum dimensions of the nodule in the anteroposterior, transverse and craniocaudal axes are necessary to follow up thyroid nodules correctly. The absence of this information can mistakenly lead to thinking that a nodule is growing significantly when in fact it is stable, for example if a single different dimension is given in two successive reports. This situation may trigger unnecessary invasive procedures, including surgery.

The objective of this study was to evaluate the current quality of thyroid ultrasound reports in the Community of Madrid to identify areas for improvement to be addressed by the Regional Endocrine Society (Society of Endocrinology, Nutrition and Diabetes of the Community of Madrid – SENDIMAD)’s thyroid cancer working group.

Methods

This transversal study was approved by the ethics committee of the Ramón y Cajal University Hospital (#178-21). Study data were collected and managed using REDCap electronic data capture tools hosted at Ramón y Cajal Health Research Institute (15, 16). REDCap (Research Electronic Data Capture) is a secure, web-based software platform designed to support data capture for research studies, providing i) an intuitive interface for validated data capture, ii) audit trails for tracking data manipulation and export procedures, iii) automated export procedures for seamless data downloads to common statistical packages, and iv) procedures for data integration and interoperability with external sources. For this study, no personal identifiers or demographic data were collected. Two different forms with predefined fields were created in the database: one for reports from 2021 and another one for reports from 2022. In both forms, the specialty of the physician issuing the report, the origin of the report (own center or outsourced test) and the hospital where the report had been evaluated were registered. The presence or absence of information on eight descriptive variables for nodules 1 cm or larger was collected: number of dimensions reported (0, 1, 2 or 3); identification of corresponding axes for the given measurements; composition; echogenicity; margins; shape; calcifications and category of sonographic suspicion. The specific ultrasound classification system used in the report was also registered. Each of the variables was categorized as consistently reported (for all nodules 1 cm or larger); reported sometimes (in some but not all nodules 1 cm or larger) or absent (not described for any of the nodules). To calculate the average of variables correctly described in each report, those described ‘sometimes’ (not reported in all nodules >1 cm) were considered as incorrectly reported. The number of dimensions was considered correctly reported if all three dimensions were given. The shape of the nodule was considered as correctly described if it was either specified in the report or it could be derived from the dimensions and axes detailed in the report. The quality of the report was considered good for assessing the need for cytological evaluation if either size in one dimension (assuming the measurement of largest dimension was provided) plus category of suspicion or size in one dimension plus composition, echogenicity, margins, shape and calcifications were described. The quality of the report was considered good to assess growth during follow-up if three dimensions with corresponding axes were given.

Results

A total of 1234 reports were included in the analyses. The distribution of such reports by institution and year of data capture (2021 vs 2022) is shown in Fig. 1. Most reports (774, 63%) evaluated were from 2021, and two participating institutions did not contribute to reports evaluated in 2022. Reports were issued by radiologists (n = 1015, 82%), endocrinologists (n = 207, 17%) or other specialists (n = 12, 1%; eight by nuclear medicine and four by primary care physicians). Most reports were issued by in-house specialists (n = 1097, 89%), whereas the rest were issued by specialists from outsourced centers (n = 137, 11%).

View Full Size

Distribution of reports according to institution and year of data capture.

Citation: European Thyroid Journal 14, 1; 10.1530/ETJ-24-0390

• Download Figure
• Download figure as PowerPoint slide

The overall quality of the reports is described in Table 1. Only 43.6% reports provided the three dimensions, and 32% identified the corresponding axes of the dimensions given. The frequency of axes reporting was associated with the number of dimensions provided (OR_trend: 2.8; 95% CI: 2.3–3.3; P < 0.001). Nodule composition and echogenicity were the features most frequently reported (79% and 72%, respectively). The rest of the evaluated features were appropriately described in less than half of the reports: margins in 40%, calcifications in 28% and shape in 32%. Only 471 (38%) reports categorized the sonographic suspicion of the nodules, and among them, the classification system most frequently used was ACR-TIRADS (63%), followed by EU-TIRADS (12%) and ATA (American Thyroid Association) (8%).

Comparison of report quality by year of data capture (2021 vs 2022)

The comparison by year of data capture is reported in Table 1. There was no difference in the median number interquartile range (IQR) of features appropriately reported between 2021 and 2022 (4 (2–6) vs 3 (2–5), P = 0.1). The distributions of the number of features appropriately reported and their Kernel density plots (overlapped) by year of data capture are shown in Fig. 2A. Only 6% and 4% reports had all the nodule features reported in 2021 and 2022, respectively. The same figures were observed for reports without any features appropriately reported. Among 2021 reports, three dimensions, echogenicity, and margins were more frequently reported (47% vs 37%, P = 0.005; 74% vs 69%, P = 0.06 and 44% vs 35%, P = 0.006, respectively). There were no differences in the reporting frequency of axes, shape, composition, or presence of calcifications. Reports from 2022 tended to categorize the sonographic suspicion of the nodules more frequently (42% vs 36%, P = 0.08). The most frequently used classification system for reporting sonographic suspicion was ACR-TIRADS in both groups (65% vs 61% in 2021 and 2022, respectively); however, the EU-TIRADS classification system was more frequently used in the 2022 reports (15% vs 9%, P < 0.001).

View Full Size

Distribution of the number of features appropriately reported and their Kernel density plots (overlapped) by year of data capture (A), origin of the report (B) and specialty of the informant (C).

Citation: European Thyroid Journal 14, 1; 10.1530/ETJ-24-0390

• Download Figure
• Download figure as PowerPoint slide

Discussion

The quality of the reports was considered good to appropriately select nodules for biopsy (either size in one dimension plus category of suspicion or size in one dimension plus composition, echogenicity, margins, shape and calcifications) in 40% of cases. In addition, the quality of the reports was considered good to appropriately assess growth during follow-up (three dimensions with axes) in 23% of cases. Therefore, most reports’ quality was insufficient to inform on the need for biopsy (60%) and/or to assess growth during follow-up (77%). Both quality criteria were met in only 13% (159) of the reports. It is unknown whether missing information was assessed by informants but omitted from the reports because of negative findings (non-suspicious features). However, omitting such information in the report leaves treating physicians with the same uncertainty as if it had not been evaluated, prompting management decisions based on insufficient data, leading to unnecessary repeated ultrasounds, biopsies, surgeries and ultimately to system inefficiency.

Thyroid nodule management is mostly decided by endocrinologists with clinical information such as the presence of compressive symptoms, lab test results, ultrasound report and, if available, cytological diagnosis and/or scintigraphy. The quality of ultrasound reports is instrumental in the management decision process. As described in previous studies, the quality of thyroid ultrasound reports before implementing report-standardization strategies was poor in our setting (18, 19, 20, 21, 22). Perhaps the poor quality of such an instrumental tool has driven endocrinologists and other specialists dealing with thyroid nodule management to train in thyroid ultrasound. In Spain, endocrinologists started to perform thyroid ultrasound a little over 15 years ago. Since then, the number of endocrinologists trained in thyroid ultrasound has been increasing, but most thyroid ultrasounds are still being done by radiologists.

In our cohort, most reports (82%) were issued by radiologists, with a median of three out of eight items correctly reported for all thyroid nodules. The quality of the reports was worse for outsourced ultrasounds and better for reports issued by endocrinologists. A previous study also found higher-quality reports among provider-performed ultrasounds than in ultrasounds performed in imaging centers or in radiology departments (21). However, our data might be biased, as endocrinologists’ reports in our 2021 cohort were often issued by the investigators of this study, who have a particular interest in thyroid nodules and thyroid cancer (members of the SENDIMAD’s thyroid cancer working group). Thus, it is possible that the quality of thyroid ultrasound reports issued by other endocrinologists is not as good. To limit bias in the endocrinologists’ reports due to awareness of quality assessment, inclusion of their own reports was only allowed in 2021 (issued before the study design) but forbidden in 2022. This limited the number of reports issued by endocrinologists in our 2022 cohort and prevented comparison with 2021 data due to low counts. The number of features correctly described in the reports issued by radiologists in 2022 improved by an average of 0.4 items over 2021 reports (3.4 vs 3.0). This observation could represent a selection bias of our 2021 cohort or a true improvement of reports over time. Anyhow, the quality of most reports was still insufficient to make clinical decisions.

We did not collect information about the expertise of the informants or select which radiologists issued the reports. Furthermore, radiologists were unaware of the report quality evaluation. Thus, our study represents real-world data from mostly tertiary academic institutions that provide care to more than half of the population of the Community of Madrid. Most reports were issued by either endocrinologists or radiologists working for the public National Health System, who have a fixed monthly salary independent of the number or quality of their reports. However, thyroid ultrasounds ordered by primary care physicians are frequently outsourced to private imaging centers, which get reimbursed per patient. No minimum quality of the report is required for reimbursement. Thus, there is a lack of incentive to provide good quality reports in private imaging centers at this time. We found that report quality was particularly poor in this setting.

We did not collect the number of nodules described in the reports or the size of the largest nodule but rather aggregate data for all nodules >1 cm. Prior studies found no association between the quality of the description and the size of the nodule (18). Furthermore, several studies considered reporting the maximum diameter of the nodule as a quality measurement. It is usually assumed that whenever a single dimension is provided, it must be the largest one. Although this sounds reasonable, it is often not true. But even if it were the largest dimension, it is still insufficient to assess growth during follow-up. Like in our series, most reports in previous studies failed to report three dimensions (18, 19, 21).

A good-quality report can precisely rule out malignancy in 50% of the thyroid nodules (23), select nodules requiring cytological evaluation (9, 10, 11), allow risk stratification of thyroid nodules with indeterminate cytology (24, 25), and provide relevant information for the use and interpretation of the results of molecular testing (26). Thus, improving the quality of ultrasound reports is key for the optimization of the institutional thyroid nodule evaluation pathway (26, 27). Previous studies have demonstrated that implementing report-standardization strategies improves the quality of thyroid ultrasound reports (12, 14, 20, 22, 28). However, more detailed reports may take longer time to evaluate and describe all relevant features (13). Thus, guideline implementation might require scheduling adjustments.

In summary, our study found that most thyroid ultrasound reports issued during 2021 and 2022 in the public health care system of the Community of Madrid provided insufficient data to assess the need for cytological evaluation and/or growth during follow-up of thyroid nodules. Such reports likely led to health care system inefficiency. The quality of outsourced thyroid ultrasound reports was particularly poor. Quality metrics need to be enforced for thyroid ultrasounds covered by the public health care system. To facilitate this, the SENDIMAD’s thyroid cancer working group is developing an application intended to generate semiautomatic standardized thyroid ultrasound reports. Future studies will address whether reporting time is reduced with the application and whether the quality of thyroid ultrasound reports improves with its use in the Community of Madrid.

Declaration of interest

The authors declare that there is no conflict of interest that could be perceived as prejudicing the impartiality of the research reported.

Funding

This work was supported by a 2021 Research in Endocrinology, Nutrition and Diabetes grant (IPI/2021/N2) from the Society of Endocrinology, Nutrition and Diabetes of the Community of Madrid (SENDIMAD).

Author contribution statement

PV was responsible for conceptualization, methodology, investigation, writing the original draft, supervision and funding acquisition. JBQS contributed to investigation, formal analysis, data curation, writing the original draft and visualization. SCM, CFC, MPMN, MCSP, EFF, ALG, MESS, PMR-M, PPR, CTF, MJRT, VAL, MLV and NPG were involved in investigation, as well as writing, reviewing and editing.