Session Item

Bayesian networks (BNs) are probabilistic graphical models that have shown potential to aid medical physicists and radiation technologists in the time-consuming procedure of error detection in radiotherapy treatment plans. Luk et al¹. developed a BN that flags potentially erroneous parameters in radiotherapy plans for humans to double check. They built the structure based on expert knowledge, then trained and internally validated it using local data from University of Washington (UW) (Elekta linear accelerator and oncology information system (OIS)). We assessed whether retraining the model with external data from an independent institute (Maastro Clinic, Maastricht, The Netherlands) with different technology and clinical profile would improve its performance.

We extracted a dataset of 5238 patients (19054 treatment plans) treated in Maastro with photon Intensity Modulated Radiation therapy (IMRT) and Volumetric Modulated Arc Therapy (VMAT) from Aria OIS (Varian Medical Systems) to retrain the BN. The patients were treated with a Truebeam linear accelerator (LINAC) (Varian Medical Systems) between 2012 and 2020. We embedded simulated errors in 3% of the plans categorized into four types: patient positioning, collimator angle, dose prescription, and general radiotherapy plans errors. We split the dataset into training (80%) and internal validation sets (20%). We assessed the performance of three different models on the internal validation set: 1) the original BN, with parameters trained on UW data; 2) a BN trained on Maastro data; and 3) a BN trained on UW and Maastro data mapping terminologies between the two datasets. 

As shown in Figure 1, the BNs trained on the three different datasets achieved similar areas under the receiver operating characteristic curve (AUCs) with overlapping confidence intervals: original 71.1 % (95% CI: 67.2% - 75.3%), Maastro 70.8% (66.5% - 75.2%) and UW+Maastro 72.5% (67.9% - 76.9%). When setting the flagging threshold to a level that would result in a 95% sensitivity, the positive predictive value (PPV) of the BNs assuming a 3% prevalence of error ranged between 3.2% and 4.3% and the percentage of values flagged as errors in order to catch 99% of them ranged between 66- and 78% (see Table 1).  

The results show that the BN trained with data from one institution (UW) could be used on detecting errors in treatment plans in another institution (Maastro), and the performance is similar to the BN trained from scratch with local data. However, the performance of BN in the new institution (Maastro) is not as good as the published result¹ (AUC 72.5% vs 89%), showing that further work is required to adjust the BN in another institution with different technology and clinical profile.

References

1.       1.PMID: 30927253