\section{Appendix}

\iffalse
\begin{figure}[htp]
    \begin{subfigure}[b]{0.5\textwidth}
    \includegraphics[width=\textwidth]{evaluation/images/generate4_1og.png}
    \caption{Original ultrasound image \newline \newline}
  \end{subfigure}
  \hfill
  \begin{subfigure}[b]{0.5\textwidth}
    \includegraphics[width=\textwidth]{evaluation/images/generate4_1.png}
    \caption{Synthesised image with added papillation. Green box is the original image and Blue box is the modified region of interest}
  \end{subfigure}
  
  \begin{subfigure}[b]{0.5\textwidth}
    \includegraphics[width=\textwidth]{evaluation/images/generate4_2og.png}
    \caption{Original ultrasound image\newline \newline}
  \end{subfigure}
  \hfill
  \begin{subfigure}[b]{0.5\textwidth}
    \includegraphics[width=\textwidth]{evaluation/images/generate4_2.png}
    \caption{Synthesised image with added papillation. Green box is the original image and Blue box is the modified region of interest}
  \end{subfigure}
  
  \begin{subfigure}[b]{0.5\textwidth}
    \includegraphics[width=\textwidth]{evaluation/images/generate4_4og.png}
    \caption{Original ultrasound image\newline \newline}
  \end{subfigure}
  \hfill
  \begin{subfigure}[b]{0.5\textwidth}
    \includegraphics[width=\textwidth]{evaluation/images/generate4_4.png}
    \caption{Synthesised image with added papillation. Green box is the original image and Blue box is the modified region of interest}
  \end{subfigure}
  
    \caption{Synthesised ultrasound images with their corresponding original image classified as realistic by experts}
    \label{eval:im:realistic}
\end{figure}

\begin{figure}[htp]
  \begin{subfigure}[b]{0.5\textwidth}
    \includegraphics[width=\textwidth]{evaluation/images/generate4_3og.png}
    \caption{Original ultrasound image\newline \newline}
  \end{subfigure}
  \hfill
  \begin{subfigure}[b]{0.5\textwidth}
    \includegraphics[width=\textwidth]{evaluation/images/generate4_3.png}
    \caption{Synthesised image with added papillation. Green box is the original image and Blue box is the modified region of interest}
    \label{eval:im:bad1}
  \end{subfigure}
  \begin{subfigure}[b]{0.5\textwidth}
    \includegraphics[width=\textwidth]{evaluation/images/generate4_5og.png}
    \caption{Original ultrasound image\newline \newline}
  \end{subfigure}
  \hfill
  \begin{subfigure}[b]{0.5\textwidth}
    \includegraphics[width=\textwidth]{evaluation/images/generate4_5.png}
    \caption{Synthesised image with added papillation. Green box is the original image and Blue box is the modified region of interest}
    \label{eval:im:bad2}
  \end{subfigure}
  \caption{Synthesised ultrasound images with their corresponding original image classified as less realistic by experts}
  \label{eval:im:badimage}
\end{figure}
\fi



\iffalse
\begin{figure}[htp]
\centering
\resizebox{0.8\textwidth}{!}{
  \begin{subfigure}[b]{0.45\textwidth}
    \includegraphics[width=\textwidth]{evaluation/images/j596_3L1-9.png}
    \caption{Segmentation of locules by the baseline model}
    \label{im:eval:multiloc1}
  \end{subfigure}
  \hfill
  \begin{subfigure}[b]{0.45\textwidth}
    \includegraphics[width=\textwidth]{evaluation/images/j596_3L1-9-1.png}
    \caption{Segmentation of locules by the baseline model with synthesised data}
  \end{subfigure}}
  \caption{Sample of a segmentation of a locule [Green represents correctly labelled pixels, red the falsely predicted segmentation and gold the false negative pixels]}
  \label{im:eval:multilocbad}
\end{figure}
\fi



\begin{table}[htp]
\centering
\resizebox{\textwidth}{!}{%
\begin{tabular}{@{}p{1cm}p{4cm}p{4cm}p{4cm}p{4cm}@{}}
\toprule
\multicolumn{1}{l}{\textbf{Machine}} & \multicolumn{1}{l}{\textbf{1}}            & \multicolumn{1}{l}{\textbf{2}}           & \textbf{3}                       & \textbf{4}                                \\ \midrule
\textbf{GPU}                                  & GM200 {[}GeForce GTX TITAN X{]}           & GA102GL {[}RTX A6000{]}                  & GA102GL {[}RTX A5000{]}          & GA102 {[}GeForce RTX 3090{]}              \\
\textbf{CPU}                                  & Intel(R) Xeon(R) CPU E5-1630 v3 @ 3.70GHz & Intel(R) Xeon(R) Gold 6230 CPU @ 2.10GHz & AMD EPYC 7502P 32-Core Processor & Intel(R) Core(TM) i9-10920X CPU @ 3.50GHz \\
\textbf{RAM}                                  & 66GB                                      & 192GB                                    & 258GB                            & 130GB                                     \\ \bottomrule
\end{tabular}%
}
\caption{Specification of machines used to carry out our experiments}
\label{machines_used}
\end{table}
\section{Clinical Considerations}

This project is a medical research study which handles and analyses ultrasound images obtained from women. Our study recruited women who attended the Rapid Access and Ovarian Cyst clinics at the Hammersmith Hospital. The main requirement for our IOTA-7 study is for women to present with at least one adnexal mass. This study has been approved by the Queen Square Research Ethics Committee, and permission was granted for the use of ultrasound images for machine learning purposes. 

All patients have provided us with a written and signed consent form which, amongst several rights, allows us to use their scans as long as they are pseudo-anonymised. To do so, we have labelled their scans with a study number for training purposes, making the patients non-identifiable, and the scans are exported from the machines anonymously. Finally, no information directly identifying the patients has been used as part of the study.
\fi
\section{Background}










\section{Conclusion}

This paper proposes a novel ultrasound image synthesiser to address dataset imbalance and under-represented adnexal masses. Our research has demonstrated that the nnU-Net baseline has room for improvements, specifically for masses with under-represented features: solid areas and papillations.
Medical experts have confirmed the realistic appearance of the synthesised images. The use of synthesised images balanced the dataset for papillations and solid areas, translating to an improvement in the performance of the model by 3.7\% and 8\% for papillations and solid areas respectively over the state-of-the-art nnU-Net ~\cite{nn_unet}. Furthermore, the integration of synthesised images also reduced the variability found between medical experts (by 35\% on average). 

\noindent\textbf{Acknowledgements:}  UKRI London Medical Imaging and Artificial Intelligence Centre for Value Based Healthcare (No.104691, \url{https://www.aicentre.co.uk/}). JB was supported by an Imperial Health Charity Pre-Doc Fellowship award 2021-2022. CL got support from the Imperial Dept. Computing Corporate Partnership Programme~\url{https://cpp.doc.ic.ac.uk/}.  

\section{Evaluation and Results}

\noindent\textbf{Dataset:} We have curated a transvaginal ultrasound dataset for this study using a GE Voluson E10, E8, S10 Expert or a Samsung Hera W10, WS80A scanner with a transvaginal convex transducer. The dataset contains 532 2D images from 222 patients with their corresponding ground truth segmentations of lesions, locules, solid areas and papillations  (Fig.~\ref{fig:ultrasound_examples}). Segmentation masks have been generated by an expert and verified/corrected by another clinical expert. 
Solid areas and papillation labels are severely under-represented and also naturally less common and hard to detect due to their small size.
We have conducted an inter-observer variability study where three experienced doctors individually segmented adnexal masses in 20 ultrasound images. Consistently for all pairs of observers, the segmentation of solid areas and papillations appeared to be the most challenging for medical professionals (up to 12.7\% decrease). We use the DSC mean and variance across observers as the human-level performance benchmark, and the tolerance for the Surface DSC is based on the variance of this study.






\begin{figure}
\centering
\begin{minipage}{.46\textwidth}
    \centering
    \resizebox{0.95\textwidth}{!}{
      \begin{subfigure}[c]{0.5\textwidth}
        \includegraphics[width=\textwidth]{dataset/images/label_1_zoomed.png}
        \caption{Lesion segmentation}
        \label{im:label1zoom}
      \end{subfigure}
      \hfill
      \begin{subfigure}[c]{0.5\textwidth}
        \includegraphics[width=\textwidth]{dataset/images/label_2_zoomed.png}
        \caption{Locule segmentation}
        \label{im:label2zoom}
      \end{subfigure}}
   
    
   
    \centering
    \resizebox{0.95\textwidth}{!}{
      \begin{subfigure}[b]{0.5\textwidth}
        \includegraphics[width=\textwidth]{dataset/images/label_3_zoomed.png}
        \caption{Solid area segmentation}
        \label{im:label3zoom}
      \end{subfigure}
      \hfill
      \begin{subfigure}[b]{0.5\textwidth}
        \includegraphics[width=\textwidth]{dataset/images/label_4_zoomed.png}
        \caption{Papillation segmentation}
        \label{im:label4zoom}
      \end{subfigure}}
      
      \caption{Ultrasound image showing examples of each adnexal mass}
      \label{fig:ultrasound_examples}
\end{minipage}%
\qquad
\begin{minipage}{.46\textwidth}
    \centering
    \begin{tikzpicture}
    \begin{axis}[xbar,
    height = 0.95\textwidth,
    width = 0.93\textwidth,
   
    axis lines = left,
   
    enlarge y limits  = 0.2,
    xmin = 0,
    xmax = 750,
   
    symbolic y coords = {Papillation, Solid area, Locule, Lesion},
    legend pos = south east,
    nodes near coords,
  ]
  \addplot coordinates { (166,Papillation)         (225,Solid area)
                         (498,Locule)  (532,Lesion) };
  \addplot coordinates { (282,Papillation)         (345,Solid area)
                         (616,Locule)  (650,Lesion) };
    \legend{$D_1$, $D_2$}

    \end{axis}
    \end{tikzpicture}
    \caption{Comparison of the class balance: original dataset $D_1$ and extended dataset with synthesised images $D_2$.}
    \label{fig:dataset_balance_compare}
\end{minipage}
\end{figure}

We experiment with training models on a dataset of only real images $D_1$ and the same dataset extended with images produced by our data synthesiser $D_2$. The training took approximately 40 hours per fold.
We evaluated each model with the same held-out test set consisting of 53 real samples, ensuring consistency and fairness when comparing the methods.
None of these test images were included in any part of the synthesis process when creating $D_2$.
As a baseline we used nnU-Net \cite{nn_unet} with region-based training to allow for overlapping labels.


We have synthesised ultrasound images with lesions, locules, solid areas and papillations. In our case, a patient can only have a papillation if there is also a solid area, a locule and a lesion present, which represents only 10\% of our dataset. This means that the dataset cannot be entirely balanced, but we have increased the number of images with solid areas and papillations from 225 to 345 and 166 to 282 respectively (Fig.~\ref{fig:dataset_balance_compare}).




















\section{Results}

\noindent\textbf{Metrics:} Segmentation performance is evaluated between ground truth and model prediction by Dice similarity coefficient (DSC), Surface Dice similarity coefficient (SDSC), Hausdorff Distance at 95th percentile (HD95) and recall.
For images in the test set where one label was not present in both the ground truth and the predicted segmentations, the DSC and SDSC and recall are set to 1 if no false positive prediction is present, \emph{i.e.}, an empty label mask is predicted. To determine whether the improvement of methods from the baseline is significant, we perform a statistical significance test for the DSC and SDSC. We will carry out a two-sided paired t-test on two models trained on the same test set. From this statistical significance test, a p-value is calculated and is compared to a threshold $\alpha$.





\begin{table}[htb]
\centering
\resizebox{\textwidth}{!}{
\begin{tabular}{@{}p{0.5cm}lllll@{}}
\toprule
& \textbf{Model}   & \textbf{DSC}           & \textbf{SDSC}          & \textbf{Recall}        & \textbf{HD95}          \\ \midrule


\multirow{5}{*}{\rotatebox{90}{\textbf{Lesions}}} &  \textbf{BL}             & 0.951 $\pm$ 0.063          & 0.961 $\pm$ 0.005          & 0.948 $\pm$ 0.080          & 10.35 $\pm$ 12.71          \\
& \textbf{BL + BN + DA}     & 0.955 $\pm$ 0.058          & 0.966 $\pm$ 0.006          & 0.947 $\pm$ 0.085          & 9.461 $\pm$ 12.79          \\
& \textbf{BL + S}           & 0.955 $\pm$ 0.056          & 0.965 $\pm$ 0.011          & \textbf{0.955} $\pm$ \textbf{0.067} & 9.379 $\pm$ 12.73           \\
& \textbf{BL + S + BN + DA} & $\textbf{0.957} \pm \textbf{0.050}^{*}$ & \textbf{0.966} $\pm$ \textbf{0.001}          & 0.953 $\pm$ 0.075          & \textbf{9.068} $\pm$ \textbf{11.32} \\
& \textbf{Expert} & $0.98 \pm 0.062$          & -          & $0.975 \pm 0.092$          & $17.88 \pm 55.267$          \\
\midrule

\multirow{5}{*}{\rotatebox{90}{\textbf{Locule}}} &
\textbf{BL}               & 0.936 $\pm$ 0.073          & 0.984 $\pm$ 0.062          & 0.934 $\pm$ 0.095          & 10.057 $\pm$ 11.65         \\
& \textbf{BL + BN + DA}     & 0.938 $\pm$ 0.074         & 0.988 $\pm$ 0.047          & 0.93 $\pm$ 0.105           & 9.549 $\pm$ 11.52          \\
& \textbf{BL + S}    & $\textbf{0.943} \pm \textbf{0.064}^*$ & \textbf{0.989} $\pm$  \textbf{0.05} & \textbf{0.944 $\pm$ 0.08} & \textbf{8.852} $\pm$ \textbf{10.17} \\
& \textbf{BL + S + BN + DA} & 0.941 $\pm$ 0.071          & 0.988 $\pm$ 0.045          & 0.936 $\pm$ 0.100          & 9.298 $\pm$ 10.79          \\ 
& \textbf{Expert} & $0.939 \pm 0.229$          & -          & $0.943 \pm 0.229$          & $4.204 \pm 17.1$          \\\midrule

\multirow{5}{*}{\rotatebox{90}{\textbf{Solid area}}} &
\textbf{BL}               & 0.736 $\pm$ 0.383          & 0.852 $\pm$ 0.339         & 0.822 $\pm$ 0.320          & 26.011 $\pm$ 18.93          \\
& \textbf{BL + BN + DA}     & $0.812 \pm 0.330^{\bullet}$          & $0.901 \pm 0.276^{*}$         & 0.854 $\pm$ 0.291          & 26.748 $\pm$ 20.98          \\
& \textbf{BL + S}           & $\textbf{0.816} \pm \textbf{0.327}^{\circ}$ & $\textbf{0.908} \pm \textbf{0.276}^{\circ}$ & \textbf{0.875 $\pm$ 0.265} & \textbf{23.957} $\pm$ \textbf{19.77}          \\
&  \textbf{BL + S + BN + DA} & $0.809 \pm 0.334^{\circ}$          & $0.898 \pm 0.292^{*}$         & 0.864 $\pm$ 0.283          & 25.432 $\pm$ 21.70          \\ 
& \textbf{Expert} & $0.853 \pm 0.345$          & -          & $0.908 \pm 0.27$         & $69.611 \pm 164.532$          \\\midrule

\multirow{5}{*}{\rotatebox{90}{\textbf{Papillation}}} &
\text{BL}               & 0.761 $\pm$ 0.395          & 0.854 $\pm$ 0.325         & 0.805 $\pm$ 0.363         & \textbf{31.746 $\pm$ 26.67}          \\
& \textbf{BL + BN + DA}     & $0.796 \pm 0.376^*$          & 0.843 $\pm$ 0.353        & \textbf{0.815 $\pm$ 0.363} & 36.436 $\pm$ 30.77 \\

& \textbf{BL + S}          & $0.798 \pm 0.380^*$ & $0.878 \pm 0.30^{\circ}$ & 0.814 $\pm$ 0.357         & 37.467 $\pm$ 27.1          \\
& \textbf{BL + S + BN + DA} & $\textbf{0.809} \pm \textbf{0.391}^*$          & $\textbf{0.882} \pm \textbf{0.357}^{\circ}$          & 0.807 $\pm$ 0.371         & 40.851 $\pm$ 28.48          \\
& \textbf{Expert} & $0.873 \pm 0.33$          & -          & $0.954 \pm 0.202$         & $25.318 \pm 66.985$          \\
\bottomrule
\\
\end{tabular}}
\caption{Evaluation and ablation study for DSC, SDSC, Recall and HD95 (in px) scores across models for locules, lesions, solid areas and papillations [\textbf{$\text{BL}$}: Baseline nnU-Net \cite{nn_unet} with region-based training, \textbf{BN}: Batch normalisation, \textbf{DA}: Intense data augmentation, \textbf{S}: With synthesised data] on dataset $D_1$ and $D_2$ for synthesiser. Significance compared to the baseline ($\text{BL}$):  $p^*$ < 0.05, $p^{\circ}$ < 0.01, $p^{\bullet}$ < 0.001)}
\label{tab:results}
\end{table}


\noindent\textbf{Results:} Figure \ref{implementation:synthesised_image_examples} shows synthesised images with their corresponding original ultrasound image. For most images, the added papillation is blended seamlessly in the image; the edges are not apparent and the intensities are coherent whilst still keeping the texture of papillations. The less realistic images arise from incompatible textures in the source and destination images. As observed, the textures are obviously different for the less realistic images leading to apparent edges. We sought an evaluation by experienced ultrasound examiners in Gynaecology who rated a selection of images. 

In Table~\ref{tab:results} we observe that the nnU-Net \cite{nn_unet} baseline model, which uses region-based training, performs very well for lesions and locules (>0.9 DSC) but worse for solid areas and papillations (<0.8 DSC). We have also trained the baseline model with more intensive data augmentation and batch normalisation, which improves the segmentation performance.

The models trained with synthesised data appear to be the best performing for all classes. 
The most significant effect is observed in solid areas where all of the metrics have improved, with the DSC and SDSC increasing by 8\% and 5.6\% respectively, making it significantly better than the baseline with p < 0.01 and considerably better than all other models.
Finally, the performance of the segmentation of papillations is also improved with the DSC and SDSC increasing by 2.1\% and 2.4\% respectively, making the model significantly better than the baseline with regards to the SDSC (p < 0.01). However, the variance and value of the HD95 has increased. We believe this is due to the fragmented nature of this label and more papillations being segmented compared to the other models. The recall is higher which confirms our hypothesis that targeted lesion synthesis can improve ultrasound image segmentation. 

The model with synthesised data was also trained with the addition of batch normalisation and intense data augmentation. The reason behind this was to apply more data augmentation onto the synthesised images and make the model more robust to all images and bridge the domain gap between the different machines used. The modifications appear to perform similarly to the baseline model with synthesised data. For solid areas, the performance is very similar to the model without BN and DA where the DSC and SDSC are still significantly better than the baseline with p < 0.01 and p < 0.05 respectively. For papillations, the DSC, SDSC and recall still remain within a single standard deviation, although the HD95 has increased.
Qualitative examples are shown in Figure~\ref{tab:appendix:goodpredictionlabel3} and \ref{im:eval:seglabel1}.  We used an Nvidia RTX A6000 for experimentation.

\iffalse
\begin{figure}[htp]
\resizebox{\textwidth}{!}{
\begin{tabular}{ >{\centering\arraybackslash}m{5cm}  >{\centering\arraybackslash}m{5cm}  >{\centering\arraybackslash}m{5cm}  >{\centering\arraybackslash}m{5cm} }

\multicolumn{4}{c}{\textbf{\Large Solid Areas}} \\[.4cm] 

\includegraphics[width=0.33\textwidth, height=40mm]{evaluation/images/bl26428_0011L1-9_copy.png} &
\includegraphics[width=0.33\textwidth, height=40mm]{evaluation/images/better26428_0011L1-9.png} &
\includegraphics[width=0.33\textwidth, height=40mm]{evaluation/images/synth26428_0011L1-9.png} & 
\includegraphics[width=0.33\textwidth, height=40mm]{evaluation/images/synth_better26428_0011L1-9.png} \\[.7cm] 

\multicolumn{4}{c}{\textbf{\Large Papillations}} \\[.4cm] 


\includegraphics[width=0.33\textwidth, height=40mm]{evaluation/images/better2j596_1L1-9-1.png} & \includegraphics[width=0.33\textwidth, height=40mm]{evaluation/images/better2j596_1L1-9.png} & 
\includegraphics[width=0.33\textwidth, height=40mm]{evaluation/images/synthj596_1L1-9.png} & 
\includegraphics[width=0.33\textwidth, height=40mm]{evaluation/images/synth_betterj596_1L1-9.png} \\[.4cm]
\\
\textbf{\Large BL} & \textbf{\Large BL + BN + DA} & \textbf{\Large BL + S} & \textbf{\Large BL + S + BN + DA} \\
\\                        
\end{tabular}}
\caption{Predicted segmentation of trained models compared to the ground truth (GT) segmentation.
[\textbf{$\text{BL}$}: Baseline nnU-Net \cite{nn_unet} with region-based training, \textbf{BN}: Batch normalisation, \textbf{DA}: Intense data augmentation, \textbf{S}: With synthesised data] [Green represents correctly labelled pixels, red the falsely predicted segmentation and gold the false negative pixels]}
\label{tab:goodpredictionlabel3}
\end{figure}
\fi

\begin{figure}[htp]
\resizebox{\textwidth}{!}{
\begin{tabular}{ >{\centering\arraybackslash}m{5cm}  >{\centering\arraybackslash}m{5cm}  >{\centering\arraybackslash}m{5cm}  >{\centering\arraybackslash}m{5cm} }

\multicolumn{4}{c}{\textbf{\Large Solid Areas}} \\[.4cm] 

\includegraphics[width=0.33\textwidth, height=40mm]{evaluation/images/better26428_0011L1-9.png} &
\includegraphics[width=0.33\textwidth, height=40mm]{evaluation/images/better26428_0011L1-9.png} &
\includegraphics[width=0.33\textwidth, height=40mm]{evaluation/images/synth26428_0011L1-9.png} & 
\includegraphics[width=0.33\textwidth, height=40mm]{evaluation/images/synth_better26428_0011L1-9.png} \\

\includegraphics[width=0.33\textwidth, height=40mm]{evaluation/images/regions425963_0028_copy.png} &
\includegraphics[width=0.33\textwidth, height=40mm]{evaluation/images/better25963_0028.png} &
\includegraphics[width=0.33\textwidth, height=40mm]{evaluation/images/synth25963_0028.png} & 
\includegraphics[width=0.33\textwidth, height=40mm]{evaluation/images/synth_better25963_0028.png}\\

\includegraphics[width=0.33\textwidth, height=40mm]{evaluation/images/regions25969_0023L1-9.png} &
\includegraphics[width=0.33\textwidth, height=40mm]{evaluation/images/better25969_0023L1-9.png} & 
\includegraphics[width=0.33\textwidth, height=40mm]{evaluation/images/synth25969_0023L1-9-1.png} & 
\includegraphics[width=0.33\textwidth, height=40mm]{evaluation/images/synth_better25969_0023L1-9.png} \\[.7cm] 

\multicolumn{4}{c}{\textbf{\Large Papillations}} \\[.4cm] 

\includegraphics[width=0.33\textwidth, height=40mm]{evaluation/images/regions425606_0016L1-9TIF-1.png} & 
\includegraphics[width=0.33\textwidth, height=40mm]{evaluation/images/better25606_0016L1-9TIF.png} & 
\includegraphics[width=0.33\textwidth, height=40mm]{evaluation/images/synth25606_0016L1-9TIF_copy.png} & 
\includegraphics[width=0.33\textwidth, height=40mm]{evaluation/images/synth_better25606_0016L1-9TIF.png}\\

\includegraphics[width=0.33\textwidth, height=40mm]{evaluation/images/regions25600_0011.png} &
\includegraphics[width=0.33\textwidth, height=40mm]{evaluation/images/better25600_0011.png} & 
\includegraphics[width=0.33\textwidth, height=40mm]{evaluation/images/synth25600_0011.png} & \includegraphics[width=0.33\textwidth, height=40mm]{evaluation/images/synth_better25600_0011.png} \\

\includegraphics[width=0.33\textwidth, height=40mm]{evaluation/images/better2j596_1L1-9-1.png} & \includegraphics[width=0.33\textwidth, height=40mm]{evaluation/images/better2j596_1L1-9.png} & 
\includegraphics[width=0.33\textwidth, height=40mm]{evaluation/images/synthj596_1L1-9.png} & 
\includegraphics[width=0.33\textwidth, height=40mm]{evaluation/images/synth_betterj596_1L1-9.png} \\[.4cm]
\\
\textbf{\Large BL} & \textbf{\Large BL + BN + DA} & \textbf{\Large BL + S} & \textbf{\Large BL + S + BN + DA} \\
\\                        
\end{tabular}}
\caption{Predicted segmentations of trained models compared to the ground truth (GT) segmentation [\textbf{$\text{BL}$}: Baseline nnU-Net \cite{nn_unet} with region-based training, \textbf{BN}: Batch normalisation, \textbf{DA}: Intense data augmentation, \textbf{S}: With synthesised data] [Green represents correctly labelled pixels, red the falsely predicted segmentation and gold the false negative pixels]}
\label{tab:appendix:goodpredictionlabel3}
\end{figure}

\begin{figure}[htp]
\centering
\resizebox{1.0\textwidth}{!}{
  \begin{subfigure}[b]{0.25\textwidth}
    \includegraphics[height=3.0cm]{evaluation/images/24893_0014-1.png}
    \caption{
    }
  \end{subfigure}
  \hfill
  \begin{subfigure}[b]{0.25\textwidth}
    \includegraphics[height=3.0cm]{evaluation/images/24893_0014.png}
    \caption{}
  \end{subfigure}
  \begin{subfigure}[b]{0.25\textwidth}
    \includegraphics[height=3.0cm]{evaluation/images/j596_3L1-9.png}
    \caption{}
    \label{im:eval:multiloc1}
  \end{subfigure}

  \begin{subfigure}[b]{0.25\textwidth}
    \includegraphics[height=3.0cm]{evaluation/images/j596_3L1-9-1.png}
    \caption{}
  \end{subfigure}}
  \caption{Sample of a segmentation of a lesion (a-b) and locule (c-d) [Green represents correctly labelled pixels, red the falsely predicted segmentation and gold the false negative pixels]. (a) Segmentation of a lesion by the baseline model. (b) Segmentation of a lesion by the baseline with synthesised data, batch normalisation and intense data augmentation model (BL + S + BN + DA). (c) Segmentation of locules by the baseline model, (d) Segmentation of locules by the baseline model with synthesised data.}
  \label{im:eval:seglabel1}
\end{figure}

\iffalse
\begin{table}[htp]
\centering
\resizebox{\textwidth}{!}{%
\begin{tabular}{@{}p{1cm}p{4cm}p{4cm}p{4cm}p{4cm}@{}}
\toprule
\multicolumn{1}{l}{\textbf{Machine}} & \multicolumn{1}{l}{\textbf{1}}            & \multicolumn{1}{l}{\textbf{2}}           & \textbf{3}                       & \textbf{4}                                \\ \midrule
\textbf{GPU}                                  & GM200 {[}GeForce GTX TITAN X{]}           & GA102GL {[}RTX A6000{]}                  & GA102GL {[}RTX A5000{]}          & GA102 {[}GeForce RTX 3090{]}              \\
\textbf{CPU}                                  & Intel(R) Xeon(R) CPU E5-1630 v3 @ 3.70GHz & Intel(R) Xeon(R) Gold 6230 CPU @ 2.10GHz & AMD EPYC 7502P 32-Core Processor & Intel(R) Core(TM) i9-10920X CPU @ 3.50GHz \\
\textbf{RAM}                                  & 66GB                                      & 192GB                                    & 258GB                            & 130GB                                     \\ \bottomrule
\end{tabular}%
}
\caption{Specification of machines used to carry out our experiments}
\label{machines_used}
\end{table}
\fi



\section{Introduction}


One in two people will develop cancer, with ovarian cancer being the most lethal gynaecological malignancy \cite{nhs_cancer_stat}. Around 75\% of women diagnosed with ovarian cancer are already at the advanced stages of the disease due to the earlier stages often showing no symptoms \cite{ovarian_cancer_stats_2}, leading to a survival rate of only $35\%$ \cite{cancerresearch}.

Current diagnostic methods for detecting ovarian cancer include tumour markers (CA-125) and the analysis of a transvaginal ultrasound image. Ultrasound classification of adnexal masses can be a challenging process. Current ultrasound based diagnostic models exist, but still rely on the extraction of ultrasound features, which is time-consuming, labor-intensive, operator-dependant and subject to error. Accurate classification of adnexal masses is necessary to reduce incorrect diagnosis and unnecessary surgeries. With an efficient workflow, population screening for ovarian cancer could be feasible, leading to earlier disease detection and increased survival rates. Evidently, there is tangible value in developing a tool able to automatically extract the features in transvaginal ultrasounds accurately and consistently.


Supervised Deep Learning is a fast-growing area of research that has been successfully applied in the medical field to predict illnesses, design treatments, and distinguish between benign and malignant masses \cite{hesamian2019deep}. For ovarian cancer diagnosis, Deep Learning could be used to automatically segment transvaginal ultrasound features and extract characteristics required by risk models such as the number of cysts and their dimensions.


A major challenge of ovarian cancer diagnosis is the accurate interpretation of ultrasound images. Although ultrasound is the imaging modality of choice, as it is accessible, non-invasive, and does not use radiation \cite{746626}, it is also ``notorious for having significant noise with a low signal-to-noise ratio'' \cite{roscoecoping}. This represents a challenge for both medical experts and segmentation algorithms. The wide variety in shape and size of adnexal masses can also pose challenges for segmentation.



In this paper we automate and regularise ultrasound feature extraction, with the aim of improving the accuracy of diagnosis and reducing unnecessary referrals.
We focus on transvaginal ultrasound images and segment four adnexal mass features: (1) lesions, (2) locules, (3) solid areas and (4) papillations -- this has not yet been attempted in literature to the best of our knowledge. The presence or absence, number of occurrences, and dimensions of adnexal masses represent the ultrasound features used by risk models to determine the probability of malignancy.
Additionally, some adnexal masses are more common than others, and some of the less common masses are the most difficult to detect, even for medical experts. This can lead to an imbalanced dataset and a poor segmentation of under-represented masses. 

\noindent\textbf{Contributions:}
Current Deep Learning methods struggle to accurately segment images when trained on imbalanced datasets.
Our contributions are as follows:
1) we propose a novel method to synthesise segmentation data with Poisson image editing~\cite{perez2003poisson} to mitigate class imbalance;
2) we extensively evaluate our method on transvaginal ultrasound data and compare it to domain experts, which we show to be highly biased in an inter-observer study;
3) models trained with our additional generated data outperform the current state-of-the-art nnU-Net~\cite{nn_unet} (+3.7\% DSC and +8\% DSC for the augmented classes), achieving excellent and high-standard segmentations for lesions and locules $\geq 0.94$ DSC, and good quality segmentation for solid areas and papillations $\approx 0.82$ DSC.

\subsubsection{Acknowledgements} Please place your acknowledgements at
\bibliographystyle{splncs04}

\section{Methodology}


Deep neural networks such as nnU-Net \cite{nn_unet} require a large amount of data to perform well. This requirement is commonly addressed with extensive data augmentation techniques such as rotating and scaling. However, this does not improve cases of imbalanced datasets.
In this work we aim to synthesise new data by extracting examples of less-seen masses and integrating them into other samples in which such features would be clinically plausible. 
To do this we cannot na\"ively copy and paste the masses from one image to another, as this would result in unrealistic image discontinuities. This simple technique was first trialled and resulted in a significant decrease of performance for solid areas and papillations (-2.7\% and -3.3\% DSC respectively). Instead we propose a novel method of synthesising data based on the Poisson image editing algorithm \cite{perez2003poisson}.
This method allows us to balance the dataset with good quality imaging, and subsequently improve the performance of the model, specifically for the augmented classes. An overview of our approach is shown in Figure~\ref{im:synthesiser}.

 





\input{implementation/implementation}
