\section{Introduction }

Cardiovascular disease (CVD) has become one of the most high-mortality diseases, for which the amount of coronary artery calcification acts as a strong indicator of CVD risk \cite{rumberger1999electron}.
In
clinical practice, CAC is quantified by the Agatston score, using dedicated cardiac CT scans, followed by a expert who manually identify CAC lesions.
 
To assist
medical professionals, previous work based on classical machine learning have attempted to design
CAD methods for computation of CAC score. Durlak et al. \cite{durlak2017growing} applied an atlas-based feature approach in combination with a random forest classifier which is used to incorporate fuzzy spatial knowledge from offline data. Isgum et al. \cite{isgum2012automatic} 
employed a nearest neighbor classifier directly and a two-stage classification with nearest neighbor as well as support vector machine classifiers. There are plenty of other research can be explored \cite{kurkure2010supervised,shahzad2013vessel,wolterink2014automatic}.

In recent years, convolutional neural networks (CNNs) have exhibited great success in Computer Vision by data-driven, especially in image classification tasks. Meanwhile, fully convolutional networks (FCNs) , as the extension of CNNs, also obtained state-of-the-art performance for segmentation problems. In the context of medical image segmentation, specifically cardiac calcification segmentation, algorithms based on deep learning have shown promise. Wolterink et al. \cite{wolterink2015automatic} first attempted to apply CNNs to CAC scoring in contrast-enhanced cardiac CT, with a two-stage network structure but only one stage using deep learning. Recently, some works used two-stage deep learning structure \cite{lessmann2018automatic,wolterink2016automatic}, with the first stage identifying CAC-suspected voxels 
and the second stage more precisely identifying CAC. Shadmi et al. \cite{shadmi2018fully} 
employed Dense-FCN, a design different from the two-stage methods, to segment the lesion directly in cardiac CT. But all the automatic CAC scoring approaches above are designed for either cardiac or chest CT only. 

More recently, multiple screening in one CT session has become a trend in clinical practice. Huang et al. \cite{huang2017densely} presented an automatic method with two CNNs that performs direct computation of CAC score in both cardiac and chest CT scans. 
On the other hand, according to the work of Wolterink et al. \cite{wolterink2016automatic}, 2.5D input has a great advantage compared with 3D input in CAC scoring, as the number of parameters are greatly reduced while retaining spatial information. Both Lessmann et al. and wolterink et al. \cite{lessmann2018automatic,wolterink2016automatic} used the 2.5D ConvNets combining features from three identical 2D ConvStacks with shared weights, each processing an input patch from a different orthogonal viewing direction (axial, sagittal and coronal). To our knowledge, none have applied the efficient 2.5D FCN architecture on multiple types of non-enhanced CT.
 
In this work, 
we propose an automatic method for CAC scoring on both ECG-synchronized cardiac and chest CT. Unlike the the methods required two cascaded networks to calculate CAC scoring, our network directly segment the calcified voxels and obtain CAC scoring. Meanwhile, we adopt a 2.5D patch input to reduce the computational overhead of 3D input. Instead of previous 2.5D methods \cite{lessmann2018automatic,wolterink2016automatic}which input patch from axial, sagittal and coronal direction, our network takes 9-channel stacks of images with corresponding 2D labels for segmentation of the corresponding center slice. 
We applied our method on a private dataset composed of 44 Cardiac CT scans and 805 chest CT scans. In comparison to experts' manual annotations, our algorithm achieved competitive results.

\section{Materials and Methods}
\subsection{Data}
A dataset of 849 CT scans was collected from several medical centers in China, which consists of 805 chest CT scans and 44 cardiac CT scans. The CT scans were acquired by different CT scanners with Philips, GE and Siemens. Each CT scan contains a sequence of slices at the thin-section slice spacing (range from 1.0 to 3.0 mm). CAC lesions were manually labeled by three experienced radiologists from different centers.

We newly connected 144 CT scans as a test set, incorporating chest CT scans and cardiac CT scans from medical centers in China. And to evaluate our network performance, lesions were delineated by experienced radiologists.


\subsection{Data preprocessing}
Since the connected dataset contains various sizes of chest CT scans and cardiac CT scans, we process all images as follows. First, we resize all CT images to $512 \times 512$ pixel resolution. Second, we randomly crop and then resize images to $512 \times 512$, where maximum size of the cropped image is $256 \times 256$. We continuously select nine processed slices as the input of our network, and for such an input, its label is the ground-truth label of its middle slice. We also process all ground-truth labels to alter the pixel label when its corresponding CT value lower than 130HU.

 
\subsection{DenseRAUnet for segmentation}
We proposed a novel FCN architecture based on dense U-Net for calcification segmentation, called DenseRAUnet. The network consists of two main components: (1) a basic network for feature extraction, and (2) three task-specific sub-network structures, incorporating \emph{Residual Atrous Unit (RAU)}, \emph{scSE block} and \emph{Extra Dense Block (EDB)}. Fig.~\ref{fig1} depicts our proposed DenseRAUnet.

\begin{figure}
\centering
\includegraphics[width=\textwidth]{fig/final.eps}
\caption{The overall structure of DenseUnet and details in Residual Atrous Unit } \label{fig1}
\end{figure}

The basic network is an encoder-decoder architecture, similar to dense U-Net. We adopt a backbone network (DenseNet-121) as the encoder sub-network. The decoder sub-network consists of three decoder modules. Each decoder module is an upsampling block followed by a scSE block, where upsampling block contains a deconvolution layer and two convolution layers, which followed by a Batch Normalization (BN) layer and an activation function called ReLU.


\noindent\textbf{Residual Atrous Unit.} Accurately segmenting various sizes of calcified areas may require different combinations of local and global information. So we consider that a simple skip connection is not enough for the complex segmentaiton task. Inspired by ASPP \cite{chen2018deeplab} and embed the idea of Inception \cite{szegedy2015going}, we further design a lateral connection called Residual Atrous Unit (RAU). Such a module is a residual block, and is used to capture multi-scale information by combining several convolutional layers with different dilation rates in parallel. As shown in Fig.～\ref{fig1}, we use a concatenation of three $3 \times 3$ dilated convolution layers with dilation rates are 2, 4, and 8 in each RAU.


\noindent\textbf{scSE block.} To take full advantage of local and global information, we added scSE block in the decoder sub-network, which is introduced in \cite{roy2018concurrent} for recalibrating the feature maps separately along channel and space.

\noindent\textbf{Extra Dense Block.} In order not to waste the image features extracted from input images, we insert an Extra Dense Block (EDB) in the first skip connection. Such a block could make more accurate use of shallow information, which do not represent input image in a high dimensional space, via adding more nonlinear into the first long connection.
\subsection{Loss Function
Inter-class imbalances are common problems when using deep learning methods for image segmentation, and even more in medical image segmentation. To solve it, we propose a new loss function, the combination of Bootstrap Loss and IoU Loss:
\begin{equation}
Loss = Bootstrap\ Loss + IoU\ Loss
\end{equation}
\noindent\textbf{Bootstrap Loss.} When we train a FCN, though images were cropped, there may be thousands of labeled pixels to predict. However, many of them may be easily distinguishable, and continuing to learn from these pixels does not improve model performance. In the context of medical image segmentation, most of such pixels are marked as background. For this reason, we design a weighted bootstrap loss, which not only forces network to focus on hard pixels but also balances positive and negative pixels during training.

Suppose there is only one processed image per mini-batch and there are a total $N$ pixels to predict. There are only two categories $c_{j}$ in the label space. Let $y_{i}$ denotes the ground-truth label of pixel $x_{i}$, and $p_{i,j}$ denotes the predicted probability that pixel $x_{i}$ belongs to the category $c_{j}$. Then, the loss function could be defined as:

\begin{equation}
l = - \left\{\alpha \frac{\sum\limits_{i \in N, j=0}log p_{i,j}1\left\{p_{i,j} < t\ and\ y_{i}=j\right\}}{\sum\limits_{i \in N, j=0}1\left\{p_{i,j} < t\ and\ y_{i}=j\right\}} + \beta \frac{\sum\limits_{i \in N, j=1}log p_{i,j}1\left\{y_{i}=j\right\}}{\sum\limits_{i \in N, j=1}1\left\{y_{i}=j\right\}}\right\}
\end{equation}
where $t$ is a threshold. Here $1\left\{\cdot\right\}$ is equal to one when the condition in parentheses, and otherwise is zero. In other words, we focus all positive pixels and drop negative pixels when they are too easy for the current model, i.e. their predicted probability greater than $t$. In practice, we hope that positive and negative pixels are balanced, hence we add $\alpha$ and $\beta$ as trade-off coefficients.

\noindent\textbf{IoU Loss.} Bootstrap loss is similar to cross entropy loss, focusing more on its own predictions of pixels and ignoring the relationship between adjacent ones. To better obtain the boundary of lesion, we add IoU in the loss function using such a relationship. Suppose there are $N$ pixels to predict. To ensure that losses are on the same magnitude, we use the following exponential form of IoU:

\begin{equation}
iou\ loss = -ln \frac{\sum\limits_{i \in N}p_{i}g_{i}}{\sum\limits_{i \in N}p_{i} + \sum\limits_{i \in N}g_{i} - \sum\limits_{i \in N}p_{i}g_{i}}
\end{equation}
where $p_{i}$ is the predicted probability of pixel $x_{i}$, $g_{i}$ is the ground-truth label of pixel $x_{i}$.

\subsection{Post-processing}
The final segmentation result of the network is obtained by a predefined threshold (here set to 0.5), and each lesion segmented by the network is considered a calcification candidate. Then each candidate is classified as CAC by thresholding with 130 HU and performing connected-components analysis. Since the CT slice thickness is mostly 1mm, calculation of the final Agatston score for the whole volume is done by the following corrected formula:

\begin{equation}
Agatston\ Score =\sum_{i} \sum_{n} f_{i, n} A_{i, n} \frac{\Delta S}{3}
\end{equation}

where $i$ is the $i$th CT slice of a CT volume, $n$ is the $n$th selected lesion, $f$ is the weighted intensity, $A$ is the lesion area, and $\Delta S$ is the slice spacing (mm).

\section{Experiments and Results} 
\noindent\textbf{Evaluation Metric.} We evaluate the pixel-level segmentation performance of the network by F1 score:
\begin{equation}
F1 = 2 \cdot \frac{Precision \cdot Recall}{Precision + Recall}
\end{equation}
We also define \emph{CAC rate} denotes the proportion of patients who was correctly predicting the CVD risk level without post-processing, and \emph{CAC filter Rate} represents the proportion of patients with post-processing.

\noindent\textbf{Implementation details.} The experiments conducted were all trained from scratch and  initialized by the Gauss method. During training, we collected one processed image as a mini-batch for each iteration and trained for 25 epochs. To optimize these experiments with fast convergence, we employed the SGD optimizer with  momentum of 0.9. The initial learning rate is 0.001 and is reduced by 0.99 times per 2000 iterations. The parameters in the loss function are experimentally set as $t = 0.9$, $\alpha = 8$ and $\beta = 1$. We implemented all the experiments via the deep learning toolki MXNet and trained on a GTX 1080 (NVIDIA) GPU.




\noindent\textbf{Ablation experiments.} We use ``Dense U-net \& Bootstrap Loss'' as the baseline for all experiments. To evaluate the effectiveness of various structures in our method, we conducted ablation experiments. First, using the bootstrap loss, we compare the role of three modules in the network. Second, we studied the effect of two loss functions through trained our network.

 
 
\begin{table}
\begin{center}
\caption{Comprasion of performance of the basic network using different tricks}
\label{tab1}
\scriptsize
\begin{tabular}{cccccccc}
\toprule
Basic network & Bootstrap & RAU & EDB & scSE & IoU & F1-Score & \\
\midrule
Dense U-Net & $\surd$ &         &         &         &         & 0.65 & \\
            & $\surd$ & $\surd$ &         &         &         & 0.68 & \\
            & $\surd$ & $\surd$ & $\surd$ &         &         & 0.69 & \\
            & $\surd$ & $\surd$ & $\surd$ & $\surd$ &         & 0.71 & \\
            & $\surd$ & $\surd$ & $\surd$ & $\surd$ & $\surd$ & \textbf{0.75} & Ours\\
\bottomrule
\end{tabular}
\end{center}
\end{table}

\begin{table}[]
\begin{center}
\caption{ Qualitative results of CAC rate and CAC filter rate for patients, Patients represents the total number of patients, CAC No. and CAC fliter No. represents the number of patients were predicted correctly by model and post-processing respectively.}
\label{tab1}
\scriptsize
\begin{tabular}{lccccc}
\toprule
Tricks &CAC No. & CAC filter No. & Patients & CAC Rate & CAC filter Rate \\
\midrule
Dense U-net & {\itshape} 101& {\itshape} 99 & {\itshape} 144 & {\itshape} 0.70 & {\itshape}0.69\\

Dense U-net+RAU & {\itshape} 104& {\itshape} 111 & {\itshape} 144 & {\itshape} 0.72 & {\itshape}0.77\\

Dense U-net+RAU+EDB  & {\itshape} 109& {\itshape} 115 & {\itshape} 144 & {\itshape} 0.76 & {\itshape}0.80\\

Dense U-net+RAU+EDB+scSE & {\itshape} 109& {\itshape} 117 & {\itshape} 144 & {\itshape} 0.76 & {\itshape}0.81\\


Our proposed method & {\itshape} \textbf{113} & {\itshape} \textbf{120} & {\itshape} 144 & {\itshape} \textbf{0.78} & {\itshape} \textbf{0.83}\\
\bottomrule
\end{tabular}
\end{center}
\end{table}

Table 1 lists the F1 scores of Dense U-Net using different tricks, Table 2 indicates the performance of corresponding network architectures in Table 1 on CAC. It is shown that all the tricks provide increase in F1 score and CAC in comparison to the baseline. We further observe that adding RAU in the network achieves more significant improvement for CAC segmentation. Comparing the results across Table 1, our method yields the best performance. From Table 2 we can also conclude that post-processing by the definition of CAC score is essential. 

\section{Conclusion}
This paper proposed an algorithm based on deep learning. Our method consists of two core elements: (1) a novel fully convolutional network, DenseRAUnet, and (2) a loss function combined bootstrap loss and IoU. We trained our network in a 2.5D-patch fashion to reduce input parameters while preserving spatial information. While trained solely on chest CT, our model achieved competitive and robust performance on both chest CT and cardiac CT which has significant higher resolution and lower spacing compared to training data, thanks to the power of residual atrous unit that enlarges the receptive field with downward compatibility. 
We aim to futher explore and extend our method to other medical image analysis challenges in future work.
\begin{figure}
\centering
\includegraphics[width=12.0cm,height=3.5cm]{fig/cac.png}
\includegraphics[width=12.0cm,height=3.5cm]{cac_r.png}
\caption{Segmentation results of chest CT (top) and cardiac CT (bottom). From left to right: the segmentation result of our model without post-processing, the result of with post-processing, ground truth.} \label{fig3}
\end{figure}

