\section{Introduction}

Deformable registration is a major challenge in medical image processing.
The result is a dense mapping showing pixel-wise non-linear correspondences between a pair of images that best aligns the input image
$ I $ onto the target image $ T $ by means of some similarity definition $ \mathcal{L} $.
Deformable registration is applied in the analysis of patient-specific temporal or anatomical changes, e.g. from pre-operative to post-operative state, or to show inter-patient variances \cite{Sotiras2013}.
Deformable registration is also performed in atlas-based segmentation, where an input image is matched onto a target image with known segmentation \cite{Cabezas2011}.

Existing registration methods can be separated into two categories.
The first category is based on non-learning methods which estimate a registration $ w $ by optimizing a cost function of the form
\begin{equation}
    \argmin_{w} \Big\{ \mathcal{L}(T, w \circ I) + \lambda \mathcal{R}(w) \Big\} ~ ,
    \label{eq:cost}
\end{equation}
where $ w \circ I $ denotes $ I $ warped by $ w $.
A common assumption of $ w $ is a displacement or velocity vector field $ \bm{u}(\bm{x}) $.
The final deformation results in $ \bm{\phi}(\bm{x}) = \bm{x} + \bm{u}(\bm{x}) $ which maps every pixel coordinate $ \bm{x} $ to other pixel coordinates.
The first term in (\ref{eq:cost}) is referred to as data term, which is typically chosen to be a pixel intensity error measure. Optimization of the data term alone is considered ill-posed.
The second term $ \mathcal{R} $, weighted by trade-off factor $ \lambda $,  is a regularizer that shapes the registration by any chosen prior, which helps solving the ill-posed problem.
Common regularization is done by enforcing smoothness onto the displacement vector field by penalizing first or higher order spatial derivatives of $ \bm{u} $ \cite{Werlberger2010}.
The result of the registration algorithm heavily depends on the cost function and therefore on the chosen prior of $ \mathcal{R} $.

The second category implicitly learns the regularization prior by training a convolutional network on a large database of domain-specific images.
Early approaches rely on ground truth registrations \cite{Sokooti2017}, which are hard to obtain especially in medical imaging.
More recent methods \cite{Balakrishnan2019} propose unsupervised registration using the spatial transformer function \cite{Jaderberg2015}.
However, these methods either only support small displacements or require segmentation maps of the image pairs during training to assist the convergence \cite{Hu2018}.
Additionally, the trained networks are limited to register images from the training domain (e.g. CT or MRI).

Inspired by the idea of deep image priors \cite{Lempitsky2018}, we subsequently propose our learning-free method for deformable medical image registration using the structure of an untrained convolutional network as regularization prior.

\section{Methods}

\begin{figure}[t]
    \centering
    \includegraphics[width=1\textwidth]{overview.pdf}
    \caption{Overview of our method. The randomly-initialized generator network $ f_\theta $ acts as parameterization of the registration field $ \bm{\phi} $. The parameters $ \theta $ are optimized for every image pair individually by gradient descent.}
    \label{fig:overview}
\end{figure}

Lempitsky et al. have recently shown that excellent performance of CNNs for inverse image problems, such as denoising, is not only based on their ability to learn image priors from data, but is also based on the structure of a convolutional image generator itself \cite{Lempitsky2018}.
They gave evidence that the structure of a network alone is sufficient to capture enough image statistics to provide state-of-the-art performance in inverse image tasks.

Leveraged by this idea, we reformulate the task of deformable image registration by using the structure of a convolutional network as regularizer (see Fig.~\ref{fig:overview}).
An image generator network $ \bm{u} = f_{\theta}(z) $ with randomly-initialized parameters $ \theta $ is interpreted as parameterization of the dense displacement field $ \bm{u} \in \mathbb{R}^{2 \times H \times W} $ from which the deformation $ \bm{\phi} = \bm{x} + \bm{u} $ between an input image $ I \in \mathbb{R}^{C \times H \times W} $ and a target image $ T \in \mathbb{R}^{C \times H \times W} $ can be obtained by adding to the identity warp $ \bm{x} $.
The input $ z \in \mathbb{R}^{C' \times H \times W} \sim \mathcal{N}(0, 0.1) $ has the same spatial dimensions as $ \bm{\phi} $ and is sampled from a random normal distribution in every iteration.
This leads to the following optimization problem
\begin{equation}
    \argmin_{\theta} \Big\{ \mathcal{L} \left( T, \left( \bm{x} + f_{\theta}(z) \right) \circ I \right) \Big\} ~ ,
    \label{eq:task}
\end{equation}
where $ \left( \bm{x} + f_{\theta}(z) \right) \circ I $ denotes the differentiable spatial transformer function \cite{Jaderberg2015}. Eq.
(\ref{eq:task}) is optimized for every image pair $ \left\{ I, T \right\} $ using the Adam gradient descent optimizer \cite{Kingma2014}.
As data term, we chose pixel-wise mean absolute error
$ \mathcal{L}(T, \bm{\phi} \circ I) = \vert (\bm{\phi} \circ I) - T \vert $.
The architecture of the image generator network $ f_{\theta} $ is chosen according to \cite{Lempitsky2018}.
It has an encoder-decoder structure with skip connections between the encoding and decoding part.
To begin the optimization from close to an identity warp, we initialize the parameters with $ \theta \sim \mathcal{N} (0, 0.01) $. 

\section{Results \& Conclusion}

\begin{figure}[t]
    \centering
    \includegraphics[height=5cm]{results_img.pdf} ~
    \includegraphics[height=5cm]{results-crop.pdf}
    \caption{Results of our approach compared to a state-of-the-art method from the Insight ToolKit (ITK). Left: Two example MRI pairs from the data set. Right: Boxplots of the means of $ \det ( J_{\bm{\phi}} ) $ and SSIM between $ T $ and $ \bm{\phi} \circ I $.}
    \label{fig:results}
\end{figure}
\enlargethispage{1\baselineskip}

We demonstrate our approach on the task of 2D brain magnetic resonance imaging (MRI) registration.
The data used in this work contain 109 pairs of MRI scans from The Cancer Genome Atlas \cite{NCI} showing lower-grade gliomas.
We use the structural similarity index (SSIM) \cite{Wang2004} between $ \bm{\phi} \circ I $ and $ T $ and the mean of the determinants of Jacobians $ \det ( J_{\bm{\phi}} ) $ \cite{Ashburner2007} of the deformation as evaluation metrics.
The latter metric shows regularity of $ \bm{\phi} $. 
We compare our method to state-of-the-art methods from the Insight ToolKit (ITK) registration framework by combining an initial affine registration and a subsequent deformable displacement field registration \cite{Avants2012}.
Results for exemplary image pairs and boxplots of results for all image pairs are shown in Fig.\,\ref{fig:results}.
Additional results including registration fields are shown in appendix \ref{sec:results_app}.

The results reveal that the structure of a convolutional network can act as regularization in deformable medical image registration with state-of-the-art performance.
This connects traditional non-learning methods and learning-based methods by using randomly-initialized convolutional networks as prior.

\midlacknowledgments{This research has received funding from the European Union as being part of the EFRE OPhonLas project.}

