\section{Appendix}\label{appendix}

\subsection{Network architecture}

In this section we give further details of our neural network architectures. For simplicity we ignore the batch dimension.

Our encoder is an approximation of the AlexNet architecture. Input is a 3-channel colored image with size $128\times128$. At each layer we reduce the image dimensions using strided convolutions, avoiding the need for max pooling operations. We use a Rectified Linear Unit (ReLU) for all non-linear activations. Whilst a deeper network, or one with more modern operations (e.g. Residual connections), would likely have helped overall performance, we aimed to keep are architecture as simple as possible to explore the relative benefits of our proposed learning approach.

\begin{table}[h!]
    \caption{AlexNet encoder architecture details.} \label{tbl:encoder}
    \vspace{.1cm}
    \setlength{\tabcolsep}{0.15cm}
    \centering
    \begin{tabular}{ c c c c c c} 
         \hline
         & Layer & Out Size & Batch Norm & Activation \\
         \hline
         C1 & Conv 2D & [64, 30, 30] & True & ReLU \\ 
         C2 & Conv 2D & [192, 14, 14] & True & ReLU \\ 
         C3 & Conv 2D & [384, 7, 7] & True & ReLU \\ 
         C4 & Conv 2D & [256, 4, 4] & True & ReLU \\ 
         C5 & Conv 2D & [64, 1, 1] & False & ReLU \\ 
         \hline
         \multicolumn{5}{r}{Parameters: 2,029,056}

    \end{tabular}
    \includegraphics[width=\linewidth]{figures/EncoderArch.png}
\end{table}

Our critic architecture is fully convolutional, and therefore contains no fully connected layers. Unlike the image encoder we find Leaky ReLU activation functions to be more effective. The critic also contains no max pooling layers, using convolutions with either stride=2 or stride=4 to reduce the image size. Input is a 3-channel colored image with size $128\times128$. In particular with the critic, we found normalization to be important. Whilst both Instance Normalization and Batch Normalization proved to be effective, all results in the paper were obtained using Batch Normalization.

\begin{table}[h!]
    \caption{Critic architecture details} \label{tbl:critic}
    \vspace{.1cm}
    \setlength{\tabcolsep}{0.15cm}
    \centering
    \begin{tabular}{ c c c c c c} 
     \hline
     & Layer & Out Size & Batch Norm & Activation \\
     \hline
     C1 & Conv 2D & [64, 30, 30] & True & Leaky ReLU \\ 
     C2 & Conv 2D & [192, 7, 7] & True & Leaky ReLU \\ 
     C3 & Conv 2D & [384, 4, 4] & True & Leaky ReLU \\ 
     C4 & Conv 2D & [256, 2, 2] & True & Leaky ReLU \\ 
     C5 & Conv 2D & [64, 1, 1] & False & Sigmoid \\ 
     \hline
     \multicolumn{5}{r}{Parameters: 1,883,713}
    \end{tabular}
    \includegraphics[width=\linewidth]{figures/CriticArch.png}
\end{table}

The parameter prediction MLP contains a single shared trunk layer followed by respective parameter branches. No weights are shared on any branch layers. After the shared layers (trunk) we resize the vector to size $n\times d$ where $n$ is the number of object predictions and $d$ number of feature dimensions. We find this effective as it enables each prediction to have its own independent feature transformation. $\mathtt{Center}$ predicts the per-object translation offset in camera space. $\mathtt{RGB}$ predicts an per-object RGB color. $\mathtt{Confidence}$ predicts a per-object confidence which is subsequently used as an opacity value for differentiable rendering with varying objects. $\mathtt{Light}$ predicts a per-scene light direction along a hemisphere above the scene. 

For experiments where a canonical mesh orientation, and ground plane are assumed, $\mathtt{Rotation}$ predicts the per-object $i,j$ rotation vector direction. The final rotation $r$ around the $y$-axis is calculated as $r=\mathrm{arctan2}(j, i)$. For experiments where this is not the case (\textsc{Bunny}, \textsc{LineMOD}) we instead using a 6D rotation representation as proposed by \cite{zhou2019continuity}. We choose this representation as it is continuous unlike other representations (Euler angles, rotation matrix, Quaternions etc.), which is more suitable for gradient decent optimizations.

We do not use any normalization in between MLP layers.

\begin{table}[h!]
    \caption{Parameter MLP network architecture.} \label{tbl:mlp}
   
    \setlength{\tabcolsep}{0.15cm}
    \centering
    \begin{tabular}{ l c c c} 
         \hline
          & Layer & Out Size & Activation \\
         \hline
         Trunk & Fully Connected & [64, 1] & ReLU \\ 
        \hline
         \multirow{2}{5em}{$\mathtt{Center}$} & Fully Connected & [n, 64] & ReLU \\ 
          & Fully Connected & [n, 3] & Linear \\ 
        \hline
         \multirow{2}{5em}{$\mathtt{Rotation}$} & Fully Connected & [n, 64] & ReLU \\ 
         & Fully Connected & [n, 2] & TanH \\ 
        \hline
         \multirow{2}{5em}{$\mathtt{RGB}$} & Fully Connected & [n, 64] & ReLU \\ 
         & Fully Connected & [n, 3] & Sigmoid \\ 
        \hline
         \multirow{2}{5em}{$\mathtt{Confidence}$} & Fully Connected & [n, 64] & ReLU \\ 
         & Fully Connected & [n, 1] & Sigmoid \\ 
        \hline
         \multirow{2}{5em}{$\mathtt{Light}$} & Fully Connected & [1, 64] & ReLU \\ 
         & Fully Connected & [1] & Linear \\ 
         \hline
         \multicolumn{4}{r}{Parameter range: 4,387 - 48,266}
    \end{tabular}
    \includegraphics[width=\linewidth]{figures/MLPArch.png}
\end{table}

\subsection{Training details}\label{sec:training_details}
All networks were implemented in the PyTorch v1.7 framework. A list of hyperparameters are shown in Tab. \ref{tab:HyperParams}. We trained all models until a convergence threshold based on the image loss is reached. Due to the nature of generator-critic min-max games, we found the critic loss to be an unreliable indicator of convergence. Network training times took between 12-36 hours on a single Nvidia 2080ti depending on task. To allow for a high batch size we utilize virtual batch sizes to avoid running into memory constraints. We achieve this by running multiple forward passes with a smaller batch size and accumulating the gradients before running a backwards pass through our network.

\begin{table}[]
    \centering
    \begin{tabular}{r|c}
        \hline
         Hyperparameter & Value  \\
         \hline
         Batch size & 128 \\
         Optimizer & Adam ($\beta_1$: 0.9, $\beta_2$: 0.999) \\
         Generator Learning rate & 1e-4 \\
         Critic learning rate & 1e-6 \\
         Image Loss $\lambda$ & 0.01 \\
         Critic loss $\lambda$ & 10 \\
         Gradient L2 clipping & 0.5 \\
    \end{tabular}
    \caption{List of hyperparameters used to train networks.}
    \label{tab:HyperParams}
\end{table}

\subsection{Real data experimental setup}\label{sec:exp_setup}
\mycfigure{RealSetup}{Real dataset acquisition pipeline. Images are first captured from either mobile (\textsc{Bunny}) or fixed (\textsc{Sphere Real, Shapes}) cameras (\textbf{a}). Camera alignment is calculated using SfM (\textbf{b}). Using reconstructed geometry and camera exterior orientations we can quickly model the scene (\textbf{c}). Finally, we pass all the parameters into a Differentiable Renderer (DR) for scene parameter learning (\textbf{d}). Note, camera alignment is only used for validation and not during the training or online phase.}

To enable real world experimentation, we recreate physical versions of our synthetic world setups. To enable object learning requires 3 steps, a) data acquisition, b) cameras alignment, c) object and scene modeling (\refFig{RealSetup}).

\textbf{Data acquisition} is performed using two standard Canon DSLR cameras fixed on tripods. We set the validation (novel-view) camera at an $\sim$90\textdegree
angle from the training camera view. To create the validation dataset objects are then placed at random within the scene and the scene is captured from each camera. A training dataset is then created by randomly placing objects into the scene and only capturing data from the training camera. We finally take an image from each camera with no objects in the scene for scene background modeling.

\textbf{Camera alignment}
To get relative exterior orientations between the two cameras we utilize a Structure-from-Motion (SfM) pipeline. Each camera is lifted from the tripod and multiple images take from various viewpoints. We ensure we densely cover the space between the two cameras. The images are input into the SfM software which produces the exterior orientations of the cameras in a common (arbitrary) coordinate system. We further generate a dense point cloud using multi-view stereo to aid object modeling. All SfM processing was performed in MetaShapes v1.6.5 using default settings.

\textbf{Object and scene modeling}
Each shape is modeled inside Blender software, based on the dense point cloud. This produces mesh objects which we pass to the DR. By doing so we ensure our mesh has the same scale as the scene.

We assume a flat ground plane and that objects are in a canonical position, with the only rotation axis being about the objects up-axis. We further transform the camera exterior orientations such that the reconstructed ground plane lies on the $XY$ plane. This is done manually as we have no ground control points to align the bundle adjustment. If a real co-ordinate system is required, ground control points could be placed into the scene avoiding the need for manual transformation (rotation and scaling) of the exterior orientations.

Once the camera alignment is complete, we can then create a ground plane mesh which we texture with the image containing no objects for training and validation cameras respectively. To ensure the same view of the ground plane we model the ground plane to the view frustum of the camera. This can be done by importing the intrinsic matrix estimated from the SfM to a virtual camera. Camera exterior and intrinsic properties are exported along with all mesh objects.

In scenarios where 3D CAD objects are available prior to data capture (e.g. industrial applications where parts are designed in CAD software before being manufactured), the object modeling process is not required.

\textbf{Bunny}
The \textsc{Bunny} dataset demonstrates a more casual data capture method. A single 3D-printed Stanford bunny is filmed from a moving mobile camera. Training data is generated by extracting frames every $n$-frames where $n=10$ in our case. Camera alignment is then computed for a sequence of frames. The first frame in the sequence is used as the input frame and the last frame is used as the novel view. As the motion is continuous, we can get relative camera positions using SfM. Object is then the same as for the \textsc{Real Sphere} and \textsc{Shapes} datasets. In order to model the scene, we require a background image, which contains no object. We found Adobe Photoshop content-aware delete tool to quickly and effectively remove the foreground object, leaving a clean background image which can be used to texture the ground-plane mesh.

\subsection{Validation Experiment}\label{sec:ValidationExperiment}
In this section we propose an experiment to verify the effect the critic has on the network optimization. Our key aim to construct an environment where the input data distribution is from a known distribution $\mathcal{D}$, and therefore the behavior of the critic (to ensure all network outputs belong to this distribution) can be evaluated. We therefore seek to find a differentiable analytical solution which calculates the distance between distribution of the current network output and $\mathcal{D}$. Such a solution would act as an Oracle critic $\mathcal{O}$ (i.e., a perfectly behaved critic).

\textbf{Setup} To aid the experiment, we first reduce our problem space to 2D, where our network must predict a position $t \in [0, 1]$ along the $x$-plane and a luminance $l \in [0, 1]$ (see \refFig{ParamSpace}).

\textbf{Data Distribution} To generate our training dataset, we sample parameters $p\in \mathbb{R}^{t, l}$ from a uniform distribution $\mathcal{D}: \{t, l\} \sim \mathcal{U}(0,1)$. Our optimization then becomes $$\argmin{} D(\hat{p} - \mathcal{D}), $$ where $D$ is a distance metric defined below.

\textbf{Oracle Critic} We define $D$ such that it allows for gradient-based parameter optimization. First, we note that a uniform distribution $\mathcal{U}$ would have a constant density across the parameter space. Therefore, we aim to penalize $\hat{p}$ varying from a constant density. This variation we refer to a discrepancy $D$. 

We can measure $D$ by testing for a low discrepancy pattern between all possible subsets of $\hat{p}$ and a random sampling $S \sim \mathcal{D}$. The size $N$ of sampling $S$ here is not strictly important, so long as it represents $\mathcal{D}$ sufficiently. In our experiments $N=300$.

First we calculate the density with a Kernel Density Estimate (KDE) $E$. Whilst it would be tempting to compute $E$ using common methods such as a box or Epechnikov approach, these would not be differentiable. Instead, we use a steep Gaussian kernel $$K = \exp(-(||\hat{p}-S||)^2).$$ Next, we compute KDE estimates $E$ for each point $e \in K$ as $$\frac{1}{2}(e_t^i + e_l^i).$$ Finally, we calculate the variance in the KDE response (discrepancy) $E$ as $$D = \frac{1}{N-1} \sum_{i=0}^{N}{(E_i - \bar{E})^2}.$$ This is the scalar value we minimize in the optimization.

When a simple L2 loss is used and no distribution is enforced (Fig. \ref{fig:ValExpIterations} top) the optimization almost always falls into the local-minima of turning the circle black ($l=0$). This is the same phenomenon we experience in the main paper for $\mathtt{NonCur}$ experiments. We observe this because the majority of gradients for $l$ point towards the black background. As a result, the overall gradient is dominated by this incorrect force. This is visible in the main paper Fig. 6 (left) as parameters falling towards the bottom of the chart. By enforcing $\hat{p} \sim \mathcal{D}$, we see that when the parameters start to fall, they are push back up, where they find more dominant L2 gradients main paper Fig. 6 (right).

A range of optimization visualizations can be found in the supplemental folder under $\mathtt{./val\_experiment}$.

\myfigure{ParamSpace}{Visualization of the parameter space when rendered.}

\subsection{Visualizing training}\label{sec:VisTraining}

In this section we visualize various training scenarios to show the effect of a real critic during training on simple problem. We present two scenarios where the parameter space is 5D ($x,y,r,g,b$) and 6D ($x,y,z,r,g,b$) respectively. For each scenario we demonstrate learning using image loss ($5D_{\mathtt{IMG}}$ / $6D_{\mathtt{IMG}}$), b) curiosity ($5D_{\mathtt{CUR}}$ / $6D_{\mathtt{CUR}}$) and image loss and curiosity ($5D_{\mathtt{OURS}}$ / $6D_{\mathtt{OURS}}$). We generate a dataset by randomly sampling position from a uniform distribution within a fixed bound and each color from a pre-computed color map. This allows us to observe if the critic is able to learn to enforce both the position and color distribution. By visualizing the step-by-step optimization, we note the following observations.

In the 5D scenario, $5D_{\mathtt{IMG}}$ follows dominant and incorrect gradients to match the background color. Furthermore, the gradients push the object towards the edge of the scene, and ultimately out of view of the camera which results in perfectly matching the background. $5D_{\mathtt{CUR}}$ learns to make predictions within the position and color distributions. As no image loss is present, there is no incentive for the network to make a prediction on a given single sample, but instead generate proposals that match the entire distribution. $5D_{\mathtt{OUR}}$ combines both the image loss (i.e., loss on a single given sample) and the critic loss (i.e., loss on the entire distribution). This results in avoiding the local and incorrect minima and results in convergence with respect to the position and color parameters. 

Similar observations are made in the 6D scenario. With the additional dimension $6D_{\mathtt{IMG}}$ also finds a local minima by reducing the size of the sphere. This is achieved in practice by pushing the sphere further away from the camera (i.e., $> z$). $6D_{\mathtt{CUR}}$ learns the new position distribution and generates plausible samples that comply with the dataset generation constraints. $6D_{\mathtt{OURS}}$ again finds the correct parameter solution for both position and color. Full training sequences can be found in the supplementary files under $\mathtt{./training\_vis/\{5d, 6d\}}$.

\myfigure{VisTraining}{Example outputs of training visualization experiments for 5D scenario. Columns 1-7 show progress during training. Columns 8 and 9 show the final models network output (col. 9) given an input (col. 8). Full videos can be found at: $\mathtt{./training\_vis/\{5d, 6d\}}$.}

\mycfigure{ValExpIterations}{Renderings of parameters at regular intervals during optimization for \textbf{a)} without curiosity and \textbf{b)} with curiosity. When image loss is used without a curiosity term the parameters degrade into an out-of-distribution solution where luminance $l$ tends towards 0. Optimization visualizations can be found in the supplementary files under $\mathtt{./val\_experiment}$.}

\mycfigure{Results2}{Additional qualitative results.}

\mycfigure{Results3}{Additional qualitative results.}

\mycfigure{Linemod1}{Additional qualitative results on LineMod dataset.}
\mycfigure{Linemod2}{Additional qualitative results on LineMod dataset.}


\section{#1}\label{sec:#2}}
\newcommand{\mysubsection}[2]{\subsection{#1}\label{sec:#2}}
\newcommand{\mysubsubsection}[2]{\subsubsection{#1}\label{sec:#2}}


\newcommand{\scalebox{0.8}{\ding{108}}}{\scalebox{0.8}{\ding{108}}}
\newcommand{\scalebox{0.8}{\ding{117}}}{\scalebox{0.8}{\ding{117}}}
\newcommand{\scalebox{0.8}{\ding{116}}}{\scalebox{0.8}{\ding{116}}}
\newcommand{$\blacktriangleleft$}{$\blacktriangleleft$}
\newcommand{\scalebox{0.8}{\ding{108}}}{\scalebox{0.8}{\ding{108}}}
\newcommand{\ding{72}}{\ding{72}}
\newcommand{\ding{89}}{\ding{89}}
\newcommand{\ding{88}}{\ding{88}}
\newcommand{\ding{70}}{\ding{70}}
\definecolor{colorA}{HTML}{4285f4}
\definecolor{colorB}{HTML}{ea4335}
\definecolor{colorC}{HTML}{fbbc04}
\definecolor{colorD}{HTML}{34a853}
\definecolor{colorE}{HTML}{ff6d01}
\definecolor{colorF}{HTML}{46bdc6}
\definecolor{colorG}{HTML}{000000}
\definecolor{colorH}{HTML}{777777}

\newcommand{\colorIcon}[2]{\textcolor{color#1}{\csname icon#2\endcsname}}
\newcommand{\colorIconAndName}[2]{\colorIcon{#1}{#2}~\text{#2}}
\newcommand{\nameInColor}[1]{\textcolor{color#1}{\texttt{#1}}}
\newcommand{\shortNameInColor}[2]{\textcolor{color#1}{\texttt{#2}}}
\newcommand{\nameAndIcon}[1]{``#1'' (denoted~{\csname icon#1\endcsname})}

