\section{Introduction}
\label{sec:intro}


\begin{figure*}[]
\setlength\tabcolsep{1pt}
\begin{tabular}{lllll}
  \multicolumn{1}{c}{Source image} & \multicolumn{1}{c}{Baseline} & \multicolumn{1}{c}{Ours} & \multicolumn{1}{c}{Exemplar} &  \\
  \multicolumn{1}{l}{\includegraphics[width=0.25\linewidth]{figs/sky_cs.jpg}}&
        \includegraphics[width=0.25\linewidth]{figs/sky_munit.jpg} &
        \includegraphics[width=0.25\linewidth]{figs/ours_A_1690_B_178.jpg} &
        \includegraphics[width=0.25\linewidth]{figs/sky_gta.jpg} & \\
  \multicolumn{1}{l}{\includegraphics[width=0.25\linewidth]{figs/road_cs.jpg}}&
        \includegraphics[width=0.25\linewidth]{figs/road_csmunit.jpg} &
        \includegraphics[width=0.25\linewidth]{figs/road_cstrans.jpg} & 
        \includegraphics[width=0.25\linewidth]{figs/road_gta.jpg}
\end{tabular}
\vspace{-0.2cm}
\caption{{\bf Global style vs dense style representations.} The baseline method (MUNIT)~\cite{huang2018munit} represents the exemplar style with a single, image-level feature vector. As such, some appearance information from the exemplar bleeds into semantically-incorrect regions, giving, for example, an unnatural bluish taint to the road and the buildings in the first row, second image. By modeling style densely, our approach better respects the semantics when applying style from the exemplar to the translated image. Our method also has finer-grained control over style. The color of the road and center line in the second row reflect exemplar appearance more accurately.}
\label{fig:intro}
\end{figure*}


Unpaired image-to-image (UI2I) translation aims to translate a source image to a target image domain by training a deep network using images from the source and target domains without ground-truth input-translation pairs. In the exemplar-based scenario (UEI2I), an additional target image exemplar is provided as input so as to further guide the style translation. Ultimately, the resulting translation should 1) preserve the content/semantics of the source image; 2) convincingly seem to belong to the target domain; and 3) adopt the specific style of the target exemplar image. 

Existing UEI2I strategies~\cite{huang2018munit, lee2018drit} encode the style of the exemplar using a global, image-level feature vector. While this has proven to be effective for relatively simple scenes, it leads to undesirable artifacts for complex, multi-object ones, as illustrated in Fig.~\ref{fig:intro}, where appearance information of the dominating semantic regions, such as sky, unnaturally bleeds into other semantic areas, such as the road, tree and buildings. Some methods~\cite{shen2019init, bhattacharjee2020dunit, jeong2021mguit, kim2022instaformer} address this by computing instance-wise or class-wise style representations. However, they require knowledge of the scene semantics, e.g., segmentation masks or bounding boxes 
during training, which limits their applicability. 

Here, by contrast, we propose to model style densely. That is, we represent the style of an image with a feature tensor that has the same spatial resolution as the content one. The difficulty in doing so lies in the fact that style information can then more easily pollute the content one, and vice versa. To prevent this and encourage the disentanglement of style and content, we utilize perceptual and adversarial losses on randomly-stylized content representations, which encourages the model to preserve fidelity and semantics in the absence of a dense style component.

Dense style representation alone is not beneficial for UEI2I as spatial arrangement of each dense style component is only applicable for its own image. Hence, we propose a cross-domain semantic correspondence module to spatially arrange/warp the dense style of the target image for the source content. To that end, we utilize the vision backbone from CLIP \cite{radford2021clip} as feature extractor and find correspondence between the features of source and target images using Optimal Transport \cite{cuturi2013sinkhorn,liu2020scot}.


As a consequence, and as shown in Fig.~\ref{fig:intro}, our approach transfers the local style of the exemplar to the source content in a more natural manner than the global-style techniques. Yet, in contrast to~\cite{shen2019init, bhattacharjee2020dunit, jeong2021mguit, kim2022instaformer}, it does not require semantic supervision during training, thanks to our dense modeling of style. To quantitatively evaluate the benefits of our approach, we introduce a metric that better reflects the stylistic similarity between the translations and the exemplars than the image-level metrics used in the literature. 

Our contributions can be summarized as follows:
\begin{itemize}
    \item We propose a dense style representation for UEI2I.
    Our method retains the source content in the translation while providing finer-grain stylistic control.
    \item We show that adversarial and perceptual losses encourage the disentanglement of our dense style and content representations.
    \item We develop a cross-domain semantic correspondence module to warp the exemplar style to the source content.
    \item We propose a localized style metric to measure the stylistic accuracy of the translation.
\end{itemize}
Our experiments evidence both qualitatively and quantitatively the benefits of our method over global, image-level style representations. We will make our code publicly available upon acceptance.


\section{Related Work}
\label{sec:related}

\newcommand\RotText[1]{\rotatebox{90}{\parbox{2cm}{\centering#1}}}

\begin{table}[t]

\resizebox{\columnwidth}{!}{%
\begin{tabular}{c|ccccccc}
\begin{tabular}[c]{@{}c@{}}Method \end{tabular} &
\begin{tabular}[c]{@{}c@{}}\parbox{1.2cm}{No \\ Label} \end{tabular} &
\begin{tabular}[c]{@{}c@{}}\parbox{1.2cm}{Multi-modal} \end{tabular} &
\begin{tabular}[c]{@{}c@{}}\parbox{1.2cm}{Exemplar guided} \end{tabular} &
\begin{tabular}[c]{@{}c@{}}\parbox{1.2cm}{Local style} \end{tabular}
\\
\hline \hline
MUNIT \cite{huang2018munit} & \cmark & \cmark & \cmark & \xmark & \\

DRIT \cite{lee2018drit} & \cmark & \cmark & \cmark & \xmark & \\

CUT \cite{park2020cut} & \cmark & \xmark & \xmark & \xmark & \\

FSeSim \cite{zheng2021lsesim} & \cmark & \cmark & \xmark & \xmark & \\

MGUIT \cite{jeong2021mguit} & \xmark & \cmark & \cmark & \cmark & \\

Ours & \cmark & \cmark & \cmark & \cmark & \\

\end{tabular}}
\vspace{-3pt}
\caption{Comparison of I2I methods.}
\label{i2i_comparison}
\end{table}

Our method primarily relates to three lines of research: Image-to-image (I2I) translation, Style Transfer, and Semantic Correspondence. Our main source of inspiration is I2I research as it deals with content preservation and translation fidelity. However, we borrow concepts from Style Transfer when it comes to adopting exemplar style and evaluating stylistic accuracy. Furthermore, our approach to swapping styles across semantically relevant parts of different images can be related to semantic correspondence.

\subsection{Image-to-image Translation}

We focus the discussion of I2I methods on the unpaired scenario, as our method does not utilize paired data. CycleGAN~\cite{zhu2017cyclegan} was the first work to address this by utilizing cycle consistency as a source of supervision in the presence of unpaired images from two domains. Recent works~\cite{park2020cut, zheng2021lsesim,jung2022src,hu2022qs} lift the cycle consistency requirement and perform one-sided translation while achieving high fidelity. To this end, they utilize contrastive losses and/or self-similarity between the source and the translation. Many I2I methods, however, are unimodal, in that they produce a single translation per input image, thus not reflecting the diversity of the target domain, especially in the presence of high within-domain variance. Although some works~\cite{zheng2021lsesim, jung2022src} extend this to multimodal outputs, they cannot adopt the style of a specific target exemplar, which is what we address.

Some effort has nonetheless been made to develop exemplar-guided I2I methods. For example, \cite{huang2018munit,lee2018drit} decompose the images into content and style components, and generate exemplar-based translations by merging the exemplar style with the content of the source image. However, these models define a single style representation for the whole image, which does not reflect the complexity of multi-object scenes. By contrast, \cite{mo2018instagan, shen2019init, bhattacharjee2020dunit, jeong2021mguit, kim2022instaformer} reason about object instances for I2I translation. Their goal is thus similar to ours, but their style representations focus on foreground objects only, and they require object-level (pseudo) annotations during training. Moreover, these methods are evaluated based on object detection scores and output diversity, but they do not report how stylistically close their translations are to the exemplars. Here, we achieve dense style transfer for all categories without requiring annotations and show that our method generates translations closer to the exemplar style while having comparable fidelity with that of state-of-the-art methods.



\subsection{Style Transfer}

Style transfer aims to bring the appearance of the content image closer to a target image. The seminal work of Gatys et al.~\cite{gatys2016gram} achieves so by matching the Gram matrices of the two images via image-based optimization.  \cite{li2017wct} provides an analytical solution to Gram matrix alignment, enabling arbitrary style transfer without image based optimization.  \cite{huang2017adain} only matches the diagonal of the Gram matrices by adjusting the channel means and standard deviations. \cite{li2017demystifystyle} shows that matching the Gram matrices minimizes the Maximum Mean Discrepancy between the two feature distributions. Inspired by this distribution interpretation, \cite{kolkin2019strotss} proposes to minimize a relaxed Earth Movers Distance between the two distributions, showing the effectiveness of Optimal Transport in style transfer. \cite{zhang2019mcut} defines multiple styles per image via GrabCut and exchanges styles between the local regions in two images. \cite{zhang2019mcut, kolkin2019strotss} are particularly relevant to our work as they account for the spatial aspect of style. \cite{li2018photowct,yoo2019wct2, chiu2022photowct2} aim to achieve photorealistic stylization using a pre-trained VGG based autoencoder. \cite{kim2020deformable, liu2021learning, yang2022industrial} model texture and geometry based style separately and learn to warp the texture-based style to the geometry of another image. However, the geometric warping module they rely on makes their methods only applicable to images depicting single objects. Unlike these works, our method operates on complex scenes.

\subsection{Semantic Correspondence}
Semantic correspondence methods aim to find semantically related regions across two different images. This involves the challenging task of matching object parts and fine-grained keypoints. Early approaches~\cite{barnes2009patchmatch, liu2010siftflow} used hand-crafted features for this task. These features, however, are not invariant to changes in illumination, appearance, and other low-level factors that do not affect semantics. Hence, they have limited ability to generalize across different scenes. \cite{liu2020scot, min2019hpf, aberman2018nbb} use ImageNet~\cite{simonyan2014imagenet} pre-trained features to address this issue and find correspondence between images containing similar objects. However, these methods do not generalize to finding accurate correspondences across images from different modalities/domains. 

Semantic correspondence has been explored in the context of image to image translation as well. In particular, \cite{zhan2021unite, zhang2020cocosnet1, zhou2021cocosnet2, zhan2022mclnet} use cross domain correspondences to guide paired exemplar-based I2I translation. In their case, both the I2I and semantic correspondence tasks benefit from the paired supervision.
Here, we also use cross domain correspondences, but unlike these works, we do so for unpaired, unsupervised, and exemplar-based I2I translation.


\section{Method}
\label{sec:method}

\begin{figure*}[]
\setlength\tabcolsep{1pt}
\includegraphics[width=1\linewidth]{figs/fig.png}
\vspace{-0.5cm}
\caption{{\bf Overview of our method.} During training, our method does not require any labels or paired images. During test time, we utilize CLIP backbone for building semantic correspondence. See Section \ref{sec:method} for definitions and explanations.}
\label{fig:model}
\end{figure*}

Let us now introduce our UEI2I approach using dense style representations. To this end, we first define the main architectural components of our model. It largely follows the architecture of~\cite{huang2018munit} and is depicted in Fig~\ref{fig:model}. Given two image domains $\mathbf{X}$, $\mathbf{Y} \subset \mathbb{R}^{3 \times H'W'}$, our model consists of two style encoders $E^s_X$, $E^s_Y: \mathbb{R}^{3 \times H'W'} \rightarrow \mathbb{R}^{S\times HW}$, two content encoders $E^c_X$, $E^c_Y: \mathbb{R}^{3 \times H'W'} \rightarrow \mathbb{R}^{C\times HW}$, two generators $G_X$, $G_Y: \mathbb{R}^{C\times HW}, \mathbb{R}^{S\times HW} \rightarrow \mathbb{R}^{3 \times H'W'} $, and two patch discriminators $D_X$, $D_Y: \mathbb{R}^{3 \times H'W'} \rightarrow \mathbb{R}^{S\times H''W''}$.

The content and style representations are then defined as follows. The content of image $\mathbf{x}$ is computed as $\mathbf{C}_x := E^c_X(\mathbf{x})$, and its dense style as $\mathbf{S}_x^{dense} := E^s_X(\mathbf{x})$. Note that the latter departs from the definition of style in~\cite{huang2018munit}; here, instead of a global style vector, we use a dense style map, which will let us transfer style in a finer-grained manner. Nevertheless, we also compute a global style for image $\mathbf{x}$ as $\mathbf{S}_x^{global} := Expand(Avg(\mathbf{S}_x^{dense}))$, where $Avg$ denotes spatial averaging and $Expand$ denotes repeating a vector across spatial dimensions. Furthermore, we define a mixed style $\textbf{S}_x^{mix} := 0.5 \mathbf{S}_x^{global} + 0.5 \mathbf{S}_x^{dense}$. As will be shown later, this mixed style will allow us to preserve the content without sacrificing stylistic control.

In the remainder of this section, we first introduce our approach to injecting a dense style to produce an image. We then turn to the problem of learning meaningful dense style representations during training, shown in the top portion of Fig.~\ref{fig:model}, and finally to that of exchanging the dense styles of the source and exemplar images at inference time, illustrated in the bottom portion of Fig.~\ref{fig:model}.

\subsection{Injecting Dense Style}
\label{sec:inject}
We define style as low level attributes that do not affect the semantics of the image. These low level attributes can include lighting, color, appearance, and texture. We also believe that a change in style should not lead to an unrealistic image and should not modify the semantics of the scene. In this work, we argue that, based on this definition, style should be represented densely. Given a dense style map, $S^{dense}$, extracted by one of the above-mentioned style encoders, we now describe how we inject such a representation in our framework to produce an image. Note that, in this section, we assume that the input dense style map is in direct spatial correspondence with the image content. We will explain in Section~\ref{sec:exchange} how we handle the different spatial arrangements of the source and exemplar images.

Accurate stylization requires the removal of the existing style as an initial step~\cite{li2017wct, li2017demystifystyle}. Thus, for our dense style to be effective, we incorporate a dense normalization that first removes the style of each region. To this end, inspired by~\cite{park2019spade, zhu2020sean, li2019pono}, we utilize a Positional Normalization Layer~\cite{li2019pono} followed by dense modulation. These operations are performed on generator activations that produce the images.

Formally, let $\mathbf{P}\in \mathbb{R}^{C' \times H W}$ denote the generator activations, with $C'$ the number of channels. We compute the position-wise means and standard deviations of $\mathbf{P}$, $\mathbf{\mu}, \mathbf{\sigma} \in \mathbb{R}^{H W}$. We then replace the existing style by our dense one via the Dense Normalization (DNorm) function
\begin{align}
    F_{DNorm}(\mathbf{P}, \mathbf{\alpha}, \mathbf{\beta}) = \frac{\mathbf{P} - \mathbf{\mu}}{\mathbf{\sigma}} \mathbf{\beta} + \mathbf{\alpha}\;
\end{align}
where the arithmetic operations are performed in an element-wise manner and by replicating $\mathbf{\mu}$ and $\mathbf{\sigma}$ $C'$ times to match the channel dimension of $\mathbf{P}$. The tensors $\mathbf{\alpha}, \mathbf{\beta} \in \mathbb{R}^{C' \times H W}$ are obtained by applying $1 \times 1$ convolutions to the dense style $\mathbf{S}^{dense}$.



\subsection{Learning Dense Style}

Let us now explain how we learn a dense style representation that accurately reflects the stylistic attributes of the exemplar. One difficulty when modeling style densely is that the image reconstruction loss will tend to encode information that goes beyond our style definition in it.  
To prevent this, we utilize perceptual and adversarial losses to regularize our dense style representation. Specifically, we use the reconstruction loss 
\begin{align}
 L_{recon} &= L_1(G_Y(\mathbf{C}_y, \mathbf{S}_y^{dense}), \mathbf{y})\;
\end{align}to let $\mathbf{S}^{dense}$ model spatially rich and dense information. In order to prevent style from modeling content, we constrain the translations to be realistic and to preserve the semantics of the source using
\begin{align}

 L_{adv\_global} &= L_{GAN}(G_Y(\mathbf{C}_x, \mathbf{S}_y^{global}), D_Y)\;,\\
 L_{adv\_mix} &= L_{GAN}(G_X(\mathbf{C}_x, \mathbf{S}_x^{mix}), D_X)\;,\\
 L_{per\_global} &= L_1(VGG(\mathbf{x}), VGG(G_Y(\mathbf{C}_x, \mathbf{S}_y^{global})))\;,\\
 L_{per\_mix} &= L_1(VGG(\mathbf{x}), VGG(G_X(\mathbf{C}_x, \mathbf{S}_x^{mix})))\;,
\end{align}
where $L_{GAN}$ denotes a standard adversarial loss, and $VGG()$ represents the VGG16 backbone up to the the Global Average Pooling layer (exclusive).
Note that adversarial and perceptual losses involve $\mathbf{S}^{mix}$ and $\mathbf{S}^{global}$ but not directly $\mathbf{S}^{dense}$; involving $\mathbf{S}^{dense}$ in the adversarial and perceptual losses tends to make the model learn to ignore the style representation. Moreover, exchanging dense style across two images requires building semantic correspondence between them to warp the style of the exemplar for the source. Thus, in order to avoid a collapsed style representation and using semantic information during training, we only exchange global styles across domains and constrain reconstructions with $\mathbf{S}^{mix}$. This circumvents the spatial arrangement mismatch between the source and exemplar images mentioned in Section~\ref{sec:inject}, thus letting us directly use $F_{DNorm}$ for dense style injection.

In addition to the above-mentioned losses, we use perceptual and GAN losses with random global style. We have observed them to encourage the model to preserve content in the absence of dense style representation. Moreover, as discussed in~\cite{huang2018munit, karras2019stylegan}, GAN loss helps matching the random latent style with the stylistic variety in the image domain.
\subsection{Exchanging Dense Style}
\label{sec:exchange}
Up to now, we have discussed how to inject dense style in an image and how to learn a meaningful dense style representation in training stage. However, one problem remains unaddressed in the test stage: The dense style extracted from an image is only applicable to that same image because its spatial arrangement corresponds to that image. In this section, we therefore propose an approach to swapping dense style maps across two images from different domains. 

Our approach is motivated by the intuition that style should be exchanged between semantically similar regions in both images. To achieve this, we leverage an auxiliary pre-trained network that generalizes well across various image modalities~\cite{radford2021clip}. Specifically, we extract middle layer features $\mathbf{F}_x, \mathbf{F}_y \in \mathbb{R}^{F \times H W}$ by passing the source and exemplar images through the CLIP-RN50 backbone~\cite{radford2021clip}. We then compute the cosine similarity between these features, clipping the negative similarity values to zero. We denote this matrix as $\mathbf{Z}_{yx}$ and use it to solve an optimal transport problem as described in \cite{liu2020scot, zhang2020cocosnet1}. We construct our cost matrix  $\mathbf{C}$ as 
\begin{align}
  \mathbf{Z}_{yx} &= \text{max}( \text{cos}(\mathbf{F}_y, \mathbf{F}_x), 0), \\
    \mathbf{C} &= \mathbf{1} - \mathbf{Z}_{yx}.
\end{align}
Thus, we use Sinkhorn's Algorithm~\cite{cuturi2013sinkhorn} to compute a doubly stochastic optimal transportation matrix $ \mathbf{A}_{yx} \in \mathbb{R}^{H W \times H W}$:
\begin{align}
     \mathbf{A}_{yx} &= arg \min_{\mathbf{A}} \langle \mathbf{A}, \mathbf{C} \rangle_F - \lambda h(\mathbf{A})\\
    \text{s.t} \quad & \mathbf{A} \mathbf{1}_{HW} = \mathbf{p}_y \; , \; \mathbf{A}^T \mathbf{1}_{HW} = \mathbf{p}_x
    \label{eq:ot}
\end{align}
where $h(\mathbf{A})$ denotes the entropy of $A$ and $\lambda$ is the entropy regularization parameter. $\mathbf{p}_x$, $\mathbf{p}_y \in \mathbb{R}^{H W \times 1}$ are the constrains on the row and column sums of $\mathbf{A_{yx}}$. The novelty of our semantic correspondence module lies in the way we choose the masses $\mathbf{p}_x$ and $\mathbf{p}_y$. 

The most straightforward choice for transportation masses $\mathbf{p}_x$ and $\mathbf{p}_y$ is the uniform distribution. However, doing so transports equal mass from every location in the images. This is problematic for us because we can see in Fig.~\ref{fig:intro} that when translation pairs have unbalanced classes, the largest semantic region can dominate the style representation and lead to undesired artifacts. In our example in Fig.~\ref{fig:intro}, the content image expects to receive style vectors for roads, buildings, and tree but the exemplar image provides style for sky and road. This results in building and tree regions being stylized by sky attributes.

To solve the unbalanced class problem, we first assume that segmentation labels $\mathbf{M_x}$ $\mathbf{M_y} \in \{0,1\}^{K \times HW}$ for $K$ classes are available. We define $\mathbf{M}^k$ as the binary mask for the $k$-th class. The number of pixels in class $k$ is defined as $\tilde{M}^k := \sum_l{M^{k,l}}$ where $l$ indexes the spatial dimension. We, then, define $\mathbf{\hat{M}_{yy}}$,$\mathbf{\hat{M}_{yx}} \in \mathbb{R}^{K \times H W}$ as
\begin{align}
    \mathbf{\hat{M}}_{yy} = \mathbf{M}_y^T \mathbf{\tilde{M}}_y\;\;\text{and} \;\; \mathbf{\hat{M}}_{yx} = \mathbf{M}_y^T \mathbf{\tilde{M}}_x 
\end{align}
where $\mathbf{\tilde{M}} \in \mathbb{R}^{K \times 1}$ is the concatenation of $\mathbf{\tilde{M}}^k$. We propose dividing the transportation mass $\mathbf{p_y}$ of each semantic region in $\mathbf{y}$ by the area of that semantic region to normalize the style based on class distribution of $\mathbf{y}$. We also multiply the mass of each semantic region in $\mathbf{y}$ by the area of the same semantic class in $\mathbf{x}$ to match the expectations of $\mathbf{x}$. We set $\mathbf{p_x}$ to be the uniform distribution.
\begin{align}
    \mathbf{\hat{p}}_y = \frac{\mathbf{\hat M}_{yx}}{\mathbf{\hat M}_{yy}}
\end{align}
However, we perform correspondence only during test time and we cannot rely on labels. Hence, we do not know the area of any of the classes. To that end, propose estimating $\mathbf{\hat{M}}_{yy}$ and $\mathbf{\hat{M}}_{yx}$ based on features $\mathbf{F}_x$ and $\mathbf{F}_y$. As such, we define $\mathbf{Z}_{xx}$ as self-similarity of $\mathbf{x}$ similarly to $\mathbf{Z}_{yx}$ and estimate a semantic area for each region in $\mathbf{x}$ and $\mathbf{y}$ as
\begin{align}
    \mathbf{R}_{yy} = \sum_l \mathbf{Z}_{yy}^l\;\;\text{and} \;\; \mathbf{R}_{yx} = \sum_l \mathbf{Z}_{yx}^l
\end{align}
where $\mathbf{Z}^l \in \mathbb{R}^{1 \times H W}$ and $l$ indexes to the second dimension of $Z$. We then compute $\mathbf{\hat p}_y$ as
\begin{align}
    \mathbf{\hat{p}}_y = \frac{\mathbf{R}_{yx}}{\mathbf{R}_{yy}}
\end{align}
which is linearly scaled to obtain a probability distribution $\mathbf{\hat{p}}_y$. Lastly, we compute $\mathbf{A}_{yx}$ as to warp $\mathbf{S}_y^{dense}$ as
\begin{equation}
    \mathbf{S}_{y \rightarrow x} = Reshape(\mathbf{S}_y^{dense} \mathbf{A}_{yx})\;.
\end{equation}
With this operation, each spatial element $\mathbf{S}_{y \rightarrow x} [{h,w}]$ can be seen as a weighted sum of spatial elements of $\mathbf{S}_y^{dense} [{h',w'}]$ with the weights being proportional to the semantic similarity between $\mathbf{F}_x^{h,w}$ and $\mathbf{F}_y^{h'w'}$. Hence, we can trade the style across semantically similar regions.
Note that this can also be thought of as an attention mechanism across two images with the queries being $\mathbf{F}_x$, the keys $\mathbf{F}_y$ and the values $\mathbf{S}_y^{dense}$. 

\section{Experiments}
\label{sec:experiments}

\subsection{Evaluation Metrics}

The current evaluation metrics in I2I do not take the spatial aspect of style into account. Hence, we propose a novel metric to evaluate the performance of our method. To that end, we use VGG until its first pooling layer, denoted as $VGG$, to extract features of size $\mathbb{R}^{V \times HW}$ from the input image $\mathbf{x}$, exemplar $\mathbf{y}$, and translation $\mathbf{x \rightarrow y}$. Our metric uses segmentation masks to compute the style similarity across corresponding classes. Using the segmentation masks as $\mathbf{M}_x^k$, we compute the Gram matrix $\mathbf{Q}_x^k$ for class $k$ in image $\mathbf{x}$ as
\begin{align}
  \mathbf{Q}_x^k & = \frac{1}{\sum_l{\mathbf{M}_x^{k,l}}} (VGG(\mathbf{x}) \odot \mathbf{M}_x^k) (VGG(\mathbf{x}) \odot \mathbf{M}_x^k)^T\;.
\end{align}
This operation is equivalent to treating each class as a separate image and computing their Gram matrices. 

We then compute the distance between the Gram matrices of corresponding classes in two images, i.e., 
\begin{align}
    \mathbf{L}^k(\mathbf{x},\mathbf{y}) & = \frac{1}{\mathbf{\tilde{M}}} \|\mathbf{Q}_x^k - \mathbf{Q}_y^k\|_F^2\;.
\end{align}
Note that $L^k(\mathbf{x},\mathbf{y})$ denotes the Maximum Mean Discrepancy (MMD) between the features of the masked regions with a degree 2 polynomial kernel~\cite{li2017demystifystyle}. Since this distance is computed based on early layer VGG features, it implies a stylistic distance between the two images~\cite{gatys2016gram}. 

However, $\mathbf{L}^k(x,y)$ is not very informative as its scale depends on the input image pair $\mathbf{x}, \mathbf{y}$. Hence, we propose a metric that takes $\mathbf{x}$, $\mathbf{y}$, and $\mathbf{x \rightarrow y}$ at the same time for better interpretability.
We express it as
\begin{align}
    \mathbf{H}^k(\mathbf{x},\mathbf{y},\mathbf{x \rightarrow y}) & = \frac{\mathbf{L}^k(\mathbf{x \rightarrow y},\mathbf{y})}{\mathbf{L}^k(\mathbf{x},\mathbf{y})} \mathbbm{1}_{\{\mathbf{\tilde{M}}_x^k > 0\}} \mathbbm{1}_{\{\mathbf{\tilde{M}}_y^k > 0\}}\;,
\end{align}
where $\mathbbm{1}_{\{\}}$ is the indicator function.

Unlike $\mathbf{L}^k$, $\mathbf{H}^k$ is more interpretable because its value would be equal to one if the translation output the content image. In an ideal translation scenario, we would expect the feature distributions of the translation and exemplar to be close to each other~\cite{kolkin2019strotss}. Hence, we would expect small values for more successful translations. 

Along with our metric, we report the standard Inception Score (IS)~\cite{salimans2016incetion_score}, Conditional Inception Score (CIS)~\cite{huang2018munit} and Frechet Inception Distance (FID)~\cite{heusel2017fid}. We also report the accuracy of a segmentation model pre-trained on the target domain and tested on source translations. 

\subsection{Implementation Details}

We evaluate our model on the GTAV~\cite{Richter_2016_gta} and Cityscapes~\cite{Cordts2016cityscapes} datasets. Images from both datasets are resized to have a short side of 256. We borrow most of the hyperparameters from~\cite{huang2018munit} but we scale all the adversarial losses by half since our method receives gradients from three adversarial losses for one source image. We do not change the hyperparameters for the perceptual losses. The entropy regularization term in Sinkhorn's Algorithm in Eq. \ref{eq:ot} is set to 0.05. During training of the I2I methods, we crop the center 224x224 pixels of the images. We report single scale evaluation with the same resolution for all the metrics. We use a pre-trained DRN\cite{yu2017drn} to report the segmentation results.


\begin{figure*}[]
\setlength\tabcolsep{1pt}
\begin{tabular}{lcllll}
 & Content & \multicolumn{1}{c}{Exemplar 1} & \multicolumn{1}{c}{Exemplar 2} & \multicolumn{1}{c}{Exemplar 3} &  \\
 & \multicolumn{1}{l}{\includegraphics[width=0.22\linewidth]{figs/content.png}}&
        \includegraphics[width=0.23\linewidth]{figs/ex1.png} &
        \includegraphics[width=0.23\linewidth]{figs/ex2.png} & 
        \includegraphics[width=0.23\linewidth]{figs/ex3.png} &  \\
 & \begin{tabular}[c]{@{}c@{}}Baseline\end{tabular}   &
        \includegraphics[width=0.23\linewidth]{figs/m_tr1.png} &
        \includegraphics[width=0.23\linewidth]{figs/m_tr2.png} & 
        \includegraphics[width=0.23\linewidth]{figs/m_tr3.png} &  \\
 & \begin{tabular}[c]{@{}c@{}}Ours\end{tabular} & 
        \includegraphics[width=0.23\linewidth]{figs/o_tr1.png} &
        \includegraphics[width=0.23\linewidth]{figs/o_tr2.png} & 
        \includegraphics[width=0.23\linewidth]{figs/o_tr3.png} &  \\
 & \multicolumn{1}{l}{}                                         &    &    &    & 
\end{tabular}
\vspace{-0.5cm}
\caption{{\bf Effect of the exemplar.} Our method can change the appearance of each semantic region differently, yet has realistic output.}
\label{fig:_begin}
\end{figure*}

\begin{figure*}[]

\begin{sideways}
\hspace{-1.5cm}
\rotatebox{180}{Source}
\hspace{-3.2cm}
\rotatebox{180}{Exemplar}
\hspace{-3.2cm}
\rotatebox{180}{Ours}
\hspace{-3.1cm}
\rotatebox{180}{MUNIT}
\hspace{-3.0cm}
\rotatebox{180}{DRIT}
\hspace{-2.8cm}
\rotatebox{180}{CUT}
\hspace{-3.1cm}
\rotatebox{180}{LSeSim}
\hspace{-3.1cm}
\rotatebox{180}{MGUIT}

\end{sideways}

\setlength\tabcolsep{1pt}
\hspace{0.3cm} 
\begin{tabular}{lllll} 

       \multicolumn{1}{l}{\includegraphics[width=0.22\linewidth]{qualt/8_18.jpg}} &
        \includegraphics[width=0.22\linewidth]{qualt/8_13.jpg} &
        \includegraphics[width=0.22\linewidth]{qualt/23_06} &
        \includegraphics[width=0.22\linewidth]{qualt/23_18.jpg} & \\
\end{tabular}
\vspace{-0.3cm}
\caption{{\bf Qualitative comparison with other methods.} In the first column, our method can disentangle road from sky and preserves the dark color for the road. In the second column, the appearance of the road and road lines in our translation are closest to those in the exemplar. In the last two columns, our model preserves the content semantics better, especially for tree and building classes.}
\label{fig:qualitative_end}
\end{figure*}


\subsection{Results}

Firstly, we evaluate the stylistic distance between exemplars and translations using our metric $\mathbf{H}^k$ that we propose. We report this metric for the most frequent six classes from GTA~\cite{Richter_2016_gta} and Cityscapes~\cite{Cordts2016cityscapes} datasets because they co-occur in pairs of images. The trend with other classes is similar and can be seen in our supplementary material. Our method outperforms the baselines in synthetic to real and real to synthetic scenarios as shown in Table \ref{local-metric}. Modeling style densely is effective at bringing the style of the regions closer to each other. 

We then evaluate the fidelity of the translations using FID~\cite{heusel2017fid} in Table~\ref{fid-div}. Our method can generate translations with high fidelity. We outperform all methods in terms of IS and CIS as well. Our method generates high variety translations, which is in line with our findings using our local style metric $\mathbf{H}^k$.

We conduct a user study on Amazon Mechanical Turk and ask the users which translation is closer to the exemplar in terms of color and appearance. We show the users one target image, and translations from the six methods in Fig.~\ref{fig:qualitative_end}. 77 users participate in the study, each one evaluating 39 comparisons of 6 translations. In total, there are 90 translation-target pairs that were randomly chosen. Out of 3003 votes, our method received the most votes (1062), see Table~\ref{tab:userstudy}. MUNIT~\cite{huang2018munit} is the second best model with 860.

\begin{table}[]
\resizebox{\columnwidth}{!}{
\begin{tabular}{l|l|l|l|l|l|l}\hline Method & Ours & MUNIT  & DRIT & CUT & FSeSim & MGUIT \\\hline
Number of votes            & 1062 & 861 & 541 & 166 & 219 & 15 \\
Ratio of votes                & 0.35 & 0.28 & 0.18 & 0.05 & 0.07 & 0.05 \\
\end{tabular}}
\caption{User study on similarity of translations with exemplars.}
\label{tab:userstudy}
\end{table}

Lastly, we measure the segmentation accuracy of pretrained models with our translations, see the last column of Table~\ref{fid-div}. We use Cityscapes pretrained DRN for translations of GTA images and vice versa. Due to the perceptual losses, our method is good at preserving content.

\begin{table}[]
\resizebox{\columnwidth}{!}{
\begin{tabular}{l|l|l|l|l|l|l||l}\hline
GTA $\rightarrow$ CS            & car  & sky  & \parbox{0.7cm}{vege-tation} & \parbox{0.7cm}{buil-\\ding} & \parbox{0.7cm}{side-\\walk} & road & Avg  \\\hline
MUNIT \cite{huang2018munit}   & 0.43 & 0.78 & 0.21  & 0.28  & 0.13    & 0.06 & 0.32 \\
DRIT  \cite{lee2018drit}      & 0.41 & 1.21 & 0.27  & \textbf{0.27}  & 0.12    & 0.08 & 0.39 \\
CUT   \cite{park2020cut}      & 0.44 & 0.92 & 0.24  & 0.36  & 0.16    & 0.13 & 0.38 \\
FSeSim\cite{zheng2021lsesim}  & 0.4  & 0.96 & 0.25  & 0.38  & 0.15    & 0.13 & 0.38 \\
MGUIT\cite{jeong2021mguit}    & 0.45 & 1.42 & 0.29  & 0.39  & 0.18    & 0.19 & 0.49 \\
Ours(DSI2I)               & \textbf{0.29} & \textbf{0.22} & \textbf{0.16}  & 0.29  & \textbf{0.08}    & \textbf{0.03} & \textbf{0.17} \\\hline
\end{tabular}}
\resizebox{\columnwidth}{!}{
\begin{tabular}{l|l|l|l|l|l|l||l}\hline
CS $\rightarrow$ GTA            & car  & sky  & \parbox{0.7cm}{vege-tation} & \parbox{0.7cm}{buil-\\ding} & \parbox{0.7cm}{side-\\walk} & road & Avg  \\\hline
MUNIT \cite{huang2018munit}   & 0.57 & 0.44 & 0.59  & 0.53  & \textbf{0.47}    & 0.35 & 0.49 \\
DRIT  \cite{lee2018drit}      & 0.66 & 0.63 & 0.53  & 0.56  & 0.51    & 0.39 & 0.55 \\
CUT   \cite{park2020cut}      & 0.88 & 0.58 & 0.75  & 0.67  & 0.72    & 0.74 & 0.72 \\
FSeSim\cite{zheng2021lsesim}  & 0.77 & 0.68 & 0.75  & 0.75  & 0.70    & 0.63 & 0.71 \\
MGUIT\cite{jeong2021mguit}    & 0.76 & 0.69 & 0.69  & 0.65  & 0.68    & 0.57 & 0.67 \\
Ours(DSI2I)               & \textbf{0.35} & \textbf{0.17} & \textbf{0.44}  & \textbf{0.41}  & 0.50    & \textbf{0.29} & \textbf{0.36} \\\hline
\end{tabular}}
\caption{Comparison with other methods. Classwise stylistic distance between translation-exemplar pairs (lower is better).}
\label{local-metric}
\end{table}

\begin{table}[]
\begin{tabular}{l|l|l|l|l}\hline
GTA $\rightarrow$ CS            & FID $\downarrow$   & IS $\uparrow$  & CIS $\uparrow$ & Seg Acc $\uparrow$ \\\hline
MUNIT  \cite{huang2018munit}                        & 47.76 & 3.19 & 1.08 & 0.79  \\
DRIT   \cite{lee2018drit}                           & 42.93 & 3.12 & 1.08 & 0.70  \\
CUT    \cite{park2020cut}                           & 50.89 & 3    & 1    & 0.62  \\
FSeSim \cite{zheng2021lsesim}                       & 48.77 & 3.08 & 1    & 0.71  \\
MGUIT  \cite{jeong2021mguit}                        & 44.36 & 2.88 & 1.03 & 0.65  \\
Ours(DSI2I)               & \textbf{42.61} & \textbf{3.43} & \textbf{1.19} & \textbf{0.82}  \\\hline
\end{tabular}
\begin{tabular}{l|l|l|l|l}\hline
CS $\rightarrow$ GTA            & FID $\downarrow$   & IS $\uparrow$  & CIS $\uparrow$ & Seg Acc $\uparrow$ \\\hline
MUNIT  \cite{huang2018munit}                        & 48.91 & 3.56 & 1.17 & 0.79  \\
DRIT   \cite{lee2018drit}                           & 48.18 & 3.82 & 1.17 & 0.72  \\
CUT    \cite{park2020cut}                           & 65.68 & 3.9  & 1    & 0.61  \\
FSeSim \cite{zheng2021lsesim}                       & 64.81 & 4.4  & 1    & 0.74  \\
MGUIT  \cite{jeong2021mguit}                        & 55.72 & 3.63 & 1.05 & 0.68  \\
Ours(DSI2I)               & \textbf{45.12} & \textbf{3.85} & \textbf{1.36} & \textbf{0.81}  \\\hline
\end{tabular}
\caption{Fidelity and diversity of the translations. Our method outperforms all others on all metrics. }
\label{fid-div}
\end{table}

\begin{table}[]
\resizebox{\columnwidth}{!}{
\begin{tabular}{l|l|l}\hline
GTA $\rightarrow$ CS               & FID $\downarrow$   & Seg Acc $\uparrow$ \\\hline
Ours(DSI2I)                      & \textbf{42.81} & \textbf{0.83}  \\
Ours(DSI2I) w/o $S^{mix}$          & 43.92 & 0.82  \\
Ours(DSI2I) w/o $S^{glb}$          & 43.73 & 0.81  \\
Ours(DSI2I) w/o $S^{mix}$, $S^{glb}$ & 45.67 & 0.78  \\\hline
\end{tabular}}
\resizebox{\columnwidth}{!}{
\begin{tabular}{l|l|l}\hline
CS $\rightarrow$ GTA               & FID $\downarrow$   & Seg Acc $\uparrow$ \\\hline
Ours(DSI2I)                      & \textbf{44.75} & \textbf{0.81}  \\
Ours(DSI2I) w/o $S^{mix}$          & 46.15 & 0.79  \\
Ours(DSI2I) w/o $S^{glb}$          & 46.71 & 0.78  \\
Ours(DSI2I) w/o $S^{mix}$, $S^{glb}$ & 50.95 & 0.76  \\\hline
\end{tabular}}
\caption{Ablation study on $S^{mix}$ and $S^{glb}$. The segmentation labels are used in the ablation study.}
\label{smix}
\end{table}
\subsection{Ablation Study}
To understand the effect of dense style independently from the semantic correspondence, we use segmentation labels as $\frac{\mathbf{M}_y^T}{\mathbf{M}_{yy}^T} \mathbf{M}_x$ instead of $\mathbf{A}_{yx}$ in Table \ref{smix}. This is equivalent to averaging $\mathbf{S}_y^{dense}$ class-wise and then swapping it between corresponding classes. Our ablations on \ref{smix} show that losses on $\mathbf{S}_{mix}$ and $\mathbf{S}_{glb}$ encourage our model to preserve content and generate high quality translations.

We assess the effectiveness of our semantic correspondence module by setting $\mathbf{p}_y$ as uniform distribution. Similar to warping $\mathbb{S}_y^{dense}$, we warp the segmentation labels $\mathbb{M}_y$ and report the accuracy with the warped labels in Table \ref{py}. We observe a drop in the accuracy of the warped labels.
\begin{table}[]
\begin{tabular}{l|l|l}\hline
Corr Acc            & GTA $\rightarrow$ CS   & CS $\rightarrow$ GTA \\\hline
Ours                & 0.59                 & 0.59               \\
Ours w/o $\mathbf{p}_y$      & 0.57                 & 0.56               \\\hline
\end{tabular}
\caption{Accuracy of semantic correspondence. Our unsupervised $\mathbf{p}_y$ increases the accuracy of correspondence.}
\label{py}
\end{table}
\section{Conclusion}
We present a framework for UEI2I that densly represents style and show how it can be learned and exchanged. This allows local stylistic changes of each semantic region. Our method does not require any labels. We quantitatively and qualitatively show the effectiveness of our method.

\textbf{Acknowledgement.} This work was supported in part by the Swiss National Science Foundation via the Sinergia grant CRSII5-180359. We also thank Ehsan Pajouheshgar for valuable discussions.
\iffalse
\begin{table}[]
\resizebox{\columnwidth}{!}{
\begin{tabular}{l|l|l|l|l|l|l||l}\hline
GTA $\rightarrow$ CS            & car  & sky  & vegt. & bldg. & sidewk. & road & Avg  \\\hline
Ours(DSI2I)                    & 0.31 & 0.24 & 0.16  & 0.22  & 0.08    & 0.03 & 0.17 \\
Ours(IN-MUNIT)                & 0.32 & 0.26 & 0.16  & 0.22  & 0.09    & 0.03 & 0.18 \\
Ours(IN-scratch)              & 0.33 & 0.25 & 0.19  & 0.25  & 0.14    & 0.06 & 0.2  \\
no\_rec                       & 0.33 & \textbf{0.2}  & 0.19  & 0.24  & 0.11    & 0.04 & 0.19 \\
no\_mix                       & 0.29 & 0.24 & 0.15  & 0.2   & 0.08    & \textbf{0.02} & 0.16 \\
no\_glb                       & 0.24 & 0.23 & 0.15  & 0.16  & 0.08    & \textbf{0.02} & 0.15 \\
no\_mix\_no\_rec              & 0.29 & 0.22 & 0.15  & 0.19  & 0.08    & \textbf{0.02} & 0.16 \\
no\_mix\_no\_rec\_no\_glb     & \textbf{0.23} & 0.26 & \textbf{0.13 } & \textbf{0.15}  & \textbf{0.06}    & \textbf{0.02} & \textbf{0.14} \\\hline
\end{tabular}}
\resizebox{\columnwidth}{!}{
\begin{tabular}{l|l|l|l|l|l|l||l}\hline
CS $\rightarrow$ GTA            & car  & sky  & vegt. & bldg. & sidewk. & road & Avg  \\\hline
Ours(DSI2I)                    & \textbf{0.25} &\textbf{ 0.19} & 0.36  & 0.32  & 0.35    & 0.31 & 0.3  \\
Ours(IN-MUNIT)                & 0.42 & 0.22 & 0.35  & 0.32  & 0.39    & 0.22 & 0.32 \\
Ours(IN-scratch)              & 0.46 & 0.22 & 0.4   & 0.38  & 0.47    & 0.28 & 0.37 \\
no\_rec                       & 0.43 & 0.23 & 0.37  & 0.34  & 0.43    & 0.25 & 0.34 \\
no\_mix                       & 0.38 & 0.2  & 0.39  & 0.31  & 0.39    & 0.21 & 0.31 \\
no\_glb                       & 0.32 & 0.23 & \textbf{0.3}   & 0.23  & \textbf{0.32}    & 0.14 & \textbf{0.26} \\
no\_mix\_no\_rec              & 0.35 & 0.21 & 0.43  & 0.32  & 0.42    & 0.21 & 0.32 \\
no\_mix\_no\_rec\_no\_glb     & 0.27 & 0.24 & 0.38  & \textbf{0.24}  & 0.33    & \textbf{0.13} & 0.27 \\\hline
\end{tabular}}
\caption{Ablation study. Classwise stylistic distance between translation-exemplar pairs (lower is better).}
\end{table}

\begin{table}[]
\resizebox{\columnwidth}{!}{
\begin{tabular}{l|l|l|l}\hline
GTA $\rightarrow$ CS            & FID $\downarrow$   & IS $\uparrow$  & CIS $\uparrow$ \\\hline
Ours(DSI2I)                    & \textbf{42.81} & 3.51 & 1.23  \\
Ours(IN-MUNIT)                & 46.3  & 3.54 & 1.12  \\
Ours(IN-scratch)              & 50.76 & 3.53 & 1.07  \\
no\_rec                       & 49.78 & 3.59 & 1.09  \\
no\_mix                       & 46.58 & 3.61 & 1.17  \\
no\_glb                       & 45.64 & 3.62 & 1.22  \\
no\_mix\_no\_rec              & 47.55 & 3.55 & 1.17  \\
no\_mix\_no\_rec\_no\_glb     & 48.92 & \textbf{3.66} & \textbf{1.29}  \\\hline
\end{tabular}}
\resizebox{\columnwidth}{!}{
\begin{tabular}{l|l|l|l}\hline
CS $\rightarrow$ GTA            & FID $\downarrow$   & IS $\uparrow$  & CIS $\uparrow$ \\\hline
Ours(DSI2I)                    & \textbf{44.75} & \textbf{3.82} & 1.35  \\
Ours(IN-MUNIT)                & 47.58 & 3.65 & 1.15  \\
Ours(IN-scratch)              & 50.27 & 3.66 & 1.11  \\
no\_rec                       & 49.23 & 3.59 & 1.13  \\
no\_mix                       & 47.15 & 3.57 & 1.19  \\
no\_glb                       & 49.44 & 3.66 & 1.32  \\
no\_mix\_no\_rec              & 46.71 & 3.59 & 1.21  \\
no\_mix\_no\_rec\_no\_glb     & 52.95 & 3.53 & \textbf{1.41}  \\\hline
\end{tabular}}
\caption{Ablation study. Fidelity and diversity of translations. Lower FID means higher fidelity. Higher IS/CIS means higher diversity.}
\end{table}
\fi

{\small
\bibliographystyle{ieee_fullname}

\section{Corrections on the Main Paper}

There is mistake in Table 3 
in the main paper. The numbers for the MGUIT model in the CS $\rightarrow$ GTA section were copied from GTA $\rightarrow$ CS, and we forgot to change them. The correct table is reported in \cref{local-metric}. Our model outperforms all the baselines.

There is a similar mistake in Table 5 of the main paper.
The last three rows in the FID column in the CS $\rightarrow$ GTA section of Table 5 were copied from the GTA $\rightarrow$ CS section of the same table, and we forgot to change them.
The correct table is provided in \cref{smix}. Our complete method yields the best results.


\begin{table}[]
\resizebox{\columnwidth}{!}{
\begin{tabular}{l|l|l|l|l|l|l||l}\hline
GTA $\rightarrow$ CS            & car  & sky  & \parbox{0.7cm}{vege-tation} & \parbox{0.7cm}{buil-\\ding} & \parbox{0.7cm}{side-\\walk} & road & Avg  \\\hline
MUNIT \cite{huang2018munit}   & 0.43 & 0.78 & 0.21  & 0.28  & 0.13    & 0.06 & 0.32 \\
DRIT  \cite{lee2018drit}      & 0.41 & 1.21 & 0.27  & 0.27  & 0.12    & 0.08 & 0.39 \\
CUT   \cite{park2020cut}      & 0.44 & 0.92 & 0.24  & 0.36  & 0.16    & 0.13 & 0.38 \\
FSeSim\cite{zheng2021lsesim}  & 0.4  & 0.96 & 0.25  & 0.38  & 0.15    & 0.13 & 0.38 \\
MGUIT\cite{jeong2021mguit}    & 0.45 & 1.42 & 0.29  & 0.39  & 0.18    & 0.19 & 0.49 \\
Ours(DSI2I)               & \textbf{0.29} & \textbf{0.22} & \textbf{0.16}  & \textbf{0.29}  & \textbf{0.08}    & \textbf{0.03} & \textbf{0.17} \\\hline
\end{tabular}}
\resizebox{\columnwidth}{!}{
\begin{tabular}{l|l|l|l|l|l|l||l}\hline
CS $\rightarrow$ GTA            & car  & sky  & \parbox{0.7cm}{vege-tation} & \parbox{0.7cm}{buil-\\ding} & \parbox{0.7cm}{side-\\walk} & road & Avg  \\\hline
MUNIT \cite{huang2018munit}   & 0.57 & 0.44 & 0.59  & 0.53  & 0.47    & 0.35 & 0.49 \\
DRIT  \cite{lee2018drit}      & 0.66 & 0.63 & 0.53  & 0.56  & 0.51    & 0.39 & 0.55 \\
CUT   \cite{park2020cut}      & 0.88 & 0.58 & 0.75  & 0.67  & 0.72    & 0.74 & 0.72 \\
FSeSim\cite{zheng2021lsesim}  & 0.77 & 0.68 & 0.75  & 0.75  & 0.70    & 0.63 & 0.71 \\
MGUIT\cite{jeong2021mguit}    & 0.76 & \underline{0.69} & \underline{0.69}  & \underline{0.65}  & \underline{0.68}    & \underline{0.57} & \underline{0.67} \\
Ours(DSI2I)               & \textbf{0.35} & \textbf{0.17} & \textbf{0.44}  & \textbf{0.41}  & \textbf{0.50}    & \textbf{0.29} & \textbf{0.36} \\\hline
\end{tabular}}
\caption{Clarification on Table 3 of the main paper. }
\label{local-metric}
\end{table}




\begin{table}[]
\resizebox{\columnwidth}{!}{
\begin{tabular}{l|l|l}\hline
GTA $\rightarrow$ CS               & FID $\downarrow$   & Seg Acc $\uparrow$ \\\hline
Ours(DSI2I)                      & \textbf{42.81} & \textbf{0.83}  \\
Ours(DSI2I) w/o $S^{mix}$          & 43.92 & 0.82  \\
Ours(DSI2I) w/o $S^{glb}$          & 43.73 & 0.81  \\
Ours(DSI2I) w/o $S^{mix}$, $S^{glb}$ & 45.67 & 0.78  \\\hline
\end{tabular}}
\resizebox{\columnwidth}{!}{
\begin{tabular}{l|l|l}\hline
CS $\rightarrow$ GTA               & FID $\downarrow$   & Seg Acc $\uparrow$ \\\hline
Ours(DSI2I)                      & \textbf{44.75} & \textbf{0.81}  \\
Ours(DSI2I) w/o $S^{mix}$          & \underline{46.15} & 0.79  \\
Ours(DSI2I) w/o $S^{glb}$          & \underline{46.71} & 0.78  \\
Ours(DSI2I) w/o $S^{mix}$, $S^{glb}$ & \underline{50.95} & 0.76  \\\hline
\end{tabular}}
\caption{Clarification on Table 5 of the main paper.}
\label{smix}
\end{table}



\section{Technical Details}
In this section, we describe the technical details of the I2I methods that we used in our comparisons. For each method, we adopt the default training strategy and hyperparameters of the method. All the models are trained for 600K iterations with 224x224 images. We use linear learning rate decay after 300K iterations as suggested in these works.

For training, we resize the input images to have a shorter side of size 256 without changing the aspect ratio and then crop the center 224x224 region. At test time, we generate translations without cropping the images. We use the evaluation code of FSeSim~\cite{zheng2021lsesim} for computing the FID. We resize the images to have a shorter side of size 299 without changing the aspect ratio when computing the FID. We borrow the code for IS/CIS from MUNIT~\cite{huang2018munit}. We report the exponential of IS and CIS as done in~\cite{huang2018munit}. The images are resized to have a shorter side of size 299 followed by taking a center crop of size 299x299 as done in~\cite{huang2018munit}. In FID, IS and CIS computations, we sample 100 random source images and 19 target images for each source images. We generate 1900 exemplar based translations as done in \cite{huang2018munit}.

We use two pre-trained DRN models~\cite{yu2017drn} for segmentation. We use the pretrainedmodels for GTA and CS from~\cite{hoffman2018cycada} and~\cite{yu2017drn}, respectively. The former is a DRN-C 26 model whereas the latter is a DRN-D 22. The images are resized to have shorter side of size 256 without center crop.

\section{Qualitative Results}

\subsection{Ablation Study}
\begin{figure*}[]

\begin{sideways}
\hspace{5.2cm}
{Source}
\hspace{-3.5cm}
{Exemplar}
\hspace{-3.3cm}
{Ours}
\hspace{-3.4cm}
{w/o $\mathbf{S}^{glb}$}
\hspace{-3.6cm}
{w/o $\mathbf{S}^{mix}$}
\hspace{-4.2cm}
{w/o $\mathbf{S}^{glb}, \mathbf{S}^{mix}$}

\end{sideways}
\setlength\tabcolsep{1pt}
\begin{tabular}{lllll}
 & \hspace{0.4cm}\includegraphics[width=0.24\linewidth]{cvpr2023-author_kit-v1_1-1/latex/ablation/0_c_A0_s_B0_B.png} &
 \includegraphics[width=0.24\linewidth]{cvpr2023-author_kit-v1_1-1/latex/ablation/0_c_A1000_s_B177_B.png} &
 \includegraphics[width=0.24\linewidth]{cvpr2023-author_kit-v1_1-1/latex/ablation/0_c_A1400_s_B190_B.png} &
 \includegraphics[width=0.24\linewidth]{cvpr2023-author_kit-v1_1-1/latex/ablation/0_c_A1925_s_B306_B.png} \\
  & \hspace{0.3cm} \includegraphics[width=0.24\linewidth]{cvpr2023-author_kit-v1_1-1/latex/ablation/0_c_A0_s_B0_A.png} &
 \includegraphics[width=0.24\linewidth]{cvpr2023-author_kit-v1_1-1/latex/ablation/0_c_A1000_s_B177_A.png} &
 \includegraphics[width=0.24\linewidth]{cvpr2023-author_kit-v1_1-1/latex/ablation/0_c_A1400_s_B190_A.png} &
 \includegraphics[width=0.24\linewidth]{cvpr2023-author_kit-v1_1-1/latex/ablation/0_c_A1925_s_B306_A.png} \\
  & \hspace{0.3cm} \includegraphics[width=0.24\linewidth]{cvpr2023-author_kit-v1_1-1/latex/ablation/munit_dense_scratch_gtavcs_c_A0_s_B0_BA.png} &
 \includegraphics[width=0.24\linewidth]{cvpr2023-author_kit-v1_1-1/latex/ablation/munit_dense_scratch_gtavcs_c_A1000_s_B177_BA.png} &
 \includegraphics[width=0.24\linewidth]{cvpr2023-author_kit-v1_1-1/latex/ablation/munit_dense_scratch_gtavcs_c_A1400_s_B190_BA.png} &
 \includegraphics[width=0.24\linewidth]{cvpr2023-author_kit-v1_1-1/latex/ablation/munit_dense_scratch_gtavcs_c_A1925_s_B306_BA.png} \\
  & \hspace{0.3cm} \includegraphics[width=0.24\linewidth]{cvpr2023-author_kit-v1_1-1/latex/ablation/munit_dense_scratch_gtavcs_noglb_c_A0_s_B0_BA.png} &
 \includegraphics[width=0.24\linewidth]{cvpr2023-author_kit-v1_1-1/latex/ablation/munit_dense_scratch_gtavcs_noglb_c_A1000_s_B177_BA.png} &
 \includegraphics[width=0.24\linewidth]{cvpr2023-author_kit-v1_1-1/latex/ablation/munit_dense_scratch_gtavcs_noglb_c_A1400_s_B190_BA.png} &
 \includegraphics[width=0.24\linewidth]{cvpr2023-author_kit-v1_1-1/latex/ablation/munit_dense_scratch_gtavcs_noglb_c_A1925_s_B306_BA.png} \\
   & \hspace{0.3cm} \includegraphics[width=0.24\linewidth]{cvpr2023-author_kit-v1_1-1/latex/ablation/munit_dense_scratch_gtavcs_nomix_c_A0_s_B0_BA.png} &
 \includegraphics[width=0.24\linewidth]{cvpr2023-author_kit-v1_1-1/latex/ablation/munit_dense_scratch_gtavcs_nomix_c_A1000_s_B177_BA.png} &
 \includegraphics[width=0.24\linewidth]{cvpr2023-author_kit-v1_1-1/latex/ablation/munit_dense_scratch_gtavcs_nomix_c_A1400_s_B190_BA.png} &
 \includegraphics[width=0.24\linewidth]{cvpr2023-author_kit-v1_1-1/latex/ablation/munit_dense_scratch_gtavcs_nomix_c_A1925_s_B306_BA.png} \\
   & \hspace{0.3cm} \includegraphics[width=0.24\linewidth]{cvpr2023-author_kit-v1_1-1/latex/ablation/munit_dense_scratch_gtavcs_noglb_norecon_nomix_c_A0_s_B0_BA.png} &
 \includegraphics[width=0.24\linewidth]{cvpr2023-author_kit-v1_1-1/latex/ablation/munit_dense_scratch_gtavcs_noglb_norecon_nomix_c_A1000_s_B177_BA.png} &
 \includegraphics[width=0.24\linewidth]{cvpr2023-author_kit-v1_1-1/latex/ablation/munit_dense_scratch_gtavcs_noglb_norecon_nomix_c_A1400_s_B190_BA.png} &
 \includegraphics[width=0.24\linewidth]{cvpr2023-author_kit-v1_1-1/latex/ablation/munit_dense_scratch_gtavcs_noglb_norecon_nomix_c_A1925_s_B306_BA.png} \\
 &  &  &  & 
\end{tabular}
\vspace{-0.5cm}
\caption{{\bf Effect of $\mathbf{S}^{mix}$ and $\mathbf{S}^{glb}$.} The adversarial and perceptual losses on $\mathbf{S}^{mix}$ and $\mathbf{S}^{glb}$ constrain the dense style representation and, thus, encourage the preservation of content, semantics, and details of the source image. As mentioned in the main paper, the labels are used to swap style across classes in our ablation study instead of the semantic correspondence module.}
\label{fig:ablation}
\end{figure*}

The losses we used during training are provided below. 

\begin{align}
\label{l_recon}
 L_{recon} &= L_1(G_Y(\mathbf{C}_y, \mathbf{S}_y^{dense}), y)\;,\\
 \label{l_adv_glb}
 L_{adv\_global} &= L_{GAN}(G_Y(\mathbf{C}_x, \mathbf{S}_y^{global}), D_Y)\;,\\
 \label{l_adv_mix}
 L_{adv\_mix} &= L_{GAN}(G_Y(\mathbf{C}_y, \mathbf{S}_y^{mix}), D_Y)\;,\\
 \label{l_adv_rand}
 L_{adv\_rand} &= L_{GAN}(G_Y(\mathbf{C}_x, \mathbf{r}), D_Y)\;,\\
 \label{l_per_glb}
 L_{per\_global} &= L_1(VGG(\mathbf{x}), VGG(G_Y(\mathbf{C}_x, \mathbf{S}_y^{global})))\;,\\
 \label{l_per_mix}
 L_{per\_mix} &= L_1(VGG(\mathbf{y}), VGG(G_Y(\mathbf{C}_y, \mathbf{S}_y^{mix})))\;,\\
 \label{l_per_rand}
 L_{per\_rand} &= L_1(VGG(\mathbf{x}), VGG(G_Y(\mathbf{C}_x,  \mathbf{r})))\;,
\end{align}

We show the qualitative effect of $\mathbf{S}^{mix}$ and $\mathbf{S}^{glb}$ in Fig.~\ref{fig:ablation}. Without using $\mathbf{S}^{mix}$ or $\mathbf{S}^{glb}$, the content component encodes less information about the image, which leads to unrealistic translations with the exemplar style. In our ablations, for w/o $\mathbf{S}^{glb}$, we remove the losses involving $\mathbf{S}^{glb}$, i.e., Eqs.~\ref{l_adv_glb} and~\ref{l_per_glb}. Likewise, for w/o $\mathbf{S}^{mix}$, we remove Eqs.~\ref{l_adv_mix} and~\ref{l_per_mix}. In all our ablations, we use the segmentation labels instead of semantic correspondence module to observe the effect of our style representation independently.





\subsection{Semantic Correspondence}

We use CLIP~\cite{radford2021clip} to build semantic correspondences between the two images. CLIP~\cite{radford2021clip} is trained with image-caption pairs from the internet, to match the global representation of the image with the language representation of the corresponding caption. Hence, it has never received pixel-level supervision or segmentation masks. Thus, CLIP is considered as a weakly supervised backbone for segmentation task since it has seen a weaker supervision signal, image-level semantic information, for a pixel-level task.

We also experiment with a pretrained DenseCL~\cite{wang2021densecl} model. DenseCL~\cite{wang2021densecl} is trained in a self supervised way to predict the intersection of two crops from two augmentations of the same image. We measure the accuracy of correspondence by warping the segmentation labels of the exemplar and then dividing the correctly classified pixels by the total number of pixels. We observe that semantic correspondence with DenseCL~\cite{wang2021densecl} is less accurate, hence we stick to using CLIP.

We use pretrained weights for the ResNet50 architecture. Specifically, we extract features from the end of the 'layer1' and 'layer3' stages of ResNet50 architecture.

\begin{table}[]
\begin{tabular}{l|l|l}\hline
Corr Acc                      & GTA $\rightarrow$ CS   & CS $\rightarrow$ GTA \\\hline
CLIP~\cite{radford2021clip}   & 0.59                 & 0.59               \\
DenseCL~\cite{wang2021densecl}& 0.55                 & 0.54               \\\hline
\end{tabular}
\caption{Accuracy of semantic correspondence with different backbones.}
\label{py}
\end{table}


\begin{figure*}[]
\setlength\tabcolsep{1pt}
\begin{tabular}{lll}
 & \hspace{2cm} Source image \hspace{3.5cm} Exemplar image \hspace{3cm} Cosine Similarity\\
 & \includegraphics[width=1\linewidth]{cvpr2023-author_kit-v1_1-1/latex/corr/corr_road.png}\\
 & \includegraphics[width=1\linewidth]{cvpr2023-author_kit-v1_1-1/latex/corr/corr_roadline.png}\\
 & \includegraphics[width=1\linewidth]{cvpr2023-author_kit-v1_1-1/latex/corr/corr_wheel.png}\\ &
\end{tabular}
\vspace{-0.5cm}
\caption{{\bf Visualization of Cosine Similarity across domains.} We choose a region centered at the red point from the source image in the first column and display the cosine similarity between the chosen source region with all the other target regions. Our correspondence module is able to relate the object parts that are not labeled in semantic segmentation annotations which is demonstrated by the correspondence of roadlines in the second row and wheel in the third row.}
\end{figure*}

\iffalse
\begin{figure*}[]
\setlength\tabcolsep{1pt}
\begin{tabular}{ll}
 & \includegraphics[width=1\linewidth]{cvpr2023-author_kit-v1_1-1/latex/corr/corr_seg_warp.png}\\&
\end{tabular}
\vspace{-0.5cm}
\caption{{\bf Visualization of the warped semantic labels.} We warp the segmentation labels of the target image in the first row for the source content. The warped labels are shown in the second column and the source labels in the last column. The source and target images are shown in Fig.~\ref{fig:ablation}, middle row}
\end{figure*}
\fi





\subsection{Comparison with Other Methods}

We provide qualitative examples for our model in the end of our supplementary material. 

\subsection{User Study}

We conducted a user study on Amazon Mechanical Turk. The users were shown an exemplar image and translations from six models, ours and five other baselines \cite{huang2018munit, lee2018drit, zheng2021lsesim, park2020cut, jeong2021mguit}, as shown in Fig. \ref{fig:userstudy}. They are asked to choose the image that looks the most similar to the exemplar image. The question they received was: 'Which image is more similar to the target image (T)? Similar images would have closer road and sky colors and would reflect the same time of the day.' We did not provide
the users with the source image S and we randomized the order of the choices in our user study. We received answers from 77 users and each user answered 39 questions. Out of 3003 answers, 1082 picked our method as the best.

\begin{figure*}[]

\setlength\tabcolsep{1pt}
\begin{tabular}{lcl}\\
      & \hspace{0.3cm} \multicolumn{1}{l}{\includegraphics[width=1\linewidth]{cvpr2023-author_kit-v1_1-1/latex/user_study/49_05.jpg}}& \\
 & \multicolumn{1}{l}{}                                         &    &    &    & 
\end{tabular}
\vspace{-0.5cm}
\caption{{\bf User study} The users were asked to pick the translation that looks the most similar to the exemplar image T. We did not provide the users with the source image S and we randomized the order of the choices in our user study. Here, (1): Ours, (2): MUNIT~\cite{huang2018munit}, (3): DRIT~\cite{lee2018drit}, (5): CUT~\cite{park2020cut}, (5): FSeSim~\cite{zheng2021lsesim},(6): MGUIT~\cite{jeong2021mguit}}
\label{fig:userstudy}
\end{figure*}

\begin{figure*}[]

\begin{sideways}
\hspace{-2cm}
{Source}
\hspace{-3.5cm}
{Exemplar}
\hspace{-3.3cm}
{Ours}
\hspace{-3.2cm}
{MUNIT}
\hspace{-3.2cm}
{DRIT}
\hspace{-2.9cm}
{CUT}
\hspace{-3.3cm}
{LSeSim}
\hspace{-3.4cm}
{MGUIT}

\end{sideways}

\setlength\tabcolsep{1pt}
\begin{tabular}{lcllll}\\

      & \hspace{0.3cm} \multicolumn{1}{l}{\includegraphics[width=0.24\linewidth]{cvpr2023-author_kit-v1_1-1/latex/supp_qualt0/1_12.jpg}}&
        \includegraphics[width=0.24\linewidth]{cvpr2023-author_kit-v1_1-1/latex/supp_qualt0/13_10.jpg} &
        \includegraphics[width=0.24\linewidth]{cvpr2023-author_kit-v1_1-1/latex/supp_qualt0/13_03.jpg} &
        \includegraphics[width=0.24\linewidth]{cvpr2023-author_kit-v1_1-1/latex/supp_qualt0/13_04.jpg} & \\
 & \multicolumn{1}{l}{}                                         &    &    &    & 
\end{tabular}
\vspace{-0.5cm}
\caption{{\bf Qualitative comparison with other methods.} The road and sky appearance in all the columns are closer to the exemplar road and sky with our method. In the second column, our method is more accurate in the appearance of cars. In the second and third columns, the roadlines are yellow in our translations, which is closer to the exemplar appearance.}
\end{figure*}

\begin{figure*}[]

\begin{sideways}
\hspace{-2cm}
{Source}
\hspace{-3.5cm}
{Exemplar}
\hspace{-3.3cm}
{Ours}
\hspace{-3.2cm}
{MUNIT}
\hspace{-3.2cm}
{DRIT}
\hspace{-2.9cm}
{CUT}
\hspace{-3.3cm}
{LSeSim}
\hspace{-3.4cm}
{MGUIT}

\end{sideways}

\setlength\tabcolsep{1pt}
\begin{tabular}{lcllll}\\

      & \hspace{0.3cm} \multicolumn{1}{l}{\includegraphics[width=0.24\linewidth]{cvpr2023-author_kit-v1_1-1/latex/qualt/8_18.jpg}}&
        \includegraphics[width=0.24\linewidth]{cvpr2023-author_kit-v1_1-1/latex/supp_qualt0/10_12.jpg} &
        \includegraphics[width=0.24\linewidth]{cvpr2023-author_kit-v1_1-1/latex/supp_qualt0/15_03.jpg} &
        \includegraphics[width=0.24\linewidth]{cvpr2023-author_kit-v1_1-1/latex/supp_qualt0/19_01.jpg} & \\
 & \multicolumn{1}{l}{}                                         &    &    &    & 
\end{tabular}
\vspace{-0.5cm}
\caption{{\bf Qualitative comparison with other methods.} Our method brings sky and road appearances closer to those of the exemplar in all cases. In the second column, our method preserves the tree whereas the other methods remove it and display sky.}
\end{figure*}

\begin{figure*}[]

\begin{sideways}
\hspace{-2cm}
{Source}
\hspace{-3.5cm}
{Exemplar}
\hspace{-3.3cm}
{Ours}
\hspace{-3.2cm}
{MUNIT}
\hspace{-3.2cm}
{DRIT}
\hspace{-2.9cm}
{CUT}
\hspace{-3.3cm}
{LSeSim}
\hspace{-3.4cm}
{MGUIT}

\end{sideways}

\setlength\tabcolsep{1pt}
\begin{tabular}{lcllll}\\

      & \hspace{0.3cm} \multicolumn{1}{l}{\includegraphics[width=0.24\linewidth]{cvpr2023-author_kit-v1_1-1/latex/supp_qualt0/22_04.jpg}}&
        \includegraphics[width=0.24\linewidth]{cvpr2023-author_kit-v1_1-1/latex/supp_qualt0/22_10.jpg} &
        \includegraphics[width=0.24\linewidth]{cvpr2023-author_kit-v1_1-1/latex/supp_qualt0/23_06.jpg} &
        \includegraphics[width=0.24\linewidth]{cvpr2023-author_kit-v1_1-1/latex/supp_qualt0/23_13.jpg} & \\
 & \multicolumn{1}{l}{}                                         &    &    &    & 
\end{tabular}
\vspace{-0.5cm}
\caption{{\bf Qualitative comparison with other methods.} Our method changes the roadlines based on the exemplar. In the first and second columns, the appearance of the roadlines is adjusted based on the exemplar whereas the other methods either leave them as white or change them to yellow for all the exemplars. In columns three and four, we can see that our method preserves the tree better than the other methods do.}
\end{figure*}

\begin{figure*}[]

\begin{sideways}
\hspace{-2cm}
{Source}
\hspace{-3.5cm}
{Exemplar}
\hspace{-3.3cm}
{Ours}
\hspace{-3.2cm}
{MUNIT}
\hspace{-3.2cm}
{DRIT}
\hspace{-2.9cm}
{CUT}
\hspace{-3.3cm}
{LSeSim}
\hspace{-3.4cm}
{MGUIT}

\end{sideways}

\setlength\tabcolsep{1pt}
\begin{tabular}{lcllll}\\

      & \hspace{0.3cm} \multicolumn{1}{l}{\includegraphics[width=0.24\linewidth]{cvpr2023-author_kit-v1_1-1/latex/supp_qualt0/24_02.jpg}}&
        \includegraphics[width=0.24\linewidth]{cvpr2023-author_kit-v1_1-1/latex/supp_qualt0/24_16.jpg} &
        \includegraphics[width=0.24\linewidth]{cvpr2023-author_kit-v1_1-1/latex/supp_qualt0/33_07.jpg} &
        \includegraphics[width=0.24\linewidth]{cvpr2023-author_kit-v1_1-1/latex/supp_qualt0/33_09.jpg} & \\
 & \multicolumn{1}{l}{}                                         &    &    &    & 
\end{tabular}
\vspace{-0.5cm}
\caption{{\bf Qualitative comparison with other methods.} Our method preserves the building pixels in the first two columns. In the last two columns tree and sky are better preserved with our method and reflect closer appearance to the exemplar.}
\end{figure*}

\begin{figure*}[]

\begin{sideways}
\hspace{-2cm}
{Source}
\hspace{-3.5cm}
{Exemplar}
\hspace{-3.3cm}
{Ours}
\hspace{-3.2cm}
{MUNIT}
\hspace{-3.2cm}
{DRIT}
\hspace{-2.9cm}
{CUT}
\hspace{-3.3cm}
{LSeSim}
\hspace{-3.4cm}
{MGUIT}

\end{sideways}

\setlength\tabcolsep{1pt}
\begin{tabular}{lcllll}\\

      & \hspace{0.3cm} \multicolumn{1}{l}{\includegraphics[width=0.24\linewidth]{cvpr2023-author_kit-v1_1-1/latex/supp_qualt0/31_01.jpg}}&
        \includegraphics[width=0.24\linewidth]{cvpr2023-author_kit-v1_1-1/latex/supp_qualt0/31_02.jpg} &
        \includegraphics[width=0.24\linewidth]{cvpr2023-author_kit-v1_1-1/latex/supp_qualt0/31_18.jpg} &
        \includegraphics[width=0.24\linewidth]{cvpr2023-author_kit-v1_1-1/latex/supp_qualt0/34_05.jpg} & \\
 & \multicolumn{1}{l}{}                                         &    &    &    & 
\end{tabular}
\vspace{-0.5cm}
\caption{{\bf Qualitative comparison with other methods.} Our method yields a high output diversity, yet preserves the trees in the first, second, and third columns. The road has the closest appearance to the exemplar with our method in the last column.}
\end{figure*}


\begin{figure*}[]

\begin{sideways}
\hspace{-2cm}
{Source}
\hspace{-3.5cm}
{Exemplar}
\hspace{-3.3cm}
{Ours}
\hspace{-3.2cm}
{MUNIT}
\hspace{-3.2cm}
{DRIT}
\hspace{-2.9cm}
{CUT}
\hspace{-3.3cm}
{LSeSim}
\hspace{-3.4cm}
{MGUIT}

\end{sideways}

\setlength\tabcolsep{1pt}
\begin{tabular}{lcllll}\\

      & \hspace{0.3cm} \multicolumn{1}{l}{\includegraphics[width=0.24\linewidth]{cvpr2023-author_kit-v1_1-1/latex/supp_qualt1/b10_14.jpg}}&
        \includegraphics[width=0.24\linewidth]{cvpr2023-author_kit-v1_1-1/latex/supp_qualt1/b12_16.jpg} &
        \includegraphics[width=0.24\linewidth]{cvpr2023-author_kit-v1_1-1/latex/supp_qualt1/b16_17.jpg} &
        \includegraphics[width=0.24\linewidth]{cvpr2023-author_kit-v1_1-1/latex/supp_qualt1/b2_08.jpg} & \\
 & \multicolumn{1}{l}{}                                         &    &    &    & 
\end{tabular}
\vspace{-0.5cm}
\caption{{\bf Qualitative comparison with other methods.} In this figure, and in the following ones, we show translations in the opposite direction, namely from GTA to CS. Even though stylistic diversity is less in the real image domain, the advantage of our method is still visible. Our method is better at matching the road and sky colors. In the second column, our method does not introduce trees instead of sky, which is common in translations of GTA images.}
\end{figure*}

\begin{figure*}[]

\begin{sideways}
\hspace{-2cm}
{Source}
\hspace{-3.5cm}
{Exemplar}
\hspace{-3.3cm}
{Ours}
\hspace{-3.2cm}
{MUNIT}
\hspace{-3.2cm}
{DRIT}
\hspace{-2.9cm}
{CUT}
\hspace{-3.3cm}
{LSeSim}
\hspace{-3.4cm}
{MGUIT}

\end{sideways}

\setlength\tabcolsep{1pt}
\begin{tabular}{lcllll}\\

      & \hspace{0.3cm} \multicolumn{1}{l}{\includegraphics[width=0.24\linewidth]{cvpr2023-author_kit-v1_1-1/latex/supp_qualt1/b17_08.jpg}}&
        \includegraphics[width=0.24\linewidth]{cvpr2023-author_kit-v1_1-1/latex/supp_qualt1/b17_19.jpg} &
        \includegraphics[width=0.24\linewidth]{cvpr2023-author_kit-v1_1-1/latex/supp_qualt1/b20_03.jpg} &
        \includegraphics[width=0.24\linewidth]{cvpr2023-author_kit-v1_1-1/latex/supp_qualt1/b26_16.jpg} & \\
 & \multicolumn{1}{l}{}                                         &    &    &    & 
\end{tabular}
\vspace{-0.5cm}
\caption{{\bf Qualitative comparison with other methods.} Aside from road and sky colors, our method is better at preserving the sky regions whereas other methods introduce trees.}
\end{figure*}

\begin{figure*}[]

\begin{sideways}
\hspace{-2cm}
{Source}
\hspace{-3.5cm}
{Exemplar}
\hspace{-3.3cm}
{Ours}
\hspace{-3.2cm}
{MUNIT}
\hspace{-3.2cm}
{DRIT}
\hspace{-2.9cm}
{CUT}
\hspace{-3.3cm}
{LSeSim}
\hspace{-3.4cm}
{MGUIT}

\end{sideways}

\setlength\tabcolsep{1pt}
\begin{tabular}{lcllll}\\

      & \hspace{0.3cm} \multicolumn{1}{l}{\includegraphics[width=0.24\linewidth]{cvpr2023-author_kit-v1_1-1/latex/supp_qualt1/b3_02.jpg}}&
        \includegraphics[width=0.24\linewidth]{cvpr2023-author_kit-v1_1-1/latex/supp_qualt1/b30_01.jpg} &
        \includegraphics[width=0.24\linewidth]{cvpr2023-author_kit-v1_1-1/latex/supp_qualt1/b30_11.jpg} &
        \includegraphics[width=0.24\linewidth]{cvpr2023-author_kit-v1_1-1/latex/supp_qualt1/b33_02.jpg} & \\
 & \multicolumn{1}{l}{}                                         &    &    &    & 
\end{tabular}
\vspace{-0.5cm}
\caption{{\bf Qualitative comparison with other methods.} In all the columns, sky is flipped to tree with other methods. Our method is better at preserving the semantics, yet has diverse outputs.}
\end{figure*}

\begin{figure*}[]

\begin{sideways}
\hspace{-2cm}
{Source}
\hspace{-3.5cm}
{Exemplar}
\hspace{-3.3cm}
{Ours}
\hspace{-3.2cm}
{MUNIT}
\hspace{-3.2cm}
{DRIT}
\hspace{-2.9cm}
{CUT}
\hspace{-3.3cm}
{LSeSim}
\hspace{-3.4cm}
{MGUIT}

\end{sideways}

\setlength\tabcolsep{1pt}
\begin{tabular}{lcllll}\\

      & \hspace{0.3cm} \multicolumn{1}{l}{\includegraphics[width=0.24\linewidth]{cvpr2023-author_kit-v1_1-1/latex/supp_qualt1/b34_12.jpg}}&
        \includegraphics[width=0.24\linewidth]{cvpr2023-author_kit-v1_1-1/latex/supp_qualt1/b38_05.jpg} &
        \includegraphics[width=0.24\linewidth]{cvpr2023-author_kit-v1_1-1/latex/supp_qualt1/b4_13.jpg} &
        \includegraphics[width=0.24\linewidth]{cvpr2023-author_kit-v1_1-1/latex/supp_qualt1/b5_12.jpg} & \\
 & \multicolumn{1}{l}{}                                         &    &    &    & 
\end{tabular}
\vspace{-0.5cm}
\caption{{\bf Qualitative comparison with other methods.} Our method has much less artifacts in sky in the first three columns. In the last column, the road has closer appearance to the exemplar with our method.}
\end{figure*}

{\small
\bibliographystyle{ieee_fullname}
