\subsubsection{Synthetic Setting}
\label{sec:denoising_synth_discussion}

\begin{figure}[t]
\centering
\includegraphics[width=\linewidth]{figures/denoising_imgs/synthetic/synth_comparison.png}
\caption{Denoising visual and quantitative comparison of synthetic data: (a) The results of data setting (100, 2). (b) The results of data setting (2000, 50). Note that in these two settings our method (Ours) does not apply data augmentation.}
\vspace{-1.5em}
\label{denoised_results_synthetic_setting}
\end{figure}


Using the synthetic data with ground truth in the setting (100, 2) shown in \autoref{denoised_results_synthetic_setting} (a), we have very limited amount of projections (i.~e., only 2) and a low dose which leads to a low \gls*{snr}. In settings like this where data are scarce and full of noise, a natural choice is to average the two available projections. Although the denoised result is not close to the noise-free reference, our method outperforms the simple average of the two available projections in terms of \gls*{ssim}~\cite{ssim}, \gls*{mse}, \gls*{psnr} and \gls*{ffl}. In the very high dose cases (i.~e., {700, 1000, 1500, 2000}), we show consistently better results than simple averaging in terms of \gls*{ffl}. For example, the quantitative comparison in \autoref{denoised_results_synthetic_setting} (b) shows that the \gls*{ffl} of our method is roughly 4 times better than simple averaging. The quantitative comparisons of these two figures show that our physically-based Noise2Noise-like model is effective.


To compare with the original Noise2Noise~\cite{noise2noise} model and Topaz-Denoise~\cite{topaz_denoiser}, we present our results from the setting (100, 50) denoted as Ours-50 in \autoref{synthetic_denoising_comparison}. The figure also shows the results of a trained Noise2Noise model, \texttt{Topaz P} and \texttt{Topaz T}. The Noise2Noise model does not learn to denoise even if we have a large amount of available data (i.~e., 5000). Therefore, we further tested the performance of the Topaz model with our synthetic dataset. The results show that the pretrained model (i.~e., \texttt{Topaz P}) generalizes well by filtering out high-frequency noise and preserves the particles, but it is worse than our method because the boundaries of particles are blurred. Our trained Topaz model (i.~e., \texttt{Topaz T}) filters out most of the noise and recognizes the positions of the particles. However, it does not preserve the shape of the particles, which makes it unusable. On the contrary, our method can work in the setting with less data (i.~e., (100, 50)), which shows that our method is advantageous over training a Noise2Noise model~\cite{noise2noise} and a Topaz-Denoise~model~\cite{topaz_denoiser} from scratch.

It is noteworthy that the Noise2Noise model and \texttt{Topaz T} are trained with data settings (100, 5000) and (100, 4500) respectively. It is not realistic nor viable to obtain such large datasets in real-world scenarios, which means these two models are not usable. However, \texttt{Topaz P} generalizes well in our setting, so we use it for data augmentation in the real-world scenario to generate more data for our denoising method.

For more results of our method under synthetic settings, we refer the reader to the supplementary material.

\begin{figure}[t]
\begin{center}
\includegraphics[width=\linewidth]{figures/denoising_imgs/synthetic/Synthetic-comparison-3.png}
\caption{Comparison of denoising the synthetic data: images in the top row show original noisy projection (NP) and clean ground truth projection (GT) and our denoising result, in bottom row are denoising results using the trained Noise2Noise result (Noise2Noise), pretrained Topaz-Denoise~(\texttt{Topaz P}), and Topaz-Denoise~trained on our noisy projections (\texttt{Topaz T}).}
\vspace{-2.5em}
\label{synthetic_denoising_comparison}
\end{center}
\end{figure}




\begin{table*}[t]
\centering\caption{SSIM, PSNR and FFL comparison} \label{SSIM_PSNR_FFL_comparison}
\resizebox{\textwidth}{!}{%
\begin{tabular}{cccccc|ccccc|ccccc}
\multicolumn{6}{c}{\textbf{SSIM}} & \multicolumn{5}{c}{\textbf{PSNR}} & \multicolumn{5}{c}{\textbf{FFL}}\\
\toprule
\multicolumn{1}{c}{}      & \texttt{Noisy}    & \texttt{Topaz}    & \texttt{Ours(T)}        & \texttt{Ours(N)}         & \texttt{Ours(T+N)}      & \texttt{Noisy}    & \texttt{Topaz}    & \texttt{Ours(T)}        & \texttt{Ours(N)}        & \texttt{Ours(T+N)}      & \texttt{Noisy}    & \texttt{Topaz}    & \texttt{Ours(T)}        & \texttt{Ours(N)}   & \texttt{Ours(T+N)}      \\
\midrule
\multicolumn{1}{r|}{\texttt{Noisy}} & 1.0000   & 0.6017   & 0.4139   & 0.7395   & \textbf{0.6547}   & 361.200  & 65.8250  & 65.8270  & 70.1500  & \textbf{66.1000}  & 0.0000   & 0.0096   & 0.0082   & 0.0032   & \textbf{0.0095}   \\
\multicolumn{1}{r|}{\texttt{Topaz}} & 0.6017   & 1.0000   & 0.6894   & 0.7184   & \textbf{0.8651}   & 65.8250  & 361.200  & 72.2000  & 68.9700  & \textbf{75.0400}  & 0.0096   & 0.0000   & 0.0012   & 0.0045   & \textbf{0.0008}   \\
\multicolumn{1}{r|}{\texttt{Ours(T)}}     & 0.4139   & 0.6894   & 1.0000   & 0.7797   & 0.8001   & 65.8270  & 72.2000  & 361.200  & 71.6200  & 76.2800  & 0.0082   & 0.0012   & 0.0000   & 0.0020   & 0.0001   \\
\multicolumn{1}{r|}{\texttt{Ours(N)}}     & 0.7395   & 0.7184   & 0.7797   & 1.0000   & 0.9098   & 70.1500  & 68.9700  & 71.6200  & 361.200  & 71.6700  & 0.0032   & 0.0045   & 0.0020   & 0.0000   & 0.0029   \\
\multicolumn{1}{r|}{\texttt{Ours(T+N)}}   & 0.6547   & 0.8651   & 0.8001   & 0.9098   & 1.0000   & 66.1000  & 75.0400  & 76.2800  & 71.6700  & 361.200  & 0.0095   & 0.0008   & 0.0001   & 0.0029   & 0.0000   \\
\bottomrule
\end{tabular}
}
\end{table*}

\begin{figure*}[t]
\begin{center}
\includegraphics[width=\textwidth]{figures/reconstruction-and-dvr-comparisson-small.png}
\caption{Tomographic reconstructions from the tilt series (left) and DVR renderings (right).}
\vspace{-1em}
\label{fig:reconstruction-and-dvr-comparison}
\end{center}
\end{figure*}

In the experiments using real micrographs, we use the tilt series from Yao \textit{et~al.}~\cite{Sai2020:CryoET-Pipeline}, which show SARS-CoV-2 virions. The dose in the dataset is 320 electrons per nm\textsuperscript{2} per projection. Therefore, without data augmentations, the data setting is (320, 1). As discussed in Section~\ref{denoising_section}, our method cannot work in a scenario with only single projection, so in our experiments, we apply a number of data augmentations. In the setting \texttt{Ours(T)}, the data augmentation with \texttt{Topaz P} is used, yielding one more projection, which means the data setting is (320, 2). In the setting denoted as \texttt{Ours(N)}, neighboring projections of a projection are aligned and used, so the data setting is (320, 3). In the setting \texttt{Ours(T+N)}, \texttt{Topaz P} and aligned neighboring projections are used, so the data setting is (320, 6). The case of \texttt{Ours(T+N)} matches exactly our \autoref{denoising_optimization} while in the case of \texttt{Ours(T)}, $P_n=\{P\}$, and in the case of \texttt{Ours(N)}, preprocessing with Topaz-Denoise~is not used (i.~e., $P_{\train}=\normalize(P_n)$). In the real setting, we do not have noise-free ground truth projections to compare against, so we first present results of the qualitative comparison.

On the top of \autoref{denoising_with_real_data}, we can see \texttt{Topaz} has block-like artifacts and \texttt{Noisy} contains more noise while others have minor visual difference. However, if we zoom-in the images, as shown in the bottom of ~\autoref{denoising_with_real_data}, we can see (1) \texttt{Topaz} has more blurring artifacts, (2) \texttt{Noisy} has the most noise, and (3) \texttt{Ours(T+N)} is better than \texttt{Ours(N)} and \texttt{Ours(T)} in that it has less high-frequency noise while preserving clear boundaries. This makes our method \texttt{Ours(T+N)} the best among these five because of its good balance between denoising and preserving boundaries from the view of quality. 

\begin{figure}[t]
\begin{center}
\includegraphics[width=\linewidth]{figures/RealDataDenoising-composite.png}
\caption{\texttt{Topaz}: the real projection denoised by \texttt{Topaz P}; \texttt{Noisy}: the real noisy projection; Top: Full views; Middle: Zoom-in of the first row. In the bottom row is another example of our denoising method, \texttt{Ours(T+N)} contains less noise while preserving boundaries.}
\vspace{-2em}
\label{denoising_with_real_data}
\end{center}
\end{figure}

We also present quantitative results in Table~\ref{SSIM_PSNR_FFL_comparison}. Based on the bottom images from \autoref{denoising_with_real_data}, we can conclude that \texttt{Topaz} contains the least noise while \texttt{Noisy} contains the most noise. With this conclusion, the values in bold in the table show that (1) \texttt{Ours(T+N)} is the closest one to \texttt{Topaz} and (2) \texttt{Ours(T+N)} has a moderate amount of noise compared to \texttt{Ours(T)} and \texttt{Ours(N)}. This means \texttt{Ours(T+N)} is a good compromise between removing noise and preserving details, echoing with our qualitative comparison.

In addition to the example on the top two rows of \autoref{denoising_with_real_data}, we give another example of denoising real data. From the last row in \autoref{denoising_with_real_data}, we can draw similar conclusions. \texttt{Topaz} has the least noise, but compromised boundaries while \texttt{Ours(T+N)} has a moderate amount of noise but also clearer boundaries and more visibility of spikes.

The application of such denoising are also useful for the tomographic reconstruction. In left part of  \autoref{fig:reconstruction-and-dvr-comparison} we show the mid-slice from the volumes reconstructed using SIRT reconstruction technique from IMOD~\cite{kremer1996computer} from original noisy projections (left), from denoised projections using \texttt{Topaz} (middle), and from denoised projections using our approach \texttt{Ours(T+N)} (right). One can clearly see that \texttt{Topaz} and our approach are better that reconstruction from the original noisy projections. Moreover, our approach retains more details which are smudged out by \texttt{Topaz}.

Additionally, in the right part of \autoref{fig:reconstruction-and-dvr-comparison}, we show \gls*{dvr} rendering of the same volumes using the same rendering parameters apart from the threshold, which was manually adapted for each case. One can see that \texttt{Topaz} cleans the volume extensively, but also removes parts of the virions and removes the high frequency details from the structures. This is not the case for results using our approach. While there is more noise present in the volume, the high-frequency details are retained. Both approaches are clearly better choice than rendering the original noisy volume.


\subsection*{More Comparisons of Denoising Synthetic Projections}
Here we present more comparisons of results of different synthetic settings to show the effectiveness and usefulness of our denoising method.

When the number of samples is small, for example, 2 or 5. Our results are close to the averages of the datasets visually. Yet, our results are better in terms of quantitative metrics as shown in \autoref{fig:num_2_dose_200} and \autoref{fig:num_5_dose_200} in which the results of data settings of (200, 2) and (200, 5) are presented.

When the dose is high (i.~e., greater than 700 electrons per nm\textsuperscript{2} per projection), our method consistently outperforms the averages of the datasets by a large margin in terms of \gls*{ffl}, which indicates our denoising method is much more correct in frequency domain. Here we present the results of the settings (700, 10) and (1500, 10). In \autoref{fig:num_10_dose_700} we can see the \gls*{ffl} is almost 50\% better than the average of the dataset and in \autoref{fig:num_10_dose_1500} the margin is more than 400\%.

Along with the results in \autoref{denoised_results_synthetic_setting}, we can conclude that despite the margins are different with different data settings, our method is advantageous over simple averaging consistently. This is especially beneficial when the data are too scarce and too noisy to train neural networks.

\begin{figure}[H]
\begin{center}
\includegraphics[trim={3.5cm 2cm 3.5cm 2cm},clip=true, width=\linewidth]{figures/denoising_imgs/synthetic/comparison_ref_sample_num=2_dose=200_normalized_metrics.png}
\caption{Denoising visual and quantitative comparison of synthetic data with data setting (200, 2)}
\label{fig:num_2_dose_200}
\end{center}
\end{figure}

\begin{figure}[h]
\begin{center}
\includegraphics[trim={3.5cm 2cm 3.5cm 2cm},clip=true,width=\linewidth]{figures/denoising_imgs/synthetic/comparison_ref_sample_num=5_dose=200_normalized_metrics.png}
\caption{Denoising visual and quantitative comparison of synthetic data with data setting (200, 5)}
\label{fig:num_5_dose_200}
\end{center}
\end{figure}


\begin{figure}[H]
\begin{center}
\includegraphics[trim={3.5cm 2cm 3.5cm 2cm},clip=true,width=\linewidth]{figures/denoising_imgs/synthetic/comparison_ref_sample_num=10_dose=700_normalized_metrics_unavg.png}
\caption{Denoising visual and quantitative comparison of synthetic data with data setting (700, 10)}
\label{fig:num_10_dose_700}
\end{center}
\end{figure}

\begin{figure}[H]
\begin{center}
\includegraphics[trim={3.5cm 2cm 3.5cm 2cm},clip=true,width=\linewidth]{figures/denoising_imgs/synthetic/comparison_ref_sample_num=10_dose=1500_normalized_metrics_unavg.png}
\caption{Denoising visual and quantitative comparison of synthetic data with data setting (1500, 10)}
\label{fig:num_10_dose_1500}
\end{center}
\end{figure}




\subsection{Denoising}
\label{denoising_related_work}
When denoising \gls*{cryo-em} micrographs, clean references are not available, so many approaches have been proposed to specifically denoise \gls*{cryo-em} micrographs, addressing this problem.

Researchers have proposed various analytical filters for denoising, such as simple averaging, Gaussian filters, and many filters in the frequency space that enhance or damp signal responses. Additionally, physics-based techniques were proposed, such as \gls*{nad}~\cite{NAD}, which takes advantage of a diffusion model to denoise 3D volumes. For a comprehensive review on filters used in \gls*{cryo-em}, we refer the reader to the review~\cite{cryo_em_filtering}.

In the era of \gls*{dl}, many neural networks are proposed to denoise \gls*{cryo-em} micrographs. Noise2Noise~\cite{noise2noise} is a general denoising approach that does not require clean data and has been adapted in CryoCARE~\cite{cryo-care} and Topaz-Denoise~\cite{topaz_denoiser}. In CryoCARE~\cite{cryo-care}, a neural network is trained as a denoiser with multiple micrographs that come from multiple consecutive exposures or multiple aligned micrographs in a tilt series or movie frames captured during a long exposure. In Topaz-Denoise~\cite{topaz_denoiser}, the researchers trained the model with a larger dataset containing thousands of micrographs acquired in different conditions, which enables better generalizability of their neural networks. Besides Noise2Noise-based approaches, \gls*{nt2c}~\cite{nt2c} trains two neural denoisers and a \gls*{gan}-based noise synthesizer to decouple noise from micrographs that are generated using a \gls*{cryo-em} simulator based on real and synthetic data.

The implicit reconstruction approach by Kniesel~\textit{et~al.}~\cite{kniesel2022stem} also incorporates a normalizing flows neural network to model the noise distribution in the \gls*{stem} data in a signal-dependent manner, allowing them to separate the noise from the clean reconstruction.

Our approach is similar to Noise2Noise~\cite{noise2noise} in the way that our model is required to output similar noisy images (i.e., noisy \gls*{em} projections). However, in our simulator, we do not involve neural networks but a physical model.


\section{TEM Implementation}
To accelerate the \gls*{tem} simulator, we developed the \gls*{gputem} simulator that exploits parallelism of the GPU hardware using the CUDA API~\cite{cuda}. A large portion of the simulator pipeline is implemented in CUDA, while there are some  stages where we resort to the CPU implementation. We make a parallel version of the phantom generator and a per-slice parallelized version the electron-particle interaction algorithm.

For the phantom generator, the most time-consuming task is computing the scattering potential of biological structures (particles). The scattering potential of one particle is calculated by the sum of the scattering potential of all atoms that form the particle. In the original \gls*{tem}~\cite{rullgaard2011simulation} simulator, this task is performed sequentially. 
To overcome this bottleneck in our \gls*{gputem}, we make a CUDA kernel for computing scattering potential for one atom, so the scattering potential of all atoms in the particle is computed in parallel at the atom level.

To calculate the electron-specimen interaction of each slice, we first determine which structures of the specimen are within the slice. Then we calculate the scattered electron wave after it interacts with these particles. This task is also performed sequentially in the original \gls*{tem}~\cite{rullgaard2011simulation} simulator. We make another CUDA kernel in our \gls*{gputem} simulator to compute the scattered electron wave after it interacts with one structure, so the electron-specimen interaction of each slice is computed in parallel at the particle level.

To implement our \gls*{difftem} simulator, we use Pytorch~\cite{paszke2019pytorch} and Taichi~\cite{hu2019taichi, hu2019difftaichi}. Pytorch is a well-known machine learning framework for parallel computing with GPU acceleration. With Pytorch, all gradients of each component in the simulator are calculated automatically and can be used for optimization with gradient descent. However, the \gls*{tem} simulator uses the multi-slice method for computing the electron-specimen interaction, which is a sequential algorithm and hard to be expressed in Pytorch operators. Therefore, to speed up and differentiate this computation, we use Taichi~\cite{hu2019taichi, hu2019difftaichi} and implement the calculation using Taichi kernels. 

Our \gls*{difftem} simulator is differentiable, so all continuous parameters of different parts of the simulator can be optimized or learned from the real data via backpropagation and gradient descent. We demonstrate this capacity through two examples: (1) \gls*{mtf} parameters estimation and (2) denoising. From the pipeline, the final projection is obtained by adding Poisson noise to the noise-free projection and applying \gls*{mtf} afterward. Therefore, to estimate \gls*{mtf}, we perform backpropagation from real projections and use gradient descent for optimization. To denoise the real projections with gradient-based optimizations, we estimate the gradients of Poisson noise by using the method proposed by Schulman \textit{et~al.}~\cite{Schulman2015:GE-SCG} and apply gradient descent.




\subsection{Performance}
We demonstrated the scalability of our GPU-TEM in two aspects: (1) number of atoms and (2) number of virion instances. The performance of each simulator for a single instance each virus type is shown in~\autoref{table:performance_atoms}. 

\begin{table}[H]
\small
\caption{Performance on w.r.t. number of atoms}
\label{table:performance_atoms}
\centering
\begin{tabular}{lccc}
\toprule
\textsc{Model} & \textsc{GPU-\gls*{tem}} & \textsc{\gls*{tem}~\cite{rullgaard2011simulation}}\\
\midrule
{\gls*{tmv}} & {4s} & {45s}\\ 
{ZIKA} & {1m46s} & {1m51s} \\ 
{SARS-CoV-2} & {} & {} \\ 
\bottomrule
\end{tabular}
\end{table}

From the~\autoref{table:performance_atoms}, the performance on single instance of ZIKA virus of our GPU-TEM is similar to the baseline~\cite{rullgaard2011simulation}, so we measured the performance on many instances of ZIKA, the results are shown in~\autoref{table:performance_instances}. 
As in~\autoref{table:performance_instances}, our GPU-TEM performs better for scenes containing high number of instances.

\begin{table}[H]
\small
\caption{Performance on many instances of ZIKA}
\label{table:performance_instances}
\centering
\begin{tabular}{lccc}
\toprule
\textsc{No. of instances} & \textsc{GPU-\gls*{tem}} & \textsc{\gls*{tem}~\cite{rullgaard2011simulation}}\\
\midrule
{1} & {\textbf{1m 46s}} & {1m 51s} \\ 
{5} & {3m 1s} & {2m 9s} \\ 
{10} & {3m 10s} & {2m 16s} \\ 
{50} & {\textbf{3m 30s}} & {3m 34s} \\ 
{100} & {3m 48s} & {2m 16s} \\ 
{500} & {\textbf{5m 42s}} & {21m 17s} \\ 
{1000} & {\textbf{6m 22s}} & {35m 27s} \\ 
{5000} & {\textbf{18m 9s}} & {4h11m } \\ 
\bottomrule
\end{tabular}
\end{table}


\subsection{MTF parameters estimation}
\begin{figure*}[t]
\begin{center}
\includegraphics[width=\textwidth]{figures/Graphs.pdf}
\caption{\gls*{mtf} parameters estimation experiment results}
\label{fig:mtf_parameters_exp}
\end{center}
\end{figure*}

The effect of \gls*{mtf} is hard to see in the real \gls*{cryo-em} images because of low \gls*{snr} so we applied both ground truth \gls*{mtf} and predicted \gls*{mtf} on the noise-free image of synthetic scene and performed evaluation. The details of the synthetic data, the number of background images, and ground truth values of \gls*{mtf} parameters $(a, b, c, \alpha, \beta)$ are shown in~\autoref{table:desc_mtf_exp}.

\begin{table}[ht]
\small
\caption{Details of experiments for estimating \gls*{mtf} parameters}
\label{table:desc_mtf_exp}
\centering
\begin{tabular}{lccc}
\toprule
\textsc{Subject} & \textsc{\gls*{mtf} paramters} & \textsc{No. of images}\\ 
\midrule
{3 \gls*{tmv}} & {0.7, 0.2, 0.1, 10, 40} & {5, 10, 20, 50}\\ 
{$\approx 10$ SARS-CoV-2} & {0.6, 1.24, -0.84, 0.97, 0.97} & {48} \\ 
{Synthetic \gls*{mtf}} & {-2.86, 2.113, -0.1318, 57.288, 25.429} & {50} \\ 
\bottomrule
\end{tabular}
\end{table}

To evaluate the effect of the \gls*{mtf} on the noise-free image, we calculated the contribution of each frequency for the noise-free image and determined which frequency band yields the largest contribution by summing the magnitude for each band. The frequency ($\frac{1}{pixel}$) bands we used are $\left\{[0.0, 0.02), [0.02, 0.04), \ldots, [0.48, 0.5)\right\}$. If the predicted \gls*{mtf} aligns well with the most contributing frequency band of the ground truth \gls*{mtf}, their effects on the noise-free images are similar. We additionally calculated the mean and maximum difference of values for both images. If two \gls*{mtf} align with the ground truth \gls*{mtf} in the same degree, the better one is the \gls*{mtf} has the mean  and maximum differences values are smaller.

The correlation between the number of background images and the performance of each loss function on synthetic data is shown in \autoref{fig:mtf_parameters_exp}(a, b). \autoref{fig:mtf_parameters_exp}(a, b) shows that the \gls*{mse} performance decreases when the number of background images is greater than $10$, the gap between the predicted \gls*{mtf} and the ground truth \gls*{mtf} increases, the mean and maximum differences also increase. With $50$ background images, the images applied predicted \gls*{mtf} is blurring, and hard to see the details of each \gls*{tmv} virions. On the contrary, \gls*{ffl} performance improves when increasing the number of background images. The predicted \gls*{mtf} with $5$ background images is more aligned with the ground truth \gls*{mtf} than $10, 20$ background images. However, the gap between it the ground truth \gls*{mtf} is larger than the gap between the \gls*{mtf} estimated from $50$ background images. The image applied the predicted \gls*{mtf} with $50$ has the smallest mean and maximum differences values. The comparison of the best \gls*{mse} and \gls*{ffl} results is shown in~\autoref{fig:mtf_parameters_exp}(a, b). It is clear that \gls*{ffl} is better choice than \gls*{mse}.

We also investigated how the change of \gls*{ffl} $\alpha$ parameter affects the \gls*{mtf} estimation for different setting. The results are shown in  \autoref{fig:mtf_parameters_exp}(c, d). The best predicted \gls*{mtf} for \gls*{tmv}, SARS-CoV-2, and synthetic setting are obtained by using $\alpha = 2.0$, $\alpha = 0.5$, $\alpha = 2.0$, for \gls*{ffl} respectively. The best $\alpha$ value for each experiment is different because of the settings (electron dose, sample preparation, etc.) of different experiments are not the same, the distribution of frequencies in background images is changed. The results show that our system works effectively for different types of \gls*{mtf} curve. The first type is the real \gls*{mtf} (\gls*{tmv} and SARS-CoV-2) gives high modulation in the most contributing frequency band, so the details of virions are not blurry too much. The second type is the synthetic \gls*{mtf}, the modulation decreases in the most contributing frequency band, so the image is applied the \gls*{mtf} is more blurry.
Our predicted \gls*{mtf} for each case is aligned well with the ground truth in the most contributing frequency band, and give the similar effect on the noise free image. 

\subsection{Denoising}
We compare the denoising capabilities of our approach in synthetic and real settings qualitatively and quantitatively.

\label{sec:denoising_synth_discussion}

\begin{figure}[H]
\begin{center}
\includegraphics[trim={3.5cm 0 3.5cm 0},clip=true,width=\linewidth]{figures/denoising_imgs/synthetic/comparison_ref_sample_num=2_dose=100_normalized_metrics.png}
\caption{Denoised result comparison of synthetic data setting (100, 2)}
\label{num_2_dose_100}
\end{center}
\end{figure}

Using a synthetic data with ground truth in the setting (100, 2)~, we have very limited amount of projections (i.~e., only 2) and a low dose which leads to a low \gls*{snr}. In settings like this where data are scarce and full of noise, a natural choice is to average the two available projections. Although the denoised result is not close to the noise-free reference, our method outperforms the simple average of the two available projections in terms of \gls*{ssim}~\cite{ssim}, \gls*{mse}, \gls*{psnr} and \gls*{ffl}. In the case of very high doses (i.~e., {700, 1000, 1500, 2000}), we show consistently better results than simple averaging in terms of \gls*{ffl}. For example, Fig.~\ref{num_50_dose_2000} shows that the \gls*{ffl} of our method is roughly 4 times better than simple averaging. The quantitative comparisons of these two figures show that our physics-based Noise2Noise-like model is effective.

\begin{figure}[H]
\begin{center}
\includegraphics[trim={3.5cm 0 3.5cm 0},clip=true,width=\linewidth]{figures/denoising_imgs/synthetic/comparison_ref_sample_num=50_dose=2000_normalized_metrics_unavg.png}
\caption{Denoised result comparison of synthetic data setting (2000, 50)}
\label{num_50_dose_2000}
\end{center}
\end{figure}

To compare with the original Noise2Noise~\cite{noise2noise} model and Topaz-Denoise~\cite{topaz_denoiser}, we present the our result of the setting (100, 50) denoted as Ours-50 in \autoref{synthetic_denoising_comparison}, in which the results of a trained Noise2Noise model, Topaz P and Topaz T are also shown. The Noise2Noise model cannot work even if we have a large amount of data available (i.~e., 5000). Therefore, we further tested the performance of Topaz models with our synthetic dataset. As we can see, the pretrained model (i.~e., \verb|Topaz P|) generalizes well in that it filters out high-frequency noise and preserves the particles, but it is worse than our method because the boundaries of particles are blurred. Our trained Topaz model (i.~e., \verb|Topaz T|) is better in that it filters out most of noise and recognizes the positions of the particles. However, it does not preserve the shape of the particles, which makes it unusable. On the contrary, our method can work in the setting with less data (i.~e., (100, 50)), which shows that our method is advantageous over training a Noise2Noise~\cite{noise2noise} model and a Topaz-Denoise~\cite{topaz_denoiser} model from scratch.

It is noteworthy that the Noise2Noise model and \verb|Topaz T| are trained with data settings (100, 5000) and (100, 4500) respectively. It is not realistic nor viable to obtain such large datasets in real-world scenarios, which means these two models are not usable. However, \verb|Topaz P| generalizes well in our setting, so we will use it to do data augmentation in the real-world scenario to generate more data for our denoising method.

For more results of our method under synthetic settings, we refer the reader to the \fl{our supplementary materials}.

\begin{figure}[H]
\begin{center}
\includegraphics[width=\linewidth]{figures/denoising_imgs/synthetic/Synthetic-comparison-3.png}
\caption{Comparison of denoising the synthetic data: images in the top row show original noisy projection (NP) and clean projection (GT), in bottom row are denosing results using Trained Noise2Noise result (Noise2Noise), pretrained Topaz Denoise (Topaz P), and Topaz Denoise trained on our images (Topaz T).}
\label{synthetic_denoising_comparison}
\end{center}
\end{figure}



\label{denoising_real_results}

In the setting of using the real data, we do not have noise-free ground truth projections to compare against to, so we first present results for qualitative comparisons.

\begin{figure}[H]
\begin{center}
\includegraphics[width=\linewidth]{figures/denoising_imgs/real_data_compare/RealDataDenoising.png}
\caption{Topaz: the real projection preprocessed by Topaz P; Noisy: the real noisy projection; Top: full views; Bottom: Zoom-in}
\label{denoising_with_real_data}
\end{center}
\end{figure}

\begin{table*}[t]
\centering\caption{SSIM, PSNR and FFL comparison} \label{SSIM_PSNR_FFL_comparison}
\resizebox{\textwidth}{!}{%
\begin{tabular}{cccccc|ccccc|ccccc}
\multicolumn{6}{c}{\textbf{SSIM}} & \multicolumn{5}{c}{\textbf{PSNR}} & \multicolumn{5}{c}{\textbf{FFL}}\\
\toprule
\multicolumn{1}{c}{}      & Noisy    & Topaz    & Ours (T)        & Ours (N)         & Ours (T+N)      & Noisy    & Topaz    & Ours (T)        & Ours (N)        & Ours (T+N)      & Noisy    & Topaz    & Ours (T)        & Ours (N)   & Ours (T+N)      \\
\midrule
\multicolumn{1}{r|}{Noisy} & 1.0000   & 0.6017   & 0.4139   & 0.7395   & \textbf{0.6547}   & 361.200  & 65.8250  & 65.8270  & 70.1500  & \textbf{66.1000}  & 0.0000   & 0.0096   & 0.0082   & 0.0032   & \textbf{0.0095}   \\
\multicolumn{1}{r|}{Topaz} & 0.6017   & 1.0000   & 0.6894   & 0.7184   & \textbf{0.8651}   & 65.8250  & 361.200  & 72.2000  & 68.9700  & \textbf{75.0400}  & 0.0096   & 0.0000   & 0.0012   & 0.0045   & \textbf{0.0008}   \\
\multicolumn{1}{r|}{Ours (T)}     & 0.4139   & 0.6894   & 1.0000   & 0.7797   & 0.8001   & 65.8270  & 72.2000  & 361.200  & 71.6200  & 76.2800  & 0.0082   & 0.0012   & 0.0000   & 0.0020   & 0.0001   \\
\multicolumn{1}{r|}{Ours (N)}     & 0.7395   & 0.7184   & 0.7797   & 1.0000   & 0.9098   & 70.1500  & 68.9700  & 71.6200  & 361.200  & 71.6700  & 0.0032   & 0.0045   & 0.0020   & 0.0000   & 0.0029   \\
\multicolumn{1}{r|}{Ours (T+N)}   & 0.6547   & 0.8651   & 0.8001   & 0.9098   & 1.0000   & 66.1000  & 75.0400  & 76.2800  & 71.6700  & 361.200  & 0.0095   & 0.0008   & 0.0001   & 0.0029   & 0.0000   \\
\bottomrule
\end{tabular}
}
\end{table*}
In \autoref{denoising_with_real_data}, we can see \verb|Topaz| has block-like artifacts and \verb|Noisy| contains more noise while the other techniques have few differences. However, if we zoom-in the images, as shown in the bottom of ~\autoref{denoising_with_real_data}, we can see (1) \verb|Topaz| has more blurring artifacts, (2) \verb|Noisy| has the most noise, and (3) \verb|T+N| is better than \verb|N| and \verb|T| in that it has less high-frequency noise while preserving clear boundaries. This makes our method \verb|T+N| the best among these five because of its good balance between denoising and preserving boundaries from the view of quality. 

We also present quantitative results in Table~\ref{SSIM_PSNR_FFL_comparison}. Based on the bottom images from \autoref{denoising_with_real_data}, we can conclude that \verb|Topaz| contains the least noise while \verb|Noisy| contains the most noise. With this conclusion, the values in bold in the three tables show that (1) \verb|T+N| is the closest one to \verb|Topaz| besides \verb|Topaz| itself and (2) \verb|T+N| has a moderate amount of noise compared to \verb|T| and \verb|N|. This means our method is a good compromise between removing noise and preserving details, echoing with our qualitative comparison.

Besides the example in \autoref{denoising_with_real_data}, we give another example of denoising real data. From \autoref{denoising_with_real_data_zoomin_another}, we can draw the similar conclusions. \verb|Topaz| has the least noise, but compromised boundaries while \verb|T+N| has moderate amount of noise but also clearer boundaries and more visibility of spikes.

\begin{figure}[H]
\begin{center}
\includegraphics[width=\linewidth]{figures/denoising_imgs/real_data_compare/RealDataDenoising-2.png}
\caption{Denoising with real data zoom-in another example}
\label{denoising_with_real_data_zoomin_another}
\end{center}
\end{figure}

\fl{TODO: add results of reconstruction}


\subsection{MTF Parameters Estimation}

Because \gls*{mtf} parameters are independent of the specimen, we can optimize these parameters by using a few crops of background from the real projections. These crops are $M \times N$ pixels in size. They are considered as the labelled projections $I_{l_1}, I_{l_2}, \ldots, I_{l_n}$. The predicted projections $I_{p_1}, I_{p_2}, \ldots, I_{p_n}$ are created using our simulator with the same configuration for the sample, electron beam and optical system. The specimen of the predicted projection contains only the background. The predicted projections have the same size as the labelled projections. Both labelled projections and predicted projections are normalized to the range $[0, 1]$. Applying \gls*{mtf} is performed in the frequency domain which is motivation for us to choose a loss function that can capture the differences of the predicted and the labelled projection in both spatial and frequency domain. In this work, we choose \gls*{ffl}~\cite{jiang2021focal} combined with \gls*{mse} by summing them into the loss function.

The computational graph for estimating the \gls*{mtf} parameters is shown in \autoref{fig:compute_graph_of_detector_parameter_estimation_noisy}.
\begin{figure}[tb]
\begin{center}
\includegraphics[width=\linewidth]{figures/detector_parameter_estimation_compute_graph_noisy.pdf}
\caption{Compute Graph of Detector Parameter Estimation: The \gls*{difftem} first receives noisy training data, computes the loss with simulated noisy data, applies the predicted \gls*{mtf}, and performs gradient descent for optimizing \gls*{mtf} parameters.}
\vspace{-1.5em}
\label{fig:compute_graph_of_detector_parameter_estimation_noisy}
\end{center}
\end{figure}

The optimization can be formally described as:
\begin{equation*}
\underset{\Theta_D}{\argmin}\ \mathbb{E}(L(\mtf_{\Theta_D}(\poisson(\tilde{I})),\  \nd)),
\end{equation*}
where $\Theta_D$ is the set of \gls*{mtf} parameters, $\tilde{I}$ a noise-free projection without \gls*{mtf}, $\nd$ is the noisy dataset, and $L$ is the loss function. $\nd$ can be the synthetic or real-world data. We conducted experiments for both. 

The \gls*{ffl} loss is defined through a weight matrix and a frequency distance matrix. 
$F_{l_1}, F_{l_2}, \ldots, F_{l_n}$ are 2D discrete Fourier transforms of labelled projections $I_{l_1}, I_{l_2}, \ldots, I_{l_n}$. $F_{p_1}, F_{p_2}, \ldots, F_{p_n}$ are 2D discrete Fourier transforms of predicted projections $I_{p_1}, I_{p_2}, \ldots, I_{p_n}$.




The \gls*{ffl} of labelled projection $I_{l_i}$ and predicted projection $I_{p_i}$ is:
\begin{equation}\label{ffl}
    \ffl(F_{l_i}, F_{p_i}) = \mean(w(F_{l_i}, F_{p_i}) * \left|F_{l_i} - F_{p_i}\right|^{2})
\end{equation}
where $\left|F_{l_i} - F_{p_i}\right|$ is frequency distance between the labelled projection $I_{l_i}$ and the predicted projection $I_{p_i}$, $w(F_{l_i}, F_{p_i}) = Normalize(\left|F_{l_i} - F_{p_i}\right|^{\alpha})$ is weighted matrix, $\alpha$ is the scaling factor for flexibility. The bigger $\alpha$ is, the more focus on optimization for hard frequencies (frequency components that are hard to synthesize). We conducted an experiment with different values for $\alpha$ and chose the best one.

The \gls*{mtf} parameters are optimized using the Adam optimizer~\cite{Kingma2014} $(\beta_{1} = 0.8, \beta_{2} = 0.992, lr = 0.1)$. We also experimented with optimization using only \gls*{mse} loss. The evaluation shows that combining \gls*{ffl} and \gls*{mse} performs best for the task.

\begin{algorithm}[tb]
\small
\begin{algorithmic}
\caption{\gls*{mtf} Parameters Optimization}
\label{mtf_optimization}
\State $I_{l_n} \gets Noisy\ Projections$
\State $I_{l_n} \gets \normalize(I_{l_n})$
\State $\Theta_D \gets random\ values$
\State $\tilde{I} \gets noise\verb|-|free\ projection\ without\ \mtf$
\State $\alpha \gets Learning\ Rate$
\State $IT \gets\ iterations$
\State $i \gets 1$
\While{$i\leq IT$}
    \State $I_{p_n} \gets \mtf(\poisson(\tilde{I}))$
    \State $I_{p_n} \gets \gets \normalize(I_{p_n})$
    \State $loss \gets \mse(I_{p_n}, I_{l_n}) + \ffl(I_{p_n}, I_{l_n})$
    \State $\Theta_D\gets \Theta_D - \alpha \frac{\partial loss}{\partial \Theta_D}$
    \State $i \gets i+1$
\EndWhile
\State \Return $\Theta_D$
\end{algorithmic}
\end{algorithm}

\subsection{Denoising}
\label{denoising_section}

As shown in \autoref{fig:electron_microscope_schematic_tem}, in the forward computation of our simulator, the input of quantization is a noise-free projection without \gls*{mtf} and the output of our simulator is a noisy projection. 

Since our simulator is differentiable, we can do inverse computations via gradient descent, which means we can re-formulate the problem of denoising into an inverse problem. That is, given noisy data and a differentiable compute pipeline, to recover the noise-free projections without \gls*{mtf}, which are in the upstream of our forward computation. The optimization can be formulated as:
\begin{equation*}
\underset{\tilde{I}}{\argmin}\ \mathbb{E}(\slf(\mtf(Q(\tilde{I})),\  \rd)),
\end{equation*}
where $\tilde{I}$ is the recovered denoised projection without \gls*{mtf}, $\slf$ the surrogate loss function devised from the work of stochastic compute graph~\cite{Schulman2015:GE-SCG}, $\rd$ the real data (e.~g., noisy SARS-CoV-2 projections), and $L$ a loss function. To get a denoised projection $I_d$ from $\tilde{I}$, we bypass the quantization and apply \gls*{mtf} directly, which means $I_d=\mtf(\tilde{I})$. 

The compute flow of the optimization of denoising is shown in \autoref{compute_graph_of_denoising}. The $\tilde{I}$ (i.~e., \verb|Projection without MTF|) is initialized with random values. The quantization models the shot noise of a detector. Given a Poisson distribution and its parameter $\lambda$, we can calculate the log likelihood of a sampled value $x\sim \poisson(\lambda)$, which is denoted as $\LL(x,\lambda)=ln(P(x|\poisson, \lambda))$ where $ln$ is the natural logarithm. After sampling with quantization and calculating log likelihoods, the results with shot noise $I_q$ (i.~e. \texttt{Noisy Projection without MTF} in \autoref{compute_graph_of_denoising}) are applied with \gls*{mtf}, which yields the predicted noisy projections $I_n$.

\begin{figure}[tb]
\begin{center}
\includegraphics[width=\linewidth]{figures/denoising_compute_graph.pdf}
\caption{Compute graph of denoising: The projection without \gls*{mtf} is the optimization target. Log likelihood values are used in our surrogate loss function along with the comparison loss that computes the difference between noisy projection predictions and noisy projections in a dataset.}
\vspace{-1.5em}
\label{compute_graph_of_denoising}
\end{center}
\end{figure}

Inspired by the surrogate loss functions by Schulman \textit{et~al.}~\cite{Schulman2015:GE-SCG}, we take the surrogate loss function as Eq.~\ref{surrogate_loss_denoising} where $\logl$ equals $\LL(I_q, \tilde{I})$ and $\mean(\cdot)$ is the operator taking the mean value of all values in a tensor.

\begin{equation}\label{surrogate_loss_denoising}
    \slf(I_n, \rd, \logl) = \mean((\rd-I_n)^2*\logl).
\end{equation}

The whole optimization algorithm is shown in \autoref{denoising_optimization}. The normalization trick in Line 5 ensures consistent and standardized contrast and thus stabilizes our optimization. Note that due to the scarcity of real noisy projections, we perform data augmentations to generate meaningful training data. For one micrograph $P$ in a tilt series, we align its neighboring micrographs, which are denoted as $P_{-}$ and $P_{+}$, using perspective-based image alignment in OpenCV~\cite{opencv_library} because we assume that neighboring micrographs have minor differences compared to $P$, which is a reasonable assumption when the tilt angle increment is small, for example, 2-3 degrees. Therefore, in \autoref{denoising_optimization}, $P_n = \{P_{-}, P, P_{+}\}$. Furthermore, we preprocess $P_n$ with Topaz-Denoise~\cite{topaz_denoiser} to generate training data that are less noisy. The data augmentation is optional in the synthetic setting because we have sufficient amount of noisy training data, but in the real setting, it is necessary because (1) the available raw data is scarce (i.~e., only one), (2) the amount of information from only one projection with a low \gls*{snr} is limited, and (3) as discussed in Section~\ref{denoising_related_work}, our method is essentially a physical-model-based Noise2Noise~\cite{noise2noise}, which means it cannot be trained with only one projection. In the denoising optimization, we use Adam~\cite{Kingma2014} as the optimizer with $(\beta_1=0.8,\beta_2=0.992,lr=1.0)$.

\begin{algorithm}[tb]
\small
\begin{algorithmic}[1]
\caption{Denoising Optimization}
\label{denoising_optimization}
\State $P_n \gets Noisy\ Aligned\ Projections$
\State $\alpha \gets Learning\ Rate$
\State $IT \gets\ iterations$
\State $P_t \gets \topazOp(P_n)$
\State $P \gets \normalize(P_n\cup P_t)$
\State $\tilde{I} \gets random\ values$
\State $i \gets 1$
\While{$i\leq IT$}
    \State $I_q \gets Q(\tilde{I})$
    \State $\logl\gets \LL(I_q, \tilde{I})$
    \State $I_n \gets \mtf(I_q)$
    \State $\loss \gets \slf(I_n, P, ll)$
    \State $\tilde{I}\gets \tilde{I} - \alpha \frac{\partial \loss}{\partial \tilde{I}}$
    \State $i \gets i+1$
\EndWhile
\State $I_d=\mtf(\tilde{I})$
\end{algorithmic}
\end{algorithm}
\subsection{Performance evaluation}
We compared the performance of the original TEM simulator and our GPU-TEM. In the phantom generator we used the model of SARS-CoV-2 virus modelled by Nguyen~\textit{et~al.}~\cite{nguyen2021modeling}, ZIKA virus \gls*{pdb}id 5IRE and \gls*{tmv} PDBid 2OM3. The details of each model are shown in~\autoref{table:desc_virus_model}. All performance experiments were done on a Windows system with one processor Intel(R) Xeon(R) CPU E5-2687W v4 3GHz and one GPU NVIDIA GeForece RTX 3090.

\begin{table}[H]
\small
\caption{Details of the models used in the performance evaluation.}
\label{table:desc_virus_model}
\centering
\begin{tabular}{lccc}
\toprule
\textsc{Virions}& \textsc{No. of pdbs} & \textsc{No. of atoms}  \\ 
\midrule
{\gls*{tmv}} & {1} & {12505} \\ 
{ZIKA} & {1} & {21149} \\
{SARS-CoV-2} & {17} & {$\approx 13$ millions} \\ 
\bottomrule
\end{tabular}
\end{table}

\subsection{MTF parameters estimation}
For estimating \gls*{mtf} parameters, we performed experiments on both synthetic and real data. The synthetic data is created from the envelope of SARS-CoV-2 virion and \gls*{tmv} virion. The real dataset is provided by Yao~\textit{et~al.}~\cite{Sai2020:CryoET-Pipeline} consists of 20 tilt series containing several SARS-CoV-2 virions each, but we only use cropped sub-image containing background for optimization. The real data was acquired using a Titan Krios microscope with K3 detector from Gatan, operated at a voltage of $300$ kV. 
The ground truth of \gls*{mtf} parameters for real data was obtained by performing curve-fitting from \gls*{mtf} curve of Gatan K3 detector\footnote{https://www.gatan.com/K3\#resources}. The ground truth values of \gls*{mtf} parameters for synthetic data \gls*{tmv} virion is provided by Rullg{\aa}rd~\textit{et~al.}~\cite{rullgaard2011simulation} in their simulation and calibrated with their real data which is the micrograph of \gls*{tmv} virions measured by a Philips CM 200 FEG \gls*{tem} microscope. We conducted experiments for each loss function (\gls*{mse}, \gls*{mse} + \gls*{ffl}) with different number of background images of synthetic data to see the correlation between performance of each loss and number of images. Then we conducted the experiments on the real data with the number of images similar to the number that gives the best performance on synthetic data. We also conducted experiments with different \gls*{ffl} loss $\alpha$ values:  $(0.0, 0.5, 1.0, 2.0)$. Beside, to demonstrate the generalization of the estimation method, we also experimented with synthetic \gls*{mtf} that has the shape is different with real \gls*{mtf}.
All estimating \gls*{mtf} parameters experiments were conducted on Linux system with four GPUs NVIDIA RTX A6000.

\subsection{Denoising}
First, we conducted experiments using a synthetic dataset generated with our simulator given an atomistic model (i.~e., \gls*{tmv} model from the research by Sachse \textit{et~al.}~\cite{SACHSE2007812}). In this setting, we obtain a clean ground truth data that are synthesized by bypassing the quantization. 

In simulation and in real electron microscopy process, the dose, measured in electrons per nm\textsuperscript{2} per projection, is a major factor that determines \gls*{snr}. We tested our denoising method with synthetic datasets with applying varying doses. The higher the dose is, the higher \gls*{snr} is. For generating a tilt series of 60 projections, 100 electrons/nm\textsuperscript{2} per projection is a typical low dose while 200 is a typical high dose. We tested datasets of doses \{100, 200, 300, 500, 700, 1000, 1500, 2000\} because it is possible to use a very high dose (e.~g., 1000) if the number of total projections is low and thus the total dose stays the same. Moreover, the number of available projections of the same scene (i.~e., same tilt angle and specimen) also affects denoising quality. We tested our method on the numbers of available projections of \{2, 5, 10, 25, 50\}. We denote varying data settings with pairs, i.e., dose, the number of available projections.

To compare with the original Noise2Noise~\cite{noise2noise} model and Topaz-Denoise~\cite{topaz_denoiser}, we trained a Noise2Noise model and a Topaz-Denoise~model with our synthetic data. The data setting for training Noise2Noise model is (100, 5000) while it is (100, 4500) for training Topaz-Denoise~model. We also use pretrained Topaz-Denoise~model to contrast the results.


In the experiments using real micrographs, we use the tilt series from Yao \textit{et~al.}~\cite{Sai2020:CryoET-Pipeline}, which is SARS-CoV-2. The dose in their data collection is 320 electrons/nm\textsuperscript{2} per projection. Therefore, without data augmentations, the setting is (320, 1). As discussed in Section~\ref{denoising_section}, our method cannot work in a scenario with only single projection, so in our experiments, we apply a number of data augmentations. In the setting \texttt{Ours(T)}, the data augmentation with Topaz P is used, yielding one more projection, which means the setting is (320, 2). In the setting denoted as \texttt{Ours(N)}, neighboring projections of a projection are aligned and used, so the data setting is (320, 3). In the setting \texttt{Ours(T+N)}, Topaz P and aligned neighboring projections are used, so the data setting is (320, 6). The case of \texttt{Ours(T+N)} matches exactly our \autoref{denoising_optimization} while in the case of \texttt{Ours(T)}, $P_n=\{P\}$, and in the case of \texttt{Ours(N)}, preprocessing with Topaz-Denoise~is not used (i.~e., $P_{train}=Normalize(P_n)$). The results of the three cases (i.~e., \texttt{Ours(T)}, \texttt{Ours(N)} and \texttt{Ours(T+N)}) will be presented along with the projection preprocessed by \texttt{Topaz P} and the noisy projection in Section~\ref{denoising_real_results}.

\subsection{Contribution}
This paper presents several technical contributions that and visualization applications:
\begin{itemize}[noitemsep,topsep=0pt]
    \item The overall architecture of the first differentiable microscopy simulator that is scalable to render realistic microscopy simulations within seconds or minutes.
    \item Learning detector parameters from real projections, to generate synthetic micrographs with the same detector characteristics that lead to similar visual appearance of the synthetic and the real projection.
    \item Utilization of deterministic and stochastic differentiation for various stages of the microscopy pipeline.
    \item Generation of large micrographs \emph{on par} with physical dimensions and complexity of real-world microscopy.
    \item A new denoising method which learns from noisy projections to create a noise-free version of real-world projections. Subsequent tomographic reconstruction then results in a noise-free volume that can be used for 3D visualization.
\end{itemize}
\section{Introduction}
\input{includes/introduction}

\section{Related work}
\input{includes/related-work}

\section{Architecture overview}
\input{includes/archiecture-overview}

\section{Learning components}
\input{includes/learning-components}

\section{Reverse-Engineering MTF}
\input{includes/MTF}

\section{Scalable TEM Simulation}
\input{includes/GPUTEM}

\section{Denoising Micrographs}
\input{includes/denoising}

\section{Conclusion}
\input{includes/conclusion}

\section*{Acknowledgment}

The authors would like to thank Sai Li and his team at School of Life Sciences, Tsinghua University, China for sharing the SARS-CoV-2 \gls*{cryo-em} data for this work.

\clearpage
\bibliographystyle{unsrt}
