\section*{Appendix 1: (Method Section of the Experiments)} 
\textbf{Paper Title}: Affective Medical Estimation and  Decision Making via Visualized Learning and Deep Learning

\section{Methods}

We have shared all the source codes of the experiments are shared online on GitHub  \\ (\href{https://github.com/mohaEs/VL4ML}{https://github.com/mohaEs/VL4ML}).
For all of the experiments, the network is developed with \textit{Keras} on \textit{Tensorflow} framework. The \textit{Adam} optimizer with a learning rate of 0.001 is used for training and the loss function is the \textit{Mean Absolute Error} between the target image and the produced output. 


\subsection{Experiment 1 - Patient Survival Prediction in ICU} 

The data used in this example is from \textit{WiDS Datathon 2020} challenge \cite{WiDS, WiDS2} available on \textit{Kaggle} and \textit{PhysioNet} \cite{physionet} website, which focuses on patient health through data from MIT’s GOSSIS (Global Open Source Severity of Illness Score) initiative \cite{GOSSIS}.
Each EHR observation has 186 columns including 3 identification information (\textit{patient id}, \textit{hospital id}, \textit{encounter id}), 16 demographic information  (e.g. \textit{icu type}, \textit{age}, \textit{bmi}, \textit{ethnicity}, \textit{hospital death}), 28 \textit{APACHE} covariates (e.g. \textit{apache 2 diagnosis}, \textit{apache 3j diagnosis}, \textit{gcs eyes apache}, \textit{gcs motor apache}, \textit{resprate apache}, \textit{sodium apache}), 52 vital signs (e.g. \textit{d1 heartrate min}, \textit{d1 resprate max}, \textit{h1 mbp max}, \textit{d1 hemoglobin max} ),  76 lab tests (e.g. \textit{h1 pao2fio2ratio max}, \textit{h1 hemoglobin min}, \textit{h1 glucose min}) and 12 APACHE predictions and comorbidity (\textit{apache 4a hospital death prob}, \textit{cirrhosis}). \\
\textbf{Inputs:} All the elements except identification information and \textit{hospital death} are used as features that create feature vectors with 183 values.  \\
\textbf{Outputs:} As labels, we use information about \textit{hospital deaths}. An image with size 23x23x3 (RGB colorful) is designed for visualization in which white color shows 0 label (survival) and Orange color shows 1 labels (fatal). Also, the surrounding pixels are reserved as RU areas with zero intensities (black color) to mean no effect. \\
\textbf{Network:} The network architecture is shown in Figure 2A, including fully connected layers, reshaping, concatenation, transposed convolution layer with different dilation rates and stride values. Dropout and batch normalization is also used to prevent overfitting.  \\
\textbf{Training:} 15830 observation samples used and train/val/test sets are selected randomly by an 80/10/10 ratio. The network with 183,336 trainable parameters  is trained with 500 maximum epochs and a batch size of 1000. Early stopping regarding to the value of training loss and validation's loss are also addressed with 30 epochs and 50 epochs patience, respectively.  

\subsection{Experiment 2- Diabetic Retinopathy Severeness Assessment}

\textbf{Type I and II:}
The dataset considered in this experiment is from the \textit{APTOS 2019 Blindness Detection} challenge on \textit{Kaggle} website \cite{APTOS}. The images were gathered from multiple clinics using a variety of cameras over an extended period of time. The associated dataset contains 3,662 retinal images and a clinician has rated each image for the severity of diabetic retinopathy on a scale from 0 to 4: 0-No DR, 1-Mild,2 - Moderate,3 - Severe,4 - Proliferative DR.\\
\textbf{Type III:} Data is obtained from \textit{DeepDRiD} challenge dataset \cite{Deepdrid}. \\
\textbf{Inputs:} All the \textit{Fundus} images are reshaped into 256x256 resolutions with colors preserved.  \\
\textbf{Outputs:} The desired output is an image with size 23x23x3, and the shape of the output image is an inner square with a narrow peripheral strip surrounding it. The rest of the pixels are reserved for \textit{RU} area using black color expressing 0 effect. Two different kind of outputs are considered for this experiment. The type I method involves creating output images based on varying intensities, while the type II method includes both intensities and colors. For type I, five different intensity levels are assigned for five different levels of disease severity. As shown in Figure 3A, RGB(1,1,1), RGB(0.8,0.8,0.8), RGB(0.6,0.6,0.6), RGB(0.4,0.4,0.4), RGB(0.2,0.2,0.2) are used for expressing different levels of disease varying from 0 to 4. For type II, five different intensity levels are considered for the peripheral strip while the color of the inner square is coded. As shown in Figure 3B, the color of the inner square is bright green (RGB(0,1,0)), dark green (RGB(0.5,1,0)), yellow (RGB(1,1,0)), orange(RGB(1,0.5,0)) and red(RGB(1,0,0)) for 0 to 5 level of disease. \\
\textbf{Network:} The network architecture is same as shown in Figure 2A for layers L4 and onward. The layers before L4 are as follows: 256x256x3 $\xrightarrow[]{\text{2D-FCL}}$ 127x127x100 $\xrightarrow[]{\text{2D-FCL}}$ 63x63x50 $\xrightarrow[]{\text{2D-FCL}}$ 31x31x50  $\xrightarrow[]{\text{flatten}}$ $\xrightarrow[]{\text{FC}}$ 50 $\xrightarrow[]{\text{FC}}$ 50 $\xrightarrow[]{\text{FC}}$ 100 (L4)$\xrightarrow[]{\text{...}}$\\
\textbf{Training:}  Only the training subset is used via nested cross-validation. 3662 samples are available and the train/val/test sets are selected randomly by an 80/10/10 ratio. The network with 2,511,359 trainable parameters is trained for 500 maximum epochs with  batch size of 100. 
Early stopping regarding to the value of training loss and validation's loss are also addressed with 30 epochs and 50 epochs patience, respectively.  


\subsection{Experiment 3- Estimating the Length-of-Stay in Hospital}

The \textit{MIT MIMIC-III} dataset  \cite{mimic} is used for this experiment because it is an openly available dataset on \textit{Physionet} platform \cite{physionet, mimic_physionet} developed by the MIT Lab for computational physiology, comprising de-identified health data associated with ~60,000 critical care patients. It includes demographics, vital signs, laboratory tests, medications, and more \cite{mimic}. LOS values are defined as the number of days between admission and discharge and just LOS observations lower than 45 are considered for this experiment. The implementation code for this experiment is published online and the details are as follows: 
\textbf{Inputs:} 52 features are selected as inputs which includes diagnosis categories, 17 primary categories of \textit{ICD-9} (e.g., digestive, skin), demographic information (e.g., age, race, religion, marital status), payment method (e.g., self-pay, medicare), hospital admission type (e.g., urgent, newborn, elective) (\url{https://github.com/daniel-codes/hospital-los-predictor}).  \\
\textbf{Outputs:} A resulting image is 45x45x3 in which each column represents a day (i.e. 45 days) and the column direction is assumed to be the time direction. If the LOS value is $M$, all columns from left to $M^{th}$ column would be cyan color (RGB(0,1,1)). The 10 rows on top and bottom parts of the image are reserved for the \textit{RU} area using black color.  \\
\textbf{Network:} The network architecture is the modified version of the network in 2A as follow. 52 $\xrightarrow[]{\text{FC}}$ 104 $\xrightarrow[]{\text{FC}}$ 52 $\xrightarrow[]{\text{FC}}$ 100  $\xrightarrow[]{\text{Reshape-> \dots ->L8}}$ 21x21x100 $\xrightarrow[]{\text{L9: 2D-TCL}}$ 43x43x100 $\xrightarrow[]{\text{L10: 2D-TCL}}$ 45x45x3\\
\textbf{Training:} 51944 samples are considered and the train/val/test sets are selected randomly by an 80/10/10 ratio. The network has  137,003 trainable parameters and is trained for 300 maximum epochs with 1000 batch size.  Early stopping regarding to the value of training loss and validation's loss are also addressed with 30 epochs and 50 epochs patience, respectively. 


\subsection{Experiment 4- Severeness assessment for COVID-19}

The aim of this experiment is binary classification of COVID-19 disease from X-ray images along with regression task of estimating the severity of disease. The extent of lung involvement by geographic ground glass opacity is selected as the severeness score.
For the COVID-19 cases, data is obtained from COVID-19 image data collection \cite{covidProspective, covidProspective_link}  and normal cases is collected from Chest X-Ray Images (Pneumonia) \cite{covid_normal, covid_normal_link}. \\
\textbf{Input:} The x-ray images are resized to 512x512 and the intensity values are scaled between 0 and 255. \\
\textbf{Output:} A 45x45x3 image is desired as the final output. COVID cases are expressed by red circles, whose radius enlargement corresponds to the severity of the disease, the larger is the circle, the more severe is the disease. Normal cases are denoted by green circles. The rest of the pixels are reserved for the \textit{RU} area\\
\textbf{Network:} The network has int total 10,116,981 trainable parameters. Regarding to Figure 2A, the new network architecture is 512x512x3 $\xrightarrow[]{\text{2D-FCL}}$ 256x256x3 $\xrightarrow[]{\text{2D-FCL}}$ 127x127x100 $\xrightarrow[]{\text{2D-FCL}}$ 63x63x50   $\xrightarrow[]{\text{flatten}}$ $\xrightarrow[]{\text{FC}}$ 50  $\xrightarrow[]{\text{FC}}$ 100 (L4)$\xrightarrow[]{\text{Reshape-> \dots ->L8}}$ 21x21x100 $\xrightarrow[]{\text{L9: 2D-TCL}}$ 43x43x100 $\xrightarrow[]{\text{L10: 2D-TCL}}$ 45x45x3. 
\\
\textbf{Training:} There are 94 labeled COVID-19 samples and the same number of normal samples are selected. Evaluation is performed via a 20-fold cross-validation, and every training session contains 0.1 data for validation. Training is performed with a length of 300 epochs and a batch size of 10.  Early stopping regarding to the value of training loss and validation's loss are also addressed with 50 epochs and 100 epochs patience, respectively. 


\subsection{Experiment 5- Alzheimer's disease progression}

\textbf{Inputs:} Multi-modal tabular data contains features extracted from \textit{MRI} and \textit{PET} sequences, \textit{demographic information} and some \textit{cognitive measurements} same as reference  \cite{neuroimage_sola, frontiers_sola}. 
\underline{MRI:} Ventricular volume, Hippocampus volume, Whole Brain volume, Entorhinal Cortical thickness, Fusiform, Middle temporal gyrus, and intracranial volume (ICV). 
\underline{PET:} FDG, Pittsburgh Compound-B (PIB), AV45. 
\underline{Cognitive Tests:} Rey Auditory Verbal Learning Test (RAVLT Immediate, RAVLT Learning, RAVLT Forgetting, RAVLT Perc Forgetting), Functional Activities Questionnaires (FAQ), Everyday Cognition (Ecog) scales: (EcogPtMem, EcogPtLang, EcogPtVisspat, EcogPtPlan, EcogPtOrgan, EcogPtDivatt, EcogPtTotal, EcogSPMem, EcogSPLang, EcogSPVisspat, EcogSPPlan, EcogSPOrgan, EcogSPDivatt, and EcogSPTotal). 
\underline{CSF:} Amyloid Beta (ABETA), Phosphorylated Tau Protein (PTAU), and Total Tau Protein (TAU). 
\underline{Risk factors:} Age, years of education, and APOE4.\\
\textbf{Outputs:} A network should automatically create an image containing colorful strips of longitudinal AD assessments where the state of the disease is displayed with different colors, AD: red, MCI: blue, CN: green.
Trajectories of cognitive statues for a 24 month (including baseline T0 and three referral sessions (T1 ($6^{th}$ month), T2 ($12^{th}$ month), T3 ($24^{th}$ month) ) is shown by a sequence of 4 strips in which each strip denotes the disease state and colored as mentioned before. Moreover, the black strip will be appended to the mentioned stripes in order to emphasize the area of \textit{RU} area.  \\
\textbf{Network:} The architecture of the deep learning network has multi-modal tabular inputs and generates a color-coded output image. The size of the output image is 23x23x3 in which the latter 3 channels indicates the \textit{RGB} channels. TThe network is designed so that initial layers concentrate on extracting complex intra-modality features utilizing fully connected layers (FC)(i.e. MRI: 7 $\xrightarrow[]{\text{FC}}$ 14 $\xrightarrow[]{\text{FC}}$ 7  $\xrightarrow[]{\text{}}$ (L3:MRI), PET: 3 $\xrightarrow[]{\text{FC}}$ 6 $\xrightarrow[]{\text{FC}}$ 3 $\xrightarrow[]{\text{}}$ (L3:PET), CSF: 3 $\xrightarrow[]{\text{FC}}$ 6 $\xrightarrow[]{\text{FC}}$ 3 $\xrightarrow[]{\text{}}$ (L3:CSF), COG: PET: 20 $\xrightarrow[]{\text{FC}}$ 40 $\xrightarrow[]{\text{FC}}$ 20 $\xrightarrow[]{\text{}}$ (L3:COG), RF: 3 $\xrightarrow[]{\text{FC}}$ 6 $\xrightarrow[]{\text{FC}}$ 3 $\xrightarrow[]{\text{}}$ (L3:RF) ). Then, feature fusion and feature extraction for the inter-modality phase would be addressed via concatenation and another fully connected layer (concatenation of all L3 layers   $\xrightarrow[]{\text{FC}}$ 100 (L4)). The rest layers are same as the network in experiment 1 and figure 2A. 
\\
\textbf{Training:} The experiment is evaluated by 10-fold cross validation and in each training session 10\% is used as the validation set. The network has 36,143 trainable parameters and is trained for 4000 epochs with a batch size of 500. Early stopping regarding to the value of training loss and validation's loss are also addressed with 500 epochs and 800 epochs patience, respectively. 



\bibliographystyle{unsrt}



\section{Introduction}

When processing and analyzing medical data, machine learning (ML) is proving to be an effective tool capable of producing clinical-grade results that are comparable to the steps taken by healthcare providers in their decision-making process \cite{IntroBook-2019,general_breast_2019, gerenral-nature-hpc, JAMA-general}. By world-wide collective knowledge and different modalities of data, AI could create a super-human physician. This is by no means a man versus machine contest, but its an assistive way and the human intellect collaborating with machines \cite{gerenral-nature-hpc} and human doctors provide dimensions that artificial intelligence is not yet capable of rendering\cite{ghassemi2018doctor}. 
 


In many sectors and applications, including medical settings, assistive AI is preferred over autonomous AI. Integrating the physician or clinician in the decision-making process on cases that are run on automated ML algorithms is key to raising the confidence level in ascertaining a given classification or prediction outcome, and to augmenting our understanding of disease state and its progression in time \cite{gerenral-nature-hpc, holzinger2016interactive}.  Moreover, patients and their relatives, who have a lot at stake, are eager to understand their health status and be confident in the future steps or plan of action they will have to consider moving forward. This type of communication between doctors and patients has made shared decision-making (SDM) highly recommended by the current healthcare guidelines \cite{JAMA-Shared}. 
This SDM concept requires patients and clinicians to work together to better understand the patient's condition and discuss and plan for a treatment in a most effective way. Therefore, it is better that such AI-based algorithms produce outcomes and predictions that are visually appealing and can be perceived intuitively, overcoming the complexity of tabulated data or graphs and bringing added insight into the opacity of the black box effect inherent to machine learning   
\cite{nature_roadmap, theis2016age, valdez2016human, Jama-challenges-ml4h}.


In accordance with the adage \textit{"a picture is worth a thousand words"} \cite{info-vis-book}, this study aims to investigate the prospect for producing novel visual displays of medical/clinical estimations made through machine learning (ML) to be communicated to patients and medical professionals in a concise and straightforward way. As means to gauge the confidence level, another aim of this method is to create possibilities for showing the uncertainty of the ML algorithm per case (i.e. subject or patient data) that has been processed. 
The main idea is to encode the information visually by using colors, intensities, patterns, and shapes rather than using numbers, probabilities, and complex diagnosis labels and graphs with so many overlapping regions. The proposed method named \textit{VL4ML} stands for \textit{visualized Learning for machine learning} and to the best of the authors' knowledge, it is a first-of-its-kind. Figures \ref{fig:intro}A and \ref{fig:intro}B demonstrate the concept of the proposed \textit{VL4ML} estimation approach, in which the machine produces its estimations in a visualized way. 


\begin{figure*}[t]
\centering
\minipage{1\textwidth}
  \includegraphics[width=\linewidth,trim={1cm 4.0cm 1cm 3.5cm},clip]{figures/figure_intro.pdf}
\endminipage
\caption{A) The conceptual demonstration of the proposed \textit{VL4ML} approach. B) Comparing the outputs of the \textit{VL4ML} approach and standard ways with an example of the target outcomes for patients whom their disease status is converted from Mild to Severe at the 4th time-point. C) The circumstance of the proposed approach.}
\label{fig:intro}
\end{figure*}


\begin{figure*}[t]
\centering
\minipage{1\textwidth}
  \includegraphics[width=\linewidth,trim={4cm 8.8cm 5.5cm 4.5cm},clip]{figures/figure_survival.pdf}
\endminipage
\caption{A) The architecture of the network developed for experiment 1, predicting the patient survival in ICU. B) Some results of experiment 1. The left and right images in each sub-figure are the target and achieved output. 
This experiment has two types of target images, the white for survive-ability (a,b,c,g) and orange for mortality (d,e,f,h) along with the surrounding RU area. The last two sub-figures are miss-detected samples. It can be observed that the system’s uncertainty affects the produced output.}
\label{fig:survival}
\end{figure*}

\subsection{Motivation and Advantages of the Generalized Construct}

Figure \ref{fig:intro}C summarizes the ecosystem of the proposed approach and its many application domains in support of its generalized construct. There are several aspects that guide the motivation behind this visualization platform, among them are: \\
\textbf{Faster Brain Analysis}: 
Because the human brain did not evolve with the ability to read and write naturally, we rely on a variety of different visual cues to help us comprehend and analyze information better. There is clear evidence that our brain understands and reacts faster to information or data that is visually relayed \cite{info-vis-1,info-vis-2}. 
Furthermore, it is found that analyzing a single image takes the brain as little as 13 milliseconds, which is much faster than analyzing any other type of information \cite{potter2014detecting}. Since minimising response-time to a risk-prone situation is critical, a method called \textit{FiToViz} was proposed as a visualisation approach the to track and identify abnormalities when a person is under a risk-prone situation. Inspired by a traffic light model of a one-class classifier, they convert the output of the one-class ML classifier to a visualized display \cite{FiToViz}.
\\
\textbf{Enhanced Memorization and Recall Ability}: 
Superior memorization and recall for pictures or images over words is empirically well supported, known as \textit{Picture Superiority Effect}, granted that the behind mechanisms are still a matter of discussion \cite{baadte2019picture}. 
The \textit{National Academy of Sciences} reports that statistically the ability of a person for remembering and recalling pictures is significantly high \cite{grady1998neural} and in a period of several days a human can remember and recall more than 2,000 pictures with at least $90\%$ accuracy in recognition tests even with short presentation times during the learning phase \cite{grady1998neural}. \\
\textbf{Augmented Decision-Making}: 
Visualization can facilitate and even augment decision-making.
The human brain responds to and processes visual data more effectively than any other data type. The brain receives $90\%$ of its information visually, and it is typically much faster at processing images than text \cite{lindsay2013human-book}, meaning that visual-data speeds up informed decision-making.
In addition, studies show that visualization of data/graphical information/results leads to increased confidence and more correct decisions \cite{tang2013effects, cardoso2016graph}. Comprehensive overviews on how visualization affects the decisions are addressed in detail in \cite{padilla2018decision-review, gotz2016data-review} and show widespread use and social impact in many domains including cognitive psychology, information visualization, and medical decision making. \\
\textbf{ Patient Preference Supports Visualization}: 
Factors that relate to a diagnosis or prognosis that is easier to understand and interpret leads to the preference for visualization of data and related results rather than having to deal with complex tabulated data. In light of this, patient-centered point-of-care systems and shared decision-making strategies strive to provide patients with meaningful visual representations of information \cite{patient_pref_ieee, patient_pref_2}.
Enhanced communication between doctor and patient can also be established with the help of visualization. Moreover, the ease for understanding and with the memorizing/recalling ability of information visually relayed would be more effective, entertaining and engaging for patients to collaborate and take part in the decision making process, especially with the pediatric population and their family members\cite{patients_pref_kid_3, patients_pref_kid_5}. \\
\textbf{Visualizing Uncertainty Injected through Machine Learning}:
In decision-making, the ability to observe the nature or the degree of uncertainty injected by the design nature of the machine learning algorithm and the mechanics of its implementation is also essential at figuring the extent at which our results can be relied upon. 
In addition, while the data and outcomes may be discrete and often correct, machine learning techniques are based explicitly on statistical analysis and the population of data, highlighting that uncertainty, out of distribution samples and risk-of-bias always exists for each new case or patient \cite{bias, datashift_uncertainty, Nature_UncertaintyQuanti}. 
This concern always present when using automated algorithms in the era of digital health and being unaware of the so-called black box effect of ML algorithms \cite{bias}. Machine-assisted medical decision-making calls for the quantification of uncertainty for each patient's dataset that is processed \cite{Nature_UncertaintyQuanti}.
Learning with a reject option, also known as rejection learning, has been studied for a long time. When there is a lot of ambiguity about a patient, ML should be able to state \textit{"I don't know"} and abstain from making prediction. In this way, to lessen the ambiguity and make a more informed diagnosis, extra human expertise might be sought (i.e. learning to defer) or additional data can be gathered \cite{defer1,defer2,defer3, defer4, Nature_UncertaintyQuanti}. 
Therefore, to present algorithm outcomes adequately in assistive way to the physicians and patients, ML methods should be able to accommodate the uncertainty introduced by the data or algorithm for each patient/case. 
It was demonstrated that uncertainty visualizing has an impact on decision-making. \cite{Nature_UncertaintyQuanti, riveiro2014effects, spiegelhalter2011visualizing, Uncertainty-Decsion-2017}.
The proposed approach is able to visualize a form of uncertainty and may be used to determine criteria for the quantification of subjective or objective uncertainty. 


\subsection{Contributions}

This work examines the creation of a pipeline that displays clinical estimations visually, embedding all the aforementioned advantages in a most practical visualized form. 
\begin{itemize}
\item In a first-of-its-kind design approach, deep learning and convolutional neural networks can be deployed to produce  clinical estimations visually.
\item As a proof of concept, and to demonstrate the generalized nature of this visualization approach, five different case studies are examined for different types of tasks including classification, regression, and longitudinal prediction. 
\item A survey analysis is conducted to obtain users’ feedback on the merits and benefits of this visualization approach.
\item The source codes for all the examples are shared online to be accessed for validation and research purposes. \textit{\url{https://github.com/mohaEs/VL4ML}}.
\end{itemize}

Thus, this work represents an important step towards an end-to-end pipeline for visualization through machine learning with the ability to deliberate on the obtained results through a visually-enhanced decision making process. 
















\section{Materials \& Methods}

\subsection{Methodology}
The main idea is to produce an image or a visual display wherein colors, intensities and patterns are to express the resulting diagnosis, prognosis and prediction outcomes.
The first step is therefore the design of the visual display itself, which if the ML model worked properly, the ML-derived visual outcome will be exactly the same as the target image which serves as the ground truth (examples are shown in Figure \ref{fig:intro}B).
Additionally, we propose to have a black area with zero values to display and reserve as the uncertainty area (\textit{RU}). Adding such an area would help researchers assess how much the machine learning algorithm has affected such a region. 
It is important to add such an area to assess if the machine learning algorithm does affect such a region and by how much, to mean that the algorithm itself adds some degree of uncertainty in the output. This criterion can be valid for both ML method developing/training and also for testing phase with new input samples. Out of distribution input samples can affect this area as well as the rest of the designed target image.

Next, we need to develop a deep learning network to generate the results in a compelling visualized form as means to overcome the burden of assessing tabulated numerical values, probabilities or categorical labels.
The design of the network is flexible, but we need to embed some layers to produce an output image as well as layers for feature extraction and fusion. In general, in most cases, the network needs to include \textit{forward convolutional layers}   (FCL), and \textit{transposed convolutional layers} (TCL) \cite{dl_cnn}. The output image should be a tensor of size NxNx3, where NxN defines the image size and 3 is to account for the RGB channels.
Obviously if the input data is not in the form of 2D or 3D tensors, reshaping the input into 2D/3D tensor form  \cite{tensorlab} becomes necessary. The network architecture as proposed is flexible, and any feature extraction and fusion technique could be utilized in order to fit the network with different applications or to improve the system's accuracy and performance for the task at hand. As an example, Figure \ref{fig:survival}A shows the network developed for experiment 1, the survival prediction in ICU. The loss criteria between the target image and network's output is the mean of absolute error.  



\subsection{Empirical Evaluation}
Five different experiments were conducted to demonstrate  adaptability of the proposed \textit{VL4ML} approach to different domains of application. Empirical evaluations are conducted to evaluate the performance of the proposed method on different tasks including classification, regression, cross-sectional and longitudinal studies and to assess the  visual displays generated to express the results of classification and predictions from which diagnosis and prognosis can be made for different medical applications. Furthermore, the  input modality as well as the target patterns, intensities, colors, and shapes vary with each experiment in order to reflect the flexibility of this visualization scheme.  
The loss criteria between the target image and network's output is the mean of absolute error and all the experiments are conducted with a  5-fold or 10-fold cross-validation scheme depending on the size of the available dataset. Training is early stopped monitoring the loss of test with 30 epochs patience. The case studies addressed here are as follows: 
\begin{enumerate}
    \item Classification of the patient survival in \textit{Intensive Care Unit (ICU)}: In this case, the tabular data from electronic health records (EHR) are fed to a network to produce visualized survival status using distinct colors in a 23x23 image. This case study is a binary classification application. 
    \item Assessing the \textit{Diabetic Retinopathy} grade: In this case, the retina \textit{Fundus} images are analyzed by a network to estimate the disease grade and image quality by visualizing intensity and colors in a 23x23 image. This experiment involves a multiclass classification as well as a multi-task application. 
    \item Estimating the patient's length-of-stay in a hospital: The EHR data is fed to a network to estimate visually the length of stay in which the amount of colored columns in a 45x45 image represents the length of stay. This is a regression case study.. 
    \item Assessing the severeness of COVID-19: Chest X-ray data is used and the results expressed through colorful circles, green for healthy state and red for disease state, where severity of disease state is proportional to the size of the red circles. This experiment is a both a classification and a regression problem.
    \item Diagnosis and prognosis of \textit{Alzheimer's Disease} (\textit{AD}): In this experiment, a multimodal tabular data are fed into a network for producing a color-coded AD prediction labels in a colored 23x23 image. This experiment is a multiclass classification and prediction in an ADNI (Alzheimer's Disease Neuroimaging Initiative) longitudinal study. 
\end{enumerate}



\subsection{Data}
Only publicly available datasets are used for the experiments and since the characteristics of each experiment is different, the details are mentioned in the corresponding subsections of the result section. 

\subsection{Limitations}
Although the proposed visualization method is practical with a design construct that is adaptable to several medical applications as the results below will demonstrate, there are some limitations to this study. The first is the lack of evaluation scenario. At this stage, we use three expert raters that had to agree on the agreement between the ML outcome and the target image, but outright misclassification/ prediction and ambiguous cases would require expert review of the baseline diagnosis and of the input measures used when making that diagnosis. Although, we illustrate the loss curves of the training and testing phases of each experiments and  confirm the convergence ability and average amount of error between the target and ML visual outcome (Figure \ref{fig:losses}),  the main goal of each experiment is the classification or regression outcome of a clinical phenotype that is presented visually. Since the rater/decision maker is the observer in our pipeline, that would be a subjective decision that could differ between the type of observer (severe/ more open and accepting of subtle change0) and expertise of the observer on the type of disease that is gauged. Thus, we need to figure out a better evaluation criteria for each task type that could include measuring the effect on the region of uncertainty.

Another limitation is the amount of participants which at this time include only 103 persons that responded worldwide to the survey. While literature strongly suggests the superiority (or user friendliness) offered through visualization  (as our our survey confirms), but the outcome of the proposed approach is subject to persons and needs more comprehensive and careful survey analysis as well as considering the target estimation/application which is visualized and analyzed. 


\begin{figure*}[t]
\centering
\minipage{1\textwidth}
  \includegraphics[width=\linewidth,trim={4cm 1cm 5.5cm 1.5cm},clip]{figures/figure_diabete2.pdf}
\endminipage
\caption{Assessing the diabetic rethinopathy in 5 levels. The left and right images in each sub-figure are target and achieved output. A) The higher intensity means healthier. 
B) Assessing the diabetic rethinopathy in 5 colors (inner square) and in 5 intensity levels (peripheral strip). 
C) Multitask scenario in which the quality of input fundus images is also assessed by an upper rectangle (High-quality: White, Low-quality: Dark gray). \\
\footnotesize{a) No DR, b) Mild DR c) Moderate DR d) Severe DR e) proliferative DR. For type I, the last three sub-figures (f,g,h) are miss-estimated samples and for type II, the last two sub-figures (g,h) are miss-estimated samples.}
}
\label{fig:results_retina}
\end{figure*}

\section{Results and Experimental Studies} 


\subsection{Experiment 1 - Patient Survival Prediction in ICU} 

In this experiment, colors are used to discriminate among classes. Using data from the first 24 hours of intensive care, this experiment seeks to predict patient survival. The data used in this example is from \textit{WiDS Datathon 2020} challenge \cite{WiDS, WiDS2} 
which focuses on patient health through data from MIT’s GOSSIS (Global Open Source Severity of Illness Score) initiative \cite{GOSSIS}. 
This dataset contains tabulated data of electronic health records (EHR) of patients. Survival and death are expressed by 0 or 1, and all the other variables, except for identification information are used as input features, creating a feature vector of 183 values.  For the target visualization, the \textit{hospital death} information is used as labels. An image with size 23x23x3 (RGB format) is designed as a target image in which a white square symbolizes a label of 0 (survival) and an Orange square symbolizes a label of 1 (fatal). Also the surrounding pixels are reserved as \textit{RU} areas with zero intensities (black color). 

Figure \ref{fig:losses}A demonstrates the training/testing loss curves and Figure \ref{fig:survival}B shows the results for a few patients of the testing set. The reported cases from (a) to (f) have estimations close to targets. Case (c) is rather interesting, this means that although the patient survived his symptoms and the information fed as input to the machine algorithm indicates that the subject had a small probability of dying but survived. For case (e), the patient had a high probability of dying and did not survive. Cases (g) and (h) are two cases in which the machine's estimation is not correct. Case (g) is estimated to have a high probability of dying but survived and it is rather a false alarm, and the last case (h) had a small probability of dying and unfortunately died.


\begin{figure*}[t]
\centering
\minipage{1\textwidth}
  \includegraphics[width=\linewidth,trim={4cm 8.45cm 5.5cm 4.5cm},clip]{figures/figure_los_covid.pdf}
\endminipage
\caption{The left and right images in each sub-figure are target and resulting output. A) Predicting the length-of-stay via coloring the first $M^{th}$ column (started from left) for LOS value equal to $M$. 
B) Predicting the COVID-19 disease severeness via circles with radius related to severeness. Normal cases (a,b), 4 scales of severeness from (c to f). \footnotesize{The last two sub-figures (g and h) for each experiment are cases with large errors.}
}
\label{fig:results_los_covid}
\end{figure*}


\subsection{Experiment 2- Retina Fundus Image Analysis}

This experiment is a multi-class classification problem for assessing the severeness level of the Diabetic Retinopathy (DR) disease by exploiting a large set of retina images taken with \textit{fundus} photography under a variety of imaging conditions. The investigated dataset is from the \textit{APTOS 2019 Blindness Detection} challenge \cite{APTOS}. The images were gathered from multiple clinics using a variety of cameras over an extended period of time. The associated dataset contains 3,662 retinal images and a clinician has rated each image for the severity of diabetic retinopathy on a scale of 0 to 4: 0-No DR, 1-Mild, 2 - Moderate, 3 - Severe, 4 - Proliferative DR.

All the \textit{Fundus} images are reshaped to 256x256 resolutions while preserving the colors. The desired output is an image with size 23x23x3 and the shape of the output image is an inner square with a narrow peripheral strip surrounding it. The rest of pixels are reserved for the \textit{RU} area using black color. Three different kind of outputs are considered for this experiment. 
\begin{itemize}
    \item Type I: five different intensity levels are assigned for 5 different levels of disease grades. Indeed as shown in Figure \ref{fig:results_retina}A, RGB(1,1,1), RGB(0.8,0.8,0.8), RGB(0.6,0.6,0.6), RGB(0.4,0.4,0.4), RGB(0.2,0.2,0.2) are assigned for showing levels of disease 0 to 4. This type of color rendering is more suitable for individuals with color blindness. 
    \item Type II, five different intensity levels are considered for peripheral strip while the color of the inner square is coded. Indeed as shown in Figure  \ref{fig:results_retina}B, the color of the inner square is bright green (RGB(0,1,0)), dark green (RGB(0.5,1,0)), yellow (RGB(1,1,0)), orange(RGB(1,0.5,0)) and red(RGB(1,0,0)) for 0 to 4 levels of disease. 
    \item Type III (multi-task): this type is an extension of type II, and in this multitask scenario, the quality of the input Fundus image is also assessed. For this reason, the design includes a rectangle whose intensity represents the quality of the Fundus image. As shown in figure \ref{fig:results_retina}C, bright white and dark gray rectangle shows high quality and low quality images, respectively. For this experiment, 800 retina Fundus images of 400 subjects from the \textit{DeepDRiD} challenge dataset is used \cite{Deepdrid}.
\end{itemize}

Figure \ref{fig:results_retina} shows the results for some illustrative cases of the test set and the loss curves of training/testing phases are shown in Figure \ref{fig:losses}B-C. 
Cases (a) through (e) for types I and II, shows successful predictions in comparison to their target ground truth. On the other hand, the network could not estimate the severity of disease accurately for cases (f), (g) and (h). It can also be inferred that color coding makes interpreting the results faster and easier. 
Another interesting observation in application II is the peripheral strip. Using color adds complexity to the network, making it harder to determine whether the exterior colors will match the inner colors. As in case (h), where the model could not create a color of orange in the inner square, it also has a hard time keeping the outer square in gray scale. Furthermore, figure \ref{fig:results_retina}C illustrates some illustrative examples of the multitask scenario in which a rectangle represents the quality of the Fundus image. Experimental results show that the multitasking scenario is also applicable for \textit{VL4ML}.






\subsection{Experiment 3- Estimating the Length-of-Stay in Hospital}

Length-of-stay (LOS) is measured in days between the time of hospital admission and the time of discharge. In hospitals, prolonged stays can be expensive for patients as well as for the health system, so identifying patients at high risk of LOS from the onset of their stay is important.
This experiment is also a regression problem and in contrast to experiment 2 it is not restricted to 5 levels. The goal of this experiment is to predict the length-of-stay for each patient at the time of admission visually and the LOS value would be any number between 0 to 40 days. The \textit{MIT MIMIC-III} dataset  \cite{mimic} is used for this experiment due to its availability to the public in the \textit{Physionet} platform \cite{physionet,mimic_physionet}. There are 52 features selected as inputs, and a 45x45x3 target image is constructed, in which each column corresponds to a day as an indicator of time. (i.e. 45 days). Indeed, if the LOS value is $M$, all columns from left to $M^{th}$ column would be in cyan color (RGB(0,1,1)). The 10 rows on top and bottom are colored as black to reflect the \textit{RU} area.  

Figure \ref{fig:results_los_covid}A shows the results for a few cases of the test set. Cases (a) through (f) shows the same length of stay in comparison to their ground truth which is on their left side, while cases (g) and (h) seems to have an error in their estimations. The training/testing loss curves are shown in Figure \ref{fig:losses}D. 


\begin{figure*}[t]
\centering
\minipage{1\textwidth}
  \includegraphics[width=\linewidth,trim={2.5cm 5.5cm 5.5cm 3cm},clip]{figures/figure_AD.pdf}
\endminipage
\caption{A) Examples of the target images for four different disease trajectories. Ruler shows the dedicated areas to time-slots T0 (baseline), T1 ($6^{th}$ month), T2 ($12^{th}$ month), T3 ($24^{th}$ month) and \textit{RU} area, respectively. Notice that, the ruler is not included in the target image. {\footnotesize a) CN from T0 to T3. b) AD from T0 to T3.  c) MCI progressed to AD at the T3. d) AD from T0 to T3. }
B) Some of the achieved results of the test set in the reported case study for AD progression. The left and right images in each sub-figure are target and predicted output. 
}
\label{fig:results_AD}
\end{figure*}


\subsection{Experiment 4- Severeness Assessment for COVID-19}

The aim of this experiment is a binary classification of COVID-19 disease from chest X-ray images along with the regression task of estimating the severity of the disease. The extent of lung involvement by geographic ground-glass opacity is selected as the severeness score. 
For the COVID-19 cases, data is obtained from COVID-19 image data collection \cite{covidProspective, covidProspective_link}  and normal cases is collected from chest X-Ray Images (Pneumonia) \cite{covid_normal, covid_normal_link}. There are 94 labeled COVID-19 samples with different severity levels with the same number of normal control samples selected. 

The target estimation images is designed such that the normal cases are illustrated by green circles, while COVID cases are shown by red circles where  radius enlargement reflects disease severity.
Figure \ref{fig:results_los_covid}B illustrates the results from some cases of the test set and the loss curves are shown in Figure \ref{fig:losses}E. 
As can be observed, healthy cases of (a) and (b) are correctly identified as normal cases using a green filled circle, and the unhealthy cases of (c) to (f) are recognized as subjects that are affected by different severity stages of COVID-19. Cases (g) and (h) are two examples where the network could not make a correct decision and also had difficulty tracing a circular pattern for the resultant shape. There are two important observations here. The first one is related to the healthy cases. While the network has two degrees of freedom for presenting the shape, which is color and radius, for healthy cases, it is restricted to color only. Meaning that if the network interferes with the size of a green circle, this should be interpreted as an error. The second important point is the degree of sharpness in the perimeter of the circle. Cases (e) and (d) are an example of this scenario, where the blurriness of the resultant circle could be affected by the uncertainty introduced by the system.  





\subsection{Experiment 5- Alzheimer's Disease Classification and Progression}

In this experiment, we investigate a multiclass classification and prediction of disease progression in Alzheimer's disease in a longitudinal study. The ML model is to generate automatically a visual display with different colored stripes to express the different states of the disease.  The colors chosen in this case are as follows:  \textit{Alzheimer's Disease (AD)}: red, \textit{Mild Cognitive impairment (MCI)}: blue, \textit{Cognitively Normal (CN)}: green.
Therefore, as shown in \ref{fig:results_AD}A, trajectories of cognitive statues over 24 months (including baseline T0 and three referral sessions (T1 ($6^{th}$ month), T2 ($12^{th}$ month), T3 ($24^{th}$ month) ) is demonstrated by a sequence of 4 stripes in which each strip denotes the disease state by its corresponding color. Stripes are followed by a black strip to denote the \textit{RU} area. Figure \ref{fig:results_AD}a shows four examples of desired target images for four different subjects with four different longitudinal trajectories.  

The clinical data used in this experiment is obtained from the \textit{Alzheimer's Disease Neuroimaging Initiative} (\textit{ADNI}) database (\href{http://adni.loni.usc.edu/}{http://ida.loni.usc.edu}).
Only the subjects that have a baseline scan and showed up for follow-up visits at 6, 12 and 24 months later, have been considered in this longitudinal study.
A total number of 1043 subjects met the follow-up condition, and were categorized into 3 classes AD, MCI, and CN, in baseline and referral sessions.
Multimodal tabular data containing features extracted from \textit{MRI} and \textit{PET} sequences, as well as \textit{demographic information} and \textit{cognitive measurements} were used as input features as in our prior works  \cite{neuroimage_sola, frontiers_sola}. 



The training/testing loss curves are plotted in Figure \ref{fig:losses}F. Figure \ref{fig:results_AD}B demonstrates some of the achieved results of the subjects in test set, including a visual representation of the disease trajectory predicted by the network. Furthermore, the method seems to provide a more realistic outcome for patients, in particular for longitudinal analysis. Fro example, while all of the cases (g),(h),(i), and (j) are for patients with steady AD status, the estimated results are different and interesting. Case (g) is diagnosed and labeled correctly, while cases (h) and (i) show different disease progression, where the ML model show that subject in (h) was MCI before it converted to AD at T1, and patient in (i) was also MCI that converted to AD around T2. Case (j) is a misdiagnosed case with moderate certainty. 
These are illustrative cases that are complex and are shown here to emphasize the challenges faced when dealing with multimodal classification and regression. 

\begin{figure*}[t]
\centering
\minipage{1\textwidth}
  \includegraphics[width=\linewidth,trim={3.5cm 1.5cm 3.5cm 1.5cm},clip]{figures/survey.pdf}
\endminipage
\caption{Addressed survey analysis. A) the distribution of people participated. B,C,D) Three exemplary questions and achieved results. The full survey and results are available in supplementary materials.  
}
\label{fig:survey}
\end{figure*}


\begin{figure*}[t]
\centering
\minipage{1\textwidth}
  \includegraphics[width=\linewidth,trim={1cm 3.5cm 2.5cm 1.5cm},clip]{figures/figure_losses.pdf}
\endminipage
\caption{The loss curves of training and testing sessions of each case study and experiment for all the folding sessions.}
\label{fig:losses}
\end{figure*}




\section{Survey Analysis}
An online survey has been created with the Qualtrics platform for collecting users' feedback on the practicality of this visualized estimation approach. The objective is to assess user preference in the ability to interpret and understand the visual display. It is also an assessment on how easy it is to remember the visual display and how easy or difficult in making a decision, especially when the outcome is ambiguous and may be infused with some degree of uncertainty. Few initial samples were given to participants to familiarize them with the meaningfulness of the visualization format and its color coding. Ten questions related to three application of survival analysis, disease progression estimation and multitask disease severity estimation were posed in this survey. 
We shared the survey on Facebook Ads and LinkedIn, and 103 individuals responded to the survey. 
Figure \ref{fig:survey}a shows the distribution of collected votes with a few sample questions and the results of the survey. Three exemplary questions and their corresponding results are also shown in Fig. \ref{fig:survey}. The full survey and results are available in supplementary materials. 
Among those who responded to the survey, 51.5\% were female, and the remaining 47.47\% were male. A large majority of participants (81.55\%) were between 25 and 44 years of age, with 15.5\% of the participants identified themselves as being healthcare professionals or working with the medical industry.
 

In the application of survival analysis, 95.15\% declared that understanding the visualized format is easy or somewhat easy to remember. Among the respondents, only 18.63\% preferred the numerical format while 83.47\% preferred the visual format. \\
With regards to the application of determining the progression of a disease, 83.49\% of the voters agree that the visual representation is easy to learn and to interpret, with 79.55\% stating that they would prefer to receive the results in visual format. With an overwhelmingly positive vote of 82.35\%, the ease for memorizing/remembering the outcome was favored through visualization. Moreover, 73.79\% voted that the visualized format speeds up the decision-making process. As for the level of uncertainty, 81.65\% stated that different level of uncertainty is visible in the visualized format. \\ In the multitask case study, again the majority of the participants (87.38\%) voted that it is easy to interpret the meaning of what was displayed to them. On the question of interpretability, only 12.62\% thought that this format was hard to interpret. And finally, on the uncertainty level, only 9.71\% stated that they were not able to discern any  level of uncertainty. 






\section{Discussions}
\label{sec:discussion}

In this study, a visualized learning method named (\textit{VL4ML}) is proposed to depict a given medical outcome as means to asses disease state and disease progression by means of classification or regression analysis. 
The \textit{VL4ML} method is based on deep convolutional neural networks and five different experiments have been conducted to demonstrate the feasibility of the proposed method and its generalized design construct that is capable of dealing with different experiments involving tasks of multimodal classification, regression analysis and predictions. Various visualization criteria, including color, intensity, shape, size and pattern encoding, have also been considered. 
According to the survey results, the visualized form is considerably in people's preferences.  
Figure \ref{fig:survey}B shows an exemplary experiment over two cases that are diagnosed with progressive AD, where the outcomes of our visualized approach is compared with the numerical results and probability values from conventional methods. 
Comparing the top image with numerical values on the bottom demonstrates that outcomes of the \textit{VL4ML} present different levels of information, sensation, and memorization impact that could be conveyed to humans and exploited when making a decision as to the visual outcome that is being evaluated.
Furthermore, according to survey study results, over 80 percent of them can detect some level of uncertainty in outcomes. 
\textit{VL4ML} embeds within it a display of the uncertainty level when making a decision, especially when the visual display provided through machine learning is different than that of the target image.

Therefore, the potentials of this novel approach in delivering a medical diagnosis or a prediction through a visual format is promising. Its simple color-coded format is seen through the survey as highly effective and informative when it comes to understanding the meaningfulness of the display and as an alternative approach for medical experts to communicate with their patients the results that were obtained, leading to a shared decision-making process between doctor and patient. 





\subsection{Future Research Directions}

While this study shows the capabilities and advantages of the proposed approach, in addition to the mentioned limitations, more interesting questions could be asked, e.g., what would be the best shape of the \textit{RU} area and its location regarding to the task types, clinical applications, and machine learning aspects? What is the best image size and the color/shape/pattern to be used for each clinical application with respect to human impression and decision-making and also with respect to gauging the accuracy of the machine? What would be the best network architecture? and more importantly how to quantify the region of uncertainty in both subjective and objective ways to unlock the mystery behind the blackbox effect of machine learning? Is uncertainty quantification independent of the designed target images and its color/shape/pattern chosen? Is checkerboard artifact \cite{checkerboard} in the output images helpful in quantifying the uncertainty, or reversely should we try to compensate  this effect through an inverse transformation? And some more questions. 

Designing contextual target images that define disease state or disease trajectory within this visualization context is flexible and is left to the imagination of other research groups that would like to replicate such work or create more sophisticated displays in relation to the disease or medical application considered (e.g. image size, color codes, shapes, etc). As an example, Figure \ref{fig:future}A shows the AD experiment for prognosis under different lifestyles (\textit{e.g.} active, rural, ...) \cite{lifestyle} and treatment plans (\textit{e.g.} drug 1, drug 2, ... ) \cite{treatment}. Fig. \ref{fig:future}B shows an example of this potential  work in which the vertical axis is also quantitatively meaningful.
But, the availability of network models, accuracy's and user's preferences should be considered. 

\begin{figure}[t]
\centering
\minipage{0.5\textwidth}
  \includegraphics[width=\linewidth,trim={14cm 10.5cm 6cm 4cm},clip]{figures/figure_future.pdf}
\endminipage
\caption{Potential future works.}
\label{fig:future}
\end{figure}



\section{conclusion}


In this paper, \textit{VL4ML}, a first-of-its-kind visualized estimation approach is offered to showcase how medical estimations under different domains of application can be assessed visually as means to gauge  disease state and disease progression by means of classification or regression analysis. 
The practicality of the suggested approach is investigated by creating deep convolutional neural networks-based networks.
In support of the flexibility afforded through this new design construct that integrates visualization to machine learning, five different experiments were explored to illustrate the ability of this generalized construct to deal with a variety of experiments under different domains of application. Color, intensity, form, size, and pattern encoding have all been examined as the potential visualization criteria. According to the results of the survey study, individuals prefer the visualized form and find it to be an easier and faster assistive  decision-making approach as well as more memorable and easier to recall. In addition, the majority of participants believe that the amount of uncertainty infused by the ML algorithm is perceivable. 





\section*{Acknowledgments}
This work was supported by the National Science Foundation (NSF) under Grants CNS-1920182, CNS-1532061,
CNS-1338922, CNS-2018611, and CNS-1551221, and NIH (NIA 1P50AG047266-01A1) through the Florida Alzheimer’s Disease Research Center.


\section*{Author contributions statement}

M.E. and S.T. are the ones that came up with the concept and  started writing the manuscript.
Required funds obtained by M.A. and T.E.. 
M.E. developed the codes and experiments. 
All of the authors discussed the findings, offered their  perspectives, shared their ideas and comments, and critically reviewed the manuscript.
M.E. and S.T are responsible for all aspects of the work.


\section*{Additional information}

The authors declare no significant competing financial, professional, or personal interests that might have influenced the performance or presentation of the work described in this manuscript.


\section*{Data Availability}
All the data used in experiments are from research-purposed publicly available datasets which are cited through the manuscript.


\section*{Computer Code Availability}
The source codes of the experiments are provided in \href{https://github.com/mohaEs/VL4ML}{https://github.com/mohaEs/VL4ML}. 







\bibliographystyle{unsrt}
