
\section{Conclusion}
Visualizations have the power to transform data-oriented practices in field work by bringing data into the field. 
This integration can bridge spatial and temporal gaps by integrating data collection and analysis.
We explore how mobile and immersive visualization could transform field analysis practices by bridging these gaps through integrated data collection and visualization platforms.
We develop preliminary insight into how visualizations might support this domain in interviews with 10 experts from 5 different domains. Our results allowed us to elicit design guidelines and open questions for data analysis systems for fieldwork.
We instantiate this feedback in FieldView, an open-source prototype platform for the mobile and situated visualization of field-collected data.
This research provides preliminary steps \review{towards characterizing the novel design space of visualization for field analysis.}



\section{FieldView} \label{sec:design_iteration}
\begin{figure}[t!]
	\centering
	\includegraphics[width=\linewidth]{FieldViewUseCase}
	\caption{We \review{iterated on our design probe using expert feedback to create the FieldView prototype}.
	
		FieldView shows how visualization can support three use cases from our interviews: localized team coordination, data quality validation, and data fusion from heterogeneous sources.}
	\label{fig:UseCases}
	\vspace{-5px}
\end{figure}
\review{Our interviews revealed design recommendations and key scenarios for how visualization can enhance field practices.} 
To provide a proof-of-concept for how visualization systems might \review{use these recommendations},
we developed FieldView, an open-source extensible prototype, on top of our design probe's base implementation. 
\review{FieldView extends the probe's data collection functionality through an intermediate cache on a portable server to support collocated teaming under limited connectivity (see \S \ref{teaming}) and basic sensor data integration (see \S \ref{fusion}). It includes situated geospatial AR visualizations designed for three operational tasks noted in our interviews: 
collocated teaming, data quality validation and refinement, and autonomous sensor integration.}
As there is little empirical guidance for immersive visualization design (see Marriot et al. for a survey \cite{marriott2018immersiveCH2}), 
these use cases demonstrate how situated analysis may enhance field operations and decision making and illustrate novel capabilities of our system 
to support field analysis (Fig. \ref{fig:UseCases}). 

Case 1 and Case 2 simulate multiple analysts collecting scorch rate data, drawing on a use case from 
our interviews with ecologists (``sampling what percentage of needles are scorched''\emph{(P6)}). As we were unable to revisit the original fieldsite, we retargeted the geocoordinates of the scorch rate data to a local fieldsite.
Case 3 simulates a desired ``synthetic vision'' (\textit{P1}) provided by seeing the physical environment with temperature data and drone-collected imagery \review{provided by our earth science collaborators and again retargeted to a new fieldsite}. 
We published these use cases as part of an open-source system designed for use with these cases and readily extensible to new data and devices based on the users' operational needs (\url{https://cmci.colorado.edu/visualab/fieldview}). 

The Gear VR lacked robust tracking,
limiting our ability to integrate data into the environment. While these capabilities can be gained through additional sensing devices, many recent headsets have these capabilities readily available. 
Our case studies use the Microsoft Hololens with freehand gestures for 
\review{interaction} as \emph{P1} reiterated challenges with gloves while in the field inhibiting touch interaction.
\review{However, as with the design probe, FieldView does not address the challenges of entering data on a mobile device while wearing gloves.}
Our modular architecture (Fig. \ref{fig:InitialSchematic_new}) allows analysts to extend FieldView to specialized use cases and technologies---such as other AR headsets or more sophisticated fusion and analytics algorithms---by changing the data source and schema and integrating custom analysis processes and visualizations.

\subsection{Case 1: Team Coordination}
Interviewees discussed the potential for the data collection and immersive visualization components of the design probe to support team coordination
by aggregating data collected by multiple team members (\S\ref{teaming}) and enabling a shared situational awareness across sites.
While remote fieldsites may complicate use of mobile devices and cloud infrastructures, our public safety and aerial firefighting experts were willing to carry lightweight
portable servers that could establish a local connection between collocated teams and mentioned using ATAK\footnote{http://syzygyintegration.com/atak-android-tactical-assault-kit/} servers for basic data exchange. 
\review{In this use case, field analysts can enter and synchronize data with local teams while offline (Fig. \ref{fig:LocalTeamCoordination}).}

\begin{figure}[t!]
	\centering








	\includegraphics[width=\linewidth]{case1_lg}
	\caption{Use Case 1: Team coordinators can assess coarse-grain coverage at a glance on the mobile device (left). They can survey team responsibilities and validate collected data (redder data points indicate greater scorch percentage) using a contextualized scatterplot (top right) and \review{compare new data to archival data (bottom right).}}

	\label{fig:LocalTeamCoordination}
	\vspace{-5px}
\end{figure}
		
A local Flask server is hosted on a Raspberry Pi (23 grams) and connects analysts to a local network.
Team members use FieldView's data collection interface to add new scorch rate samples alongside image references drawn from archival data. 
\review{All analysts connected to the server can add new data to a local database that synchronizes with the server's intermediate database while within range, allowing the entire team to view data collected within range of the server even when no internet connection is available} \review{(\S \ref{teaming})}.
\review{When a team member enters a new scorch measure, the visualization updates and all team members connected to the server have access to the new data.}
Team members engaged with the mobile platform can quickly view coverage of a surveyed area on the mobile application's map view, 
while analysts engaged in immersive analysis can see incoming data overlaid on the field (Fig. \ref{fig:LocalTeamCoordination}). 
Our overlay visualization uses GPS data from the analysts' mobile device and the HoloLens' internal depth sensors to localize data over an explicit collection grid, supporting stratified sampling \review{(\S \ref{locationData}).}
Outdoor and large-scale tracking issues with current AR headsets and mobile GPS granularity can limit the utility of this approach, but analysts mentioned they generally use more precise GPS devices that help mitigate these limitations. 

\review{Analysts first explore the 

	mobile visualization to get an overview of data coverage with a simple heatmap of the sampled area.
They can move to a relevant field location and launch the immersive visualization to see where data was collected within that site and to map teams in the field to operational goals.}
Within the immersive view, analysts can visualize scorch rates using an embedded scatterplot applied to the field grid with points color mapped to scorch intensity values (Fig. \ref{fig:LocalTeamCoordination}). This approach minimizes the amount of space consumed by the visualization while contextualizing data in the environment and supporting stratified sampling practices.
Analysts can select collected data points \review{via freehand gestures} to display archival image data collected at that location and the \review{team member} who uploaded the data on demand. \review{As new data is uploaded, older data from the same grid location forms a smaller point underneath the most recent measure to track changes over time.}

This prototype extends current field analysis capabilities to address three design considerations identified in our interviews.
First, the dynamically-updated gridded scatterplot paired with  archival data increases the local team coordinator's awareness of the team's operations across the field site
(\S\ref{locationData}).
With an updated data model and contextually relevant data situated in the physical environment, field analysts can visualize coverage and synchronize efforts across limited human resources in real time. 
Second, 
new data is combined with preplanning data in real time, allowing analysts to update and extend data in existing databases based on mobile inputs (\S\ref{fusion}).
\review{Finally, the simple, embedded scatterplot provides at-a-glance insight into collected data to allow the team to adjust and adapt their operations in response to incoming data. }

\subsection{Case 2: Data Quality Validation}
\begin{figure}
\centering
	\includegraphics[width=\linewidth]{MissingData_lg}
\caption{Use Case 2: 

	Scorch data is visualized in a scatterplot embedded within the field sampling grid with one point per grid to encode scorch rates, mitigating occlusion of the surrounding environment. Blue regions within the grid allow analysts to rapidly identify gaps in data coverage.}
\label{fig:MissingData}
\vspace{-5px}
\end{figure}

Incomplete and poorly collected field data costs analysts time and money, but
is difficult to detect while in the field despite analysts being well-positioned to correct the errors. 
Automated methods for detecting these errors are limited, and current mobile visualizations do not provide sufficient resolution to effectively detect most errors in the field. 
Further, understanding the context of collected data can enable rapid identification and correction of data collection errors. 
For example, detecting anomalies while in the field can call analysts' attention to sensors that may be occluded due to weather or dirt or that may be improperly configured. 
By visualizing data in the context of the operational area, field analysts can identify and rectify these errors as they occur, leading to
substantial savings in both time and budget.    

\review{Analysts can validate field data in FieldView at a global scale using a mobile map overview visualization (Fig. \ref{fig:LocalTeamCoordination}) to identify regions with missing or unusual measures. They can then move to that region and} 
locate 
missing data highlighted on a AR detail visualization (Fig. \ref{fig:MissingData}).
FieldView highlights gridcells without collected data in blue \cite{songs}, while existing data are encoded in the gridded scatterplot from \S6.1.
\review{When the analyst enters data for the missing region in the mobile app, they see the blue highlight replaced with an appropriately color-coded datapoint in real-time.}
When coupled with the offline teaming infrastructure from \S6.1,  this functionality allows analysts to coordinate data validation efforts with team members performing distributed data collection.

\review{Analysts can also use FieldView to explore patterns across an environment to look for anomalies that might suggest stale data.}
For example, analysts can survey scorch rate data across the fieldsite and compare the distribution of sphere colors against the 
expected data distribution of the visible environment to find inconsistencies.
Once those inconsistencies are located, analysts can walk directly to any problematic areas, assess the source of the inconsistency (e.g., by using imagery data from Case 1), and update data. 


This use case addresses \review{data quality assessment tasks mentioned by all participants:}

data completeness and correctness (\S\ref{quality}).
Visualizing data within the environment allows analysts to readily bring expertise and contextual information to bear on on-going data collection to identify and remedy simple anomalies. Future extensions of these approaches could integrate automated or comparative solutions into quality analysis. For example, our modeling expert envisioned integrating these approaches with automated processing algorithms used by glacial researchers to call attention to salient irregularities in their data. Forest ecologists envisioned direct visualizations of statistical power metrics overlaid on the physical environment to assess anomalies based on data density and other measures.
Our approach enables analysts to rectify errors by providing the means to identify the presence of anomalies in mobile overviews and detailed investigation using IA.

\subsection{Case 3: Data Fusion Across Perspectives}

\begin{figure}[t!]
	\centering
	\subfloat[Overhead view of distant imagery with a wedge object as for geospatial reference]{\centering \includegraphics[width=0.47\linewidth]{DataFusionDistant_July_lg} \label{fig:DataFusionDistant}}
	\,\,\,\,
	\subfloat[Overhead view of close range imagery with a dotted dropdown object for geospatial reference]{\centering \includegraphics[width=0.47\linewidth]{DataFusionNear_lg} \label{fig:DataFusionNear}}
	\caption{Use Case 3: Visualizing geotagged sensor data (temperature encoded by color) and imagery
	
		allows 
	
		analysts to leverage autonomous sensing data in real time.
		We complement situated visualizations with a heads-up display (HUD) in the left and right peripheries to help find datapoints beyond the line of sight.}
	\label{fig:DataFusionUseCase}
	\vspace{-5px}
\end{figure}

Autonomous data collection is increasingly central to field data \review{and can help mitigate data entry challenges due to bulky equipment such as gloves}.
Aerial firefighters want to extend their own vision with imagery captured by drones and supplement that imagery with additional measures collected using on-board sensors; earth scientists wish to examine a forest canopy from both the air and the ground
 (\S\ref{fusion}).
Analysts can use FieldView to explore autonomous data both within the field of view and from nearby devices. We demonstrate these visualizations using a dataset of overhead imagery from drones used by our earth science collaborators during a recent field operation coupled with synthetic temperature data. 
Field analysts can use these visualizations to increase their situational awareness using data from autonomous devices and to coordinate action between human and robot teams.

\review{Analysts can visualize distant imagery through an adapted Wedge technique \cite{gustafson2008wedge}.

Within the immersive visualization, analysts see drone data \review{collected within their field of view} as scatterplot points positioned at the points of collection, with color of the image border and wedge object encoding temperature using a red-blue color scale interpolated in CIELAB.
They can then select a point via freehand gesture to see the source image, which connects the image to its physical referent using a color-coded wedge

\review{\footnote{Stereo viewing in AR-HMDs allow these points to appear at the point of collection, but positional depth cues are lost when rendered to a 2D figure.
	
	} } (Fig. \ref{fig:DataFusionDistant})}.
\review{To avoid information overload and keep the center of the analyst's view unoccluded, images render on either the left or right side of the field of view.}

\review{To provide more comprehensive awareness of local conditions, analysts can also view data from devices outside their immediate field of view}
\review{

	via a heads-up display (HUD) along left and right periphery.}
To keep these visualizations simple (\S\ref{quality}) and to further minimize any unnecessary occlusion (\S \ref{requirements}),
data in the HUD is rendered using a variant of SidebARs \cite{siu2013sidebars,bork2018towards}: data from each collection site is visualized as a semi-transparent point, with points projected along a vector from the measurement point to the viewer. 
The color of each point corresponds to the current temperature reading. 
This design mirrors the needs of wildland firefighters who note that the intensity and spread of a wildland fire can change quickly and having data from remote sensors will enable them to react to changes in the environment.

Our interviews found that data fusion with both human-collected and autonomous sensors providing
multiple perspectives on a fieldsite is 
key for increasing situational awareness beyond current capabilities.
Our design choices aim to balance simplicity and situational awareness for field analysts engaging with automated sensors. 
While our approach builds on techniques for visualizing data beyond the immediate field of view, future techniques could more deeply consider how visualizations could enable collaboration between analysts and autonomous systems. 
\section{Introduction}

Data-oriented decision making is transforming practices in a broad variety of domains. 
Applications in earth science, geology, and emergency response all leverage data collected in the field to help describe the state of complex environments.
Field analysts collect data to model the changing state of field sites over time and to share information across teams to increase situational awareness and deepen scientific and operational understanding.
However, current practices for working with data in the field rely heavily on decoupled solutions that separate data analysis from the spatial and temporal contexts that data describes. Data is collected in the field but analyzed elsewhere, such as in operations centers or remote laboratories.
This separation limits analysts' abilities to use this data in on-going operations: remote analysts lack the context and local situational awareness of people in the field, while people in the field lack tools to access, analyze, and act on data while in the field. 
However, bringing analytics tools into the field creates new design challenges for data analysis systems. This work explores challenges and opportunities for using visualization to support data-driven decision making in the field. 

The lack support for analytics in the field means that decision making typically occurs in remote operations centers either during the operation or at a remote site at the end of the operation. 
Current practices for fieldwork require analysts to first preplan their operations based on data from previous collection efforts and archival data streams. Analysts then collect new data either on mobile devices or in field notebooks, physically transport data to a central location to synchronize
with other sources, and replan subsequent collection efforts and operational practices based on the newly revised data.This workflow decontextualizes collected data from the environment, prevents analysts in time-critical situations from reacting to new data, and obscures possible errors in data collection by creating spatial and temporal gaps between data collection and analysis (Fig. \ref{fig:SpatialTemporal}).
Spatial gaps in understanding arise when remote analysts lack the physical context 
surrounding field data, resulting in a lack of visual grounding for remote analysis and a lack of shared context during real-time operations. 
Temporal gaps stem from an inability to analyze data during an operation, resulting in stale data being used in dynamic environments, an inability to make timely decisions, and the potential to jeopardize a day's work or even the entire operation due to data quality issues and reduced global awareness of the state of the broader operations.


We explore how mobile and immersive technologies can help bridge spatial and temporal gaps between data collection and analysis in fieldwork. Specifically,
we examine how coupling these technologies might increase contextual and situational awareness in field analysis practices in order to improve data collection, sharing, analysis, and decision making by better connecting field analysts with their
data. 
Mobile technologies offer portable connectivity to access and synchronize data while in the field and can offer at-a-glance overviews that may help summarize current state of the operation.\todo{cite Brehmer's mobile vis paper from VIS last year} However, mobile technologies suffer from several limitations that may prove challenging for field use. For example, interacting with mobile devices is challenging while wearing gloves and forces analysts to divide their attention between the visualization and environment. Immersive visualization techniques can resolve these issues by visually embedding data in the visual environment \todo{cite situated analysis paper}; however, we have limited insight into how to design effective augmented reality visualizations. Our goal in this paper is to elicit insight into how visualization tools can be designed to effectively support the needs of field data analysis to provide preliminary steps towards a deeper integration of data into field operations. 

This paper presents an exploratory investigation into how visualization tools could support increased situated awareness and optimize data-driven decision making in the field.
Building on preliminary conversations with field workers in domains ranging from ecology to emergency response, we created an interactive design probe architected to support an integrated data collection and visual analysis workflow through a combination of mobile and immersive visualization tools. 
We use this probe to elicit formal recommendations for visual analytics tools for field work and develop a preliminary prototype based on these recommendations and three target scenarios identified during the interviews where a lack of access to field data causes significant operational challenges. Our results suggest new opportunities for leveraging visualization to empower field operations through data. 

\begin{figure}[t]
	\centering
	\includegraphics[width=\columnwidth]{SpatialTemporal}
	\caption{Temporal and spatial gaps hindering decision making in the field. The temporal gap is characterized by an inability to make decisions until after concluding operations in the field. The spatial gap refers to challenges introduced by removing collected data from the physical environment, reducing local situational awareness during decision making.}
	\label{fig:SpatialTemporal}
\end{figure}

\textbf{Contributions: }\begin{comment} We introduce a novel field data collection and analysis workflow 
that enables situated decision-making in the field. 
We conducted interviews with field analysts in domains 
such as environmental science and aerial firefighting to understand limitations in current field data use and synthesize critical design requirements for 
real-time data analysis tools. 
We implement these requirements in
an extensible prototype 
leveraging commodity technologies to support field analysis. 
Building on scenarios identified by analysts, we ground our prototype in three use cases 
demonstrating how our 
approach may address limitations in field analysis to increase operational efficiency.
\end{comment}
We establish preliminary design considerations and constraints for integrated data collection and analysis tools in the field.
We first worked with scientists and public safety officials to identify challenges in current field data practices. 
We use these limitations to develop a design probe demonstrating how mobile and immersive technologies might overcome these limitations.
 We use this probe to scaffold interviews with 10 domain experts about the potential of complimentary mobile and immersive visualizations to transform field operations.
After synthesizing these interviews, we refined the workflow introduced in the design probe to develop FieldView, an open-source prototype system to address use cases inspired by common themes and scenarios identified by analysts (Figure \ref{fig:teaser}).


\begin{comment}

\section{Introduction}
Data-oriented decision making is transforming practices in a broad variety of domains. Applications in earth science, geology, and emergency response all leverage data collected in the field to help describe the state of complex environments. 
Quantifying these environments allows analysts to model the changing state of the world over time and to share information across teams to increase situational awareness and deepen scientific and operational understanding. 
However, current practices for working with data in the field rely heavily on decoupled solutions that separate data analysis from the spatial and temporal contexts that data describes: data is collected in the field but analyzed elsewhere, such as in operations centers or remote laboratories.
This separation can limit field analysts' abilities to integrate current data into analysis and decision making practices, introducing significant time and efficiency costs. 

We address this challenge by exploring how mobile and immersive technologies can help bridge spatial and temporal gaps between data collection and analysis in fieldwork. Specifically, we explore how we can couple these technologies to increase contextual and situational awareness in field analysis practices in order to improve data collection, sharing, analysis decision making with relevant visualizations. We focus on how mobile data collection, cloud storage and fusion, mobile device visualizations, and augmented reality (AR) visualizations enable analysts to visualize data as it is collected in order to integrate contextual information about the immediate environment into data interpretation and operational planning.
Our approach fuses data from multiple sources to provide field analysts with access to new and existing data from other teams in the field and archival sources.
Analysts can then leverage at-a-glance mobile visualizations and detailed immersive interactive visualizations to develop new hypotheses, identify potential errors or gaps in coverage, and adapt operational practices while in the field.

Current practices for fieldwork require analysts to first preplan their operations based on archival data from both previous collection efforts and archival data streams.
Analysts then collect new data either on mobile devices or in field notebooks, physically transport data to a central location to synchronize with other sources, and replan subsequent collection efforts and operational practices based on the newly revised data. 
This workflow decontextualizes collected data from the environment, prevents analysts in time-critical situations from reacting to new data, and obscures possible errors in data collection. 
These challenges are exacerbated when data comes from multiple sources: analysts have limited access to additional data collected from other field teams or independent sensors. 
Our proposed workflow leverages cloud datastores to integrate data from multiple sources as data becomes available, leveraging both local and global data storage to overcome connectivity limitations in the field.
We additionally extend existing practices for field data analysis by allowing analysts to conduct rapid inference using mobile visualizations or immersive embedded visualizations that contextualize data within the physical environment and leverage visual context to support nuanced analyses.
This workflow increases contextual awareness to improve data-oriented operational goals, provide real time data quality assessment, and support synchronous operations between field teams. 

We implement and evaluate this approach in FieldView, an extensible mobile data collection and visualization \ds{prototype}
designed to bridge the spatial and temporal gaps between data collection and analysis in the field. 
We developed FieldView in collaboration with field analysts in domains ranging from forest ecology to public safety. 
We discuss how our approach addresses several challenges in field analysis and refine our approach 
using observations from ten domain experts. We demonstrate our approach using three scenarios for field analysis inspired by our discussions with field analysts and unsupported by existing systems: data-mediated teaming, data quality verification, and
fusion with autonomous sensors. By coupling mobile data collection and immersive analytics, our approach can foster improved data collection, interpretation, and decision making by providing field analysts with increased access to incoming data and allowing analysts to interpret that data with respect to aspects of the environment only available in the field. 

\textbf{Contributions: } We introduce a novel field data collection and analysis workflow 
\ds{that enables situated} decision-making in the field. 
We conducted interviews with field analysts in domains 
\ds{such as} environmental science and aerial firefighting to understand limitations in current field data use and synthesize critical design requirements for 
real-time data analysis tools. 
We implement \ds{these requirements} in
an extensible prototype 
leveraging commodity technologies to support field analysis. 
Building on scenarios identified by analysts, we ground our prototype in three use cases 
demonstrating how our 
approach \ds{may} address limitations in field analysis to increase operational efficiency.










\end{comment}
\section{\review{Qualitative Study:} Designing for Field Analytics}


We used our design probe to ground semi-structured interviews with 10 field analysts to understand how visualization systems can bridge spatial and temporal gaps in field data practices. 
Interviews ranged from 1-2 hours and were conducted in five unique sessions.
Six interview subjects were in emergency response domains,
two were earth scientists in forest ecology, one was a computer scientist using field data for climate modeling, and one specialized in robotic sensing with drones.

We first conducted a step-by-step interactive demonstration of the design probe's workflow with participants, starting with an overview of functionality. 
Then, we walked participants through example usage in field operations\review{---assessing terrain using the geospatial altitude dataset described in \S \ref{sec:InitialPrototype}---}interacting with the mobile application and immersive visualizations throughout this walk-through. 
We collected transcripts of comments during the participants' initial interactions with the probe and, following the demonstration, asked a series of interview questions.
Our interview questions (provided in the supplemental materials) focused on analysts' backgrounds; current use of data in the field including collection, storage, and analysis; responses to the design probe; and envisioned uses for field analysis based on the probe.



We analyzed the transcribed interviews using thematic analysis to identify common practices in field analytics across different domains that visualization systems
can support. 
We found that, though our participants ranged greatly in technical expertise and domain focus, they identified several common themes in potential uses for field visualization.
In this section, we discuss these themes (summarized in Fig. \ref{fig:Feedback}), how analysts envision using the design probe to support needs expressed in each theme, and how our approach could be extended to better support field analysis.
\review{Though we interviewed only 10 domain experts, we found common themes across groups that spoke to critical opportunities for visualization to enhance field practices.}

We elicit design recommendations and critical use cases for technology-mediated field analysis from these interviews.
\begin{comment}
\begin{table*}[htbp]
	\caption{Feedback on Initial Prototype}
	\label{tab:schema}
	\begin{tabular}{c | p{1.45in}p{2.55in}p{2.55in}}
		\toprule[1.5pt]
		& \textbf{Observation} & \textbf{Quote/Example} & \textbf{Design Recommendation}\\
		\midrule[1.5pt]
		\multirow{2}{*}{\rotatebox[origin=c]{90}{\textbf{Loc-based}}} & Prominent use of Geospatial Data 
		& 
		``take \textit{x} number of plots in different categories'' in gridcells to ensure sufficient sampling (\textit{P6}) & Support of applications using uniform sampling in a grid representation of the space \\ \cmidrule(lr{1em}){2-4}
		& Close coupling of visual data \& physical environment
		& Render geospatial datapoints at their physical reference points.
		 & Data points projected into physical space would enhance at-a-glance decision-making.
		  \\
		\midrule
		\multirow{3}{*}[-2ex]{\rotatebox[origin=c]{90}{\textbf{ Local/Remote Teaming}}} &
		No existing solutions collaborate on updated data in low connectivity &
		``rescue workers all have cell phones and we can get so much data until we don't have cell service and then you can't do anything'' (P8) &
		NEED A GUIDELINE \\
		\cmidrule(lr{1em}){2-4}
		& Label data based on the person/team entering the data &
		NEED AN EXAMPLE &
		Labeling data values by person/team entering the data \\
		 \cmidrule(lr{1em}){2-4}
		& High-range portable router to collate data from remote teams &
		Support decision-making based on a more complete snapshot of the data at intermediate stages &
		Local cloud support in the field is feasible and supports decision-making based on data from disparate sources \\
		\midrule
		\multirow{2}{*}[-4ex]{\rotatebox[origin=c]{90}{\textbf{ Validation }}} &
		Easily resolvable inconsistencies in data &
		Analysts validate most data by ``looking at the size of the file and saying 'that looks about right''' (P10) &
		Integrated visualization solutions should prioritize identification of missing/poorly-collected data \\
		 \cmidrule(lr{1em}){2-4}
		& Systems could highlight potential data quality issues &
		``When I am engaging in the data in a coimplex way...I am at home at my computer. The things I want to do in the field are pretty simple'' (P6) &
		Future research should consider how embedded visualizations could enable quick identification of data quality issues. \\
		\midrule
		\multirow{4}{*}[-23ex]{\rotatebox{90}{\textbf{Data Fusion}}} &
		Archival data helps in planning the day's operations &
		Aerial firefighters: want land ownership markers on the site; Public safety officials: want highlighting of key geographical features; Earth scientists: want to compare tree health data against prior efforts. &
		The ability to overlay data from different stages/sources would enable comparison and analysis of changes. \\
		 \cmidrule(lr{1em}){2-4}
		& Integrating visualization other team member locations would operational efficiency &
		NEED AN EXAMPLE &
		This integration of team member locations would allow better coordiation of effort in a single site. \\
		 \cmidrule(lr{1em}){2-4}
		& Need to fuse data from robotic drones and physical sensors &
		In analyzing fire spread ``the more data [they] get, the harder it is to process and analyze and make decisions off of'' (P2) &
		Integration of sensor/drone data through visualization in the field could significantly improve use of devices already in the field. \\
		 \cmidrule(lr{1em}){2-4}
		& Visualization of sensor data would enable consideration of new types of data. &
		In search and rescue, ``when someone provides me with a cool sensor and I can get real-time data on gas concentrations on a hazmat incident and I can have it broadcast in, I'll be stoked'' (P8) &
		The ability to visualize drone/sensor data in situ would encourage use of a broad range of sensors whose data would aid in field decision making. \\
		\bottomrule
	\end{tabular}
\end{table*}
\end{comment}

\begin{figure*}
\centering
\includegraphics[width=1.95\columnwidth]{FieldViewFeedbackSummary}
\caption
	We synthesized design recommendations for how data collection and visualization might enhance data use in field operations. 
	
	These insights fall into four primary categories that can 

	guide designers to scenarios and design \review{considerations} for field analysis tools. }
	
\label{fig:Feedback}
\vspace{-5px}
\end{figure*}

\subsection{Reliance on Location-Based Data}
\label{locationData}
\review{All} analysts responded positively to the design probe's emphasis on geospatial data. 
Though \review{experts in each domain} leveraged different types of data, analysts generally stressed the importance of \review{spatial} data for field decision making.
For example, \review{seven} analysts used image data labeled with geospatial locations to increase global awareness of the field site.
\review{The two} earth scientists employ stratified sampling to collect data at regular intervals in a geographic area of interest
by subdividing a fieldsite into regular grids and ``taking \emph{x} number of plots in different categories'' in each gridcell to ensure sufficient sampling \emph{(P6)}.
Aerial firefighters use land ownership data defined by property boundaries to guide their operations. 
\review{The two earth scientists, computer scientist, and roboticist} leverage sensor-based or manually-entered environmental data such as temperature, humidity, glacial velocity, and tree diameter to reconstruct an environment and model spatiotemporal patterns.

The focus on maps and their fusion with the data in the immersive visualizations allows analysts to visualize data about the full space of operations. 
\review{The six participants in field-oriented domains (fire fighters, search-and-rescue, and earth scientists) remarked that the simple visualizations enabled appropriate levels of analysis while providing opportunities to dig deeper into the data through interaction. The appropriate level varied depending on the target task and context. For example, firefighters wanted access to terrain maps, ownership labels, and other archival data in preplanning, but only simple, minimally obtrustive shapes that indicated decision-relevant information (e.g., simple temperature alerts or points indicating team member or payload locations) during operations}. 
However, participants wanted to see how data could be more tightly interwoven into the physical space. 
\review{All participants emphasized that the best way to improve field practices through data would be to provide a more informed picture of the operational environment: ``the more situational awareness, the better'' \emph{(P8)}.}
This coupling could improve the integration of contextual awareness into data analysis and better support operational decision making. 

\noindent
\textbf{Design Recommendation: } Geospatial data projected onto its location in physical space could enhance data sensemaking and at-a-glance decision making. 

\noindent
\textbf{Key Tasks: } Analyze stratefied samples of local data and overhead imagery, navigate relevant property boundaries and physical structures.

\subsection{Teaming under Limited Connectivity}
\label{teaming}
All participants conduct field operations in teams. Those teams can be distributed over a wide space (e.g., around the perimeter of a wildfire) or collocated (e.g., collecting tree cover measurements in a specific fieldsite).
Currently, \review{all noted} data sharing between teams is done through direct communication in collocated teams or by radio commands from an operations center.
None 
had solutions for sharing and collaboratively analyzing updated field data across teams in real time. 


\review{All but the roboticist (9/10 partipants)} were excited about the potential of analytics tools to support teaming in both collocated and distributed environments and support consistency checks across team members. 
For example, the modeling expert noted an incident where two teams at a fieldsite calibrated their sensing equipment differently, leaving their data unusable. 
Shared data between these team members would allow for at-a-glance identification of these potential anomalies. 
For distributed teams, aerial firefighters noted that recent advances have allowed them to share basic geocoordinates alongside radio transmissions to help improve situational awareness, but that these techniques are extremely limited. 
They anticipated labeling data values based on team entry in the map visualizations would significantly expand their awareness of the state and progression of a fire and let them more readily adapt operations through communication between teams.

\review{Participants in emergency response and climate modeling} saw the probe's two-phase caching scheme as beneficial in some instances, but insufficient for scenarios 
where the entire fieldsite lacks connectivity.
In these sites, even collocated team members may struggle to share data.
Our search-and-rescue expert observed that ``[rescue workers] all have cell phones and we can get so much data until we don't have cell service and then you can't do anything'' (\emph{P8}).
The climate modeler typically uploads data and receives an automated email with summary statistics to confirm the data has been stored. 
\review{When mobile on a fieldsite, this confirmation pipeline fails making} 

	``a validation of field data...the first thing to try to support'' \emph{(P9)}. 
Aerial firefighters noted a willingness to carry a high-range portable router to collate data from collocated teams, similar to the peer-to-peer infrastructure in Siren \cite{jiang2004siren}. Field analysis systems may be able to use \review{this approach for}
visualization to improve data sharing among local teams.

\noindent
\textbf{Design Recommendation:} Integrating intermediate shared data caching can support collocated teaming in environments with no connectivity through the visualization of both cached and cloud data.

\noindent
\textbf{Key Tasks: }Calibrate between sensors and teams, increase communication and global awareness between teams.


\subsection{Data Quality Validation}
\label{quality}
Participants 
advised against attempting all operation center functionality in AR.
For example, one earth scientist stated ``when I am engaging in the data in a complex way... I am at home at my computer. The things I want to know in the field are pretty simple'' \emph{(P6)}.
Interviewees emphasized the need to avoid overwhelming analysts with too much data while in the field \review{and to focus on operational tasks that analysts can assess and address in context such as validating the quality and consistency of incoming data.}

\review{Nine analysts} noted that data collection errors force them to return to the field and repeat data collection if errors are found while at the fieldsite or to discard large amounts of data if errors are undetected until later analysis.
For example, our modeling expert remarked that mistakes in data collection for glacial research were unresolvable as they required flying equipment back to remote locations in Greenland or Antarctica.
Earth scientists remarked that measurements of tree circumference---typically taken at chest height---could introduce substantial variation depending on the scientist's height.
While these inconsistencies are easy to resolve, they are currently impossible to detect in the field. \review{Four analysts specifically mentioned} validating data by ``looking at the size of the file and saying ``that looks about right'' \emph{(P10)}.


Analysts saw the design probe as a way to immediately evaluate collected data for errors in quality, \review{with the four participants in the sciences} noting that these visualizations would save substantial time and money in field operations. 
They further remarked that the simplicity of the probe's visualizations would allow analysts to quickly locate errors using familiar representations and to verify consistency by exploring high-level patterns in existing data. 
Quick glances at the mobile visualizations provide a sense of data coverage, while the immersive visualizations allow analysts to dig deeper into anomalies and compare those variations side-by-side with the conditions of the environment to remedy collection issues.


\review{Four emergency responders and the climate modeler} noted that systems may further aid in data quality validation by automatically highlighting potential quality issues, a challenge that could be addressed by coupling automated and visual analysis. 
Analysts recommended field analytics systems more deeply consider how embedded visualizations might aid in quickly identifying problematic areas in data collection to increase data quality and efficiency. 
Such technologies could allow analysts to actively update their goals and operational strategies as a function of the quality and completeness of incoming data. 
 

\noindent
\textbf{Design Recommendation: } Analysis systems should explicitly support detecting, contextualizing, and remedying low quality data.

\noindent
\textbf{Key Tasks: } Identify missing data, detect anomalies, monitor incoming data.

\subsection{Data Fusion Across Perspectives}
\label{fusion}
\review{All ten field analysts} rely on archival data, such as environmental surveys, topological maps, or data from prior efforts, to plan each day's operations. 
They noted that the design probe's visualization systems could improve efficiency by bringing these sources into the field. 
For example, aerial firefighters wanted to overlay land ownership markers on the operational site.
Public safety officials wanted to see key geological features, such as altitude shifts. 
Earth scientists wanted to couple visualizations of tree health from prior field surveys to study changes over time.
Updating visualizations during data collection may allow analysts to inform operations and change data collection practices on the fly.
\review{Nine of the ten participants} felt that extending the immersive visualization suite to integrate other teams' locations could further improve operational efficiency, allowing analysts to better coordinate operations across teams collecting data over a single site. 

While the design probe relies heavily on archival and manually collected data, analysts wanted to also fuse this data with data collected by robotic drones and other physical sensors.
\review{All participants saw the distributed notebook interface as an improved method for merging data across teams in many scenarios; however, the emergency responders and earth scientists felt the notebook collection interface was potentially problematic when gloves were required for safety or due to environmental conditions (e.g., working in active burn zones or tropical bogs). Integration with automated sensors helps overcome these limitations and expands both the data variety and field coverage used in analysis: sensors can stream more data faster and from more locations.}
In wildland firefighting,
sensor readings of temperature, wind, and humidity significantly aid analysts in estimating wildfire spread, but ``the more data [analysts] get, the harder it is to process and analyze and make decisions off of'' \emph{(P2)}. 
Experts noted that the design probe's approach could significantly improve the integration of such sensing devices by allowing analysts to fuse archival data with incoming data streams currently only accessible to operations centers. 
For example, 
image data collected by drones can provide multiple perspectives
that extend the area an analyst can see. 
In earth science, this integration would allow analysts to simultaneously study canopies from above and below to estimate forest health.
All analysts noted how practices could be improved by visualizing sensor data in real-time, 
and that visualizations of this data would bring new kinds of data into the field.
Our search and rescue expert said, ``there's only so much you can learn from a picture or video... When someone provides me with a cool sensor and I can get real-time data on gas concentrations on a hazmat incident and I can have it broadcast in, I'll be stoked'' \emph{(P8)}.
Analysts recommended that fused views should also allow for separating data from different sources, giving field analysts the ability to distinguish preplanning data from data collected by team members and autonomous sensing technologies.

\noindent
\textbf{Design Recommendation: } 
Visualizations should incorporate data from other sources, including autonomous drones, to increase awareness of both local and global operations.

\noindent
\textbf{Key Tasks: } Couple data and images from multiple sources, stream real-time data from autonomous sensors into the field. 

\section{Discussion \& Future Work}

This work 
\review{provides formative insight into how visualization systems can enhance field work,}
focusing on solutions that integrate data collection and analysis through combined mobile and immersive visualizations.
We developed 
a design probe to ground 
interviews with ten domain experts and identified four key design themes and several analysis tasks relevant to field visualizations.
We embodied these themes in our FieldView prototype
to provide a proof-of-concept of how visualization might enhance field work in three scenarios: \review{teaming, data quality validation, and visualizing autonomous sensor data.}
Our results provide a formative basis for designing visualization tools for field analytics and raise new questions for future tools, including: 
\vspace{-6px}
\begin{itemize}
	\item What kinds of analysis and decision making processes should field analytics tools support? \vspace{-6px}
	\item How can we optimize visualization designs distributed across mobile and immersive technologies? \vspace{-6px}
	\item How can we design integrated solutions for effective collection and analysis workflows in the field? \vspace{-6px}
	\item \reviewnew{How can visualizations better consider hardware constraints?}
\end{itemize}
\vspace{-6px}

\textit{What kinds of analysis and decision making processes should field analytics tools support?} 
Our interviews revealed common tasks that field analysis tools can support, such as evaluating data quality, supporting distributed teams, and integrating data from drones and other automated sensors. 
These interviews also yielded critical design recommendations for \review{visualizing this} data.
For example, visualizations should incorporate archival data with collected data, 
display this data 
in its physical context, mitigate information overload, and consider constraints of outdoor environments.
FieldView provides a proof-of-concept for how systems might reconcile the spatial gap by visualizing diverse data in the physical environment it was collected.
This approach also helps alleviate temporal gaps by allowing analysts to make timely decisions while out in the field.
\review{Our interviews uncovered} scenarios, datasets, and components of visualization designs that can support field analysis, but we still lack empirical insight into which tasks require field support and which tasks are best left to the command center. 
Future investigations into specific domains may provide frameworks for 
\review{optimizing} such distributed sensemaking.

\textit{How can we optimize visualization designs distributed across mobile and immersive technologies?}
Evolving AR, robotic, and collaborative technologies offer new challenges for effective data visualization, especially in scenarios where 
data comes from heterogeneous sources.
Immersive AR allows analysts to utilize their implicit sense of the environment (e.g., the trajectory of wildfire smoke or the physical bridge being inspected) alongside data.
Mobile solutions enable field analysts to engage with complex visualizations in the field, but all experts emphasized the need for understanding situational data at a glance.
Analysts needed both quick overviews of the data and the ability to make sense of geospatial data in more detail supported by physical context.
Positive feedback from the design probe 
suggests significant potential for integrated 
mobile and immersive visualizations; however we have little understanding of how to optimize visualizations to leverage these differing form factors simultaneously.
Future research should further explore 
combined mobile overview and detailed immersive visualizations, similar to the David \& Goliath approach to complimentary smartwatch and large display visualizations \cite{horak2018david}.

\textit{How can we design integrated solutions for effective collection and analysis workflows in the field?}
With our design probe and FieldView, we built example workflows that could 
empower field analysts with visualizations for data-oriented decision making.
Despite significant excitement about how field visualization tools could transform existing practices, we were limited in our ability to evaluate FieldView in the wild due to scientists' reluctance to take new technology into the field, stemming from the high time and financial costs of field operations and legal restrictions around head-mounted displays in public safety.
We anticipate that these barriers will change in the near future.
Technological solutions for emergency response are receiving considerable political attention, including U.S. Congressional support through a 2019 Act to ``promote the use of the best available technology to enhance the effective and cost-efficient response to wildfires'' \cite{dingell2019conservation}.
Our interviews also exposed several key design challenges for using situated visualizations in the field, such as balancing complexity and simplicity and coupling data with context. 
However, 
there is little empirical or grounded guidance for crafting immersive visualizations.
Our interviews suggest a need to balance visualization simplicity with support for a broad set of tasks and data sources.
Future work should explore ways of providing necessary information in immersive visualizations
while retaining a relatively unobscured view of the physical environment.

\textit{\reviewnew{How can visualizations better consider hardware constraints?}}
Our work also demonstrated the importance of 
\reviewnew{new hardware platforms for field analysis and visualization. }
In our interviews, we learned of technologies already being incorporated in the field that would further bolster the proposed workflow and system architecture.
For example, earth scientists generally find mobile phone GPS insufficiently precise for many \reviewnew{measurements}; however, they 
\reviewnew{have workarounds that could inform more precise geospatial data integration.}
The localized connectivity provided by the high range portable router described by aerial firefighters would enable better teaming when using an integrated system such as FieldView.
Continued collaboration with field researchers about the technology already in use could yield \reviewnew{field-ready VA systems.} 
Visualization tools should consider how to best integrate these technologies to support richer and more robust analyses. \review{Further, though FieldView employs the Microsoft HoloLens, excessive weight, 

	reduced contrast in natural light, and challenges with spatially mapping large outdoor environments make existing HMDs 

	unreliable for real-world deployments. While we have begun exploring pass-through \reviewnew{solutions,}

	fieldwork offers novel design challenges for AR hardware.}

\begin{comment}
We could not conduct a formal deployment due to limitations such as legislative barriers and the relative infrequency of field operations. 

Toward establishing an open research area in field analytics, we have identified some data-oriented issues that field researchers and emergency responders face.
A key first step in this domain is to consider what types of decisions need to be made in the field rather than in a lab or remote operations center.
Through interviews, we have concluded that visual analytics systems in the field should particularly focus on supporting identification of low quality data and fusion of heterogeneous data from other team members and robots.
These interviews also yielded some interesting design recommendations for such systems: incorporation of archival data alongside newly collected data, visualization of data alongside its physical context, mitigating information overload, and consideration of outdoor constraints.
Iterations of our FieldView prototype demonstrate some possible instantiations of collaborators' envisioned use cases.
We anticipate that FieldView will provide first steps towards overcoming spatial and temporal gaps to field data use and its extensible, open-source architecture will scaffold the development of future field analysis systems.  
\end{comment}

\begin{comment}
In the process of enumerating how visualizations might support this domain, we found
While our work offers insight into how mobile and immersive technologies may support field analysis, the limitations in our present approach provide interesting opportunities for future research.
For example, we were limited in our ability to evaluate the utility of the system during real field operations due to the lack of data available: current practice are sufficiently limited that gaining access to real geotagged datasets proved prohibitive. 
Instead, we worked with experts to retarget what data was available and to generate reasonable simulations. 
We could not conduct a formal deployment due to limitations such as legislative barriers and the relative infrequency of field operations. 
However, close collaboration with experts during the design iteration suggest the promise of FieldView for creating new opportunities for data engagement in the field. 
We are actively working with field analysts to increase accessible data and introduce novel deployment strategies to allow future systems to more readily overcome these barriers to evaluation.

Evolving AR, robotic, and collaborative technologies offer new challenges for effective data visualization, especially in scenarios where data accountability is critical. 
Immersive AR allows users to utilize their implicit sense of the environment---the trajectory of wildfire smoke or the physical bridge being inspected---alongside relatively sparse, contextualized information.
Mobile solutions enable field analysts to engage with complex visualizations in the field, but all experts 
emphasized the need for understanding situational data at a glance.
We have little understanding of how to optimize visualizations to leverage these differing form factors.
Future research could illuminate how field analysis systems can balance the richness offered by both data and context with analysts' need for simple at-a-glance information and how mobile and immersive visualizaiton modalities could work in tandem. 

\begin{comment}
\todo[inline]{Not sure this paragraph is needed}
Our visualizations also focus on fully immersive representations. While these representations allow analysts to quickly engage with contextualized data, the current designs may also occlude the analysts' field of view. Our interviews suggested that this degree of visual occlusion is not a significant limitation for our envisioned applications; however, we were unable to conduct a full field deployment to confirm this hypothesis. 
Future extensions of this work should more deeply explore the role of occlusion in immersive analytics, and systems can adapt to the environment to mitigate potential limitations caused by occluding relevant visual information.
\end{comment}

\begin{comment}
Existing hardware has limited tracking capabilities, limiting the precision and resolution of collected spatial data. 
Earth scientists generally do not find mobile phone GPS sufficiently precise for many of their needs; however, they have strategies for accounting for these limitations that could inform more precise geospatial integration in future systems. 
Our current fusion approach relies on cloud-based indexing of spatial locations. However, more sophisticated algorithms can enable rapid fusion of more complex information types \cite{khaleghi2013multisensor}. 
Applications of this approach should actively consider spatial fidelity and other deployment platform limitations as key factors in visualizing field data. 
For example, integrating measurement uncertainty into visualized data and developing solutions to account for hardware limitations is important future work. 


This work elicits some important future directions for field analytics and immersive analytics research.
Future work should carefully consider the tradeoffs of providing users with sufficient visualizations to make decisions without overloading with too much data.
This includes the tradeoff between presenting raw data for the user to interpret and algorithmically processed data that presents derived values.
We present an integrated mobile and immersive visualization suite as the two modalities are highly complimentary, though future work in collaboration with domain experts should further explore the role of each in visualizing field data.
Continued formative research should elicit analysts workflows in the field when provided with a field analytics system such as FieldView.
Further extensions of the system itself should incorporate drone and sensor data in real time, integrate the standalone GPS tracking devices and high range routers being brought out into the field, and employ more feasible headsets than the Gear VR or HoloLens for immersive visualization.
\end{comment}
\section{Design Probe Implementation}
\label{sec:InitialPrototype}
\begin{figure}
	\centering
	\includegraphics[width=\linewidth]{InitialSchematic_new}
	\caption{An overview of our 
	
		 design probe architecture---data is collected and merged with cached data on a mobile device and synchronized with a cloud datastore when the device is connected. Cached data can be viewed on the mobile device or explored in a portable immersive headset. 
		
		
		
	 }
	\label{fig:InitialSchematic_new}
	\vspace{-5px}
\end{figure}

We used \review{our preliminary interviews} to construct a design probe
to ground interviews about visualization design for fieldwork (Fig. \ref{fig:InitialSchematic_new}). 
The probe allows analysts to input data on mobile devices, synchronizing to a local database and then to a remote data store when \review{connected} (\textbf{R1}).
Analysts can visualize cached data on their mobile devices using standard overview visualizations (\textbf{R3}) or in immersive interactive visualizations (\textbf{R2}) using either a portable see-through display (e.g., Fig. \ref{fig:MissingData}) or the mobile device's camera (e.g., Fig. \ref{fig:InitialImmersiveVis}).
Our initial probe
used an Android device and Samsung Gear VR headset to \review{provide portable and integrated data collection and analysis} 
(\textbf{R4}; Fig. \ref{fig:InitialSchematic_new}). 
\review{We architected our probe using a geospatial dataset of altitude measures at various sites (seeded with the set of mountains in Colorado with elevation greater than 14,000 feet) and image data collected using the phone's camera, corresponding to data used by public safety and wildland fire experts.} 



\noindent\textbf{Target Workflow: }
We illustrate our design probe's workflow using
a scenario \review{synthesized from our preliminary interviews} where an analyst is in the field collecting geotagged data.
Analysts sync their data during preplanning, and 
can analyze the new data \review{collected during an operation} using an overview+detail paradigm distributed across devices: mobile visualizations provide rapid overviews of data collected by all teams involved in an operation while immersive detail visualizations show data related to the local environment. \review{As the analyst and other team members collect data, these visualizations update in real-time to remedy temporal analysis gaps.}
By situating data in the same physical space the data represents, 
analysts can overcome spatial analysis gaps by analyzing data in the environment. Analysts can continue this collect-and-analyze workflow over the course of the operation and refine their practices based on observations developed in real-time.

We support this workflow by integrating mobile, cloud, and immersive technologies, using an Android application for data collection and mobile visualization and immersive visualizations built in Unity.
Analysts first synchronize a remote target database with the mobile application, which extracts and displays a list of existing data \review{and creates an input form based on the preplanning schema}.
Analysts \review{navigate the target environment, adding datapoints 
to a local database that syncs to the remote database when possible.}
They can then explore this data at a glance through three mobile overview visualizations---a bar chart, scatterplot, and heatmap---to get a quick understanding of the full data corpus.
These visualizations \review{increase global situational awareness by providing an overview of data from the fieldsite.}

From within the mobile application, analysts can transition to an immersive detailed view of this data.
\review{

	The analyst sees datapoints projected into the physical environment

	using geolocation tags from the mobile device.}
These visualizations enable analysts to more fully explore the data (Fig. \ref{fig:InitialImmersiveVis}).
\review{Analysts can pull up details on demand by tapping the D-pad on the side of the headset while looking at a point of interest, and can pan the visualizations by sliding a finger across the D-pad.}
Our implementation supports the requirements enumerated in initial interviews (\S \ref{requirements}) as follows: 

\noindent
\textbf{R1--Offline \& Distributed Data Collection:} 
\review{The disparate storage media used in the field make it difficult to combine data across teams and with archival sources, requiring substantial curation prior to analysis.}  
Mobile data collection platforms can substantially improve distributed data collection \cite{tomlinson2009use,pascoe2000using}; however, limited connectivity in the field can hinder these approaches.
We build on prior mobile data collection efforts for distributed teams and for limited connectivity.
We leverage a two-phase approach similar to Truong et al. \cite{truong2007escape} to allow analysts to locally cache new data offline and to integrate that data with active cloud datastores when connected. 

The application uses the remote database schema to populate fields in a field notebook interface allowing numeric, text, time, GPS, and image data collection, 
\review{standardizing} data entry across distributed teams (Fig. \ref{fig:DataCollectionA}). 
As analysts add data, new datapoints immediately sync to the local datastore. 
To support distributed collection efforts, new entries are pushed to the remote cloud database when the device is online, and any new entries in the remote database are pulled to the local datastore.
When offline, the application caches the new data and flags the data as unsynchronized to provide insight into the freshness of the available data.
To enable ready extensibility, we implement our cloud datastore using a MySQL database hosted on a standard web server. 
Our dataflow uses standard HTTP message passing where analysts could easily integrate custom server-side 
functionality tailored to particular domains or operations. 


\begin{figure}
	\centering
	\subfloat[]{\centering \includegraphics[width=0.245\linewidth]{DataCollection} \label{fig:DataCollectionA}}
	\subfloat[]{\centering \includegraphics[width=0.245\linewidth]{LocationList_lg} \label{fig:DataCollectionB}} 
	\subfloat[]{\centering \includegraphics[width=0.245\linewidth]{MobileVis_lg} \label{fig:DataCollectionC}}
	\subfloat[]{\centering \includegraphics[width=0.245\linewidth]{MobileMap_lg} \label{fig:DataCollectionD}}
	\caption{The design probe's mobile application provides (a) a field notebook-style data collection interface, (b) a corresponding list of local data points collected in the field, and (c \& d)  interactive mobile overview visualizations based on remote server data.}
\label{fig:DataCollection}
\vspace{-5px}
\end{figure}



\noindent
\textbf{R2--Merge Environmental Context \& Analysis:}
Recent advances in mobile and immersive analytics (e.g., \cite{elmqvist2013ubiquitous,willett2017embedded}) allow analysts to explore data in real time to increase situational awareness and to ground those analyses within the environmental context.
However, these techniques are currently tailored to specific applications and do not 
integrate data collection and analytics as is critical for 
field operations. 
Our probe allows analysts to launch either mobile or immersive interactive visualizations directly from the Android application. 
Mobile visualizations provide rapid overviews of the data, allowing for increased situational awareness over large geographic regions. 
Analysts can use these visualizations to identify coverage gaps in current data collection efforts. 
However, mobile visualizations suffer from 
several potential limitations 
(see \S \ref{requirements}) including decontextualization and limited screen space.

We overcome these limitations by allowing analysts to use immersive visualizations for more detailed exploration within the local environment.
These visualizations use AR to provide greater detail about data describing the immediate local environment. 
To understand how analysts might use these visualizations, our design probe uses three immersive visualization strategies:
a low-coupling billboarding approach (Fig. \ref{fig:InitialImmersiveVis}a) that provides 
a flat interactive representation of locations as scatterplot points similar to a conventional heads-up display,
a low-coupling 3D representation of the data (Fig. \ref{fig:InitialImmersiveVis}b), and
a high-coupling approach that employs spatial mapping 
to allow the user to walk around the visualizations and engage with data that keeps its physical placement as the analyst moves (Fig. \ref{fig:InitialImmersiveVis}c).
This approach of overlaying data on top of the physical environment \review{provides analysts} immediate understanding of the data's real world context by collocating data with the physical environment it describes.
\review{True high-coupling in some scenarios goes beyond spatial location to incorporate object position and structure. This mapping would require advanced computer vision techniques outside of the scope of our design probe.}

\noindent
\textbf{R3--Mitigate Information Overload:}
We couple mobile and immersive AR visualization using an overview+detail paradigm
to mitigate information overload 
\cite{ellis2007taxonomy}.
Mobile visualizations provide an overview of the data from the entire operational area while the immersive visualizations allow exploration of detailed subsets relevant to the active area of exploration.
\review{Within our mobile visualizations, analysts can quickly visualize important data dimensions using a map, bar chart, or scatterplot (Fig. \ref{fig:DataCollection}).}  
These simple designs adhere to analysts expressed needs to quickly ``sanity check'' analyses and are tailored to the limitations of mobile devices \cite{wiehr2013drivesense}, while providing simple, familiar data representations \review{to ground our discussion of the design space using this probe}. \review{To support rapid insight into local and remote data, }our visualizations encode remote data values as blue marks, cached data as green, and unsynchronized data in red (Fig. \ref{fig:InitialImmersiveVis}). 
\begin{comment}
Removed for VIS: Design probe
Our immersive visualizations consisted of 2D and 3D bar charts, scatterplots, and heatmaps, with 3D representations designed to contextualize data for comparisons within the physical 3D environment. 
Analysts interact with objects using a gaze-based raycast cursor to select objects and the Gear VR's touchpad directional pad to access details about that data on demand. 
Panning along the D-pad allows analysts to relocate the visualization to adjust the position of data given the Gear VR's limited tracking capabilities and to move billboarded visualizations within the field of view to avoid occlusion.
Given the HoloLens' more advanced tracking of headset orientation, analysts can navigate the visualization and access details by looking toward a particular datapoint.
\end{comment} 



\noindent
\textbf{R4--Use in Outdoor Environments:}
using the rear-facing camera of the phone for pass-through AR. 
We use AR rather than VR
due to the potential for AR to bind data and context \cite{willett2017embedded} and to mitigate potential occlusion issues that would inhibit active operation in an outdoor environment. 
We also implement our probe on the Microsoft HoloLens to showcase the benefits of using an AR headset with spatial mapping, notably embodied navigation and freehand gestural input.
\review{Though gestural interaction allows for hands-free data exploration, it does not resolve issues associated with data entry on a mobile device while wearing gloves.
Manual data can be input by multiple team members and the probe's architecture allows sensor data to stream to the remote database. Future iterations could explore integration of sensor-specific visualizations and verbal dictation using the phone or HMD microphone to automatically log data.}





\begin{figure}[t!]
	\centering
	\subfloat[Billboarded map in 2D on Gear VR] {\includegraphics[width=0.3\linewidth]{2DAR_lg}}
	\hfill
	\subfloat[3D bar chart with altitude data on Gear VR] {\includegraphics[width=0.3\linewidth]{3DAR_lg}} \hfill

	\subfloat[Embedded bar chart on MS HoloLens] {\includegraphics[width=0.35\linewidth]{14ersOnFarrand_lg}}
	\caption{Our design probe used three immersive visualizations to increase contextual awareness: \review{(a) a billboarded map, (b) a 3D bar chart, and (c) an embedded bar chart. We used simple visualizations in the probe based on R3 and to minimize bias from design in our qualitative study}.}
\label{fig:InitialImmersiveVis}
\vspace{-5px}
\end{figure}


\section{Introduction}

Data-oriented decision making is transforming practices in a broad variety of domains. 
For example, with the substantial number of wildfires in western North America, technology-based solutions for emergency response are receiving considerable attention, including U.S. Congressional support through a 2019 Act to ``promote the use of the best available technology to enhance the effective and cost-efficient response to wildfires'' \cite{dingell2019conservation}.
Applications in earth science, geology, and emergency response all leverage data collected in the field to help describe the state of complex environments.
Field operations in these domains typically rely on a collaboration between people in the field and remote workers in a command central, though current field data analysis efforts focus only on supporting decision making in the command center. 
This limits analysts' abilities to use this data in on-going operations: analysts in the command center lack the context and local situational awareness of people in the field, while people in the field lack tools to access, analyze, and act on data while in the field. 
This work explores the needs, feasibility, and use cases of visualizations in the field.

Due to the lack support for analytics in the field, decision making typically occurs in remote operations center either during the operation or at a remote site at the end of the operation. 
For this reason, field operations are plagued by issues caused by both  spatial and temporal gaps between data collection and data analysis (Fig. \ref{fig:SpatialTemporal}).
Spatial gaps in understanding arise due to remote operators not having the physical context of the data collected that field workers do, lack of visual grounding to communications from the command center to the field and unreliable connectivity in remote areas.
Temporal gaps stemming from an inability to analyze data until back in a lab result in field operation on stale data in dynamic environments, an inability to make timely decisions, and could jeapordize a day's work or even the entire operation.

In designing visualization systems to the constraints onf field operations, we first need to consider the decision making needs while in the field.
Though a lab or command central is better suited to the global view of the operation on a desktop computer, the field is better suited to make decisions on lower level tasks while out in the field.
That is, field workers can make quick sense of the data alongside its physical context in real time and update plans accordingly.
Being in the field collecting data, field workers also have the opportunity to visualize data within its spatial context.
This paper focuses on the requirements of solutions that could support this situated awareness to optimize decision making in the field.
Building on preliminary conversations with field workers, we created an interactive design probe architected to support an integrated data collection and visual analysis workflow through a combination of mobile and immersive visual analytics tools. 
We use this probe to elicit formal recommendations for visual analytics tools for field work and develop a preliminary prototype based on these recommendations. 
We seek to identify important decisions better suited to be made in the field than a lab or operations center, what data would enable this decision-making, and how visual analytics systems could support field workers.

\begin{figure}[t]
	\centering
	\includegraphics[width=\columnwidth]{SpatialTemporal}
	\caption{Temporal and spatial gaps hindering decision making in the field. The temporal gap is characterized by an inability to make decisions until after concluding operations in the field. The spatial gap refers to challenges introduced by removing collected data from the physical environment, reducing local situational awareness during decision making.}
	\label{fig:SpatialTemporal}
\end{figure}

\textbf{Contributions: }\begin{comment} We introduce a novel field data collection and analysis workflow 
that enables situated decision-making in the field. 
We conducted interviews with field analysts in domains 
such as environmental science and aerial firefighting to understand limitations in current field data use and synthesize critical design requirements for 
real-time data analysis tools. 
We implement these requirements in
an extensible prototype 
leveraging commodity technologies to support field analysis. 
Building on scenarios identified by analysts, we ground our prototype in three use cases 
demonstrating how our 
approach may address limitations in field analysis to increase operational efficiency.
\end{comment}
We conduct a series of interviews to establish some design considerations and constraints for integrated data collection and analysis tools in the field, summarized in Fig. \ref{fig:teaser}.
We first worked with scientists and public safety officials to identify challenges in current field data practices. 
We use these limitations to develop a design probe demonstrating how mobile and immersive technologies might overcome these limitations.
Using a design probe, we interviewed 10 domain experts about the potential of complimentary mobile and immersive visualizations to transform field operations.
After synthesizing these interviews, we refined the workflow introduced in the design probe to develop FieldView, an open-source prototype system to address use cases inspired by common themes and scenarios identified by analysts.


\begin{comment}

\section{Introduction}
Data-oriented decision making is transforming practices in a broad variety of domains. Applications in earth science, geology, and emergency response all leverage data collected in the field to help describe the state of complex environments. 
Quantifying these environments allows analysts to model the changing state of the world over time and to share information across teams to increase situational awareness and deepen scientific and operational understanding. 
However, current practices for working with data in the field rely heavily on decoupled solutions that separate data analysis from the spatial and temporal contexts that data describes: data is collected in the field but analyzed elsewhere, such as in operations centers or remote laboratories.
This separation can limit field analysts' abilities to integrate current data into analysis and decision making practices, introducing significant time and efficiency costs. 

We address this challenge by exploring how mobile and immersive technologies can help bridge spatial and temporal gaps between data collection and analysis in fieldwork. Specifically, we explore how we can couple these technologies to increase contextual and situational awareness in field analysis practices in order to improve data collection, sharing, analysis decision making with relevant visualizations. We focus on how mobile data collection, cloud storage and fusion, mobile device visualizations, and augmented reality (AR) visualizations enable analysts to visualize data as it is collected in order to integrate contextual information about the immediate environment into data interpretation and operational planning.
Our approach fuses data from multiple sources to provide field analysts with access to new and existing data from other teams in the field and archival sources.
Analysts can then leverage at-a-glance mobile visualizations and detailed immersive interactive visualizations to develop new hypotheses, identify potential errors or gaps in coverage, and adapt operational practices while in the field.

Current practices for fieldwork require analysts to first preplan their operations based on archival data from both previous collection efforts and archival data streams.
Analysts then collect new data either on mobile devices or in field notebooks, physically transport data to a central location to synchronize with other sources, and replan subsequent collection efforts and operational practices based on the newly revised data. 
This workflow decontextualizes collected data from the environment, prevents analysts in time-critical situations from reacting to new data, and obscures possible errors in data collection. 
These challenges are exacerbated when data comes from multiple sources: analysts have limited access to additional data collected from other field teams or independent sensors. 
Our proposed workflow leverages cloud datastores to integrate data from multiple sources as data becomes available, leveraging both local and global data storage to overcome connectivity limitations in the field.
We additionally extend existing practices for field data analysis by allowing analysts to conduct rapid inference using mobile visualizations or immersive embedded visualizations that contextualize data within the physical environment and leverage visual context to support nuanced analyses.
This workflow increases contextual awareness to improve data-oriented operational goals, provide real time data quality assessment, and support synchronous operations between field teams. 

We implement and evaluate this approach in FieldView, an extensible mobile data collection and visualization \ds{prototype}
designed to bridge the spatial and temporal gaps between data collection and analysis in the field. 
We developed FieldView in collaboration with field analysts in domains ranging from forest ecology to public safety. 
We discuss how our approach addresses several challenges in field analysis and refine our approach 
using observations from ten domain experts. We demonstrate our approach using three scenarios for field analysis inspired by our discussions with field analysts and unsupported by existing systems: data-mediated teaming, data quality verification, and
fusion with autonomous sensors. By coupling mobile data collection and immersive analytics, our approach can foster improved data collection, interpretation, and decision making by providing field analysts with increased access to incoming data and allowing analysts to interpret that data with respect to aspects of the environment only available in the field. 

\textbf{Contributions: } We introduce a novel field data collection and analysis workflow 
\ds{that enables situated} decision-making in the field. 
We conducted interviews with field analysts in domains 
\ds{such as} environmental science and aerial firefighting to understand limitations in current field data use and synthesize critical design requirements for 
real-time data analysis tools. 
We implement \ds{these requirements} in
an extensible prototype 
leveraging commodity technologies to support field analysis. 
Building on scenarios identified by analysts, we ground our prototype in three use cases 
demonstrating how our 
approach \ds{may} address limitations in field analysis to increase operational efficiency.










\end{comment}
\section{Introduction}

Data-oriented decision making is transforming practices in a broad variety of domains. 
Applications in earth science \review{\cite{eeckhaut2007use}}, geology \review{\cite{pavlis2010computer}}, and emergency response \review{\cite{palen2016crisis}} all leverage data collected in the field to describe the state of complex environments.
Field analysts collect data to model the changing state of field sites and share information across teams to increase situational awareness and deepen scientific and operational understanding.
However, \review{our interviews with field workers across multiple domains reveal that} current practices for working with data in the field
rely heavily on decoupled solutions that separate data analysis from the spatial and temporal contexts that data describes. 
Data is collected in the field but analyzed in operations centers or remote laboratories.
\review{Analysts report that this} separation limits \review{their} abilities to use this data in on-going operations: remote analysts lack the context and local situational awareness of people in the field; people in the field lack tools to access, analyze, and act on data while in the field. 
This work explores challenges and opportunities for using visualization to support data analysis in the field. 

The lack of support for analytics in the field means that decision making typically occurs in remote operations centers either during or after operations.
Current practices for fieldwork require analysts to first preplan their operations based on data from previous collection efforts and archival data streams. Analysts then collect new data either on mobile devices or in field notebooks, physically transport data to a central location to synchronize
with other sources, and replan subsequent collection efforts and operational practices based on the newly revised data \review{\cite{cutter2003gi,pavlis2010computer}}.
This workflow decontextualizes collected data from the environment, prevents analysts from reacting to new data, and obscures possible errors in data collection by creating spatial and temporal gaps between data collection and analysis (Fig. \ref{fig:SpatialTemporal}).
Spatial gaps in understanding arise when remote analysts lack the physical context 
surrounding field data, resulting in a lack of visual grounding for remote analysis and 
limited shared context during real-time operations. 
Temporal gaps stem from an inability to analyze data during an operation, resulting in stale data being used in dynamic environments and an inability to make timely decisions.
Temporal gaps could jeopardize a day's work or even the entire operation due to data quality issues and reduced global awareness of the state of the broader operation. \review{While domain experts noted that these gaps are acceptable for some complex analyses (e.g., those requiring heavy computation or in-depth exploration), improving access to field data could transform many aspects of field practices by expanding field analysts' situational awareness.}


We explore how mobile and immersive analytics tools can \review{begin to} bridge spatial and temporal gaps between data collection and analysis in fieldwork. Specifically,
we examine how coupling these technologies might increase contextual and situational awareness in field practices 
to improve data collection, sharing, analysis, and decision making by better connecting field analysts with their data. 
Mobile technologies offer portable connectivity to access and synchronize data in the field and can offer at-a-glance overviews that help summarize \review{an operation's} current state
\cite{brehmer2019visualizing}.
However, mobile technologies suffer from several limitations 
\review{in the field.} 
For example, interacting with mobile devices is challenging while wearing gloves and forces analysts to divide their attention between the visualization and environment. Immersive visualization techniques can resolve these issues by visually embedding data in the physical environment \cite{willett2017embedded}; however, we have limited knowledge of how to design effective AR visualizations. Our goal in this paper is to elicit insight into how visualization tools can be designed to effectively support the needs of field data analysis to provide preliminary steps towards a deeper integration of data into field operations. 

This paper presents an exploratory investigation into how visualization tools could support increased situated awareness and optimize data-driven decision making in the field.
Building on preliminary conversations with field workers in domains ranging from earth science to emergency response, we created an interactive design probe architected to support an integrated data collection and visual analysis workflow
\review{combining} mobile and immersive tools.
We use this probe to elicit formal recommendations for visual analytics tools for field work and develop a preliminary prototype based on these recommendations.
The prototype 
\review{addresses} three target scenarios identified during the interviews where a lack of access to field data causes significant operational challenges: \review{team coordination, data quality validation, and integrating autonomous sensors}. Our results suggest new opportunities for leveraging visualization to empower field operations through data. 

\begin{figure}[t]
	\centering
	\includegraphics[width=\columnwidth]{SpatialTemporal}
	\caption{Current field practices separate data collection and analysis, leading to temporal (separating data collection and analysis by time) and spatial (decontextualizing data from the environment) gaps that 
	
		hinder effective analysis and decision making with field data. 
	
	}
	\label{fig:SpatialTemporal}
	\vspace{-5px}
\end{figure}

\textbf{Contributions: }\begin{comment} We introduce a novel field data collection and analysis workflow 
that enables situated decision-making in the field. 
We conducted interviews with field analysts in domains 
such as environmental science and aerial firefighting to understand limitations in current field data use and synthesize critical design requirements for 
real-time data analysis tools. 
We implement these requirements in
an extensible prototype 
leveraging commodity technologies to support field analysis. 
Building on scenarios identified by analysts, we ground our prototype in three use cases 
demonstrating how our 
approach may address limitations in field analysis to increase operational efficiency.
\end{comment}
\review{Our primary contribution is a formative qualitative study used to establish preliminary design considerations, constraints, and scenarios for integrated data collection and analysis tools in the field. }
We 
worked with scientists and public safety officials to identify challenges in current field data practices
\review{and} 
develop a design probe demonstrating how mobile and immersive technologies might overcome these limitations.
This probe scaffolded interviews with 10 domain experts about the potential of complimentary mobile and immersive visualizations to transform field operations.
After synthesizing these interviews, we refined the workflow introduced in the design probe to develop FieldView, an open-source prototype system to address 
\review{three critical use scenarios identified by analysts (teaming, data quality validation, and data fusion;} Fig. \ref{fig:teaser}).


\begin{comment}

\section{Introduction}
Data-oriented decision making is transforming practices in a broad variety of domains. Applications in earth science, geology, and emergency response all leverage data collected in the field to help describe the state of complex environments. 
Quantifying these environments allows analysts to model the changing state of the world over time and to share information across teams to increase situational awareness and deepen scientific and operational understanding. 
However, current practices for working with data in the field rely heavily on decoupled solutions that separate data analysis from the spatial and temporal contexts that data describes: data is collected in the field but analyzed elsewhere, such as in operations centers or remote laboratories.
This separation can limit field analysts' abilities to integrate current data into analysis and decision making practices, introducing significant time and efficiency costs. 

We address this challenge by exploring how mobile and immersive technologies can help bridge spatial and temporal gaps between data collection and analysis in fieldwork. Specifically, we explore how we can couple these technologies to increase contextual and situational awareness in field analysis practices in order to improve data collection, sharing, analysis decision making with relevant visualizations. We focus on how mobile data collection, cloud storage and fusion, mobile device visualizations, and augmented reality (AR) visualizations enable analysts to visualize data as it is collected in order to integrate contextual information about the immediate environment into data interpretation and operational planning.
Our approach fuses data from multiple sources to provide field analysts with access to new and existing data from other teams in the field and archival sources.
Analysts can then leverage at-a-glance mobile visualizations and detailed immersive interactive visualizations to develop new hypotheses, identify potential errors or gaps in coverage, and adapt operational practices while in the field.

Current practices for fieldwork require analysts to first preplan their operations based on archival data from both previous collection efforts and archival data streams.
Analysts then collect new data either on mobile devices or in field notebooks, physically transport data to a central location to synchronize with other sources, and replan subsequent collection efforts and operational practices based on the newly revised data. 
This workflow decontextualizes collected data from the environment, prevents analysts in time-critical situations from reacting to new data, and obscures possible errors in data collection. 
These challenges are exacerbated when data comes from multiple sources: analysts have limited access to additional data collected from other field teams or independent sensors. 
Our proposed workflow leverages cloud datastores to integrate data from multiple sources as data becomes available, leveraging both local and global data storage to overcome connectivity limitations in the field.
We additionally extend existing practices for field data analysis by allowing analysts to conduct rapid inference using mobile visualizations or immersive embedded visualizations that contextualize data within the physical environment and leverage visual context to support nuanced analyses.
This workflow increases contextual awareness to improve data-oriented operational goals, provide real time data quality assessment, and support synchronous operations between field teams. 

We implement and evaluate this approach in FieldView, an extensible mobile data collection and visualization \ds{prototype}
designed to bridge the spatial and temporal gaps between data collection and analysis in the field. 
We developed FieldView in collaboration with field analysts in domains ranging from forest ecology to public safety. 
We discuss how our approach addresses several challenges in field analysis and refine our approach 
using observations from ten domain experts. We demonstrate our approach using three scenarios for field analysis inspired by our discussions with field analysts and unsupported by existing systems: data-mediated teaming, data quality verification, and
fusion with autonomous sensors. By coupling mobile data collection and immersive analytics, our approach can foster improved data collection, interpretation, and decision making by providing field analysts with increased access to incoming data and allowing analysts to interpret that data with respect to aspects of the environment only available in the field. 

\textbf{Contributions: } We introduce a novel field data collection and analysis workflow 
\ds{that enables situated} decision-making in the field. 
We conducted interviews with field analysts in domains 
\ds{such as} environmental science and aerial firefighting to understand limitations in current field data use and synthesize critical design requirements for 
real-time data analysis tools. 
We implement \ds{these requirements} in
an extensible prototype 
leveraging commodity technologies to support field analysis. 
Building on scenarios identified by analysts, we ground our prototype in three use cases 
demonstrating how our 
approach \ds{may} address limitations in field analysis to increase operational efficiency.










\end{comment}
\section{Related Work}
\label{sec:RelatedWork}
Bridging the gap between data collection and analysis for field work requires
understanding how contextual awareness can inform analysis 
and the technologies and approaches that might enable such solutions. 
We focus on developing methods that provide field analysts ready access to important information to improve the quality of their data collection practices, interpretation, and operational decision making.
We build on 
work in situational awareness and contextual computing in HCI as well as mobile and immersive analytics 
to support these goals. 

\subsection{Contextual Awareness}
Current practices for analyzing field data limit contextual and situational awareness (SA). Technologies may consider many forms of SA. For example, Endesley et al. models three hierarchical phases of SA in dynamic systems \cite{endsley1995toward}: (1) perceiving elements in the environment, (2) comprehending the active situation based on disjoint elements, and (3) projecting the future status of a situation based on the status and dynamics of the current situation.
Systems can increase SA across these phases by designing for change detection, preparedness for interruption, goal reorientation, and detection of missed changes \cite{john2008staying}.


Increasing SA is especially critical in technologies for highly dynamic situations where new data frequently changes operational strategies, such as search-and-rescue and emergency response.
For example, Cao et al. 
found that people trusted their own situated understanding over the advice of agencies in predicting wildfire spread \cite{cao2017smoke}.
Kim et al. increase field responder SA by visualizing location-based data on mobile devices, allowing users to ``see through the fog'' \cite{kim2008mobile}. 
While these solutions increase awareness of areas invisible to users, they require divided attention and map complex data to a limited visual space.
We address these limitations by exploring how AR may supplement existing mobile data collection and visualization approaches, bypassing the need for device-to-environment context shifts by situating data directly within the physical environment to increase contextual awareness.

We ground our exploration of 
\review{SA} in visualization systems. These systems allow analysts to bring domain expertise and contextual awareness to data exploration and decision making by allowing analysts rather than algorithms to synthesize insights from available data.
For example, Chan et al. uses multiple spatially aware displays to increase SA in a command center \cite{Chan2016}.
Visual analytics systems may support 
\review{field work} by reducing reliance on verbal/radio communication, integrating information from multiple sources, and combining both streaming and manual data entry \cite{goubran2016modelling} as well as improving collaborative teaming \cite{kulyk2008situational}.
These technologies aim to provide domain experts 
with sufficient knowledge about the global state of the operational environment to make data-informed decisions in dynamic environments.
However, 
these solutions generally target a remote operator with a global view of the task at hand but limited understanding of the context of field data.
We instead aim to put these visualizations in the field, considering how mobile technologies and immersive visualization can support field analysis needs 
while mitigating limiting factors of current mobile approaches like divided attention and limited \review{screen} size.

\subsection{Mobile Data Collection and Visualization} 
\label{sec:MobileRelatedWork}
Mobile devices support portable data collection and sensing across a 
variety of applications \cite{ganti2011mobile,reades2007cellular,lane2010survey,Parikh2006Mobile,choe2015sleeptight,jo2017touchpivot,chen2019marvist}. 
Using mobile devices for data entry can enhance data quality over traditional physical map-based methods \cite{ryan1999enhanced, fletcher2003handheld}.
For example, Pascoe et al. discuss user interfaces for field data collection that
minimize the attention needed to use a device while maximizing benefits of contextual awareness \cite{pascoe2000using}. 
Tomlinson et al. 
show how mobile devices can support distributed data collection efforts \cite{tomlinson2009use}.
ESCAPE provides a middle-ware 
for exchanging information between mobile devices in emergency response \cite{truong2007escape}.
These studies show that simplicity and contextual awareness are critical components of successful field data collection. 

However, analyzing data on mobile devices can be difficult due to the limited display, storage, and computation capabilities of these technologies \cite{chittaro2006visualizing, langner2018v, melo2017hdr}. These challenges are exacerbated in field scenarios and extended to include other challenges such as decision time and privacy concerns \cite{flentge2008designing}.
Solutions like Siren leverage context-aware paradigms to support peer-to-peer messaging based on environmental triggers, choosing to reduce cognitive load through automated analysis based on contextual data \cite{jiang2004siren}.
Though mobile technologies have limited display size and resolution, they readily support location-based visual analysis, including mobile tourist guides \cite{burigat2005location}, overview-detail map visualizations \cite{karstens2005presenting} and mobile device-mediated navigation tools \cite{burigat2007geographical, rakkolainen20013d}. These visualizations generally focus on a few critical data attributes 
to increase analysts' abilities to get important information at a glance \cite{wiehr2013drivesense}. 
While these systems provide useful at-a-glance overviews for general analysis or detail views for well-defined tasks, they use a small form factor that divides both data and analyst attention between the screen and the operational environment, limiting contextual integration into analysis and decision making. 
We build on these approaches to support location-based data collection and overview analysis and extend these approaches with immersive AR visualizations to enable deeper engagement with data. These AR visualizations blend data and context to more closely couple data exploration with the operational environment for applications ranging from forest ecology to public safety.


\subsection{Immersive Visualization for Fieldwork}
Immersive Analytics (IA) uses embodied data analysis for data understanding and decision making, generally through 3D virtual or augmented reality.
IA offers opportunities that standard 2D displays do not, including benefits associated with situated analytics, embodied data exploration, collaboration, and increased engagement \cite{marriott2018immersive}. 
\review{Prior work in IA} focuses on the benefits of situated analytics, especially how the use of augmented reality (AR) can enable contextually-aware data analysis by outfitting the physical environment with virtual data
(see Schmalsteig \& H\"ollerer for a survey \cite{schmalstieg2016augmented}).
These techniques can leverage data
from disparate environments for situated analysis \cite{elmqvist2013ubiquitous}.
Such techniques include mid-air displays for map navigation \cite{dancu2015map} 
and cross-device displays for collaborative visualization \cite{badam2017visfer}.

Systems have used AR to directly annotate the environment with data for applications in construction and architectural oversight \cite{irizarry2013infospot}, energy aware smart homes \cite{jahn2010energy}, manufacturing \cite{Lukosch2015Providing,bell2002annotated}, and situated learning \cite{Santos2016,vcopivc2016playing}.
Other systems use AR to highlight critical information along 3D surfaces including rock folds \cite{gazconimmersive}, mammograms \cite{douglas2017augmented}, and physical bodies \cite{hoang2017augmented}. 
In our target domain, visualization overlays on proximal referents can help highlight previously searched paths in search-and-rescue \cite{willett2017embedded}.
These techniques offer interactive methods for engaging with spatial data that may outperform other mobile technologies such as tablets \cite{bach2018hologram}.


In fieldwork, these technologies may help resolve the ``field map shuffle'' where researchers constantly reference several potentially out-of-date sources for geographical 
\review{data} \cite{pavlis2010computer}.
For example, McCaffrey et al. 
theorize how portable stereo rendering might enable field-based visualization for complex geospatial models \cite{mccaffrey2005unlocking}. Pavlis et al. extend this vision to suggest how immersive visualization may improve map synthesis \cite{pavlis2010computer}. 
However, these efforts offer purely theoretical insight into such designs. 
More recent works have 
begun to explore preliminary designs for targeted analysis problems in field research.
For example, Gazc{\'o}n et al. 
allow field researchers to augment an existing view of mountains with lines highlighting geological folding patterns \cite{gazconimmersive}.
Ramakrishnaraja et al. 
visualize sensor data in AR to increase SA for oil and gas workers \cite{ramakrishnaraja2017unaware}.
However, these systems focus on a narrow set of well-defined analysis tasks and do not consider how field analysts can integrate incoming information. We extend these ideas to enable holistic situated analysis for field domains. We do so by first systematically understanding how immersive situated analysis in tandem with mobile devices
may better support data-oriented fieldwork.


\section{Preliminary Requirements Analysis}
\label{requirements}
\review{We aim to understand how visual analytics can improve field operations by bridging spatial and temporal gaps in field data usage.}
To characterize the problem space, we enumerate the anticipated requirements of \review{field VA systems} and identify limitations in existing approaches 
\review{in} preliminary \review{unstructured} interviews with four field analysts: one in public safety, two in wildland fire, and one in geological science.
\review{We synthesized challenges and requirements noted by all participants to derive four preliminary design considerations. Our design probe embodies these design considerations to elicit insight into how visual analytics can enhance field practices.} \reviewnew{While we found no studies specific to field data needs, several points raised in the interviews exemplify general data and technological challenges in emergency response. }

In these interviews, analysts \review{universally} identified \reviewnew{four} primary limitations that prevent them from adequately leveraging data in 
\review{the field:} divided attention, small form-factors, \reviewnew{touchscreen input,} and reliance on operations centers. \reviewnew{The first two factors echo concerns for mobile devices in GIS work \cite{burigat2007geographical}.}
Analysts currently preplan daily goals based on archival data from prior operations before entering the field, using mobile devices and paper to collect new data in the field.
\review{Due to limited support for data collection in the field, experts' current data consists of digital images collected during pre-op aerial fly-overs (wildland fire and public safety), archival data printed onto paper maps (e.g., topography, structural locations, and prior survey data; all domains), simple quantitative measurements from portable sensors streamed to a portable drive (e.g., canopy density, soil acidity; geological science and wildland fire), and hand-written qualitative observations (geological science). Analysts reported only having preplanning data and handwritten observations while on a mission and had no methods for engaging in any analysis tasks in the field.}
They \review{instead} synchronize and analyze \review{new} data after returning to the operations center or at a remote site after leaving the field, \reviewnew{forcing operations centers to resolve data collection inconsistencies \cite{Chan2016}}.
Current mobile-only solutions offer insights into how data could inform field practices, but require analysts to use notes and memories to integrate environmental context back into their analyses. 
Mobile visualizations offer a potential alternative, but 
\review{divide attention} between the display and the operational environment, introducing hazards similar to texting and driving \cite{lamble1999cognitive}.

These limitations lead to heavier reliance on external analysis at an operations center or field site, but limited connectivity can cause information loss and substantial delays in data exchange. 
Operations centers also lack the contextual awareness of field analysts \reviewnew{as noted in Flentge et. al. \cite{flentge2008designing}},
while field analysts lack the awareness of multiple fieldsites granted to remote operators \reviewnew{as noted in Kim et. al. \cite{kim2008mobile}}. 
\review{As a result,} field analysts \review{enumerated three options for using data in current practices:} operate agnostic of field collected data, use stale data from daily preplanning, or rely on strictly verbal communications from the operations center. 
\review{All analysts felt that visual analytics tools could transform these options by increasing situational awareness in the field. The discussions made clear that field practices offer a rich design space that differs from conventional visual analytics tools. However, analysts had a limited sense of the capabilities of current devices for situated analysis and had difficulty reasoning about how these technologies might best augment their workflows. Our interviews elicited four design considerations for field analysis agreed upon by all participants: }  

\vspace{3pt}
\noindent
\textbf{R1--Offline \& Distributed Data Collection \review{\& Analysis}:} Field analysts often work in remote locations with limited to no connectivity\review{, inhibiting fully cloud-based solutions} \reviewnew{\cite{flentge2008designing}}. 
Most field analysts collect and log data in the field, store that the data on portable thumbdrives, and fuse that data with data from other efforts back at the remote operations center or lab. 
\review{Analysts noted that variations in data collection practices used by different teams can lead to format inconsistencies that further inhibit data sharing between teams.} \review{Aggregating and} visualizing new data in real time would improve decision making for team operations when connected and provide timely perspectives on their current situations when offline. 

\noindent
\textbf{R2--Merge Environmental Context \& Analysis:} Data collection in fieldwork attempts to capture important aspects of a fieldsite to build a broader understanding of the changing environments, functions, and scenarios present in the world \reviewnew{\cite{cutter2003gi}}. 
However, not all aspects of an environment can be quantified and captured in a database. 
Therefore, building effective tools for situated field analysis requires support for analysts to assess incoming data in tandem with its environmental context to increase contextual awareness and enrich decision making.

\noindent
\textbf{R3--Mitigate Information Overload:} Analysts often multitask in the field. They therefore need to get a quick sense of how their data fits into the current context without having to spend significant amounts of time immersed in the data. \reviewnew{Traditional desktop analysis systems are too complex for use in the field \cite{cutter2003gi,flentge2008designing}}. 
As a result, situated field analysis systems should balance specificity with flexibility: analysts need to rapidly generate important insights without incurring substantial cognitive overhead to build these insights. 

\noindent
\textbf{R4--Use in Outdoor Environments:} Operations in outdoor environments require analysts to carry any necessary equipment. Field analytics systems must be sufficiently portable \review{and, when possible,} robust 
to bulky equipment like gloves \reviewnew{\cite{burigat2007geographical}}.


\review{While we intended to use an iterative, user-centered design process, the lack of shared understanding about capabilities of IA made it difficult to elicit meaningful information about specific tasks and potential designs.}
To this end, we \review{used our results to build} a design probe 
to scaffold in-depth interviews about visual analytics for fieldwork.



\section{Introduction}
This template is for papers of VGTC-sponsored conferences such as IEEE VIS, IEEE VR, and ISMAR which are published as special issues of TVCG. The template does not contain the respective dates of the conference/journal issue, these will be entered by IEEE as part of the publication production process. Therefore, \textbf{please leave the copyright statement at the bottom-left of this first page untouched}.

\section{Using the Style Template}

\begin{itemize}
\item If you receive compilation errors along the lines of ``\texttt{Package ifpdf Error: Name clash, \textbackslash ifpdf is already defined}'' then please add a new line ``\texttt{\textbackslash let\textbackslash ifpdf\textbackslash relax}'' right after the ``\texttt{\textbackslash documentclass[journal]\{vgtc\}}'' call. Note that your error is due to packages you use that define ``\texttt{\textbackslash ifpdf}'' which is obsolete (the result is that \texttt{\textbackslash ifpdf} is defined twice); these packages should be changed to use ifpdf package instead.
\item Note that each author's affiliations have to be provided in the author footer on the bottom-left corner of the first page. It is permitted to merge two or more people from the same institution as long as they are shown in the same order as in the overall author sequence on the top of the first page. For example, if authors A, B, C, and D are from institutions 1, 2, 1, and 2, respectively, then it is ok to use 2 bullets as follows:
\begin{itemize}
\item A and C are with Institution 1. E-mail: \{a\,$|$\,c\}@i1.com\,.
\item B and D are with Institution 2. E-mail: \{b\,$|$\,d\}@i2.org\,.
\end{itemize}
\item The style uses the hyperref package, thus turns references into internal links. We thus recommend to make use of the ``\texttt{\textbackslash autoref\{reference\}}'' call (instead of ``\texttt{Figure\~{}\textbackslash ref\{reference\}}'' or similar) since ``\texttt{\textbackslash autoref\{reference\}}'' turns the entire reference into an internal link, not just the number. Examples: \autoref{fig:sample} and \autoref{tab:vis_papers}.
\item The style automatically looks for image files with the correct extension (eps for regular \LaTeX; pdf, png, and jpg for pdf\LaTeX), in a set of given subfolders (figures/, pictures/, images/). It is thus sufficient to use ``\texttt{\textbackslash includegraphics\{CypressView\}}'' (instead of ``\texttt{\textbackslash includegraphics\{pictures/CypressView.jpg\}}'').
\item For adding hyperlinks and DOIs to the list of references, you can use ``\texttt{\textbackslash bibliographystyle\{abbrv-doi-hyperref-narrow\}}'' (instead of ``\texttt{\textbackslash bibliographystyle\{abbrv\}}''). It uses the doi and url fields in a bib\TeX\ entry and turns the entire reference into a link, giving priority to the doi. The doi can be entered with or without the ``\texttt{http://dx.doi.org/}'' url part. See the examples in the bib\TeX\ file and the bibliography at the end of this template.\\[1em]
\textbf{Note 1:} occasionally (for some \LaTeX\ distributions) this hyper-linked bib\TeX\ style may lead to \textbf{compilation errors} (``\texttt{pdfendlink ended up in different nesting level ...}'') if a reference entry is broken across two pages (due to a bug in hyperref). In this case make sure you have the latest version of the hyperref package (i.\,e., update your \LaTeX\ installation/packages) or, alternatively, revert back to ``\texttt{\textbackslash bibliographystyle\{abbrv-doi-narrow\}}'' (at the expense of removing hyperlinks from the bibliography) and try ``\texttt{\textbackslash bibliographystyle\{abbrv-doi-hyperref-narrow\}}'' again after some more editing.\\[1em]
\textbf{Note 2:} the ``\texttt{-narrow}'' versions of the bibliography style use the font ``PTSansNarrow-TLF'' for typesetting the DOIs in a compact way. This font needs to be available on your \LaTeX\ system. It is part of the \href{https://www.ctan.org/pkg/paratype}{``paratype'' package}, and many distributions (such as MikTeX) have it automatically installed. If you do not have this package yet and want to use a ``\texttt{-narrow}'' bibliography style then use your \LaTeX\ system's package installer to add it. If this is not possible you can also revert to the respective bibliography styles without the ``\texttt{-narrow}'' in the file name.\\[1em]
DVI-based processes to compile the template apparently cannot handle the different font so, by default, the template file uses the \texttt{abbrv-doi} bibliography style but the compiled PDF shows you the effect of the \texttt{abbrv-doi-hyperref-narrow} style.
\end{itemize}

\section{Bibliography Instructions}

\begin{itemize}
\item Sort all bibliographic entries alphabetically but the last name of the first author. This \LaTeX/bib\TeX\ template takes care of this sorting automatically.
\item Merge multiple references into one; e.\,g., use \cite{Max:1995:OMF,Kitware:2003} (not \cite{Kitware:2003}\cite{Max:1995:OMF}). Within each set of multiple references, the references should be sorted in ascending order. This \LaTeX/bib\TeX\ template takes care of both the merging and the sorting automatically.
\item Verify all data obtained from digital libraries, even ACM's DL and IEEE Xplore  etc.\ are sometimes wrong or incomplete.
\item Do not trust bibliographic data from other services such as Mendeley.com, Google Scholar, or similar; these are even more likely to be incorrect or incomplete.
\item Articles in journal---items to include:
  \begin{itemize}
  \item author names
	\item title
	\item journal name
	\item year
	\item volume
	\item number
	\item month of publication as variable name (i.\,e., \{jan\} for January, etc.; month ranges using \{jan \#\{/\}\# feb\} or \{jan \#\{-{}-\}\# feb\})
  \end{itemize}
\item use journal names in proper style: correct: ``IEEE Transactions on Visualization and Computer Graphics'', incorrect: ``Visualization and Computer Graphics, IEEE Transactions on''
\item Papers in proceedings---items to include:
  \begin{itemize}
  \item author names
	\item title
	\item abbreviated proceedings name: e.\,g., ``Proc.\textbackslash{} CONF\_ACRONYNM'' without the year; example: ``Proc.\textbackslash{} CHI'', ``Proc.\textbackslash{} 3DUI'', ``Proc.\textbackslash{} Eurographics'', ``Proc.\textbackslash{} EuroVis''
	\item year
	\item publisher
	\item town with country of publisher (the town can be abbreviated for well-known towns such as New York or Berlin)
  \end{itemize}
\item article/paper title convention: refrain from using curly brackets, except for acronyms/proper names/words following dashes/question marks etc.; example:
\begin{itemize}
	\item paper ``Marching Cubes: A High Resolution 3D Surface Construction Algorithm''
	\item should be entered as ``\{M\}arching \{C\}ubes: A High Resolution \{3D\} Surface Construction Algorithm'' or  ``\{M\}arching \{C\}ubes: A high resolution \{3D\} surface construction algorithm''
	\item will be typeset as ``Marching Cubes: A high resolution 3D surface construction algorithm''
\end{itemize}
\item for all entries
\begin{itemize}
	\item DOI can be entered in the DOI field as plain DOI number or as DOI url; alternative: a url in the URL field
	\item provide full page ranges AA-{}-BB
\end{itemize}
\item when citing references, do not use the reference as a sentence object; e.\,g., wrong: ``In \cite{Lorensen:1987:MCA} the authors describe \dots'', correct: ``Lorensen and Cline \cite{Lorensen:1987:MCA} describe \dots''
\end{itemize}

\section{Example Section}

Lorem\marginpar{\small You can use the margins for comments while editing the submission, but please remove the marginpar comments for submission.} ipsum dolor sit amet, consetetur sadipscing elitr, sed diam
nonumy eirmod tempor invidunt ut labore et dolore magna aliquyam erat,
sed diam voluptua. At vero eos et accusam et justo duo dolores et ea
rebum. Stet clita kasd gubergren, no sea takimata sanctus est Lorem
ipsum dolor sit amet. Lorem ipsum dolor sit amet, consetetur
sadipscing elitr, sed diam nonumy eirmod tempor invidunt ut labore et
dolore magna aliquyam erat, sed diam
voluptua~\cite{Kitware:2003,Max:1995:OMF}. At vero eos et accusam et
justo duo dolores et ea rebum. Stet clita kasd gubergren, no sea
takimata sanctus est Lorem ipsum dolor sit amet. Lorem ipsum dolor sit
amet, consetetur sadipscing elitr, sed diam nonumy eirmod tempor
invidunt ut labore et dolore magna aliquyam erat, sed diam
voluptua. At vero eos et accusam et justo duo dolores et ea
rebum. Stet clita kasd gubergren, no sea takimata sanctus est.

\section{Exposition}

Duis autem vel eum iriure dolor in hendrerit in vulputate velit esse
molestie consequat, vel illum dolore eu feugiat nulla facilisis at
vero eros et accumsan et iusto odio dignissim qui blandit praesent
luptatum zzril delenit augue duis dolore te feugait nulla
facilisi. Lorem ipsum dolor sit amet, consectetuer adipiscing elit,
sed diam nonummy nibh euismod tincidunt ut laoreet dolore magna
aliquam erat volutpat~\cite{Kindlmann:1999:SAG}.

\begin{equation}
\sum_{j=1}^{z} j = \frac{z(z+1)}{2}
\end{equation}

Lorem ipsum dolor sit amet, consetetur sadipscing elitr, sed diam
nonumy eirmod tempor invidunt ut labore et dolore magna aliquyam erat,
sed diam voluptua. At vero eos et accusam et justo duo dolores et ea
rebum. Stet clita kasd gubergren, no sea takimata sanctus est Lorem
ipsum dolor sit amet. Lorem ipsum dolor sit amet, consetetur
sadipscing elitr, sed diam nonumy eirmod tempor invidunt ut labore et
dolore magna aliquyam erat, sed diam voluptua. At vero eos et accusam
et justo duo dolores et ea rebum. Stet clita kasd gubergren, no sea
takimata sanctus est Lorem ipsum dolor sit amet.

\subsection{Lorem ipsum}

Lorem ipsum dolor sit amet (see \autoref{tab:vis_papers}), consetetur sadipscing elitr, sed diam
nonumy eirmod tempor invidunt ut labore et dolore magna aliquyam erat,
sed diam voluptua. At vero eos et accusam et justo duo dolores et ea
rebum. Stet clita kasd gubergren, no sea takimata sanctus est Lorem
ipsum dolor sit amet. Lorem ipsum dolor sit amet, consetetur
sadipscing elitr, sed diam nonumy eirmod tempor invidunt ut labore et
dolore magna aliquyam erat, sed diam voluptua. At vero eos et accusam
et justo duo dolores et ea rebum. Stet clita kasd gubergren, no sea
takimata sanctus est Lorem ipsum dolor sit amet. Lorem ipsum dolor sit
amet, consetetur sadipscing elitr, sed diam nonumy eirmod tempor
invidunt ut labore et dolore magna aliquyam erat, sed diam
voluptua. At vero eos et accusam et justo duo dolores et ea
rebum. 

\begin{table}[tb]
  \caption{VIS/VisWeek accepted/presented papers: 1990--2016.}
  \label{tab:vis_papers}
  \scriptsize%
	\centering%
  \begin{tabu}{%
	r%
	*{7}{c}%
	*{2}{r}%
	}
  \toprule
   year & \rotatebox{90}{Vis/SciVis} &   \rotatebox{90}{SciVis conf} &   \rotatebox{90}{InfoVis} &   \rotatebox{90}{VAST} &   \rotatebox{90}{VAST conf} &   \rotatebox{90}{TVCG @ VIS} &   \rotatebox{90}{CG\&A @ VIS} &   \rotatebox{90}{VIS/VisWeek} \rotatebox{90}{incl. TVCG/CG\&A}   &   \rotatebox{90}{VIS/VisWeek} \rotatebox{90}{w/o TVCG/CG\&A}   \\
  \midrule
	2016 & 30 &   & 37 & 33 & 15 & 23 & 10 & 148 & 115 \\
  2015 & 33 & 9 & 38 & 33 & 14 & 17 & 15 & 159 & 127 \\
  2014 & 34 &   & 45 & 33 & 21 & 20 &   & 153 & 133 \\
  2013 & 31 &   & 38 & 32 &   & 20 &   & 121 & 101 \\
  2012 & 42 &   & 44 & 30 &   & 23 &   & 139 & 116 \\
  2011 & 49 &   & 44 & 26 &   & 20 &   & 139 & 119 \\
  2010 & 48 &   & 35 & 26 &   &   &   & 109 & 109 \\
  2009 & 54 &   & 37 & 26 &   &   &   & 117 & 117 \\
  2008 & 50 &   & 28 & 21 &   &   &   & 99 & 99 \\
  2007 & 56 &   & 27 & 24 &   &   &   & 107 & 107 \\
  2006 & 63 &   & 24 & 26 &   &   &   & 113 & 113 \\
  2005 & 88 &   & 31 &   &   &   &   & 119 & 119 \\
  2004 & 70 &   & 27 &   &   &   &   & 97 & 97 \\
  2003 & 74 &   & 29 &   &   &   &   & 103 & 103 \\
  2002 & 78 &   & 23 &   &   &   &   & 101 & 101 \\
  2001 & 74 &   & 22 &   &   &   &   & 96 & 96 \\
  2000 & 73 &   & 20 &   &   &   &   & 93 & 93 \\
  1999 & 69 &   & 19 &   &   &   &   & 88 & 88 \\
  1998 & 72 &   & 18 &   &   &   &   & 90 & 90 \\
  1997 & 72 &   & 16 &   &   &   &   & 88 & 88 \\
  1996 & 65 &   & 12 &   &   &   &   & 77 & 77 \\
  1995 & 56 &   & 18 &   &   &   &   & 74 & 74 \\
  1994 & 53 &   &   &   &   &   &   & 53 & 53 \\
  1993 & 55 &   &   &   &   &   &   & 55 & 55 \\
  1992 & 53 &   &   &   &   &   &   & 53 & 53 \\
  1991 & 50 &   &   &   &   &   &   & 50 & 50 \\
  1990 & 53 &   &   &   &   &   &   & 53 & 53 \\
  \midrule
  \textbf{sum} & \textbf{1545} & \textbf{9} & \textbf{632} & \textbf{310} & \textbf{50} & \textbf{123} & \textbf{25} & \textbf{2694} & \textbf{2546} \\
  \bottomrule
  \end{tabu}%
\end{table}

\subsection{Mezcal Head}

Lorem ipsum dolor sit amet (see \autoref{fig:sample}), consetetur sadipscing elitr, sed diam
nonumy eirmod tempor invidunt ut labore et dolore magna aliquyam erat,
sed diam voluptua. At vero eos et accusam et justo duo dolores et ea
rebum. Stet clita kasd gubergren, no sea takimata sanctus est Lorem
ipsum dolor sit amet. Lorem ipsum dolor sit amet, consetetur
sadipscing elitr, sed diam nonumy eirmod tempor invidunt ut labore et
dolore magna aliquyam erat, sed diam voluptua. At vero eos et accusam
et justo duo dolores et ea rebum. Stet clita kasd gubergren, no sea
takimata sanctus est Lorem ipsum dolor sit amet. 

\subsubsection{Duis Autem}

Lorem ipsum dolor sit amet, consetetur sadipscing elitr, sed diam
nonumy eirmod tempor invidunt ut labore et dolore magna aliquyam erat,
sed diam voluptua. At vero eos et accusam et justo duo dolores et ea
rebum. Stet clita kasd gubergren, no sea takimata sanctus est Lorem
ipsum dolor sit amet. Lorem ipsum dolor sit amet, consetetur
sadipscing elitr, sed diam nonumy eirmod tempor invidunt ut labore et
dolore magna aliquyam erat, sed diam voluptua. At vero eos et accusam
et justo duo dolores et ea rebum. Stet clita kasd gubergren, no sea
takimata sanctus est Lorem ipsum dolor sit amet. Lorem ipsum dolor sit
amet, consetetur sadipscing elitr, sed diam nonumy eirmod tempor
invidunt ut labore et dolore magna aliquyam erat, sed diam
voluptua. At vero eos et accusam et justo duo dolores et ea
rebum. Stet clita kasd gubergren, no sea takimata sanctus est. Lorem
ipsum dolor sit amet.

\begin{figure}[tb]
 \centering
 \includegraphics[width=\columnwidth]{paper-count-w-2015-new}
 \caption{A visualization of the 1990--2015 data from \autoref{tab:vis_papers}. The image is from \cite{Isenberg:2017:VMC} and is in the public domain.}
 \label{fig:sample}
\end{figure}

\subsubsection{Ejector Seat Reservation}

Duis autem~\cite{Lorensen:1987:MCA}\footnote{The algorithm behind
Marching Cubes \cite{Lorensen:1987:MCA} had already been
described by Wyvill et al. \cite{Wyvill:1986:DSS} a year
earlier.} vel eum iriure dolor in hendrerit
in vulputate velit esse molestie consequat,\footnote{Footnotes
appear at the bottom of the column.} vel illum dolore eu
feugiat nulla facilisis at vero eros et accumsan et iusto odio
dignissim qui blandit praesent luptatum zzril delenit augue duis
dolore te feugait nulla facilisi. Lorem ipsum dolor sit amet,
consectetuer adipiscing elit, sed diam nonummy nibh euismod tincidunt
ut laoreet dolore magna aliquam erat volutpat.


\paragraph{Confirmed Ejector Seat Reservation}

Ut wisi enim ad minim veniam, quis nostrud exerci tation ullamcorper
suscipit lobortis nisl ut aliquip ex ea commodo
consequat~\cite{Nielson:1991:TAD}. Duis autem vel eum iriure dolor in
hendrerit in vulputate velit esse molestie consequat, vel illum dolore
eu feugiat nulla facilisis at vero eros et accumsan et iusto odio
dignissim qui blandit praesent luptatum zzril delenit augue duis
dolore te feugait nulla facilisi.

\paragraph{Rejected Ejector Seat Reservation}

Ut wisi enim ad minim veniam, quis nostrud exerci tation ullamcorper
suscipit lobortis nisl ut aliquip ex ea commodo consequat. Duis autem
vel eum iriure dolor in hendrerit in vulputate velit esse molestie

\subsection{Vestibulum}

Vestibulum ut est libero. Suspendisse non libero id massa congue egestas nec at ligula. Donec nibh lorem, ornare et odio eu, cursus accumsan felis. Pellentesque habitant morbi tristique senectus et netus et malesuada fames ac turpis egestas. Donec dapibus quam vel eros mattis, id ornare dolor convallis. Donec at nisl sapien. Integer fringilla laoreet tempor. Fusce accumsan ante vel augue euismod, sit amet maximus turpis mattis. Nam accumsan vestibulum rhoncus. Aenean quis pellentesque augue. Suspendisse sed augue et velit consequat bibendum id nec est. Quisque erat purus, ullamcorper ut ex vel, dapibus dignissim erat.

Quisque sit amet orci quam. Lorem ipsum dolor sit amet, consectetur adipiscing elit. Aliquam pharetra, nunc non efficitur convallis, tellus purus iaculis lorem, nec ultricies dolor ligula in metus. Class aptent taciti sociosqu ad litora torquent per conubia nostra, per inceptos himenaeos. Aenean eu ex nulla. Morbi id ex interdum, scelerisque lorem nec, iaculis dui. Phasellus ultricies nunc vitae quam fringilla laoreet. Quisque sed dolor at sem vestibulum fringilla nec ac augue. Ut consequat, velit ac mattis ornare, eros arcu pellentesque erat, non ultricies libero metus nec mi. Sed eget elit sed quam malesuada viverra. Quisque ullamcorper, felis ut convallis fermentum, purus ligula varius ligula, sit amet tempor neque dui non neque. Donec vulputate ultricies tortor in mollis.

Integer sit amet dolor sit amet turpis ullamcorper varius. Cras volutpat bibendum scelerisque. Maecenas mauris dolor, gravida eu elit et, sodales consequat tortor. Integer id commodo elit. Pellentesque sollicitudin ex non nulla molestie eleifend. Mauris sagittis metus nec turpis imperdiet, vel ullamcorper nibh tincidunt. Sed semper tempus ex, ut aliquet erat hendrerit id. Maecenas sit amet dolor sollicitudin, luctus nunc sit amet, malesuada justo.

Mauris ut sapien non ipsum imperdiet sodales sit amet ac diam. Nulla vel convallis est. Etiam dapibus augue urna. Aenean enim leo, fermentum quis pulvinar at, ultrices quis enim. Sed placerat porta libero et feugiat. Phasellus ullamcorper, felis id porta sollicitudin, dolor dui venenatis augue, vel fringilla risus massa non risus. Maecenas ut nulla vitae ligula pharetra feugiat non eu ante. Donec quis neque quis lorem cursus pretium ac vulputate quam. Cras viverra tellus vitae sapien pretium laoreet. Pellentesque fringilla odio venenatis ex viverra, quis eleifend tortor ornare. Ut ut enim nunc. Vivamus id ligula nec est dignissim eleifend.

Nunc ac velit tellus. Donec et venenatis mauris. Lorem ipsum dolor sit amet, consectetur adipiscing elit. Ut vitae lectus vel ante mollis congue. Vestibulum at cursus velit. Curabitur in facilisis enim. Vestibulum eget dui aliquet risus laoreet laoreet. Phasellus et est id magna interdum venenatis. Donec luctus vehicula justo sed laoreet. Quisque tincidunt suscipit augue, in molestie sem accumsan sed.
\section{Conclusion}

Lorem ipsum dolor sit amet, consetetur sadipscing elitr, sed diam
nonumy eirmod tempor invidunt ut labore et dolore magna aliquyam erat,
sed diam voluptua. At vero eos et accusam et justo duo dolores et ea
rebum. Stet clita kasd gubergren, no sea takimata sanctus est Lorem
ipsum dolor sit amet. Lorem ipsum dolor sit amet, consetetur
sadipscing elitr, sed diam nonumy eirmod tempor invidunt ut labore et
dolore magna aliquyam erat, sed diam voluptua. At vero eos et accusam
et justo duo dolores et ea rebum. Stet clita kasd gubergren, no sea
takimata sanctus est Lorem ipsum dolor sit amet. Lorem ipsum dolor sit
amet, consetetur sadipscing elitr, sed diam nonumy eirmod tempor
invidunt ut labore et dolore magna aliquyam erat, sed diam
voluptua. At vero eos et accusam et justo duo dolores et ea
rebum.


\acknowledgments{
The authors wish to thank A, B, and C. This work was supported in part by
a grant from XYZ (\# 12345-67890).}

\bibliographystyle{abbrv-doi}

