\section{Introduction}

Climate change is one of the most pressing threats globally with the potential to cause frequent or prolonged droughts in many areas of the world \citep{LeHouerou1996}. Infrequent or unpredictable rainfall and higher evapotranspiration due to increased temperature may lead to reduced soil moisture \citep{Kingston2009}. Soil moisture is critical to the growth of almost all arable crops globally impacting small subsistence farmers up to large industrial agricultural companies. Irrigation is required in many regions to maintain suitable soil moisture for crops and is one of the largest fresh water usages in the world \citep{Frenken2012}. Adequate soil moisture is critical for the optimal growth of crops and maximising yield. Non-optimal soil moistures can lead to crop failure which is a threat to the livelihood of farmers and can seriously endanger the robustness of the food supply chain that is vital in the modern world. 

Soil moisture has traditionally been monitored using ground sensors that are buried underground and can report the soil moisture in that soil column \citep{Pagay2016}. These sensors can be extremely costly both in direct equipment cost but also the labour required to embed them down to generally below one meter. These sensors then still come with the draw back that they can only report the soil moisture in that one specific location so a great number of sensors would be required to provide an accurate measure of soil moisture across a large area bringing with it an associated high cost. To avoid this, methods of interpolation have been developed that attempt to estimate the soil moisture between sensors by using algorithms based on values and distance from nearby sensors and other features at the target location \citep{Wahba1990}. Machine and deep learning have become recognised as a tool that can be used to help fight climate change across numerous domains \citep{Rolnick2019}.


\section{Background}

Soil moisture prediction into the future was previously performed through physical models. By taking an input of precipitation and temperature physical models or simple regression models could be used to predict soil moisture using predictions of weather. These models tended to have low accuracy and predict on a low-resolution providing generalisations of soil moisture on a coarse temporal scale. With the advent of machine learning model performance greatly improved and now fine scale climatic predictions such as nowcasting \citep{Agrawal2019} which provides on the spot predictions into very near timescales are possible. But predicting further into the future in a usable way is still a difficult problem. Recently deep learning has become a tool that excels in the field of time series prediction particularly with the use of recurrent neural networks (RNNs) \citep{Rumelhart1986} and long-short term memory RNNs (LSTMs) \citep{Hochreiter1997}. LSTMs have been shown to perform well in prediction of soil moisture values into the future, with increased predictive ability leading to savings in water usage \citep{Adeyemi2018}. For predicting soil moisture not just at sensor locations machine learning frameworks have been applied on satelite imagery using radar or vegetation information \citep{Efremova2019,Abbes2019} but these predictions are only accurate at surface level while many crop species are reliant on deeper soil moistures.

Vegetation health indices such as the normalised difference vegetation index (NDVI) \citep{Rouse1973} and the normalised difference water index (NDWI) \citep{Gao1996} have been used in agriculture and geospatial fields for years. These indices also provide proxy measure of other information about the available water content in plants and soil. Being able to predict these indices in the future as a time series problem would provide information about crop health and other features in the future. Previously NDVI has been predicted using different machine learning frameworks which are able to perform adequately at the task \citep{Das2016,Nay2018,Stas2016}.

Interpolation methods tend to rely on spatial features alone with methods such as inverse distance weighting, splining and kriging all seeing use \citep{Li2011}. These rely on the distance between measurements and provide some form of weights to determine the impact that distance has on the prediction. Combining interpolation methods with soil moisture predictions from LSTM's it becomes possible to predict soil moisture across entire areas into the future with a small number of sensors in a deep learning framework. 

Our aim was to build a pipeline (SMArtCast: \textbf{S}oil \textbf{M}oisture \textbf{Ar}tificial In\textbf{t}elligence fore\textbf{Cast}) that can take soil moisture data from sparse sensors and provide predictions of soil moisture across an entire area up to two weeks into the future at all sensor depths using satellite imagery to supplement features. This would be capable of providing advanced warning of soil moistures that are deleterious to crops as well as to provide insights into the effectiveness or requirements of irrigation leading to the maximisation of crop yields and to possible reduction in freshwater usage. We present evidence that using deep learning methodologies it is possible to predict multiple features into the future and interpolate soil moisture across an entire satellite image.


\section{Materials and Methods}
\label{gen_inst}
As input data, we use soil moisture measurements installed on 200 hectares of land.  Input measurements are taken from the embedded soil moisture sensors, each installed at 10 centimeters depth up until 120 cm depth; rainfall and temperature data; and high resolution satellite imagery with 13 spectral bands. 

\subsection{Soil Moisture Prediction}
\begin{figure}[!h]
    \centering
    \includegraphics[width = 8cm, height = 4cm]{ICLR1}
    \caption{Structure of the LSTM encoder decoder model for soil moisture prediction.}
    \label{fig:SM_lstm}
\end{figure}

The input of the proposed LSTM model is a matrix of features that include soil moisture, soil temperature, soil salinity and rainfall and provides an output of 14 days of soil moisture predictions, 1 per day of predictions. The model architecture is based on sequence to sequence learning using an LSTM encoder - decoder \citep{Sutskever2014}. The encoder layer and decoder layer both had 200 neurons and a \textit{tanh} activation function. The output of the decoder is then passed through a time distributed dense layer with 100 neurons and a final dense layer with a single neuron both with linear activation functions (see Fig. \ref{fig:SM_lstm}).


\subsection{Satellite imagery time series prediction}
Satellite imagery of the study site is acquired and then transformed into the normalised difference vegetation and water indices (NDVI and NDWI). These indices rely on the satellite image bandwidths in the Red, near infrared (NIR) and short-wave infrared (SWIR). 

\begin{equation}
 \begin{aligned}
  NDVI=\frac{NIR-Red}{NIR+Red}, \;\; NDWI= \frac{NIR-SWIR}{NIR+SWIR} 
 \end{aligned}
\end{equation}

These indices are per pixel calculations that are independent of the surrounding pixels meaning they can be translated into a simple time series prediction problem. The NDVI or NDWI images are converted into flattened matrices and then stacked to create a time series of each individual pixel through time. The model takes an input of the 5 previous NDVI images. Due to the irregular time interval of satellite imagery the 5 images were each coded with number of days from the target predicted image so that the output predictions are provided with a desired number of days in the future. The model again uses the sequence to sequence LSTM encoder-decoder structure \citep{Sutskever2014} with 50 neurons in each of the LSTM layers and then dense layers of 20 and 1 neurons. The output is a vector of the same length as the input with a single dimension representing one prediction per pixel. Once per pixel predictions are made on the latest imagery those predictions are reshaped into the structure of the initial image (Fig. \ref{fig:NDVI_lstm}). 

\begin{figure}[!h]
    \centering
    \includegraphics[width = 10cm]{ICLR4}
    \caption{Structure of the LSTM encoder decoder model for prediction of NDVI time series}
    \label{fig:NDVI_lstm}
\end{figure}


\subsection{Interpolation}
We used standard Gaussian kriging interpolation methods to determine soil moisture at all points between the sensors \citep{Yao2013} based on a linear distance function out from each sensor. We used soil moisture sensors and satellite imagery to supplement other features and provide a map to predict across. This was done on current data and then the outputs of the above predictions were used to produce future interpolation maps.

\section{Results and Discussion}

The LSTM predictions of soil moisture perform differently depending on the soil moisture depth that is being predicted. Generally, the average testing Root Mean Squared Error (RMSE) across the 14 days of prediction is highest at shallowest depths decreasing at the lower depths. It ranged between 2.4\% soil moisture error (value error) and 0.4\% soil moisture error with a mean of 1.23\% on soil moistures that vary between 15\% and 60\%. This outperforms previously published multiple linear regression models at 2\% \citep{Qiu2003} and is on par or outperforms \citet{Adeyemi2018}.


The LSTMs for NDVI and NDWI predictions were able to perform at a training RMSE of 0.027 and a testing RMSE of 0.065 on NDVIs that range between 0 and 1. The NDWI model has a training Mean Absolute Error (MAE) of  0.014 and a testing MAE of 0.02 on NDWI values that range between -0.3 to 0.55. 

We have observed variance in interpolation accuracy depending on the depth that was being predicted with the kriging scores (a proxy for accuracy varying between 0 and 1) between 0.82 – 0.97 with an average of 0.93 across all depths. 

The per pixel prediction maps are plotted in 3D across the X, Y and Z axes and coloured by soil moisture. Each soil moisture depth has its own predictive map produced and these are stacked so that depths can be viewed together (Fig.\ref{fig:3d}).

\begin{figure}[h]
    \centering
    \includegraphics[width = 12cm]{ICLR3}
    \caption{Left) Soil moisture interpolation across the satellite image at a single depth. Right) the interpolated soil moisture at all depths with each depth stacked on top of each other.}
    \label{fig:3d}
\end{figure}


The potential to predict soil moisture across wide areas into the future can help agriculturalists on all scales. Informed knowledge of soil moisture can allow for more precise irrigation regimes or targeted irrigation only in areas that need it leading to savings in water usage. While water is an essential part of agriculture it is also required for drinking water and nearly all manufacturing processes, with climate change the available freshwater supply is expected to reduce leading to increased competition for fresh water \citep{Elliott2014} and new methods will be required for reducing water usage of all industries. Advanced warning of soil moistures that prevent a risk to crops can help increase yield, as the human population increases and with climate change some land becomes unusable for agriculture \citep{Zhang2011} or even human habitation increased crop yields will be required to maintain an adequate food supply. Precision agricultural regimes backed up by technology and artificial intelligence can work to make agricultural industries more sustainable. 



\newpage


