
\section{Introduction}

Software is bloated. From single Unix commands \cite{Holzmann2015} to web browsers \cite{Qian2020}, most applications embed a part of code that is unnecessary to their correct operation. Several debloating tools have emerged in recent years~\cite{Sharif2018,Qian2020,Valero2020,Jiang2016,Rastogi2017,Qian2019} to address the security and maintenance issues posed by excessive code at various granularity levels. However, these works do not analyze the evolution of bloat over time. Understanding software bloat in the perspective of software evolution \cite{Hilton0M18,Spinellis17,TeytonFPB14} is crucial to promote debloating tools towards software developers. In particular, developers, when proposed to adapt a debloating tool, wonder if a piece of bloated code might be needed in coming releases, or what is the actual issue with bloat.\looseness=-1

This work proposes the first longitudinal analysis of software bloat. We focus on bloat among software dependencies \cite{Cox2019,Gustavsson20,BoldiG21, Valero2019} in the Java/Maven ecosystem. 
Bloated dependencies are software libraries that are unnecessarily part of software projects, i.e.\@\xspace, when the dependency is removed from the project, it still builds successfully.
In previous work \cite{Valero2020}, we showed that the Maven ecosystem is permeated with bloated dependencies, and that they are present even in well maintained Java projects. Our study revealed that software developers are keen on removing bloated dependencies, but that removing code is a complex socio-technical decision, which benefits from solid evidence about the actual benefits of debloating.

Motivated by these observations about bloated dependencies, we conduct a large scale empirical study about the evolution of these dependencies in Java projects. We analyze the emergence of bloat, the evolution of the dependencies statuses, and the impact of bloat on maintenance.
We have collected a unique dataset of \np{\nbCommitsNum} versions of dependency trees  from \np{\nbProjectsNum} open-source Java projects.
Each version of a tree is a snapshot of one project's dependencies, for which we determine a status, i.e.\@\xspace bloated or used.  
We rely on \textsc{DepClean}\xspace, the state-of-the-art tool to detect bloated dependencies in Maven projects. 
We analyze the evolution of  \np{\nbDependenciesNum} distinct dependencies per project and we observe that  \percent[q]{40493}{48469} of them are bloated at one point in time, in our dataset.

Our longitudinal analysis of bloated Java dependencies investigates both the evolution of bloat, as well as the relation between bloat and regular maintenance activities such as dependency updates.
We present original quantitative results regarding the evolution of bloated dependencies. We first show a clear increasing trend in the number of bloated dependencies.
Next, we investigate how the usage status of dependencies evolves over time.
This analysis is a key contribution of our work where we demonstrate that a dependency that is bloated is very likely to remain bloated over subsequent versions of a project.
We present the first observations about the impact of regular maintenance activities on software bloat.
Besides, we analyze the impact of Dependabot, a popular dependency management bot, on these activities. 
We show that developers regularly update bloated dependencies, and that many of these updates are suggested by Dependabot.
Furthermore, we systematically investigate the root of the bloat emergence,  and find that \ShowPercentage{1868}{\totalOriginDirect} of the bloated dependencies are bloated as soon as they are added in the dependency tree of a project.

\begin{figure*}[!t]
\minipage[t]{0.33\textwidth}%
    \includegraphics[origin=c,width=0.9\textwidth]{figs/pom.pdf}
    \caption{Dependency declaration.}
    \label{fig:pom}
\endminipage\hfill
\minipage[t]{0.33\textwidth}%
    \includegraphics[origin=c,width=0.9\textwidth]{figs/dt.pdf}
    \caption{Dependency tree.}
    \label{fig:dt}
\endminipage\hfill
\minipage[t]{0.33\textwidth}%
    \includegraphics[origin=c,width=0.9\textwidth]{figs/ut.pdf}
    \caption{Dependency usage.}
    \label{fig:du}
\endminipage
\end{figure*}


To summarize, the contributions of this paper are:
\begin{itemize}
    \item A longitudinal analysis of software dependencies' usage in \np{\nbCommitsNum} versions of Maven dependency trees. Our results confirm the generalized presence of bloated dependencies and show their increase over time. 
    \item A quantitative analysis of the stability of bloated dependencies: \np[\%]{89.2} of direct dependencies remain bloated. This is a concrete insight that motivates debloating dependencies. 
    \item A novel analysis of unnecessary updates on bloated dependencies made by a software bot. We find that developers often accept Dependabot's suggestions without considering if the dependency is actually used or not.
    \item A qualitative manual analysis of the origin of bloated dependencies, that reveals that adding dependencies is the principal reason that originates this type of software bloat. 
   
\end{itemize}

\section{Background}

\begin{figure*}[t]
  \centering
  \includegraphics[origin=c,width=0.9\textwidth]{figs/pipeline.pdf}
  \caption{Overview of our data collection pipeline. From a set of \np{\nbTotalProject} Java projects on GitHub, we analyze the usage status of the dependencies in \np{\nbProjectsNum} Maven projects over time, to produce a dataset of \np{\nbCommitsNum} dependency trees.}
  \label{fig:overview}
\end{figure*}

In this work, we consider a software project as a collection of Java source code files and configuration files organized to be build with Maven.\footnote{\url{https://maven.apache.org}}
In this section, we present the key concepts for the analysis of a project $p$ in the context of the set of its software dependencies, denoted as $\mathcal{D}$.

\begin{definition}
\textbf{Maven dependency: }
A Maven dependency defines a relationship between a project $p$ and another compiled project $d \in \mathcal{D}$. 
Dependencies are  compiled JAR files, uniquely identified with a triplet (\texttt{G:A:V}) where \texttt{G} is the \texttt{groupId}, \texttt{A} is the \texttt{artifactId}, and \texttt{V} is the \texttt{version}.
Dependencies are defined within a scope, which determines at which phase of the Maven build cycle the dependency is required (e.g.\@\xspace, \texttt{compile}, \texttt{test}, \texttt{runtime}).
\end{definition}

A Maven project declares a set of direct  dependencies in a specific configuration file known as \texttt{pom.xml}\xspace (acronym for “Project Object Model”), located at the root of the project.
\autoref{fig:pom} shows an excerpt of the dependency declaration in the \texttt{pom.xml}\xspace of a project $p$.
In this example, developers explicitly declare the usage of three dependencies: $d_1$, $d_2$, and $d_3$.
Note that the \texttt{pom.xml}\xspace of a Maven project is a configuration file subject to constant change and evolution: developers usually commit changes to add, remove, or update the version of a dependency.

\begin{definition}
\textbf{Direct dependency: }
The set of direct dependencies $\mathcal{D}_\text{direct} \subset \mathcal{D}$ of a project $p$ is the set of dependencies declared in $p$'s \texttt{pom.xml}\xspace file.
Direct dependencies are  declared in the \texttt{pom.xml}\xspace by the developers, who explicitly manifest the intention of using the dependency.
\end{definition}

\begin{definition}
\textbf{Transitive dependency:}
The set of transitive dependencies $\mathcal{D}_\text{transitive} \subset \mathcal{D}$ of a project $p$ 
is the set of dependencies obtained from the transitive closure of direct dependencies. 
Transitive dependencies are resolved automatically by Maven, which means that developers do not need to explicitly declare these dependencies.
\end{definition}

\begin{definition}
\textbf{Dependency tree: }
The dependency tree of a Maven project $p$ is a direct acyclic graph of the dependencies of $p$, where $p$ is the root node and the edges represent dependency relationships between $p$ and the dependencies in $\mathcal{D}$.
\end{definition}

To construct the dependency tree, Maven relies on its specific dependency resolution mechanism.\footnote{\url{https://maven.apache.org/guides/introduction/introduction-to-dependency-mechanism.html}}
First, Maven determines the set of declared dependencies based on the \texttt{pom.xml}\xspace file of the project. 
Then, it fetches the JARs of the dependencies that are not present locally from external repositories, e.g.\@\xspace, Maven Central.\footnote{\url{https://mvnrepository.com/repos/central}} 

\autoref{fig:dt} illustrates the dependency tree of the project $p$, which \texttt{pom.xml}\xspace file is in \autoref{fig:pom}.
The project has three direct dependencies, as declared in its \texttt{pom.xml}\xspace, and three transitive dependencies, as a result of the Maven dependency resolution mechanism.
$d_4$ and $d_5$ are induced transitively from $d_1$, whereas the transitive dependency $d_6$ is induced from $d_3$.
Note that all the bytecode of these transitive dependencies is present in the classpath of project $p$, and hence they will be packaged with it, whether or not they are actually used by $p$.

\begin{definition}
\textbf{Bloated dependency: }
A dependency $d \in \mathcal{D}$ in a software project $p$ is said to be bloated if there is no path in the dependency tree of $p$, between $p$ and $d$, such that none of the elements in the API of $d$ are used, directly or indirectly, by $p$. 
\end{definition}

We introduced the concept of bloated dependencies in 2020~\cite{Valero2020}.
Although they are present in the dependency tree of software projects, bloated dependencies are useless and, therefore, developers can consider removing them.

\begin{definition}
\textbf{Dependency usage status: }
The usage status of a dependency $d \in \mathcal{D}$ determines if $d$ is \textit{used} or \textit{bloated} w.r.t.\@\xspace to $p$, at a specific time of the development of $p$.
\end{definition}

\autoref{fig:du} shows an hypothetical example of the dependency usage tree of project $p$.
Suppose that $p$ directly calls two sets of instructions in the direct dependency $d_1$ and the transitive dependency $d_6$.
Then, the subset of instructions called in $d_1$ also calls instructions in $d_4$.
In this case, the dependencies $d_1$, $d_4$, and $d_6$ are used by $p$, while dependencies $d2$, $d3$, and $d5$ are bloated dependencies.

Figures \ref{fig:pom}, \ref{fig:dt}  and \ref{fig:du} illustrate the status of a project's dependencies at some point in time. Yet, the \texttt{pom.xml}\xspace file, the dependency tree, and the status of dependencies are prone to change for several reasons. 
For example, a dependency that was used can become bloated after a dependency migration or after some refactoring activities that remove the usage link between the project and some of its dependencies. It is also possible that developers add dependencies in the \texttt{pom.xml}\xspace file or that more transitive dependencies appear in the tree, e.g., when updating the direct dependencies.
This work investigates these software evolution changes and their impact on bloat and maintenance.

\section{Study Design} 

In this section, we present the research protocols that we follow to conduct our empirical study, including the research questions (RQs), the tooling utilized to detect bloated dependencies, the data collection, and our methodology to address each RQ. 

\subsection{Research Questions}\label{sec:rqs}

In this paper, we study four different aspects of bloated dependencies. Our analysis is guided by the following research questions.
\vspace{-1em}
\begin{itemize}[leftmargin=24pt]
    \item[\textbf{RQ1}.] \textbf{How does the amount of bloated dependencies evolve across releases?}
    With this first question, we aim at consolidating the body of knowledge about software bloat. 
    Several recent studies have shed light on the massive presence of bloat in different types of software projects~\cite{Jiang2016,Bruce2020,Qian2019,Sharif2018,Rastogi2017}. The growth of bloat is an important motivation for these works. Yet, this growth has never been assessed nor quantified. Our first research question addresses this lack, analyzing the evolution of the amount of bloat over time. 
    \item[\textbf{RQ2}.] \textbf{How does the usage status of each dependency evolve across time?}
    Tools that remove bloated code are designed under the assumption that a piece of code that is bloated at some point in time will always be bloated, hence it makes sense to remove it.
    In this second research question, we investigate whether this assumption holds true in the case of bloated Java dependencies. We analyze how the usage status of dependencies evolves over time, from used to bloated, or vice versa. 
    \item[\textbf{RQ3}.] \textbf{Do developers maintain dependencies that are bloated?}
    Bloated dependencies needlessly waste time and resources, e.g.\@\xspace, space on disk, build time, performance. 
    However, one of the major issues related to this type of dependency is the unnecessary maintenance effort.
    In this research question, we investigate how often developers modify the \texttt{pom.xml}\xspace to update  dependencies that are actually bloated.
    \item[\textbf{RQ4}.] \textbf{What development practices change the usage status of dependencies?}
    The emergence of bloat is due to various code maintenance activities, e.g.\@\xspace, refactoring the code, or modifying the \texttt{pom.xml}\xspace.
    In this research question, we expand the quantitative analysis of the status of each dependency and perform an in-depth analysis of the causes of dependency bloat.
\end{itemize}
\vspace{-1em}

\subsection{Detection of Bloated Dependencies}

To analyze the status of dependencies of Maven projects, we rely on \textsc{DepClean}\xspace.\footnote{\url{https://github.com/castor-software/depclean}}
This is an open-source tool that implements a practical way of detecting bloated dependencies in the complete dependency tree of a Java Maven project.
\textsc{DepClean}\xspace runs a static analysis, at the bytecode level, to detect the usage of direct and transitive dependencies.
To do so, \textsc{DepClean}\xspace constructs a static call-graph of API members' calls among the bytecode of the project and its dependencies.
Then, it determines which dependencies are referenced, either directly by the project or indirectly via transitive dependencies.
If none of the API members of a dependency are referenced, \textsc{DepClean}\xspace reports the dependency as bloated, i.e.\@\xspace, the dependency is not necessary to build the project.
\textsc{DepClean}\xspace generates a report with the status of each dependency, a list of API members that are used at least once, for each used dependency. The tool also generates a modified version of the \texttt{pom.xml}\xspace without bloated dependencies. 

\subsection{Data Collection}\label{sec:data_collection}

The dataset used in our study consists of a collection of subsequent versions of Maven dependency trees \cite{durieux21}.
Each dependency in these trees is analyzed in order to determine its status: used or bloated.
\autoref{fig:overview} summarizes the process we follow to collect this dataset.
Rounded rectangles represent procedures, non-rounded rectangles represent intermediate data results.

\ding{182} \textbf{\textit{Collect.}}
Our data collection pipeline starts from the list of Java projects extracted from GitHub by Loriot~et al.\@\xspace~\cite{styler}. 
The authors queried the GitHub API on June 9th of 2020, and provide a list of GitHub URLs including all projects that use Java as the primary programming language. 
From this list, we keep only projects with more than $5$ stars.
This initial dataset contains a total of \np{\nbTotalProject} Java projects.
Then, we inspect the projects' files and select those containing a single \texttt{pom.xml}\xspace file in the root of the repository, to focus our longitudinal analysis on  single-module Maven projects. 
This first data collection step provides a set of \np{34560} Java projects.

\ding{183} \textbf{\textit{Filter.}}
In this second step, we check all the commits on the \texttt{pom.xml}\xspace file  to determine the version of the project declared in the \texttt{pom.xml}\xspace.
Each time the version of the project changes and it is not a SNAPSHOT or a beta-version, we consider that the commit represents a new release.
We sort the list of projects by the number of releases and then we select the first \np{\nbDatasetProjectsNum} projects.
We focus on release commits since a release represents a stable version of the project, which is a suitable moment to consider the presence of bloated dependencies.
In addition to the project releases, we collect the commits that have been created by Dependabot,\footnote{\url{https://dependabot.com}} a popular software bot that automatizes the update of dependencies on GitHub~\cite{Dey2020}.
The goal is to  determine how many bloated dependencies have been updated as a result of a pull request not made by a human.
We identify \np{\nbDatasetDependabotNum} Dependabot commits for \percent[q]{143}{500} projects.
At the end of this step, we have a total of \np{\nbDatasetProjectsNum} projects, as well as \np{\nbDatasetCommitsNum} commits, including \np{\nbDatasetReleasesNum} release commits.

\ding{184} \textbf{\textit{Analyze.}}
The final and most complex step in our pipeline is to analyze the status of dependencies in the \np{\nbDatasetCommitsNum} commits.
We perform the following tasks: 
1) clone the repository and checkout the commit,
2) compile the project using Maven,
3) if the project compiles, then we execute \textsc{DepClean}\xspace on the commit to obtain the dependency usage status. We analyze dependencies that have a \texttt{compile} or \texttt{test} scope. 
The compilation task is the most crucial and difficult task because it involves downloading dependencies, having the correct version of Java and having a proper project state, i.e.\@\xspace, the Java code needs to be valid.
We mitigate those problems by compiling the projects with Java 11 and then with Java 8.
By trying to compile with Java 8 when the project does not compile with Java 11, we increase the number of successful compilations by around \np[\%]{20}.
We also use a proxy for Maven that caches and looks for dependencies in five different repositories to increase the chances to resolve them.
In total, the proxy cached \np{198611} dependencies and \np[Gb]{165} of data.
As side effects, the proxy speeds up the resolution of dependencies and increases the reproducibility of the study, i.e.\@\xspace, Maven will always resolve the same dependencies even if we recompile the projects after several years.

This final step of our pipeline outputs the definitive dataset for our longitudinal study:  the dependency usage trees of \percent[p]{\nbCommitsNum}{\nbDatasetCommitsNum} commits collected from \percent[p]{435}{500} projects. These trees capture the history of \np{\nbDependenciesNum} dependency relationships, including \np{1987} direct dependencies and  \np{23442} transitive dependencies. 
Among the commits,  \percent[p]{29822}{47276}  are project releases and \percent[p]{1693}{2017} are Dependabot commits. 
We have kept only the projects for which we can successfully analyze at least two dependency tree versions.
The dataset consists of a \texttt{JSON} file per commit for each project, containing the status of each dependency at every point in time. 
The dataset and the scripts are available in our experiment repository.\footnote{\url{https://github.com/castor-software/longitudinal-bloat}}

\input{tables/ts_descriptive}

\autoref{ts_descriptive} shows descriptive statistics of our dataset. 
The 435~projects have been active for periods ranging from five months to \np{235} months (12 years and 7 months), with most of them in the range \np{48.5} months (1st Qu.) to \np{109.5} months (3rd Qu.).
The number of dependency trees analyzed for each project ranges from \np{2} to \np{819} (Median = \np{58}, 1st Qu. = \np{41}, 3rd Qu. = \np{79}).
The table also reports  the number of direct dependencies in the oldest analyzed commit (Median = \np{5}, 1st Qu. = \np{3}, 3rd Qu. = \np{10}), and transitive dependencies (Median = \np{10.5}, 1st Qu. = \np{2}, 3rd Qu. = \np{56}).
The last two lines in the table give the number of direct dependencies in the most recent analyzed commit (Median = \np{10}, 1st Qu. = \np{5}, 3rd Qu. = \np{18}), and transitive dependencies (Median = \np{25}, 1st Qu. = \np{6.5}, 3rd Qu. = \np{82.5}).

\subsection{Methodology for RQ1}\label{sec:methodology_rq1}



In RQ1, we analyze the evolution of the number of bloated dependencies over time.
We start with a global analysis of the bloat trend in direct and transitive dependencies.
To do so, we aggregate the total number of bloated dependencies in all projects on a monthly basis and compute the average values.
Next, we look at each project separately and assign a bloat evolution trend to each of them.
We represent the number of dependencies at each commit in a project as a time series.
Let $p$ be a Maven project, $\mathcal{B}_p = b_1, b_2, ...b_n$ represents a time series of length $n$. 
A time step in this series represents one commit that modifies the \texttt{pom.xml}\xspace of $p$.
Each $b_i$ is the total number of bloated dependencies reported by \textsc{DepClean}\xspace at the $i^{th}$ commit on the \texttt{pom.xml}\xspace.
We collect two series for each project, for bloated-direct and bloated-transitive dependencies. 

For each project $p$, we determine the overall trend for the evolution of the number of bloated dependencies: increase, decrease or stable. 
The following function over $\mathcal{B}_p$ shows how we determine the trend for a project:
\[
  f(\mathcal{B}_p) =
  \begin{cases}
    \text{\emph{inc}} & \text{if $slope(lm(\mathcal{B}_p)) > 0 \wedge \exists b_j \in \mathcal{B}_p : b_j < b_{j-1}$} \\
    \text{\emph{dec}} & \text{if $slope(lm(\mathcal{B}_p)) < 0 \wedge \exists b_j \in \mathcal{B}_p : b_j > b_{j-1}$} \\
    \text{\emph{stable}} & \text{if $\forall b_i \in \mathcal{B}_p : b_i = b_{i-1}$} 
  \end{cases}
\]

We notice that several projects do not have a monotonic trend in the number of bloated dependencies (i.e.\@\xspace the value increases and decreases at different time intervals). 
To account for projects that have a non-monotonic number of bloated dependencies, we fit a simple linear regression model, denoted as \emph{lm}, and determine the trend of the time series based on the sign of the \emph{slope} of the regression line. 
A project labelled as \emph{inc} is a project for which the sign of the \emph{slope} is positive, i.e.\@\xspace, the number of bloated dependencies increase over time.
A project labelled as \emph{dec} is a project for which the sign of the \emph{slope} is negative, i.e.\@\xspace, the number of bloated dependencies decreases over time.
If the number of bloated dependencies is the same across all the data points in the time series of a project, we label it as \emph{stable}

\subsection{Methodology for RQ2}\label{sec:methodology_rq2}

In this research question, we analyze the evolution of the usage status of the \np{\nbDependenciesNum} dependencies in our dataset.
Given a dependency $d \in \mathcal{D}$, present in the dependency tree of a project $p$, we collect the status of $d$ at each analyzed commit (see data collection  \autoref{sec:data_collection}). This provides a sequence of usage statuses for $d$ and serves as the basis to determine the occurrence of  transitional patterns between used and bloated statuses.

Let $\mathcal{V}_d$ be a vector representing the history of usage statuses of dependency $d$ across the releases of a project, where each release is ordered by its date.
We label the usage status of a dependency $d$ as \texttt{B} if it is a bloated dependency, or \texttt{U} if it is a used dependency. 

\autoref{fig:meth_rq2} illustrates a transition in the usage status of a dependency from used (\texttt{U}) to bloated (\texttt{B}).
In this case, the dependency is identified as used at the two first releases of the project, then it becomes bloated at the third release, and stays as such.
Therefore, the usage pattern for this dependency results in $[\texttt{U},\texttt{U},\texttt{B},\texttt{B}]$.
Since we are interested in analyzing transitional patterns, the consecutive elements of the same category in the vector can be compressed to a single status, e.g.\@\xspace, the previous example is represented as $[\texttt{U},\texttt{B}]$.

\begin{figure}[t]
	\centering
	\includegraphics[origin=c,width=0.4\textwidth]{figs/methrq2.pdf}
	\caption{Example of a dependency analyzed over time.  It has a transition of usage status: from used at version 1.1.0 to bloated at  1.1.1 (RQ2). The dependency has two subsequent updates after bloated: at versions 1.1.1 and 2.0.0 (RQ3).} 
	\label{fig:meth_rq2}
	\vspace{-0.4cm}
\end{figure}

In this research question, we focus on analysing the occurrence of five transitional patterns: $[\texttt{U}]$, $[\texttt{B}]$, $[\texttt{U,B}]$, $[\texttt{B,U}]$, and \textit{fluctuating}. 
In the cases where the usage status of a dependency  flickers over time, we consider the status of the dependency as \textit{fluctuating}.

\subsection{Methodology for RQ3}\label{sec:methodology_rq3}

We conjecture that developers could save some maintenance efforts in the absence of bloated dependencies.
In this research question, we investigate how many times developers update the version of dependencies that are in fact bloated.
This type of change in the \texttt{pom.xml}\xspace of a project is an unnecessary engineering effort that could be avoided.
We analyze two types of commits: the commits where the developers update the version of the project to a new stable version (e.g.\@\xspace, 1.0.0), and the Dependabot commits.
Dependabot\footnote{\url{https://dependabot.com}} is a dependency management bot very active on GitHub. 
It creates pull requests that update the dependencies to remove known vulnerabilities.
Dependabot was launched on May 26, 2017 with support for Ruby and JavaScript, and now it is supporting more than ten languages, including Java since August, 2018.

We analyze Dependabot commits because they only contain edits on the dependency versions in the \texttt{pom.xml}\xspace. 
It provides a clean point of analysis to detect the impact of a dependency update.
And it allows us to study how many bloated dependencies are updated by developers as a result of the suggestion of automatic bots.

\autoref{fig:meth_rq2} illustrates a typical case of a dependency that continues to be updated even after it becomes bloated. 
The dependency is used by the project until version 1.1.0. 
Afterward, the dependency is no longer used, but it is still updated twice,  to version 1.1.1 and then to version 2.0.0.

To answer this research question, we consider the \np{\nbTotalProjectsWithDependabotUpdates} commits in our dataset that perform dependency updates in projects that have at least one Dependabot commit. 
We obtain the number of times a dependency is updated by a developer, by Dependabot, and how many of those updates are performed on bloated dependencies.
For the dependency usage analysis, we tag each dependency as used or bloated. 
We count every time the version of a direct dependency is updated, and we count separately the number of updates applied on bloated dependencies.
In the example of \autoref{fig:meth_rq2}, we count one update on a used dependency (when the used dependency is updated to version 1.1.0), and two updates on a bloated dependency (when the bloated dependency is updated to version 1.1.1 and version 2.0.0).
Using this approach, we can compare the ratio of updates made by developers and by Dependabot. 

\subsection{Methodology for RQ4}\label{sec:methodology_rq4}

In this research question, we investigate the origins of bloated dependencies. 
Each time a bloated dependency appears for the first time in a project's history, we first determine if it was used in the commit that immediately precedes the apparition of bloat.
If the dependency was used in the previous commits, we  determine in which class it was used.
By analyzing a dependency at the time it appears as bloated, we can identify what causes the emergence of bloat.
We have identified four different situations:
\vspace{-.3em}
\begin{enumerate}[leftmargin=0.6cm,noitemsep]
    \item New dependency (\texttt{ND}): The bloated dependency was not present in the previously analyzed commit. It indicates that the dependency was introduced in the project but never used.
    \item Removed code (\texttt{RC}): The bloated dependency was present in the previously analyzed commit and all the classes where the dependency was used are removed.
    \item Updated code (\texttt{UC}): The bloated dependency was present in the previously analyzed commit, yet at least one class  where the dependency was used is still present in this commit. It means that the code has been updated to remove the usage of the dependency but the \texttt{pom.xml}\xspace still contains the dependency.
    \item New version (\texttt{NV}): The bloated dependency was present in the previously analyzed commit and the version of the dependency changed. In the case of transitive dependency, the parent dependency has been updated and the project does not use  the transitive dependency anymore. 
\end{enumerate}
\vspace{-.3em}
For each of the \np{\nbCommitsNum} dependency trees, we identify the bloated dependencies.
Then, we check the status of the dependency in the previous commit.
If the dependency is not present in the previous commit, we consider the origin as \texttt{ND}.
Otherwise, we check in the previous commit in which classes the bloated dependency is used.
We then compare those classes with the new commit. 
If all classes are removed, we consider the origin of the bloat as \texttt{RC}.
If at least one of the classes is still present, we consider the origin of the bloat as \texttt{UC}.
Additionally, we compare the version of the bloated dependency with the previous commit. 
If the version  changes, and at least one class is still present, we mark the origin as \texttt{UC} and \texttt{NV}, 
since both reasons could be the origin of the bloat.

\section{Results}

In this section, we answer the four RQs presented in \autoref{sec:rqs}.

\subsection{RQ1. Bloat Trend}

In this research question, we analyze the evolution of the number of bloated dependencies over time. 
We hypothesize that this number tends to grow.
Following the protocol described in \autoref{sec:data_collection}, we analyze the usage status of each dependencies in \np{\nbCommitsNum} dependency trees along the history of \np{\nbProjectsNum} projects, as reported by \textsc{DepClean}\xspace.
We assign bloat trend labels to each project, according to the three categories defined in \autoref{sec:methodology_rq1}.

\autoref{fig:trend_bloat_average} shows the monthly evolution trend of the number of bloated-direct and bloated-transitive dependencies, from January 2011 to November 2020.
The y-axis is the average number of bloated dependencies of the \np{\nbProjectsNum} projects.
Each data point represents the average of bloat measured each month.
The lines represent linear regression functions, fitted to show the trend of bloated-direct and bloated-transitive dependencies, at a $95\%$ confidence interval.

\FPset{nbTransitiveFirst}{1,695}
\FPset{nbTransitiveLater}{286,228}

We observe that bloated-transitive dependencies have a clear tendency to grow over time, whereas bloated-direct dependencies grow at significantly lower pace.
For example, the number of bloated-transitive dependencies in 2011 was \nbTransitiveFirst, and by the end 2020 this number grew up to \nbTransitiveLater ~(increase > $250\times$).
The bloat is more pervasive and variable (SD = 17.2) among transitive dependencies, representing a larger share in comparison with direct dependencies that are less numerous and less variable (SD = 1.3).
We conclude that, overall, the amount of bloat increases, being more notable for transitive dependencies.



\begin{figure}[htb]
	\centering
	\includegraphics[origin=c,width=0.43\textwidth]{figs/trendbloataverage.pdf}
	\caption{Trend of the average number of bloated-direct and bloated-transitive dependencies per month.}
	\label{fig:trend_bloat_average}
	\vspace{-0.25cm}
\end{figure}

\FPset{increasingProject}{245}
\FPset{decreasingProject}{106}
\FPset{stableProject}{84}

\FPset{increasingProjectTransitive}{286}
\FPset{decreasingProjectTransitive}{113}
\FPset{stableProjectTransitive}{36}

\autoref{fig:trend_bloat_average} shows an overall growing trend for the number of bloated dependencies. Now, we look in more details at each project separately. 
We count the number of projects that have different trend of bloated dependencies.
\autoref{fig:lineplot_projects} shows examples of time series of projects in our dataset for which the bloated-direct dependencies are labelled according to each category (increasing, decreasing, and stable).
The name of the projects correspond to the <\textit{user}>/<\textit{repository}> on GitHub.
The x-axis is the date of the analyzed commits.
The y-axis represents the number of bloated dependencies detected.
For instance, the time series of the project \href{https://github.com/zapr-oss/druidry}{zapr-oss/druidry} has a total of \np{51} commits on the \texttt{pom.xml}\xspace (i.e.\@\xspace, data points in the time series), and it is labelled as \emph{inc} w.r.t.\@\xspace to both the direct and transitive dependencies because both series tend to continuously increase over time.

\begin{figure}[htb]
	\centering
	\includegraphics[origin=c,width=0.45\textwidth]{figs/lineplotprojects.pdf}
	\caption{Example of projects in the three classes of bloat trend defined in \autoref{sec:methodology_rq1}.}
	\label{fig:lineplot_projects}
	\vspace{-0.25cm}
\end{figure}

\autoref{fig:barplot_trend} shows the distribution of the trend of bloated-direct and bloated-transitive dependencies.  
The x-axis indicates the number of projects with bloated-direct dependencies in each specific evolution trend, given on the y-axis. Each bar in the plot is partitioned in three parts that correspond to the share of projects with a given trend for the number of bloated-transitive dependencies.
For example, the top bar of \autoref{fig:barplot_trend} shows (i) that the number of bloated-direct dependencies tends to increase for \percent[p]{\increasingProject}{435} projects; and (ii) among these \np{\increasingProject} projects, \np{180} also have a number of bloated-transitive dependencies that tends to increases, \np{59} of these projects have a decreasing number of bloated-transitive dependencies and \np{6} projects have a stable number of bloated-transitive dependencies.
The bar in the middle of the figure indicates that the number of bloated-direct dependencies tends to decrease for \percent[p]{\decreasingProject}{435} projects and the bottom bar shows that this type of bloat is stable for \percent[p]{\stableProject}{435} projects because no new bloated dependencies are introduced in the \texttt{pom.xml}\xspace.

Looking at the partitions of each bar in \autoref{fig:barplot_trend}, we first observe that whatever the trend for the number of bloated-direct dependencies, the number of bloated-transitive dependencies can evolve in any way. Yet, the majority of projects have an increasing number of bloated dependencies among their transitive dependencies.
In total, \percent[p]{\increasingProjectTransitive}{435} projects have an increasing number of bloated-transitive, whereas for \percent[p]{\decreasingProjectTransitive}{435} projects this number decreases.
The number of projects with stable transitive-dependencies, \percent[p]{\stableProjectTransitive}{435}, is relatively low.

\begin{figure}[b]
	\centering
	\includegraphics[origin=c,width=0.475\textwidth]{figs/barplottrendstacked.pdf}
	\caption{Distribution of the number of projects with increasing, decreasing, and stable trend of bloated-direct and bloated-transitive dependencies.}
	\label{fig:barplot_trend}
\end{figure}


Interestingly, from the \np{\stableProject} projects with a stable number of bloated-direct dependencies, \percent[p]{41}{\stableProject} of the bloated-transitive dependencies increase and \percent[p]{18}{\stableProject} decreases (e.g.\@\xspace, as in the project \href{https://github.com/percy-java-selenium}{percy/percy-java-selenium} in \autoref{fig:lineplot_projects}).
This result indicates that the usage status of dependencies change regardless of the modification of the \texttt{pom.xml}\xspace.
The transition from used to bloated in transitive dependencies becomes unnoticed.
In other words, even if developers update only the version of direct dependencies, without doing anything else, then the bloat grows naturally due to the inflation of the rest of the dependency tree.
It happens, for example, in the project \href{https://github.com/jpmml/jpmml-sparkml/}{jpmml/jpmml-sparkml} when a developer updates \texttt{spark-mllib\_2.11} from version 2.0.0 to 2.2.0, introducing \np{133} new transitives dependencies.

On the other hand, we observe that for \percent[p]{65}{\decreasingProject} out of  the \np{\decreasingProject} projects with a decreasing number of bloated-direct dependencies,  the number of bloated-transitive increases. 
It indicates that even in projects for which direct dependencies decreases, the number of bloated-transitive dependencies can increase and eventually lead to a global growth of bloated dependencies for the project.

\vspace{+0.30cm}
\begin{mdframed}[style=mpdframe]
\textbf{Answer to RQ1:} 
The number of bloated-direct dependencies and bloated-transitive dependencies increases over time for \ShowPercentage{\increasingProject}{435} and \ShowPercentage{286}{435} of the projects, respectively.
This result suggests that bloated dependencies tend to naturally emerge and grow through software evolution and maintenance.
\end{mdframed}
\vspace{-0.15cm}

\subsection{RQ2. Usage Patterns}

This research question addresses an essential concern when developers think about removing bloat: is a piece of software identified as bloat at one point in time prone to usage in future revisions? We answer this question through a post-mortem analysis of the transitioning in the usage status of dependencies across the evolution of the studied projects.
Our hypothesis is that dependencies do not change their usage status very frequently, i.e.\@\xspace, a dependency that is used in one commit is used in future commits, and similarly for bloated dependencies.
If our hypothesis holds, then it indicates that developers can be more confident when removing bloated dependencies.\looseness=-1

We analyzed the five usage patterns described in \autoref{sec:methodology_rq2}. \autoref{fig:tterns} shows one concrete example for each pattern. The examples are taken from our dataset and the patterns are illustrated on the period January 2017 to December 2020.
The y-axis shows the name of the direct dependency, with the pattern in square brackets.
For example, we analyze the usage status of the direct dependency \texttt{h2} in the project \href{https://github.com/dieselpoint/norm}{dieselpoint/norm}, from May 2018 to October 2020. 
As we can observe, this dependency was always reported as bloated.
On the other hand, the dependency \texttt{json} in project 
\href{https://github.com/PAXSTORE/paxstore-openapi-java-sdk}{PAXSTORE/paxstore-openapi-java-sdk} was reported as bloated in first four analyzed commits, September 2018 to November 2019, and then it was used in all the subsequent releases of the project.\looseness=-1

\autoref{fig:barplot_patterns} shows the distribution of the five transitional usage patterns among the \np{1987} direct  and  \np{23442} transitive dependencies in our dataset.
The x-axis represents the percentage of occurrence of each pattern with respect to the total.
The top bar of the plot indicates that \np[\%]{64.3} of the direct dependencies are used through their whole lifespan, whereas \np[\%]{29.9} are always bloated.
This means that \np[\%]{94.2} of direct dependencies never change their status through the evolution of the software projects.
This also means that most bloated-direct dependencies are bloated by the time they are added in the dependency tree and are likely to remain bloated forever. 
We conjecture that this happens as a side effect of some development practices, such as copy-pasting of \texttt{pom.xml}\xspace files, the use of Maven Archetypes, or the deliberate addition of dependencies when setting up the development environment.

The bottom bar of the plot shows a similar stability for the status of transitive dependencies: \np{91.1}\% of transitive dependencies do not change their usage status over their lifespan. A key difference here is that most of the dependencies are always bloated: \np{78.3}\% of the transitive dependencies are bloated from the start, whereas \np{12.8}\% are always used.
We hypothesize that most transitive dependencies are unnoticed by the developers. Consequently, they are not managed and stay in the dependency tree for no reason in most cases.\looseness=-1

\vspace{-0.15cm}
\begin{figure}[htb]
	\centering
	\includegraphics[origin=c,width=0.43\textwidth]{figs/examplepatterns.pdf}
	\caption{Example of direct dependencies with distinct usage patterns. Each dependency belongs to a different project, the status of the dependency is analyzed at each commit that changes the \texttt{pom.xml}\xspace of the project.}
	\label{fig:example_patterns}
	\vspace{-0.25cm}
\end{figure}

A key motivation for this research question is to determine whether a dependency identified as bloated is likely to stay bloated.
We compute the percentage of dependencies bloated from the start (\texttt{B}) or that remain bloated after being used (\texttt{UB}), with respect to the total number of dependencies that are bloated at some point in the future, i.e.\@\xspace, (\texttt{B}+\texttt{UB})/\texttt{(B}+\texttt{UB}+\texttt{BU}+\textit{fluctuating}).
We find that \np[\%]{89.2} of bloated-direct dependencies and \np[\%]{93.3} of bloated-transitive dependencies remain bloated over time. 

\begin{figure}[htb]
	\centering
	\includegraphics[origin=c,width=0.475\textwidth]{figs/barplotpatterns.pdf}
	\caption{Percentage of occurrence of usage patterns of bloated-direct and bloated-transitive dependencies.}
	\label{fig:barplot_patterns}
\end{figure}

\begin{mdframed}[style=mpdframe]
\textbf{Answer to RQ2:} 
The usage status is mostly constant over time: \np{94.2}\% of the direct and of \np{91.1}\% of the transitive dependencies do not change status through their lifespan i.e.\@\xspace they are either always used or always bloated.
When a dependency is detected as bloated, it stays bloated in \np[\%]{89.2} of the cases if it is direct, and \np[\%]{93.3} if is transitive.
This indicates that developers can confidently take a debloating action when detecting bloated dependencies
\end{mdframed}

\subsection{RQ3. Unnecessary Updates}



In this research question, we investigate how the update of dependencies, a regular maintenance practice for all software projects, more and more encouraged by automatic bots, relates to bloated dependencies.
We hypothesize that developers invest some effort in updating some of these dependencies, while this is not required.
To verify this hypothesis, we count how many times bloated-direct dependencies are updated in the \texttt{pom.xml}\xspace and compare it to the number of updates of used-direct dependencies. 
The methodology for this count is described in \autoref{sec:methodology_rq3}.
We analyze separately the updates  performed manually by developers and the updates suggested by Dependabot that are eventually accepted by a developer. 


Figures \ref{fig:barplot_updates} and \ref{fig:barplot_dependabot_updates} present our main results for this research question. 
Those plots present the number of dependency updates on direct dependencies made by developers and by Dependabot respectively.
We focus on the \np{143} projects in our dataset that have at least one Dependabot commit.
All the projects do not use Dependabot since its Java support is relatively recent (August 2018). 
The total number of updates on direct dependencies in these projects is \np{\nbTotalProjectsWithDependabotUpdates}, of which \np{\nbDepDevUpdatesNum} ~have been performed by humans and \np{\nbDependabotUpdatesNum} ~have been suggested by Dependabot.

\autoref{fig:barplot_updates} shows that, over a total of \np{\nbDepDevUpdatesNum} updates on direct dependencies made by developers,
\percent[p]{9403}{\nbDepDevUpdatesNum} are preformed on used dependencies, and \percent[p]{2659}{\nbDepDevUpdatesNum} are made on bloated dependencies.
These \np{2659} unnecessary updates represent a significant effort, as updating dependencies is a non trivial maintenance task \cite{Kula2018}.
\autoref{fig:barplot_dependabot_updates} shows the number of updates on direct dependencies made by accepting a suggestion from Dependabot. 
From \autoref{fig:barplot_dependabot_updates}, \percent[p]{2452}{\nbDependabotUpdatesNum} of Dependabot updates are performed on non-bloated dependencies and \percent[p]{716}{\nbDependabotUpdatesNum} on bloated dependencies.
Overall, we observe that developers perform a significantly larger number of dependency updates than Dependabot. Yet, the most interesting fact is that  developers and Dependabot perform the same ratio of updates on bloated dependencies, \np[\%]{22.0} and \np[\%]{22.6} respectively.


\begin{figure}[t]
	\centering
	\input{figs/chart_bloat_update}
	\caption{Number of updates made by developers on direct dependencies in projects that use Dependabot.}
	\label{fig:barplot_updates}
\end{figure}

\begin{figure}[t]
	\centering
	\input{figs/chart_bloat_dependabot_update}
	\caption{Number of updates made by Dependabot on direct dependencies in projects that use Dependabot.}
	\label{fig:barplot_dependabot_updates}
\end{figure}





The consequences of updating a bloated dependency are not only about the time and effort wasted by the developer. We have observed that a possible side-effect of these unnecessary updates is the increase of the total number of bloated dependencies in the project.
In RQ1, we showed that the number of bloated dependencies increases over time, with a strong trend for transitive dependencies. 
In fact, a portion of this increasing transitive bloat is introduced through the update of direct dependencies, i.e.\@\xspace, the new version has more dependencies.
Note that this scenario can happen even when updating a bloated-direct dependency.
We have observed this phenomenon in our dataset. 
The \np{\nbTotalBloatUpdatesNum} updates on bloated-direct dependencies have introduced \np{1883} new bloated-transitive dependencies.

\vspace{+0.30cm}
\begin{mdframed}[style=mpdframe]
\textbf{Answer to RQ3:} 
\np[\%]{22.0} of developer updates and \np[\%]{22.6}
of Dependabot accepted updates are performed on  bloated-direct dependencies, which represents a total of   \np{\nbTotalBloatUpdatesNum} updates over \np{143} projects. 
This is novel evidence that software bloat artificially increases maintenance effort and that dependency bots need to be improved to detect bloated dependencies.
\end{mdframed}

\subsection{RQ4. Bloat Origin}


In this research question, we investigate what type of maintenance activity is at the origin of bloat emergence. 
In other words, we perform an in-depth analysis of the usage patterns \texttt{B} and \texttt{UB} presented in RQ2 by categorizing the origin of the bloat in  four possible activities: new dependency (\texttt{ND}), removed code (\texttt{RC}), updated code (\texttt{UC}), and new version (\texttt{NV}) as described in \autoref{sec:methodology_rq4}.
\autoref{tab:origin_bloat} summarizes the number of occurrences of activities that introduce bloat for direct and transitive dependencies.
In total, we analyze the \np{25359}  dependencies that become bloated at some point in time (\np{1987} directs, \np{23442} transitives) and determine in what condition they become bloated. This corresponds to  \np{\totalOriginDirect} and \np{\totalOriginTransitive} transitions to bloat, on direct and transitive dependencies respectively. 


We observe that the primary origin of bloat is the addition of new dependencies \texttt{ND}, with \percent[p]{1868}{\totalOriginDirect} such additions that lead to more bloated-direct  dependencies and \percent[p]{33370}{\totalOriginTransitive} new dependencies that introduce more bloated-transitive dependencies.
This result confirms our findings in RQ2, where we observed that the status of most dependencies does not change  over time, which hinted to the fact that bloated dependencies are bloated as soon as they appear in the dependency tree. 
Additionally, the larger number of \texttt{ND} that grow the number bloated-transitive dependencies  is consistent with the results of RQ1, where we showed a larger increase of bloated-transitive dependencies than bloated-direct ones. This new result consolidates the finding with the root cause of the transitive bloat.
The second most frequent origin of bloated dependencies is different for direct and transitive dependencies. The action of removing code
\texttt{RC} is the second most frequent cause of the emergence of bloated-direct dependencies, with $8,8\%$ of the cases. Updating code \texttt{UC} is the second most important root cause for bloated-transitive dependencies. 
While these two actions are similar in nature (evolve the code base), we did not find a clear explanation for the difference between the types of bloated dependencies.
Updating to a new version of a dependency \texttt{NV} is the least frequent cause of bloat emergence. The rarity of this cause is explained by the fact that it can only happen in very specific conditions, when the new version of the dependency changes drastically. 

\begin{table}[t]
\small
    \caption{Number of occurrence for each origin of bloat. The occurrences are separated between the new bloated-direct and transitive dependencies.}
    \vspace{-0.25cm}
    \label{tab:origin_bloat}
    \centering
    \begin{tabular}{@{}lrr@{}}
    \toprule
Origin                    & Bloated-direct & Bloated-transitive  \\
\midrule
New dependency (\texttt{ND}) & \percent{1868}{\totalOriginDirect} & \percent{33370}{\totalOriginTransitive} \\
Removed code  (\texttt{RC})  & \percent{194}{\totalOriginDirect} & \percent{206}{\totalOriginTransitive} \\
Updated code (\texttt{UC})   & \percent{153}{\totalOriginDirect} & \percent{495}{\totalOriginTransitive} \\
New version (\texttt{NV})  & \percent{47}{\totalOriginDirect}  & \percent{124}{\totalOriginTransitive} \\
\bottomrule
    \end{tabular}
\vspace{-0.2cm}
\end{table}

We now illustrate the different situations of bloat introduction with real-world case studies observed in our dataset.
The most frequent cause of bloat introduction is a new transitive dependency in the dependency tree  (\texttt{ND}), which is never used.
For example, this happens in the project \href{https://github.com/couchbase/couchbase-java-client}{couchbase/couchbase-java-client} at the commit \href{https://github.com/couchbase/couchbase-java-client/commit/47ac44fa1aa2eb0271b4823fbe9f7d6a69818a95}{47ac44}, where the dependency 
\texttt{jackson-\allowbreak{}databind}, 
which is induced transitively when 
\texttt{encryption\allowbreak{}:1.0.0} has been added to the \texttt{pom.xml}\xspace.
\texttt{jackson-\allowbreak{}databind} is used in the class 
\texttt{Hashicorp\allowbreak{}VaultKey\allowbreak{}StoreProvider} which is never used by the \href{https://github.com/couchbase/couchbase-java-client}{couchbase/couchbase-java-client} and, therefore,
\texttt{jackson-\allowbreak{}databind}
is a bloated-transitive dependency in this project.

This case occurs with direct dependencies as well.
For example,  the direct dependency 
\texttt{jackson-core} is added as a direct dependency in the \texttt{pom.xml}\xspace of project \href{https://github.com/jenkinsci/elasticbox-plugin}{jenkinsci/elasticbox-plugin}, at commit \href{https://github.com/jenkinsci/elasticbox-plugin/commit/0083584a760c6800ea63bae8283761cadf2ef448}{008358}. Yet, the dependency is never used in the code of the project.
One year and 4 months later a pull-request, \href{https://github.com/jenkinsci/elasticbox-plugin/pull/41}{\#41}, fixes the bloat issue by removing the dependency directly. 
However, at the time of writing this paper, the pull-request has not been merged.

Projects are evolving, adding and removing code is part of the life cycle of a project. A consequence of code removal can be to eliminate the need for a dependency.
Yet, developers currently have no tool support to determine that a dependency can also be removed as part of their maintenance activities. Consequently, the dependency is likely to become bloated (\texttt{RC}).
For example, we observed that scenario happens in the project \href{https://github.com/apache/commons-lang}{apache/commons-lang}. 
The commit \href{https://github.com/apache/commons-lang/commit/def3c4672b1f99b35ec1fb1aaa316e6043463d73}{def3c4} introduces the dependency 
\texttt{bcel}, which contains annotations to document thread safety.
However, the commit \href{https://github.com/apache/commons-lang/commit/796b051f28ca96f1dbbd5dfe4b5cae5579d0d14e}{796b05} removes all classes where these annotations were used.
According to the commit, more discussions were needed to design the annotation, and the maintainers reverted partially the changes to release a new version.
A developer removed the bloated dependency after five months (see commit \href{https://github.com/apache/commons-lang/commit/66226ec1c2ff33e138189463001c649dbb404f56}{66226e}).

A similar scenario occurs when developers  update classes (\texttt{UC}).
For example, the commit \href{https://github.com/jenkinsci/remoting/commit/62aad35f64c46d0c41a5d464ccba8c78b784f9d7}{62aad3} introduces   the annotation \texttt{Ignore\allowbreak{}JRERequirement} on a method in the project \href{https://github.com/jenkinsci/remoting/}{jenkinsci/remoting}. 
However, this method is updated and deprecated in the commit \href{https://github.com/jenkinsci/remoting/commit/49c67eef8616c7bc3588263d4ecc2dbcb51d5bb8}{49c67e}.
The annotation \texttt{Ignore\allowbreak{}JRERequirement} is removed and the dependency
\texttt{animal-\allowbreak{}sniffer-\allowbreak{}annotation} 
became bloated.

The project \href{https://github.com/apache/commons-dbcp/}{apache/commons-dbcp} contains an interesting case of bloat introduced when a dependency is updated (\texttt{NV}).
In the commit \href{https://github.com/apache/commons-dbcp/commit/3550adaf1c87af8ec9b80830f9d9e9d2a1e19691}{3550ad}, the direct dependency 
\texttt{geronimo-\allowbreak{}transaction} 
is detected as bloated.
However, this dependency was not bloated in the previous commit \href{github.com/apache/commons-dbcp/commit/d7aa662fbbb99e536ae28c47d0c4e1d51e39d5b9}{d7aa66}, when the project was using the version \texttt{1.2-beta} of
\texttt{geronimo-\allowbreak{}transaction}. The dependency  was updated to  version \texttt{2.2.1} with commit \href{github.com/apache/commons-dbcp/commit/3550adaf1c87af8ec9b80830f9d9e9d2a1e19691}{3550ad}. 
This new version brought major changes in the dependency and, in \texttt{2.2.1}, all the classes used by the project had been move in a transitive dependency of 
\texttt{geronimo-\allowbreak{}transaction}. 
Therefore, the direct dependency towards
\texttt{geronimo-\allowbreak{}transaction}
became bloated.

\vspace{+0.15cm}
\begin{mdframed}[style=mpdframe]
\textbf{Answer to RQ4:} The addition of new dependencies is the root cause of the emergence of 
\ShowPercentage{1868}{\totalOriginDirect} of the bloated-direct dependencies. 
Still, \ShowPercentage{347}{\totalOriginDirect} of the cases appear after code updates or removals.
This indicates that new dependencies should be carefully reviewed the first time they are added, and we recommend developers to check the usage status of dependencies when removing code.\looseness=-1
\end{mdframed}
\vspace{-0.2cm}

\section{Implications}\label{sec:implications}


Our findings provide practical, empirically justified implications for improving dependency management \cite{Cox2019,Gustavsson20}.
The results of RQ1 and RQ2 show that bloated dependencies are likely to remain bloated in the future.
This is empirical evidence that can motivate developers and increase their confidence when they are faced with the opportunity to remove bloated dependencies. Motivation comes from the benefits associated with reducing the number of dependencies of the project and hence reduce associated maintenance activities. Confidence comes with the strong likelihood that the dependency that is removed will not be necessary in the future.

Our results show that there exist many practical difficulties related to the way of handling software dependencies.
This can raise the awareness of developers about the importance of understanding what dependencies are more likely to become bloated, and how their projects can reduce the size of dependency trees without breaking the build.
In particular, the use of tools, such as \textsc{DepClean}\xspace, to automatically detect and suggest changes in the build files can contribute to a better awareness of developers about the state of their dependencies. 
For example, we recommend to include a bloat analysis before release to ensure that no bloat is shipped and deployed. 
This is for reducing the size of the released binary and for all projects that depend on it, i.e.\@\xspace, the number of transitive dependencies will be reduced.\looseness=-1

In RQ3, we present original results of the negative impact of bloated dependencies on the maintenance of the projects. 
In particular, we shed a new light on the limitations of dependency bots, such as Dependabot, and provide evidence that developers accept bots' suggestions when updating dependencies without checking if the dependency is actually used.
Bot creators should consider improving their tools to automatically detect bloat and suggest the removal of unused dependencies.
On the same line, compilers and IDEs should also warn developers when dependencies are not used anymore and when a dependency is introduced without encountering its counterpart usage on code.

Our dataset and our case studies on the origin of bloat provide valuable references for the rapid identification of practices that result in dependency bloat.
Those references can be used to build dedicated bots that ask for additional checks, e.g. when a new dependency appears in the dependency tree, or to establish guidelines for developers when they maintain \texttt{pom.xml}\xspace files. 

\section{Threats to validity}



\textit{\textbf{Internal Validity.}}
The first internal threat relates to the detection of bloated dependencies.
The results of our study are tied to the accuracy of \textsc{DepClean}\xspace to find bloated dependencies in Maven projects.
This tool is based on advanced static analysis. 
Therefore, some usages that rely on Java dynamic features might be missed, reporting some used dependencies as bloated.
For example, \texttt{lombok} is a Java library that relies on annotations to manipulate the bytecode at compilation time, adding boilerplate code constructs such as getters and setters.
This mechanism makes the dependency to be flagged as bloated by \textsc{DepClean}\xspace, since no reference to this dependency remains in the bytecode of the compiled project.
Nevertheless, we consider that \textsc{DepClean}\xspace is a solid tool, evaluated on millions of dependencies, used in industry, and developers have removed hundreds of bloated dependencies thanks to its analysis~\cite{Valero2020}.
The second threat relates to the representativeness of the data and the analysis performed.
We mitigate those threats by collecting a large dataset of projects from multiple domains and  released across several years.
This allows us to draw general conclusions about the evolutionary trend of bloated dependencies, regardless of the existence of some false positives, which are known to be hard to detect using static analysis~\cite{Livshits2015,Landman2017,Reif2018}.

\textit{\textbf{External Validity.}} 
When conducting this study, we focus on bloated Java dependencies in projects that build with Maven.
As explained in \autoref{sec:data_collection}, the analysis of the dependency trees of \np{\nbProjectsNum} projects requires compiling and analyzing the bytecode at different time periods.
There were cases where the compilation failed for several reasons, making it difficult to obtain the complete history of dependency changes.
As we consider a large number of open-source projects, we believe our results are generalizable in this specific domain. 
Meanwhile, additional studies with proprietary projects or other programming languages should be considered to consolidate these very first result about software bloat evolution.

\textit{\textbf{Construct Validity.}}
This threat is related to the rationality of the questions asked.
We investigate the evolution of bloated dependencies over time.
To achieve this goal, we focus on four aspects: bloat trend, usage patterns, unnecessary updates, and bloat origins.
We believe that these are rational questions that provide unique and novel insights for researchers and developers

\section{Related work} 

\textit{\textbf{Software Bloat.}}
Previous research on software bloat has mainly focused on reducing C/C++ binaries to mitigate the security risks associated with unnecessary code~\cite{Sharif2018, Qian2019, Quach2018}.
Holzmann~\cite{Holzmann2015} reports the historical growth in the size of the \texttt{true} command in Unix systems.
Similarly, we observed that the number of bloated dependencies tends to grow over time, whether or not there is a need for it.
In the last years, there is a recent resurgence of interest in debloating Java bytecode~\cite{Jiang2016, Bruce2020,Macias2020,Valero2020traces,Valero2020}.
These tools remove Java bytecode using static and dynamic analysis. 
In contrast, our study focuses on the evolution and the emergence of bloat in Java projects, while spotting some of the current research gaps and tooling for effective dependency management.
Other studies have focused on eliminating bloat in source code~\cite{Vazquez2019}, binary shared libraries~\cite{Agadakos2020}, highly configurable programs~\cite{Koo2019}, or containers~\cite{Rastogi2017}.
Other works have focused on improving the debloat process through various optimizations techniques~\cite{Xin2020, Azad2019,Sun2018,AntignacSS17,HagueLH19}.
As far as we know, we are the first to conduct a longitudinal study to analyze software bloat.

\textit{\textbf{Bloated Dependencies.}}
Our work follows up on our previous study of bloated dependencies ~\cite{Valero2020}. 
Our quantitative and qualitative study of bloated dependencies in the Maven ecosystem, revealed the importance of the phenomenon in Maven Central. Our interactions with software developers showed that removing bloated dependencies is perceived as a valuable contribution.
Here we extend this previous study in two ways.
First, we perform a study of bloated dependencies with distinct study subjects on a chronological basis. 
This brings novel insights on the evolution of bloated dependencies through the history of software projects, corroborating the importance of maintaining \texttt{pom.xml}\xspace files.
We bring novel evidence in favor of removing bloated dependencies.
Second, we perform a unique study on the interaction between maintenance activities and the emergence of bloat. These new results contribute to understanding the origin of bloat as well as estimating the maintenance effort unnecessarily invested when performing dependency updates.

\textit{\textbf{Software Bots.}}
Erlenhov et al.\@\xspace~\cite{Erlenhov20} perform an empirical study about the interaction between practitioners and software bots. They found that there is currently a lack of general-purpose smart bots that go beyond simple automation tools, such as dependency version updating. This is in line with our results, as we have seen that dependency bots do not perform advanced dependency usage analysis, sending unnecessary warnings that could be avoided. 
Wessel et al.\@\xspace~\cite{wessel2020inconvenient} rise attention on the inconvenient side of software bots.
They present empirical evidence that pull requests made by bots are, in some cases, perceived as disruptive and unwelcoming by developers.
Thus, motivating our work on reducing the number of warnings caused by bots dedicated to automatically update dependencies.
\vspace{-0.15cm}



\section{Conclusion}
This paper presented the first large-scale longitudinal study about the evolution of software bloat, with a focus on bloated Java dependencies. 
We collected a unique dataset of \np{\nbCommitsNum} dependency tree versions, tagged with usage dependency status, from \np{\nbProjectsNum} Java projects hosted on GitHub. 
Through the analysis of \np{\nbDependenciesNum} dependencies, we provided evidence about an essential phenomenon: \np[\%]{89.2} of the dependencies that become bloated over evolution stay bloated over time.
As a consequence, developers spend significant time updating dependencies that are actually bloated. We find that \np[\%]{22} of dependency updates are made on bloated dependencies. These updates include a significant number of updates suggested by Dependabot.
We also demonstrate that bloated dependencies are primarily originated from the addition of new dependencies that are never used, rather than from code changes.

Our work paves the way to better understand the importance of debloating tools, such as \textsc{DepClean}\xspace, to handle the increasing phenomenon of software bloat.
In particular, evidence that bloated code stays bloated is important for developers who need to decide if they should remove code. 
Our novel findings about the role of Dependabot on the unnecessary maintenance effort provide concrete insights to improve the suggestions that this single bot shares with developers.




\begin{acks}
This work is partially supported by the Wallenberg AI, Autonomous Systems, and Software Program (WASP) funded by Knut and Alice Wallenberg Foundation and by
the TrustFull project funded by the Swedish Foundation for
Strategic Research.
\end{acks}












\subsection{Definitions}






\[
  f(\mathcal{T}_p) =
  \begin{cases}
   
   
    \text{\emph{inc}} & \text{if $slope(lm(\mathcal{T}_p)) > 0 \wedge \exists t_j \in \mathcal{T}_p : t_j < t_{j-1}$} \\
    \text{\emph{dec}} & \text{if $slope(lm(\mathcal{T}_p)) < 0 \wedge \exists t_j \in \mathcal{T}_p : t_j > t_{j-1}$} \\
    \text{\emph{stable}} & \text{if $\forall t_i \in \mathcal{T}_p : t_i = t_{i-1}$} 
   
  \end{cases}
\]


