\section{Introduction}

Please follow the steps outlined below when submitting your manuscript to
the IEEE Computer Society Press.  This style guide now has several
important modifications (for example, you are no longer warned against the
use of sticky tape to attach your artwork to the paper), so all authors
should read this new version.

\subsection{Language}

All manuscripts must be in English.

\subsection{Dual submission}

Please refer to the author guidelines on the ICCV 2021 web page for a
discussion of the policy on dual submissions.

\subsection{Paper length}
Papers, excluding the references section,
must be no longer than eight pages in length. The references section
will not be included in the page count, and there is no limit on the
length of the references section. For example, a paper of eight pages
with two pages of references would have a total length of 10 pages.
{\bf There will be no extra page charges for ICCV 2021.}

Overlength papers will simply not be reviewed.  This includes papers
where the margins and formatting are deemed to have been significantly
altered from those laid down by this style guide.  Note that this
\LaTeX\ guide already sets figure captions and references in a smaller font.
The reason such papers will not be reviewed is that there is no provision for
supervised revisions of manuscripts.  The reviewing process cannot determine
the suitability of the paper for presentation in eight pages if it is
reviewed in eleven.  

\subsection{The ruler}
The \LaTeX\ style defines a printed ruler which should be present in the
version submitted for review.  The ruler is provided in order that
reviewers may comment on particular lines in the paper without
circumlocution.  If you are preparing a document using a non-\LaTeX\
document preparation system, please arrange for an equivalent ruler to
appear on the final output pages.  The presence or absence of the ruler
should not change the appearance of any other content on the page.  The
camera ready copy should not contain a ruler. (\LaTeX\ users may uncomment
the \verb'\iccvfinalcopy' command in the document preamble.)  Reviewers:
note that the ruler measurements do not align well with lines in the paper
--- this turns out to be very difficult to do well when the paper contains
many figures and equations, and, when done, looks ugly.  Just use fractional
references (e.g.\ this line is $095.5$), although in most cases one would
expect that the approximate location will be adequate.

\subsection{Mathematics}

Please number all of your sections and displayed equations.  It is
important for readers to be able to refer to any particular equation.  Just
because you didn't refer to it in the text doesn't mean some future reader
might not need to refer to it.  It is cumbersome to have to use
circumlocutions like ``the equation second from the top of page 3 column
1''.  (Note that the ruler will not be present in the final copy, so is not
an alternative to equation numbers).  All authors will benefit from reading
Mermin's description of how to write mathematics:
\url{http://www.pamitc.org/documents/mermin.pdf}.

\subsection{Blind review}

Many authors misunderstand the concept of anonymizing for blind
review.  Blind review does not mean that one must remove
citations to one's own work---in fact it is often impossible to
review a paper unless the previous citations are known and
available.

Blind review means that you do not use the words ``my'' or ``our''
when citing previous work.  That is all.  (But see below for
tech reports.)

Saying ``this builds on the work of Lucy Smith [1]'' does not say
that you are Lucy Smith; it says that you are building on her
work.  If you are Smith and Jones, do not say ``as we show in
[7]'', say ``as Smith and Jones show in [7]'' and at the end of the
paper, include reference 7 as you would any other cited work.

An example of a bad paper just asking to be rejected:
\begin{quote}
\begin{center}
    An analysis of the frobnicatable foo filter.
\end{center}

   In this paper we present a performance analysis of our
   previous paper [1], and show it to be inferior to all
   previously known methods.  Why the previous paper was
   accepted without this analysis is beyond me.

   [1] Removed for blind review
\end{quote}

An example of an acceptable paper:

\begin{quote}
\begin{center}
     An analysis of the frobnicatable foo filter.
\end{center}

   In this paper we present a performance analysis of the
   paper of Smith {\em et al.} [1], and show it to be inferior to
   all previously known methods.  Why the previous paper
   was accepted without this analysis is beyond me.

   [1] Smith, L and Jones, C. ``The frobnicatable foo
   filter, a fundamental contribution to human knowledge''.
   Nature 381(12), 1-213.
\end{quote}

If you are making a submission to another conference at the same time,
which covers similar or overlapping material, you may need to refer to that
submission in order to explain the differences, just as you would if you
had previously published related work.  In such cases, include the
anonymized parallel submission~\cite{Authors14} as additional material and
cite it as
\begin{quote}
[1] Authors. ``The frobnicatable foo filter'', F\&G 2014 Submission ID 324,
Supplied as additional material {\tt fg324.pdf}.
\end{quote}

Finally, you may feel you need to tell the reader that more details can be
found elsewhere, and refer them to a technical report.  For conference
submissions, the paper must stand on its own, and not {\em require} the
reviewer to go to a tech report for further details.  Thus, you may say in
the body of the paper ``further details may be found
in~\cite{Authors14b}''.  Then submit the tech report as additional material.
Again, you may not assume the reviewers will read this material.

Sometimes your paper is about a problem which you tested using a tool which
is widely known to be restricted to a single institution.  For example,
let's say it's 1969, you have solved a key problem on the Apollo lander,
and you believe that the ICCV70 audience would like to hear about your
solution.  The work is a development of your celebrated 1968 paper entitled
``Zero-g frobnication: How being the only people in the world with access to
the Apollo lander source code makes us a wow at parties'', by Zeus {\em et al.}.

You can handle this paper like any other.  Don't write ``We show how to
improve our previous work [Anonymous, 1968].  This time we tested the
algorithm on a lunar lander [name of lander removed for blind review]''.
That would be silly, and would immediately identify the authors. Instead
write the following:
\begin{quotation}
\noindent
   We describe a system for zero-g frobnication.  This
   system is new because it handles the following cases:
   A, B.  Previous systems [Zeus et al. 1968] didn't
   handle case B properly.  Ours handles it by including
   a foo term in the bar integral.

   ...

   The proposed system was integrated with the Apollo
   lunar lander, and went all the way to the moon, don't
   you know.  It displayed the following behaviours
   which show how well we solved cases A and B: ...
\end{quotation}
As you can see, the above text follows standard scientific convention,
reads better than the first version, and does not explicitly name you as
the authors.  A reviewer might think it likely that the new paper was
written by Zeus {\em et al.}, but cannot make any decision based on that guess.
He or she would have to be sure that no other authors could have been
contracted to solve problem B.
\medskip

\noindent
FAQ\medskip\\
{\bf Q:} Are acknowledgements OK?\\
{\bf A:} No.  Leave them for the final copy.\medskip\\
{\bf Q:} How do I cite my results reported in open challenges?
{\bf A:} To conform with the double blind review policy, you can report results of other challenge participants together with your results in your paper. For your results, however, you should not identify yourself and should not mention your participation in the challenge. Instead present your results referring to the method proposed in your paper and draw conclusions based on the experimental comparison to other results.\medskip\\

\begin{figure}[t]
\begin{center}
\fbox{\rule{0pt}{2in} \rule{0.9\linewidth}{0pt}}
  
\end{center}
   \caption{Example of caption.  It is set in Roman so that mathematics
   (always set in Roman: $B \sin A = A \sin B$) may be included without an
   ugly clash.}
\label{fig:long}
\label{fig:onecol}
\end{figure}

\subsection{Miscellaneous}

\noindent
Compare the following:\\
\begin{tabular}{ll}
 \verb'$conf_a$' &  $conf_a$ \\
 \verb'$\mathit{conf}_a$' & $\mathit{conf}_a$
\end{tabular}\\
See The \TeX book, p165.

The space after {\em e.g.}, meaning ``for example'', should not be a
sentence-ending space. So {\em e.g.} is correct, {\em e.g.} is not.  The provided
\verb'{\em e.g.}' macro takes care of this.

When citing a multi-author paper, you may save space by using ``et alia'',
shortened to ``{\em et al.}'' (not ``{\em et.\ al.}'' as ``{\em et}'' is a complete word.)
However, use it only when there are three or more authors.  Thus, the
following is correct: ``
   Frobnication has been trendy lately.
   It was introduced by Alpher~\cite{Alpher02}, and subsequently developed by
   Alpher and Fotheringham-Smythe~\cite{Alpher03}, and Alpher {\em et al.}~\cite{Alpher04}.''

This is incorrect: ``... subsequently developed by Alpher {\em et al.}~\cite{Alpher03} ...''
because reference~\cite{Alpher03} has just two authors.  If you use the
\verb'{\em et al.}' macro provided, then you need not worry about double periods
when used at the end of a sentence as in Alpher {\em et al.}.

For this citation style, keep multiple citations in numerical (not
chronological) order, so prefer \cite{Alpher03,Alpher02,Authors14} to
\cite{Alpher02,Alpher03,Authors14}.

\begin{figure*}
\begin{center}
\fbox{\rule{0pt}{2in} \rule{.9\linewidth}{0pt}}
\end{center}
   \caption{Example of a short caption, which should be centered.}
\label{fig:short}
\end{figure*}

\section{Formatting your paper}

All text must be in a two-column format. The total allowable width of the
text area is $6\frac78$ inches (17.5 cm) wide by $8\frac78$ inches (22.54
cm) high. Columns are to be $3\frac14$ inches (8.25 cm) wide, with a
$\frac{5}{16}$ inch (0.8 cm) space between them. The main title (on the
first page) should begin 1.0 inch (2.54 cm) from the top edge of the
page. The second and following pages should begin 1.0 inch (2.54 cm) from
the top edge. On all pages, the bottom margin should be 1-1/8 inches (2.86
cm) from the bottom edge of the page for $8.5 \times 11$-inch paper; for A4
paper, approximately 1-5/8 inches (4.13 cm) from the bottom edge of the
page.

\subsection{Margins and page numbering}

All printed material, including text, illustrations, and charts, must be kept
within a print area 6-7/8 inches (17.5 cm) wide by 8-7/8 inches (22.54 cm)
high.

Page numbers should be included for review submissions but not for the 
final paper. Review submissions papers should have page numbers in the 
footer with numbers centered and .75 inches (1.905 cm) from the bottom 
of the page and start on the first page with the number 1.

Page numbers will be added by the publisher to all camera-ready papers 
prior to including them in the proceedings and before submitting the 
papers to IEEE Xplore. As such, your camera-ready submission should 
not include any page numbers. Page numbers should automatically be 
removed by uncommenting (if it's not already) the line
\begin{verbatim}
\end{verbatim}
near the beginning of the .tex file.

\subsection{Type-style and fonts}

Wherever Times is specified, Times Roman may also be used. If neither is
available on your word processor, please use the font closest in
appearance to Times to which you have access.

MAIN TITLE. Center the title 1-3/8 inches (3.49 cm) from the top edge of
the first page. The title should be in Times 14-point, boldface type.
Capitalize the first letter of nouns, pronouns, verbs, adjectives, and
adverbs; do not capitalize articles, coordinate conjunctions, or
prepositions (unless the title begins with such a word). Leave two blank
lines after the title.

AUTHOR NAME(s) and AFFILIATION(s) are to be centered beneath the title
and printed in Times 12-point, non-boldface type. This information is to
be followed by two blank lines.

The ABSTRACT and MAIN TEXT are to be in a two-column format.

MAIN TEXT. Type main text in 10-point Times, single-spaced. Do NOT use
double-spacing. All paragraphs should be indented 1 pica (approx. 1/6
inch or 0.422 cm). Make sure your text is fully justified---that is,
flush left and flush right. Please do not place any additional blank
lines between paragraphs.

Figure and table captions should be 9-point Roman type as in
Figures~\ref{fig:onecol} and~\ref{fig:short}.  Short captions should be centered.

\noindent Callouts should be 9-point Helvetica, non-boldface type.
Initially capitalize only the first word of section titles and first-,
second-, and third-order headings.

FIRST-ORDER HEADINGS. (For example, {\large \bf 1. Introduction})
should be Times 12-point boldface, initially capitalized, flush left,
with one blank line before, and one blank line after.

SECOND-ORDER HEADINGS. (For example, { \bf 1.1. Database elements})
should be Times 11-point boldface, initially capitalized, flush left,
with one blank line before, and one after. If you require a third-order
heading (we discourage it), use 10-point Times, boldface, initially
capitalized, flush left, preceded by one blank line, followed by a period
and your text on the same line.

\subsection{Footnotes}

Please use footnotes\footnote {This is what a footnote looks like.  It
often distracts the reader from the main flow of the argument.} sparingly.
Indeed, try to avoid footnotes altogether and include necessary peripheral
observations in
the text (within parentheses, if you prefer, as in this sentence).  If you
wish to use a footnote, place it at the bottom of the column on the page on
which it is referenced. Use Times 8-point type, single-spaced.

\subsection{References}

List and number all bibliographical references in 9-point Times,
single-spaced, at the end of your paper. When referenced in the text,
enclose the citation number in square brackets, for
example~\cite{Authors14}.  Where appropriate, include the name(s) of
editors of referenced books.

\begin{table}
\begin{center}
\begin{tabular}{|l|c|}
\hline
Method & Frobnability \\
\hline\hline
Theirs & Frumpy \\
Yours & Frobbly \\
Ours & Makes one's heart Frob\\
\hline
\end{tabular}
\end{center}
\caption{Results.   Ours is better.}
\end{table}

\subsection{Illustrations, graphs, and photographs}

All graphics should be centered.  Please ensure that any point you wish to
make is resolvable in a printed copy of the paper.  Resize fonts in figures
to match the font in the body text, and choose line widths which render
effectively in print.  Many readers (and reviewers), even of an electronic
copy, will choose to print your paper in order to read it.  You cannot
insist that they do otherwise, and therefore must not assume that they can
zoom in to see tiny details on a graphic.

When placing figures in \LaTeX, it's almost always best to use
\verb+\includegraphics+, and to specify the  figure width as a multiple of
the line width as in the example below
{\small\begin{verbatim}
   \usepackage[dvips]{graphicx} ...
   \includegraphics[width=0.8\linewidth]
                   {myfile.eps}
\end{verbatim}
}

\subsection{Color}

Please refer to the author guidelines on the ICCV 2021 web page for a discussion
of the use of color in your document.

\section{Final copy}

You must include your signed IEEE copyright release form when you submit
your finished paper. We MUST have this form before your paper can be
published in the proceedings.

{\small
\bibliographystyle{ieee_fullname}

\section{Introduction}
Security inspection is a process of checking assets against set criteria and the evaluation of security systems and access controls to ensure safety, which is important to uncover any potential risks in various scenarios, such as public transportation and sensitive departments. In practice, the inspectors are required to monitor the scanned X-ray images acquired by the security inspection machine to uncover prohibited items, such as guns, ammunition, explosives, corrosive substances, toxic and radioactive substances. However, the inspectors struggle to localize prohibited items hidden in messy objects accurately and efficiently, which poses a great threat to safety. 

\begin{figure}[t]
\centering
\includegraphics[width=0.9\linewidth]{figures/overlapping.pdf}
\caption{Comparisons between the natural image (left) and X-ray image (right).}
\label{fig:overlapping}
\end{figure}

In recent years, due to the substantial development of deep learning and computer vision technologies\cite{ren2015faster, liu2016ssd, tian2019fcos, DBLP:conf/cvpr/JiWZDWZLH20, DBLP:conf/eccv/JiDZWWZHL20, DBLP:conf/eccv/LiDZWLWZ20, DBLP:conf/mm/CaiDZWWWL20}, automatic security inspection of prohibited items becomes possible. The security inspectors can quickly identify the locations and categories of prohibited items relying on computer vision technology. Most of the previous object detection algorithms in computer vision are designed to detect objects in natural images, which are not optimal for detection in X-ray images. In addition, X-rays have strong penetrating power, different materials in the object absorb X-rays to different degrees, resulting in different colors. Meanwhile, the contours of the occluder and the occluded objects in the x-ray are mixed together. As shown in Figure \ref{fig:overlapping}, compared with natural images, X-ray images have a quite different appearance and edges of objects and background, which brings new challenges in appearance modeling for X-ray detection. To advance the developments of prohibited items detection in X-ray images, some recent attempts devote to construct security inspection benchmarks \cite{mery2015gdxray,akcay2017evaluation,akcay2018using,miao2019sixray,wei2020occluded}. However, most of them fail to meet the requirements in real-world applications for three reasons. (1) Existing datasets only contain a small number and very few categories of prohibited items ({\em e.g.}, \textit{knife}, \textit{gun} and \textit{scissors}). For example, some common prohibited items such as \textit{powerbank}, \textit{lighter} and \textit{sprayer} are not included. (2) Some real-world scenarios require high security level based on accurate predictions of masks and categories of prohibited items. The image-level or bounding box-level annotations in previous datasets are not sufficient to train algorithms in such scenarios. (3) Detecting prohibited items hidden in messy objects is one of the most significant challenges in security inspection. Nevertheless, few studies are developed towards this goal due to the lack of comprehensive datasets covering such cases. 

To that end, we collect a large-scale prohibited item detection dataset (PIDray) towards real-world applications. Our PIDray dataset covers $12$ common prohibited items in X-ray images. Some example images with annotations are shown in Figure \ref{fig:samples}, where each image contains at least one prohibited item with both the bounding box and mask annotations. Notably, for better usage, the test set is divided into three subsets, {\em i.e.}, \textit{easy}, \textit{hard} and \textit{hidden}. The \textit{hidden} subset focuses on the prohibited items deliberately hidden in messy objects ({\em e.g.}, change the item shape by wrapping wires). To the best of our knowledge, it is the largest dataset for the detection of prohibited items to date. 

Meanwhile, we also present the selective dense attention network (SDANet) to construct a strong baseline, which consists of two modules, {\em i.e.}, the dense attention module and the dependency refinement module. The dense attention module uses both the spatial and channel-wise attention mechanisms to exploit discriminative features, which is effective to locate the deliberately prohibited items hidden in messy objects. The dependency refinement module is constructed to exploit the dependencies among multi-scale features. Extensive experiments on the proposed dataset show that our method performs favorably against the state-of-the-art methods. Especially, our SDANet achieves $1.5\%$ and $1.3\%$ AP improvements over Cascade Mask R-CNN \cite{DBLP:journals/corr/abs-1906-09756} for object detection and instance segmentation on the {\em hidden} subset, respectively.

The main contributions of this work are summarized as follows. (1) Towards the prohibited item detection in real-world scenarios, we present a large-scale benchmark, {\em i.e.}, PIDray, formed by $47,677$ images in total. To the best of our knowledge, it is the largest X-ray prohibited item detection dataset to date. Meanwhile, it is the first benchmark aiming at cases where the prohibited items are deliberately hidden in messy objects. (2) We propose the selective dense attention network, formed by the dense attention module and the dependency refinement module. The dense attention module is used to capture the discriminative features in both spatial and channel-wise, and the dependency refinement module is constructed to exploit the dependencies among multi-scale features. (3) Extensive experiments are conducted on the proposed dataset to verify the effectiveness of the proposed method compared to the state-of-the-art methods.

\begin{figure}[t]
\centering
\includegraphics[width=1.0\linewidth]{figures/image_samples.pdf}
\caption{Example images in the PIDray dataset with $12$ categories of prohibited items. Each image is provided with image-level and instance-level annotation. For clarity, we show one category per image.}
\label{fig:samples}
\end{figure}

\section{Related Work}
\label{section:related work}

\begin{table*}[t]
  \centering
  \caption{Comparison of the dataset statistics with existing X-ray benchmarks. ``Total'' and ``Prohibited'' indicate the number of total images and the images containing prohibited items in the dataset, respectively. \textbf{C}, \textbf{O}, and \textbf{I} represent Classification, Object Detection, and Instance Segmentation respectively. \textbf{S}, \textbf{A}, and \textbf{R} represent Subway, Airport, and Railway Station respectively.}
  \vspace{4pt}
  \label{table:comparison}
  \setlength{\tabcolsep}{1.0mm}{
    \begin{tabular}{c|c|c|c|c|c|c|c|c|c|c|c}
    \hline
    \multirow{2}[4]{*}{Dataset} & \multirow{2}[4]{*}{Year} & \multirow{2}[4]{*}{Classes} & \multicolumn{2}{c|}{Images} & \multicolumn{3}{c|}{Annotations} & \multirow{2}[4]{*}{Type} & \multirow{2}[4]{*}{Scene} & \multirow{2}[4]{*}{Application} & \multirow{2}[4]{*}{Availability} \\
\cline{4-8}          &       &       & Total & Prohibited & Image & Bbox  & Mask  &       &       &       &  \\
    \hline
    GDXray \cite{mery2015gdxray} & $2015$  & $3$     & $8,150$ & $8,150$ & \checkmark     & \checkmark     &       & Real  & -     & C+O   & \checkmark \\
    Dbf$_{6}$ \cite{akcay2017evaluation}  & $2017$  & $6$     & $11,627$ & $11,627$ & \checkmark     & \checkmark     &       & Real  & -     & C+O   &  $\times$\\
    Dbf$_{3}$ \cite{akcay2018using}  & $2018$  & $3$     & $7,603$ & $7,603$ & \checkmark     & \checkmark     &       & Real  & -     & C+O   &  $\times$\\
    Liu~{\em et al.}~\cite{Jinyi2019xray}    & $2019$  & $6$     & $32,253$ & $12,683$ & \checkmark     & \checkmark     &       & Real  & S     & C+O   &  $\times$\\
    SIXray \cite{miao2019sixray} & $2019$  & $6$     & $\bf1,059,231$ & $8,929$ & \checkmark     & \checkmark     &       & Real  & S     & C+O   & \checkmark \\
    OPIXray \cite{wei2020occluded} & $2020$  & $5$     & $8,885$ & $8,885$ & \checkmark     & \checkmark     &       & Synthetic & A     & C+O   & \checkmark \\
    \hline
    \hline
    Ours  & $2021$  & $\bf12$    & $47,677$ & $\bf47,677$ & \checkmark     & \checkmark     & \checkmark     & Real  & S+A+R &    C+O+I   & \checkmark \\
    \hline
    \end{tabular}}%
\end{table*}%


\subsection{Prohibited Items Benchmarks}
When the X-ray passes through an object, different materials absorb the X-ray to different degrees due to its strong penetrating power. Therefore, different materials show different colors in X-ray images. This ability makes it difficult to detect overlapping data. In addition, the difficulties caused by natural images still exist, including intra-class differences, data imbalance, and occlusion.

To advance robust prohibited item detection methods, previous works collect a few datasets. \cite{mery2015gdxray} propose a public dataset called GDXray for nondestructive testing. GDXray contains three types of prohibited items: \textit{gun}, \textit{shuriken} and \textit{razor blade}. Since there is almost no complex background and overlap, it is easy to recognize or detect objects in this dataset. Compared with GDXray, Dbf6 \cite{akcay2017evaluation}, Dbf3 \cite{akcay2018using} and OPIXray \cite{wei2020occluded} contain complicated background and overlapping-data, but the number of images and the number of prohibited items are still insufficient. Recently, \cite{Jinyi2019xray} construct a dataset containing $32,253$ X-ray images, of which $12,683$ images include prohibited items. This dataset contains $6$ types of items, but none of them are strictly prohibited, such as \textit{mobile phones}, \textit{umbrellas}, \textit{computers}, and \textit{keys}. \cite{miao2019sixray} release a large-scale security inspection benchmark named as SIXray, which contains $1,059,231$ X-ray images with image-level annotation. However, fewer images contain prohibited items in the dataset ({\em i.e.}, only $0.84\%$). In addition, the dataset contains $6$ categories of prohibited items, but only $5$ categories are actually annotated. Different from the aforementioned datasets, we propose a new large-scale security inspection benchmark that contains over $47k$ images with prohibited items and $12$ categories of prohibited items with pixel-level annotation. Towards real-world application, we focus on detecting deliberately hidden prohibited items.


\subsection{Object Detection}
Object detection is one of the fundamental tasks in the computer vision community. Modern object detectors are generally divided into two groups: two-stage and one-stage detectors. 

\textbf{Two-stage Detectors.} R-CNN \cite{girshick2014rich} is one of the first works to show that CNN can dramatically improve the detection performance. However, each regional proposal is processed separately in RCNN, which is very time-consuming. Fast-RCNN\cite{girshick2015fast} proposes the ROI pooling layer, which can extract fixed-size features for each proposal from the feature map of the full image. Faster R-CNN \cite{ren2015faster} introduces the RPN network to replace selective search, which inspires a lot of later work. For example, FPN \cite{lin2017feature} combines low-resolution features with high-resolution features through a top-down pathway and lateral connections. Mask R-CNN \cite{he2017mask} adds a mask branch on the basis of Faster-RCNN\cite{ren2015faster} to improve the detection performance through multi-task learning. Cascade R-CNN \cite{cai2018cascade} applies the classic cascade architecture to Faster R-CNN \cite{ren2015faster}. Libra R-CNN \cite{pang2019libra} develops a simple and effective framework to eliminate the imbalance in the detection training process.

\textbf{One-stage Detectors.} OverFeat \cite{sermanet2013overfeat} is one of the first deep learning based one-stage detectors. After that, different one-stage object detectors are proposed, including SSD \cite{liu2016ssd}, DSSD \cite{fu2017dssd}, and YOLO series \cite{redmon2016you,redmon2017yolo9000,redmon2018yolov3}. RetinaNet \cite{lin2017focal} greatly improves the accuracy of one-stage detector, making it possible for one-stage detector to surpass two-stage detector. Recently, anchor-free approaches have attracted wide attention of researchers by using key points to represent the objects, including CornerNet \cite{law2018cornernet}, CenterNet \cite{duan2019centernet}, and FCOS \cite{tian2019fcos}. These methods eliminate the need for anchors and provide a simplified detection framework. 

\subsection{Attention Mechanism}
Recently, attention mechanism has been widely used in a variety of tasks, such as neural machine translation, image captioning, and visual question answering. The essence of the attention mechanism is to imitate human visual attention, which can quickly filter out discriminative information from a large number of information. In order to obtain more discriminative information, various attention mechanisms have been proposed. SENet \cite{hu2018squeeze} proposes the Squeeze-and-Excitation module to model the interdependence between channels. CBAM \cite{woo2018cbam} models the inter-channel relation and the inter-spatial relation of features. Non-Local network \cite{wang2018non} can capture the remote dependency of any two locations directly, which calculates the weighted sum of the features of all positions in the input feature map as the response of a certain position. As many previous works \cite{lin2017feature,liu2018path} show the importance of multi-scale feature fusion, we think it is the key technology to solve the problem of prohibited item detection. In X-ray images, many important details of objects are missing, such as texture and appearance information. Moreover, the contours of objects overlap, which also brings great challenges to detection. Multi-scale feature fusion considers the low-level layers with rich detail information and the high-level layers with rich semantic information, which can better detect the prohibited item. Therefore, we propose a selective dense attention network. Specifically, we learn the relations between feature maps across different stages at inter-channel and inter-pixel positions.

\begin{figure}[t]
\centering
\includegraphics[width=1.0 \linewidth]{figures/testset.pdf}
\caption{Examples of test sets with different difficulty levels in the proposed PIDray dataset. From top to bottom, the degree of difficulty gradually increases. }
\label{fig:testset_sample}
\end{figure}

\section{The PIDray Dataset}
In this section, we provide details of the constructed PIDray dataset, including the data collection, annotation, and statistical information.

\subsection{Data Collection}
The PIDray dataset was collected in different scenarios (such as airports, subway stations, and railway stations), where we were allowed to place a security inspection machine. We recruited volunteers who did not mind displaying their packages in the dataset (we promise to use it only for scientific research and not for business). We use 3 security inspection machines from different manufacturers to collect X-ray data. Images generated by different machines have certain differences in the size and color of the objects and background. After sending the package to the security inspection machine, the machine will completely cut out the package by detecting the blank part of the image. Generally speaking, the image height is fixed while the image width relies on the size of the package being scanned.

The complete collection process is as follows: when the person is required for security inspection, we randomly put the pre-prepared prohibited items in the package he or she is carrying. At the same time, the rough area of the object was saved, so that the subsequent annotation work can be carried out smoothly. There are a total of $12$ categories of prohibited items defined in the dataset, namely \textit{gun}, \textit{knife}, \textit{wrench}, \textit{pliers}, \textit{scissors}, \textit{hammer}, \textit{handcuffs}, \textit{baton}, \textit{sprayer}, \textit{powerbank}, \textit{lighter} and \textit{bullet}. To keep diversity, we prepare $2\sim15$ instances for every kind of prohibited item. We spend more than three months collecting a total of $47,677$ images for the PIDray dataset. Finally, the distribution of each category in the dataset is summarized in Figure \ref{fig:distribute}. All images are stored in PNG format. 

\begin{figure}[t]
\centering
\includegraphics[width=0.99\linewidth]{figures/distribution.pdf}
\caption{Class distribution of the PIDray dataset. The blue bar represents the number of each class in the PIDray dataset.}
\label{fig:distribute}
\end{figure}

\begin{table}[t]
  \centering
  \small
  \fontsize{10}{8}\selectfont
  \caption{Statistics of the PIDray dataset.}
  \vspace{4pt}
  \label{table:partition_dataset}
  \setlength{\tabcolsep}{3.7mm}{
    \begin{tabular}{ccccc}
    \toprule
    \multirow{2}[3]{*}{Mode} & \multirow{2}[3]{*}{Train} & \multicolumn{3}{c}{Test} \\
\cmidrule{3-5}          &       & Easy  & Hard  & Hidden \\
    \midrule
    Count & $29,457$ & $9,482$  & $3,733$  & $5,005$ \\
    \midrule
    Total & \multicolumn{4}{c}{$47,677$} \\
    \bottomrule
    \end{tabular}}%
    
\end{table}%

\begin{figure*}[t]
\centering
\includegraphics[width=0.95\linewidth]{figures/Structure.pdf}
\caption{Network architecture. (a) The overall architecture of the proposed selective dense attention network. (b) The selective dense attention module. (c) The dependency refinement module.}
\label{fig:structure}
\end{figure*}

\subsection{Data Annotation}
We recruited some volunteers to annotate the collected data. In order to enable them to identify prohibited items from X-ray images more quickly and accurately, some training courses have been organized. We first organized 5 volunteers to filter out images from the dataset that contain no prohibited items. At the same time, they also need to annotate the image-level labels, which can facilitate the later annotation work. In terms of annotation, we organized over $10$ volunteers to label our dataset using the labelme tool\footnote{\url{http://labelme.csail.mit.edu/Release3.0/}} for two months. Each image takes about $3$ minutes to annotate, and each volunteer spends about $10$ hours to annotate the image every day. During the annotation process, we label both the bounding box and the segmentation mask of each instance. After multiple rounds of double-check, the errors are minimized as much as possible. Finally, we generate high-quality annotations for each image.

\subsection{Data Statistics}
As far as we know, the PIDray dataset is the largest X-ray prohibited item detection dataset to date. It contains 47,677 images and 12 classes of prohibited items. As presented in Table \ref{table:partition_dataset}, we split those images into 29,457(roughly 60\%) and 18,220(remaining 40\%) images as training and test sets, respectively. In addition, according to the difficulty degree of prohibited item detection, we group the test set into three subsets, {\em i.e.}, \textit{easy}, \textit{hard} and \textit{hidden}. Specifically, the \textit{easy} mode means that the image in the test set contains only one prohibited item. The \textit{hard} mode indicates that the image in the test set contains more than one prohibited item. The \textit{hidden} mode indicates that the image in the test set contains deliberately hidden prohibited items. As shown in Figure \ref{fig:testset_sample}, we provide several examples in the test set with different difficulty levels. 

\section{Selective Dense Attention Network}
As discussed above, the previous works usually employ feature pyramid \cite{lin2017feature} to exploit multi-scale feature maps in the network, which focuses on fusing features only in adjacent layers. After that, the succinct heads ({\em e.g.}, a simple convolutional layer) are applied on the pooled feature grid to predict bounding boxes and masks of instances. However, the performance suffers from scale variation of objects in complex scenes. Our goal is to learn the importance of multi-scale feature maps based on top-down feature pyramid structure \cite{lin2017feature}. In this section, we will introduce the architecture and components of the proposed Selective Dense Attention Network (SDANet) in detail.

\subsection{Network Architecture}
As shown in Figure \ref{fig:structure}(a), following the feature pyramid, our network further makes full use of multi-scale feature maps by the following two critical steps: 1) Fusing information from different layers by two selective attention modules. 2) Enhancing the fused features by the dependency refinement module. 

Note that the two steps are performed on the feature map in each layer. After combining both the original and enhanced maps, the multi-scale representation is fed into the Region Proposal Network (RPN) for final prediction.

Inspired by the work of \cite{DBLP:conf/cvpr/LiW0019}, we propose two selective attention modules to extract channel-wise and spatial attention of different feature maps in the pyramid respectively, including the Selective Channel-wise Attention module (SCA) and the Selective Spatial Attention module (SSA). As shown in Figure \ref{fig:structure}(b), each feature map in the pyramid is fed into SCA and SSA respectively. At the $i$-th layer, the output enhanced feature is calculated by element-wise summation of features after the two modules. To implement the SCA and SSA modules, we first fuse features in different layers through element-wise operations, {\em i.e.}, $\hat{X} = \sum_{i=1}^{n}X_{i}$. Thus we achieve a global semantic representation among different maps. Note that, we resize the multi-stage features $\left\{X_{1},\cdots,X_{n}\right\}$ to the same scale as the $i$-th layer feature before feeding them into the two modules. Then, we obtain enhanced features by aggregating feature maps with various attentions, which is described in detail as follows.

\subsection{Selective Channel-wise Attention}
As shown in Figure \ref{figure:channel_attention}, we employ the global average pooling (GAP) layer to obtain global channel information based on the base feature $\hat{X}$. After that, we use the fully connected (FC) layer to squeeze global channel information by reducing the channel dimension ({\em e.g.}, from $256$ to $128$). Further, we obtain the channel-wise attention weights $\{\omega^{c}_{i}\}_{i=1}^n$ of different feature maps adaptively by adding FC layers and softmax operation for each layer. Finally, the enhanced feature map $V_{C}$ is obtained by the attention weight on each layer, {\em i.e.}, $V_{C} = \sum_{i=1}^{n}\omega^{c}_{i}\cdot X_{i}$.

\subsection{Selective Spatial Attention}
As shown in Figure \ref{figure:spatial_attention}, we use both the average pooling and maximum pooling operations on the feature map $\hat{X}$ to generate two different spatial context descriptors, {\em i.e.}, $\text{Avg}(\hat{X})$, $\text{Max}(\hat{X})$. Given the concatenated context descriptors, we can obtain the spatial attention weights by adding convolutional layers and softmax operation for each layer. Finally, the feature map $V_{S}$ is obtained by the attention weight on each layer, {\em i.e.}, $V_{S}(x,y) = \sum_{i=1}^{n}\omega^{s}_{i}(x,y)\cdot X_{i}(x,y)$, where $(x,y)$ indicates the index of pixel in feature map.

\begin{figure}[t]
\centering
\includegraphics[width=1.0\linewidth]{figures/Channel_attention.pdf}
\caption{Illustration of selective channel-wise attention module (SCA).}
\label{figure:channel_attention}
\end{figure}

\begin{figure}[t]
\centering
\includegraphics[width=1.0\linewidth]{figures/Position_attention.pdf}
\caption{Illustration of selective spatial attention module (SSA).}
\label{figure:spatial_attention}
\end{figure}

\subsection{Dependency Refinement}
After obtaining the aggregated features with both channel and spatial attention, we develop the Dependency Refinement (DR) module to generate more discriminative feature maps. Non-local representation \cite{wang2018non} can capture long-range dependencies effectively, which further improves the accuracy. As shown in Figure \ref{fig:structure}(c), we first aggregate global context features, and then establish relationships between different channels. Finally, the global context feature is merged into features of all positions by a fusion module.

\begin{table*}[t]
  \centering
  \small
  \caption{The evaluation results on the proposed PIDray dataset. COCO mmAP (\%)  is used to evaluate performance of all methods.}
  \vspace{4pt}
  \setlength{\tabcolsep}{2.85mm}
    \begin{tabular}{c|c|cccc|cccc}
    \hline
    \multirow{2}{*}{Method} & \multirow{2}{*}{Backbone} &  \multicolumn{4}{c|}{Detection AP}     & \multicolumn{4}{c}{Segmentation AP} \\
\cline{3-10}          &       & Easy  & Hard  & Hidden & Overall & Easy  & Hard  & Hidden  & Overall \\
    \hline
    FCOS  & ResNet-101-FPN  &  $61.8$  &  $51.7$  &  $37.5$  &  $50.3$  & -     & -     & -     & - \\
    RetinaNet  & ResNet-101-FPN  &  $61.8$  &  $52.2$  &  $40.6$  &  $51.5$  & -     & -     & -     & - \\
    Faster R-CNN & ResNet-101-FPN  & $63.3$  & $57.2$  & $42.1$  & $54.2$  & -     & -     & -     & - \\
    Libra R-CNN & ResNet-101-FPN  & $64.7$  & $58.8$  & $42.9$  & $55.5$  & -     & -     & -     & - \\
    Mask R-CNN & ResNet-101-FPN  & $64.7$  & $59.0$    & $43.8$  & $55.8$  & $57.6$  & $50.2$  & $35.2$  & $47.7$  \\
    SSD512 & VGG16  & $68.1$  & $58.9$  & $45.7$  & $57.6$  & -     & -     & -     & - \\
    Cascade R-CNN & ResNet-101-FPN  & $69.3$  & $62.8$  & $48.0$    & $60.0$  & -     & -     & -     & - \\
    Cascade Mask R-CNN & ResNet-101-FPN  & $70.9$  & $64.0$    & $48.0$    & $61.0$  & $59.2$  & $51.5$  & $36.1$  & $48.9$  \\
    SDANet(ours) & ResNet-101-FPN & $\textbf{71.2}$ & $\textbf{64.2}$ & $\textbf{49.5}$ & $\textbf{61.6}$  & $\textbf{59.9}$ & $\textbf{52.0}$ & $\textbf{37.4}$ & $\textbf{49.8}$  \\
    \hline
    \hline
    Cascade Mask R-CNN & ResNet-101-BiFPN  & $68.0$  & $61.1$    & $46.9$    & $58.7$  & $58.0$  & $49.8$  & $35.3$  & $47.7$  \\
    Cascade Mask R-CNN & ResNet-101-PAFPN  & $70.4$  & $63.4$    & $46.7$    & $60.2$  & $59.2$  & $51.4$  & $35.0$  & $48.5$  \\
    Cascade Mask R-CNN & ResNet-101-FPN  & $70.9$  & $64.0$    & $48.0$    & $61.0$  & $59.2$  & $51.5$  & $36.1$  & $48.9$  \\
    SDANet(ours) & ResNet-101-FPN & $\textbf{71.2}$ & $\textbf{64.2}$ & $\textbf{49.5}$ & $\textbf{61.6}$  & $\textbf{59.9}$ & $\textbf{52.0}$ & $\textbf{37.4}$ & $\textbf{49.8}$  \\
    \hline
    \end{tabular}%
  \label{table:baseline_results}%
\end{table*}%

\begin{table*}[t]
  \centering
  \small
  \caption{Effectiveness of various designs. All models are trained on the PIDray {\it training} subset and tested on the PIDray hidden {\it test} set. The accuracies are indicated by ``detection AP/segmentation AP''.}
  \vspace{4pt}
  \setlength{\tabcolsep}{2.4mm}{
\begin{tabular}{ccc|cccccccc}
    \hline
    SCA & SSA & DR& AP    & $\text{AP}_{50}$  & $\text{AP}_{75}$  & $\text{AP}_{S}$   & $\text{AR}_{1}$   & $\text{AR}_{10}$  & $\text{AR}_{100}$ & $\text{AR}_{S}$ \\
    \hline
          &       &       &       $48.0/36.1$    & $62.7/58.9$  & $54.0/40.4$    & $57.0/43.5$    & $56.0/42.9$    & $57.6/44.0$  & $57.6/44.0$  & $57.6/44.0$ \\
    \checkmark     &       &             & $48.3/36.5$  & $63.5/59.3$  & $54.3/41.2$  & $57.2/43.9$  & $56.2/43.4$  & $57.9/44.4$  & $57.9/44.4$  & $57.9/44.4$ \\
         &    \checkmark   &            & $48.3/36.2$  & $63.2/59.6$  & $54.6/40.1$  & $57.4/43.8$  & $56.6/43.3$  & $58.1/44.3$  & $58.1/44.3$  & $58.1/44.3$ \\
    \checkmark     & \checkmark     &            & $48.9/36.7$  & $63.8/60.0$  & $55.4/40.8$  & $58.3/44.3$  & $57.4/43.8$  & $59.3/45.0$  & $59.3/45.0$  & $59.3/45.0$ \\
    \checkmark     & \checkmark     & \checkmark        & $49.5/37.4$  & $64.5/60.6$  & $55.7/42.2$  & $58.5/44.8$  & $57.2/44.1$  & $59.5/45.5$  & $59.5/45.5$    & $59.5/45.5$ \\
    \hline
    \end{tabular}}
  \label{table:ablation_study}%
\end{table*}%


\section{Experiment}
We conduct extensive experiments on the PIDray dataset to compare the proposed method with several state-of-the-art algorithms. Then, the ablation study is used to show the effectiveness of the proposed modules in our method. Finally, we verify the effectiveness of the proposed method on general detection datasets.
\subsection{Implementation Details}
We employ the MMDetection toolkit\footnote{\url{https://github.com/open-mmlab/mmdetection}} to implement our method, which is performed on a machine with two NVIDIA Tesla V100 cards. Our method is implemented in Pytorch. For a fair comparison, all the compared methods are trained on the training set and evaluated on the test set of the PIDray dataset. The proposed SDANet is based on Cascade Mask-RCNN \cite{DBLP:journals/corr/abs-1906-09756}, where the ResNet-101 network is used as the backbone. According to our statistics, the average resolution of the images in our dataset is approximately $500 \times 500$. Therefore, we resize the image to $500 \times 500$ for compared detectors for a fair comparison. The entire network is trained with a stochastic gradient descent (SGD) algorithm with a momentum of $0.9$ and a weight decay of $0.0001$. The initial learning rate is set as $0.02$ and the batch size is set as $2$. Unless otherwise specified, other parameters involved in the experiment follow the settings of MMdetection.

\subsection{Evaluation Metrics}
According to the evaluation metric of MS COCO\cite{lin2014microsoft}, we evaluate the performance of the compared methods on our PIDray dataset using both the AP and AR metrics. The scores are averaged over multiple Intersection over Union (IoU). Notably, we use $10$ IoU thresholds between $0.50$ and $0.95$. Specifically, the AP score is averaged across all $10$ IoU thresholds and all $12$ categories. In order to better assess a model, we look at various data splits. $\text{AP}_{50}$ and $\text{AP}_{75}$ scores are calculated at IoU $=0.50$ and IoU $=0.75$ respectively. Note that many prohibited items are small (area $<32^2$) in the PIDray dataset, which is evaluated by the $\text{AR}_{S}$ metric. Besides, the $\text{AR}$ score is the maximum recall given a fixed number of detections ({\em e.g.}, $1,10,100$) per image, averaged over $12$ categories and $10$ IoUs. 

\begin{figure*}[t]
\centering
\includegraphics[width=1.0\linewidth]{figures/results_test.pdf}
\caption{Comparison between the proposed SDANet method and Cascade Mask R-CNN \cite{DBLP:journals/corr/abs-1906-09756}. GT indicates Ground-truth, Cascade indicates the results generated by Cascade Mask R-CNN, and Ours indicates the results generated by SDANet.}
\label{figure:visual_results}
\end{figure*}

\subsection{Overall Evaluation}
As presented in Table \ref{table:baseline_results}, we firstly compare our method with a few state-of-the-art object detectors. It can be seen that our SDANet achieves the best performance in terms of all the subsets in the PIDray dataset. For example, compared with the biggest competitor Cascade Mask R-CNN \cite{DBLP:journals/corr/abs-1906-09756}, our method achieves $1.5\%$ and $1.3\%$ AP gain for the two sub-tasks on the hidden test set, which shows the effectiveness of the proposed selective dense attention module. As shown in Figure \ref{figure:visual_results}, our method achieves higher accuracy than Cascade Mask R-CNN \cite{DBLP:journals/corr/abs-1906-09756}. The visual results show that SDANet can effectively detect prohibited items, especially those that have been deliberately hidden.

To verify the effectiveness of the proposed selective dense attention scheme, we compare our method with the previous multi-scale feature fusion strategies including FPN \cite{lin2017feature}, PAFPN \cite{DBLP:conf/cvpr/LiuQQSJ18}, and BiFPN \cite{DBLP:conf/cvpr/TanPL20}. FPN \cite{lin2017feature} provides a top-down pathway to fuse multi-scale features, while PAFPN \cite{DBLP:conf/cvpr/LiuQQSJ18} adds an additional bottom-up pathway on top of FPN. BiFPN \cite{DBLP:conf/cvpr/TanPL20} is weighted bi-directional feature pyramid network, which allows easy and fast multi-scale feature fusion. As presented in Table \ref{table:baseline_results}, our method outperforms existing multi-scale feature fusion strategies. We speculate that this is attributed to two reasons. First, two selective attention modules can aggregate semantic information across multi-layers densely. Second, the dependency refinement module can further capture long-range dependencies among different feature maps. The results indicate that our method can detect deliberately hidden data effectively. 

\subsection{Ablation Study}
Since this work focuses on detecting prohibited items that are hidden deliberately, we conduct the ablation study to analyze the influence of the proposed modules on the hidden test set of the PIDray dataset. 

As presented in Table \ref{table:ablation_study}, we report how the performance of our SDANet is improved when we add the module one by one in the baseline Cascade Mask R-CNN \cite{DBLP:journals/corr/abs-1906-09756}. Firstly, the selective channel-wise attention module improves the baseline method by $0.3\%$ detection AP and $0.4\%$ segmentation AP. Then, the performance continuously improves by $0.6\%$ detection AP and $0.2\%$ segmentation AP when incorporating the selective spatial attention modules. Finally, the dependency refinement module contributes to a $0.6\%$ and $0.7\%$ improvement in terms of detection AP and segmentation AP, respectively. 

We also compare the dependency refinement module with the existing attention mechanisms(e.g. SE and CBAM). Table \ref{table:DR_comparison} shows the results of all models. The results show that DR has obvious advantages in detecting deliberately hidden items.
\begin{table}[t]
\setlength{\belowcaptionskip}{-0cm}
  \centering
  \caption{Comparison of dependency refinement (DR) and other attention mechanisms on the hidden test set.}
  \vspace{4pt}
    \setlength{\tabcolsep}{7.3mm}{
    \begin{tabular}{c|cc}
    \hline
    Method & Det AP & Seg AP \\
    \hline
    ours w/o DR & 48.9  & 36.7 \\
    \hline
    +SE & 49.1  & 36.7 \\
    +CBAM & 47.0  & 35.8 \\
    +DR   & \textbf{49.5} & \textbf{37.4} \\
    \hline
    \end{tabular}}%
  \label{table:DR_comparison}%
\end{table

\begin{table}[t]
  \caption{Evaluation results on the MS COCO and PASCAL VOC detection datasets.}
  \vspace{4pt}
    \centering
    \setlength{\tabcolsep}{5.5mm}{
      \begin{tabular}{c|c|c}
      \hline
      Method & MS COCO  & PASCAL VOC\\
      \hline
      baseline&   42.9    &  81.5\\
      \hline
      SDANet  &   {\bf 43.5}   &  {\bf 82.5} \\
      \hline
      \end{tabular}}%
    \label{tab:general_datasets}%
  \end{table}%
\subsection{Evaluation on General Detection Dataset}
Finally, we also conduct some experiments on general detection datasets to evaluate the effectiveness of SDANet on the natural image. The experiment uses MS COCO\cite{lin2014microsoft} and PASCAL VOC\cite{everingham2010pascal}, which are well-known data sets in the field of natural image detection. The experimental results are shown in Table \ref{tab:general_datasets}. We follow the training and testing pipelines in MMDetection. Compared with the baseline method(Cascade Mask R-CNN), we have achieved 0.6 AP and 1.0 AP gain on MS COCO and PASCAL VOC, respectively. Experimental results demonstrate that our method is not only suitable for the detection of prohibited items, but also effective in general scenarios. 

\section{Conclusion}

In this paper, we construct a challenging dataset(namely PIDray) for prohibited item detection, especially dealing with the cases that the prohibited items are hidden in other objects. PIDray is the largest prohibited items detection dataset so far to our knowledge. Moreover, all images are annotated with bounding boxes and masks of instances. To learn the importance of multi-scale feature maps, we propose the selective dense attention network. The experiment on the PIDray dataset proves the superiority of our method. We hope that the proposed dataset will help the community to establish a unified platform for evaluating the prohibited item detection methods towards real applications. For future work, we plan to extend the current dataset to include more images as well as richer annotations for comprehensive evaluation. 

\section*{Acknowledgement}
We would like to thank Ruyi Ji, Jiaying Li, Xu Wang, and others for their help in data collection and annotation. 
{\small
\bibliographystyle{ieee_fullname}
