\section{Introduction}
Named entity linking (NEL) is a task of linking a mention of an entity in a text to the correct reference entity in the knowledge base (KB) \cite{rao2012entity, yang2015smart, sorokin-gurevych-2018-mixing, kolitsas2018endtoend, Logeswaran2019ZeroShotEL, Wu2020ScalableZE, Li2020Efficient, Sevgili2021Neural}. Here we focus on NEL in a specific setting, with the motivation to probe the difficulty of dealing with namesakes:
\begin{enumerate}[topsep=0pt,itemsep=-1ex,partopsep=1ex,parsep=1ex]
    \item The mention of interest is assumed to be located in the text.
    \item Only the local context surrounding the mention of interest is used for the linking, no other mentions in the text are used.
    \item KB is fixed and built on reliable data.
    \item Both KB and the pool of mentions are mostly composed of namesakes.
    \item A mention may have the corresponding entity in KB, or may not. We call the former mention \textit{familiar}, and the latter \textit{stranger}.
\end{enumerate}
The point 1 means that the named entity recognition task is assumed to be done, leaving us NEL in a narrow sense \cite{rao2012entity, Wu2020ScalableZE, Logeswaran2019ZeroShotEL}. The point 2 makes the problem better defined. Using other mentions of the same entity or related entities can help NEL, but our focus is on imitating the more difficult cases of lone mentions. The point 3 leaves out the question of growing or improving KB by the encountered mentions (familiar or stranger). KB built on reliable data allows to isolate the effect of KB \textit{pollution} by intentionally adding wrong data. The point 4 makes it easier to reveal NEL errors.

Both the points 3 and 4 are satisfied by choosing recent dataset \textbf{Namesakes} \cite{namesakes2021, Vasilyev2021Namesakes}, as a dataset with human labeled ambiguously named entities. 
Another recent dataset  - Ambiguous Entity Retrieval (AmbER) \cite{Chen2021Evaluating} - does include subsets of identically named entities (for the purpose of fact checking, slot filling, and question-answering tasks), but it is automatically generated. Most existing NEL-related datasets do not focus on highly ambiguous names \cite{ratinov-etal-2011-local, hoffart-etal-2011-robust, Ferragina2012FastAA, Ji2017OverviewOT, guo2018}. 

In this paper we focus on presenting our NEL method on KB entities and mentions taken from Namesakes. Our contribution:
\begin{enumerate}[topsep=0pt,itemsep=-1ex,partopsep=1ex,parsep=1ex]
    \item We introduce a simple and practical representation of entity in KB, and explore NEL to such representations on example of highly ambiguous mentions from Namesakes.
    \item We suggest an adjustment of KB based on its entities, and show that it helps in reducing NEL errors.
\end{enumerate}
In Section \ref{sec:NEL} we introduce our entity representation and NEL for KB with such representations. In Section \ref{sec:Evaluation_on_Namesakes} we explain how we use Namesakes dataset in our NEL evaluation experiments. In Section \ref{sec:Experiments} we present the experiments and results.

\section{Named Entity as a Set of Embeddings}
\label{sec:NEL}
\subsection{Knowledge Base Entity}
\label{sec:KB_Entity}
A named entity can be described in very different contexts. The same person can be a scientist and a dissident, the same location can be described by its nature and by its social events and so on. This is the motivation to represent an entity by multiple embeddings - at least if each embedding is created from a mention in specific context. 

Our representation of KB entity $E_a$ is composed of:
\begin{enumerate}[topsep=0pt,itemsep=-1ex,partopsep=1ex,parsep=1ex]
    \item Normalized embeddings $e_i$, their norms $|e_i|$ and assigned thresholds $t_i$, initially set $t_i=-1$. Here $i<=N_E$, restricted by clustering.
    \item Entity threshold $T$.
    \item Entity surface names $s_k$.
    \item References to similar entities $E_b$.
\end{enumerate}
In this and the next subsections we explain the details of this representation, and our NEL procedure that uses it.

A fundamental element used in building an entity is an embedding of a mention of this entity in some context. We tuned a pretrained BERT \cite{Devlin2018BERT} language model ’bert-base-uncased', accessed via the transformers library (\citep{Wolf2020Transformers}), on generic random news, with named entities located in the texts. The only goal of tuning is to enhance LM performance on the mentions of named entities, without changing LM goal or making LM specialized on any particular set of named entities. The tuning and inference are as following:
\begin{enumerate}[topsep=0pt,itemsep=-1ex,partopsep=1ex,parsep=1ex]
    \item At tuning only the named entities - all the located mentions within the input window in the text - serve as the labels for prediction. In input the mentions are being either left unchanged (probability 0.5) or replaced by another random mention from the same text.
    \item At inference the text is kept as it is, and the model is run only once on each input-size chunk of the text. The embeddings are picked up from the first token of the entity surface form - for each named entity that happened to occur in the input-size chunk.
\end{enumerate}
For more details see Appendix \ref{sec:Appendix_TuningLM}.

There can be many mentions available for building a KB entity, even if only reliable verified mentions are used. If the number of the embeddings obtained from the mentions is higher than $N_E$, the embeddings are clustered and only $N_E$ 'central' embeddings (closest to the centers of the clusters) are stored. We use agglomerative clustering (Appendix \ref{sec:Appendix_Settings}). The surface names of all the mentions used for creating an entity are stored in the entity.

\subsection{Linking a Mention to KB}
\label{sec:KB_Linking}

In linking a mention to KB we use only the normalized embedding $e$ of the mention. We define similarity $S$ of the mention to an entity by the scalar products with its embeddings $e_i$:
\begin{equation}
\label{eq:Similarity}
  S = \max_i[(e*e_i) / max(T, t_i)]
\end{equation}
Here all the thresholds $t_i$ are set to $-1$, and are irrelevant unless adjusted as described in Section \ref{sec:KB_Adjustment}; the entity's threshold $T$ is defined further below.

The KB entity with the highest similarity $S$ is the candidate for linking the mention to. We set a \textit{linking threshold} $T_L$: The mention is linked to the candidate-entity only if 
\begin{equation}
\label{eq:Threshold_Global}
  S >= T_L
\end{equation}
Otherwise the mention is left \textit{unlinked} (unassociated with any KB entity). It is natural to assume $T_L=1$, but our results will show that we had to lower it.

The entity's threshold $T$ is defined from the assumption that any embedding of the entity would have to successfully link to the entity (with $T_L=1$):
\begin{equation}
  T = \min_j[\max_{i \neq j}[(e_j*e_i)]]
\end{equation}
This definition makes sense only if there are at least two embeddings in the entity, hence we create a KB entity only if there are at least two mentions available (our observations in Section \ref{sec:Experiments} suggest a more strict requirement).

When linking a mention to KB we find the similarities not to all KB entities, but only to the entities that have a surface name at least somewhat similar to the mention. For this purpose, each entity $E_a$ stores references to its most 'similar' entities $E_b$. For an entity with surface names $s_k$ we select its similar entities using a (non-symmetric) \textit{surface-similarity}, which we define as 
\begin{equation}
  L = \max_m \sum_k \sum_{w\in s_k\cap s_m} l(w)
\end{equation}
where $s_m$ are the surface names of another KB entity, w is any word in $s_k$ also existing in $s_m$, and $l(w)$ is the number of characters in the word $w$. 
For each KB entity $E_a$ we find $N_S=10$ entities $E_b$ having the highest $L>0$ (may be less than 10 because of the requirement $L>0$), - references to these 'surface-similar' entities are stored in the entity $E_a$.

KB stores a map of all words from all the surface names to the entities that have any such word in their surface names:
\begin{equation}
  map: w \rightarrow \{E_a | w \; \text{in} \; s_k \; \text{in} \; E_a\}
\end{equation}
When linking a mention to KB, all KB entities that are mapped from at least one word of the mention's surface name are considered as the candidates for linking. The similarities of the mention's embedding to these candidates are calculated then by Eq.\ref{eq:Similarity}; the mention is linked to the candidate with the strongest similarity if exceeds the threshold Eq.\ref{eq:Threshold_Global}.
Generally, KB entities with similar (by some measure) embeddings can be considered in selecting the candidates, but we focus here on the namesakes.

\subsection{Knowledge Base Adjustment}
\label{sec:KB_Adjustment}
Can we improve KB right after it is created, even without using any knowledge about the texts and mentions on which NEL will be used or evaluated? We suggest adjusting the thresholds $t_i$ in each entity $E_a$ by considering the relation of $E_a$ with its similar entities $E_b$.
We impose a requirement for any embedding $e$ from any entity $E_b$: the embedding $e$ should not be able to link to the entity $E_a$ (because $E_a$ and $E_b$ are different entities).

KB is adjusted to satisfy this requirement, by iterating through all KB entities $E_a$; for each entity $E_a$ iterating through its similar entities $E_b$; for each pair $E_a$ and $E_b$ iterating through the embeddings $e_i$ of $E_a$ and $e_j$ of $E_b$. The threshold $t_i$ for $e_i$ is adjusted as:
\begin{equation}
  t_i \rightarrow c (e_k*e_i)  \;\; \text{if} \;  (e_k*e_i) > \max(T, t_i)
\end{equation}
Here $T$ is the threshold of the entity $E_a$. We use $c = 1.01$, just enough to make the linking impossible.

\section{Evaluation on Namesakes}
\label{sec:Evaluation_on_Namesakes}
Namesakes dataset consists of three parts \cite{namesakes2021}: 
\begin{enumerate}[topsep=0pt,itemsep=-1ex,partopsep=1ex,parsep=1ex]
    \item Entities: human-labeled mentions of named entities from Wikipedia entries.
    \item News: human-labeled mentions of named entities from news.
    \item Backlinks: mentions of entities linked to the entries used in Entities.
\end{enumerate}
According to \cite{Vasilyev2021Namesakes}, the mentions in all the parts are selected with the goal of creating high ambiguity of their surface names.

We are creating KB from Entities, and using News and Backlinks as sources of the mentions for evaluating NEL. These mentions - 'evaluation mentions' - is a mix of familiar and stranger mentions. The stranger mentions appear for two reasons: First, a part of the labeled mentions in News have the same surface names as the labeled mentions in Entities, but represent entities not existing in Entities.
Second, the requirement to have a certain minimal number of mentions for creating a KB entity can leave some mentions in both News and Backlinks without their counterpart KB entities. 

Performance of NEL evaluation can be represented by three indicators:
\begin{enumerate}[topsep=0pt,itemsep=-1ex,partopsep=1ex,parsep=1ex]
    \item Fraction $F_{FW}$ of familiar mentions linked to incorrect KB entity.
    \item Fraction $F_{FN}$ of familiar mentions not linked to KB.
    \item Fraction $F_{SL}$ of stranger mentions linked to KB.
\end{enumerate}
For clarity: if NEL of $N_F$ familiar mentions resulted in linking $N_{FW}$ mentions to wrong KB entities, and in not linking $N_{FN}$ mentions to KB, then $F_{FW} = N_{FW} / N_F$ and $F_{FN} = N_{FN} / N_F$. And if NEL of $N_S$ stranger mentions resulted in $N_{SL}$ linked mentions, then $F_{SL} = N_{SL} / N_S$. 

The lower each of these indicators, the better. The first indicator accounts for the worst kind of error: the mention is familiar, but it is wrongly identified. In a scenario of growing KB this would also lead to degrading KB quality.
The second indicator accounts for the most innocent error: the mention is not identified (despite it could be), but at least no wrong identity is given.
The third indicator accounts for the errors that are as bad as the first kind, with an arguable excuse that the stranger mentions are more difficult for NEL.

\section{Experiments}
\label{sec:Experiments}
\subsection{Linking to Entities of Namesakes}
\label{sec:KB_Entities}
We present here evaluation results in terms of the three indicators introduced in the previous section.
In Figure \ref{fig:NEL_mentions_KB_Entities_News} we show the level of NEL errors for evaluating all the mentions from News.
\begin{figure}[th]
\includegraphics[width=0.48\textwidth]{NEL_mentions_KB_Entities_News.png}
\caption{Dependency of NEL errors on number of mentions. KB is made of Namesakes Entities; evaluation mentions are from Namesakes News. Max number of embeddings per entity $N_E=4$. Threshold $T_L=0.825$.}
\label{fig:NEL_mentions_KB_Entities_News}
\end{figure}
We observe that the minimal number of mentions allowed to create a KB entity plays an important role in reducing the amount of errors, even though the entity mentions were clustered into 4 embeddings.

\begin{figure}[th]
\includegraphics[width=0.48\textwidth]{NEL_mentions_KB_Entities_Backlinks.png}
\caption{Evaluation is as in Figure \ref{fig:NEL_mentions_KB_Entities_Backlinks}, except that the evaluation mentions are from Backlinks.}
\label{fig:NEL_mentions_KB_Entities_Backlinks}
\end{figure}
In Figure \ref{fig:NEL_mentions_KB_Entities_Backlinks} we show again the dependency of NEL errors on the number of mentions per KB entity, but now we evaluate linking of Backlinks mentions to KB (this is the only difference between the settings for the figures \ref{fig:NEL_mentions_KB_Entities_Backlinks} and \ref{fig:NEL_mentions_KB_Entities_News}). 
We observe a comparable level of errors in linking a familiar mention to wrong KB entity and in wrongly linking a stranger mention to KB, but there is a much higher fraction of unlinked familiar mentions. We speculate that the reason is in the less context usually given for a named entity mention in a Wikipedia backlink, as opposed to a mention in the news.

\begin{figure}[th]
\includegraphics[width=0.48\textwidth]{NEL_EmbLowRange_News.png}
\caption{NEL errors for linking mentions from News to KB. Min number of mentions allowed to create KB entity: 10. Threshold $T_L=0.825$.}
\label{fig:NEL_EmbLowRange_News}
\end{figure}
\begin{figure}[th]
\includegraphics[width=0.48\textwidth]{NEL_EmbLowRange_Backlinks.png}
\caption{Evaluation is as in Figure \ref{fig:NEL_EmbLowRange_News}, except that the evaluation mentions are from Backlinks.}
\label{fig:NEL_EmbLowRange_Backlinks}
\end{figure}
In Figures \ref{fig:NEL_EmbLowRange_News} and \ref{fig:NEL_EmbLowRange_Backlinks} we show that the limit on the number of stored embeddings can be as low as 4 (at least judging by evaluation on Namesakes). We suggest that it may be helpful to store more embeddings, depending on the type and cleanness of the data involved in creating KB. We return to this issue in Section \ref{sec:Full_EntityPages}.

\begin{figure}[th]
\includegraphics[width=0.48\textwidth]{NEL_threshold_KB_Entities_News.png}
\caption{Dependency of NEL errors on the linking threshold $T_L$. Evaluation mentions are from News. KB entities are created with min number of mentions 10, and with max number of embeddings $N_E=4$.}
\label{fig:NEL_threshold_KB_Entities_News}
\end{figure}

\begin{figure}[th]
\includegraphics[width=0.48\textwidth]{NEL_threshold_KB_Entities_Backlinks.png}
\caption{Evaluation is as in Figure \ref{fig:NEL_threshold_KB_Entities_News}, except that the evaluation mentions are from Backlinks.}
\label{fig:NEL_threshold_KB_Entities_Backlinks}
\end{figure}

Increase of the linking threshold $T_L$, as expected, suppresses wrong linking, but increases the fraction of unlinked mentions. We show an example of such dependency in Figure \ref{fig:NEL_threshold_KB_Entities_News} for NEL applied to the mentions from News, and in Figure \ref{fig:NEL_threshold_KB_Entities_Backlinks} for NEL applied to the mentions from Backlinks. The choice of $T_L$ should be guided by a required trade-off between the errors $F_{SL}$ vs $F_{FW}$ and $F_{FN}$.  


\subsection{Linking to Polluted KB}
\label{sec:Pollution_Adjustment}

We consider here the effect of lowering KB quality on NEL. When creating KB entities, we add erroneous mentions, imitating the real life situation of not fully reliable sources. Such \textit{polluted} KB should increase NEL errors. We also expect that KB adjustment described in Section \ref{sec:KB_Adjustment} can alleviate the effect of the pollution. 

We use for pollution the "Other"-tagged mentions from Namesakes Entities \cite{Vasilyev2021Namesakes}: these mentions have the surface names of the entity (Wikipedia entry) but represent some different entity. 
The pollution can be characterized by the fraction of the polluted KB entities, and by the average fraction of the wrong mentions used in creating a polluted KB entity. We set both these pollution levels to 0.2, meaning that we pollute each 5th entity - creating it with 20\% added wrong mentions.

In Figure \ref{fig:NEL_PollAdg_KB_Entities_News} we show how much KB pollution affects NEL, and how much our KB adjustment alleviates this effect.
\begin{figure}[th]
\includegraphics[width=0.48\textwidth]{NEL_PollAdg_KB_Entities_News.png}
\caption{NEL errors for linking mentions from News to KB original (solid line), polluted (dotted) and adjusted after pollution (dashed). The evaluation on KB original is as in Figure \ref{fig:NEL_mentions_KB_Entities_News}.}
\label{fig:NEL_PollAdg_KB_Entities_News}
\end{figure}
The pollution strongly increases the fraction of familiar mentions linked to wrong KB entities. As seen in Figure \ref{fig:NEL_PollAdg_KB_Entities_News}, this happens regardless of the minimal number of mentions used in creating an entity. Remarkably, KB adjustment reverses this effect and restores the original level of such NEL errors (falling to zero at higher number of mentions). Pollution affects also NEL of stranger mentions, but the errors increase only at low number of mentions, and KB adjustment does not help in this case.

\begin{figure}[th]
\includegraphics[width=0.48\textwidth]{NEL_PollAdg_KB_Entities_Backlinks.png}
\caption{NEL errors for linking mentions from Backlinks to KB original (solid line), polluted (dotted) and adjusted after pollution (dashed). The evaluation on KB original is as in Figure \ref{fig:NEL_mentions_KB_Entities_Backlinks}.}
\label{fig:NEL_PollAdg_KB_Entities_Backlinks}
\end{figure}
Figure \ref{fig:NEL_PollAdg_KB_Entities_Backlinks} shows the evaluation for linking Backlinks mentions to the clean, polluted and polluted-and-adjusted KB.
Here help of KB adjustment is not essential, maybe because, again, the mention's context in a Wikipedia backlink is usually not as helpful as in the news.


\subsection{Number of Embeddings in Very Polluted KB}
\label{sec:Full_EntityPages}

The main reason for having a limited number $N_E$ of embeddings representing KB entity is saving the space in KB. But without such limitation, would it make sense to keep more embeddings for improving NEL? We can attempt to answer this question if we have many mentions available for creating an entity.

In Namesakes Entities the number of labeled mentions per entity (of Wikipedia entry) is limited, and even the text suggested to the labelers was cut \cite{Vasilyev2021Namesakes}. In order to increase the number of available mentions, for each entry in Entities of Namesakes, we will use here the whole Wikipedia text, and for building an entity we will use all the mentions in the text that match (by the surface form) at least one of the labeled mentions. In some entries there are hundreds of such mentions. Creating entities from so loose mentions inevitably makes KB very 'polluted', and we would expect NEL making many errors, - but the main point here is to observe the dependency of NEL errors on the number of embeddings per entity.

In Figure \ref{fig:NEL_FullAdg_KB_Entities_News} we show such dependency for linking mentions from News.
\begin{figure}[th]
\includegraphics[width=0.48\textwidth]{NEL_FullAdg_KB_Entities_News.png}
\caption{Dependency of NEL errors on the maximal number of embeddings $N_E$ per entity. The evaluation is done by linking the mentions from News on KB polluted by using \textbf{all} unlabeled mentions from full text Wikipedia entries. The dashed line is for adjusted KB. Min number of mentions is 10; threshold $T_L=0.825$.} 
\label{fig:NEL_FullAdg_KB_Entities_News}
\end{figure}
Interestingly, a higher number of embeddings is not only useless for improving NEL (at least in this case of extremely polluted KB), but on opposite it makes NEL worse. It could be expected that the adjustment of KB (dashed lines) can change this pattern, by suppressing the 'bad' embeddings in KB entities. But the adjustment has only small effect for the evaluation on News. 

\begin{figure}[th]
\includegraphics[width=0.48\textwidth]{NEL_FullAdg_KB_Entities_Backlinks.png}
\caption{Evaluation is as in Figure \ref{fig:NEL_FullAdg_KB_Entities_News}, except that the evaluation mentions are from Backlinks.}
\label{fig:NEL_FullAdg_KB_Entities_Backlinks}
\end{figure}
Evaluation on Backlinks is shown in Figure \ref{fig:NEL_FullAdg_KB_Entities_Backlinks}.
Here increasing the number of embeddings $N_E$ helps a only a little and at low values. 
KB adjustment does decrease the level of NEL errors, but still does not make higher numbers of embeddings useful.
The observation that a limited number of embeddings is enough to deliver the same performance is important for our method, and holds for the mentions both from News and Backlinks.

\begin{figure}[th]
\includegraphics[width=0.48\textwidth]{NEL_FullAdg_KB_EntitiesLR_News.png}
\caption{Evaluation is as in Figure \ref{fig:NEL_FullAdg_KB_Entities_News}, except that max number of embeddings is at lower range.}
\label{fig:NEL_FullAdg_KB_EntitiesLR_News}
\end{figure}




For the low range of the number $N_E$ of the stored embeddings we did not need to have a large number of mentions, and we have already made our observations in Figures \ref{fig:NEL_EmbLowRange_News} and \ref{fig:NEL_EmbLowRange_Backlinks}. Out of curiosity we show the same range in Figures \ref{fig:NEL_FullAdg_KB_EntitiesLR_News} and \ref{fig:NEL_FullAdg_KB_EntitiesLR_Backlinks} for the 'very polluted' KB considered in this subsection. The figures confirm that too low number of embeddings leads to more NEL errors.  

\begin{figure}[th]
\includegraphics[width=0.48\textwidth]{NEL_FullAdg_KB_EntitiesLR_Backlinks.png}
\caption{Evaluation is as in Figure \ref{fig:NEL_FullAdg_KB_Entities_Backlinks}, except that max number of embeddings is at lower range.}
\label{fig:NEL_FullAdg_KB_EntitiesLR_Backlinks}
\end{figure}

In this subsection we considered extremely polluted KB, which has lead, together with the challenging nature of Namesakes, to a very high level of errors, especially for stranger mentions. The reason for this is that combining all right and wrong (but named similarly) mentions from a full Wikipedia page makes the embeddings within the created KB entity further from each other, and hence decreases the entity's threshold $T$. NEL of a familiar mention may still have a better chance to be linked to the correct KB entity, which may still provide a higher similarity. But NEL of a stranger mention would simply link it wrongly to KB, because all the thresholds are lower.  
Despite higher errors, having more available mentions per entity allowed us to cover a wide range of the number of embeddings per entity. 

Altogether, through all our experiments we observed a big difference between NEL from Backlinks and NEL from News. We speculate that the lack of good context in many Backlinks mentions is the reason: a link in Wikipedia is often made for a named entity mentioned fleetingly, in some enumeration-like auxiliary sentence of no importance and almost no relation to the main context. 

\section{Conclusion}
We introduced a simple and practical representation of named entities in KB. We reviewed NEL performance for linking text mentions of named entities to such KB, using Namesakes dataset \cite{Vasilyev2021Namesakes} as the source both for the mentions and for building the KB. As a dataset of ambiguous named entities, Namesakes makes NEL difficult and helps to reveal the errors.

We observed that a requirement of a minimal number of mentions for creating KB entity is important for NEL performance: a requirement of minimum 10 mentions gives much better results than more relaxed settings (Figures \ref{fig:NEL_mentions_KB_Entities_News}, \ref{fig:NEL_mentions_KB_Entities_Backlinks}).
We described a KB adjustment based on only KB data; we have shown that the adjustment helps to reduce NEL errors when KB entities are polluted by admix of wrong mentions (Figures \ref{fig:NEL_PollAdg_KB_Entities_News}, \ref{fig:NEL_PollAdg_KB_Entities_Backlinks}).
Through the paper we considered NEL with static KB. In future we are looking forward to explore more general scenarios.

\section*{Acknowledgments}
We thank Randy Sawaya for review of the paper and valuable feedback.

