\section{Background}

\subsection{Children's Perception of Privacy}

Privacy has been described as a ``concept in disarray''\cite{solove2008understanding} reflecting the complex and often overlapping conceptualisations. O'Hara proposed ``seven veils'' of privacy, spanning the conceptual, factual, phenomenological, preferential, societal, legal, and moral~\cite{ohara2016seven}. Practical research in privacy has typically spanned multiple such veils; Livingstone suggests that privacy is related to the keeping of information out of the public domain or the effective determination of \emph{what} personal information is made available to \emph{whom}~\cite{livingstone2006children}. Nissenbaum's theory of Contextual Integrity (CI) frames privacy as  ``the right to appropriate flow of personal information"~\cite{nissenbaum2004privacy} where what is ``appropriate'' is based on particular contexts and relationships.  CI has been found to be a useful practical framework to interpret people's perceptions of privacy~\cite{barkhuus2012mismeasurement,kummer2016private}.

Managing privacy is becoming increasingly challenging, and children are particularly susceptible in part due to them having little sense of the risks posed by the accumulation of personal data over time~\cite{Pangrazio:2017:MDM:3097286.3097338}. \comm{The combination of such a lack of understanding, with incentives put in place by apps and platforms to get people to share their data, have yielded the so-called `privacy paradox', a behaviour studied in teenagers~\cite{brandimarte2013misplaced} and adults~\cite{barth2017privacy} alike, in which individuals act in ways contrary to their stated privacy preferences and concerns~\cite{jones2009us,Jia:2015:RLP:2675133.2675287,Pangrazio:2017:MDM:3097286.3097338}.} Recent studies showed that although teenagers from 14 to 18 were typically concerned about being personally identified by their personal data, they failed to perceive the potential threat of re-identification via the particular fragments they shared, e.g., images or geo-location~\cite{Pangrazio:2017:MDM:3097286.3097338}. 

Whilst there has been extensive research on \comm{teenagers'} perceptions of privacy, relatively little has been done for children aged below 11. They have often been perceived as being too young to understand privacy or exercise digital independence online~\cite{livingstone2018parentsc}.  However, developmental research has shown that children can develop a ``theory of mind" by the age of 4 \cite{colwell2016secret}, which enables children to recognise the differences between concepts in their minds and those in others, and hence to grasp complicated concepts like ``secrecy''~\cite{colwell2016secret} or ``deception''~\cite{chandler1989small}. Contrary to common expectations, children value their privacy because this enables them to enjoy their experience online \cite{Silva:2017:PCT:3091478.3091479, kumar2018cscw}, such as socialising with their friends and families or experimenting with new games. Existing literature provided us with us with useful insights into children's perception of risks \cite{zhang2016nosy, kumar2018cscw}, however, they they do not tell us how children make these judgements - or whether children can in fact identify a threat and its origination.


\subsection{Managing Children's Privacy Online}
Tools developed for managing children's privacy on mobile platforms \comm{usually comprise} features that monitor and restrict children's online activities~\cite{wisniewski2017parental,livingstone2018parentsa,ofcom2017}. Children often find these tools overly restrictive and invasive of their personal privacy \cite{Ghosh:2018:SVS:3173574.3173698,wisniewski2017parental}. Co-design studies with children under 12 show that they prefer technologies that facilitate their risk coping skill development, promote communication and interaction with their parents, and emphasise restrictions around monitoring~\cite{mcnally2018co}. However, such tools are still scarce. 

Several recent studies have recognised the importance of supporting children's learning through approaches like interactive storytelling~\cite{Zhang-Kennedy:2016:TIE:2930674.2935984}, game playing~\cite{maqsood2018day} or co-learning with parents~\cite{hashish2014involving}. They have shown the effectiveness of increasing children's awareness of related online safety issues, such as online personal identity or content appropriateness. This demonstrates that a knowledge scaffolding approach to  a child's means of dealing with privacy risk coping could provide a useful addition to safeguarding their cybersafety.


\subsection{Developmental Stages of Privacy Concepts}

Children's ability to recognise privacy risks may be intrinsically limited by their developmental stage. Children aged 3-5 start to build up friendships; however, at this stage, family interactions are dominant, and many of their online activities are still parent-guided \cite{digital2017}, peer pressure is less of an influence~\cite{ann2000developmental}. Children from 6 to 9, \comm{however}, start to learn about the complexity of relationships and feel the need to fit in to peer social groups~\cite{ann2000developmental}. They are also more involved in online activities and enjoy playing games with their friends~\cite{digital2017}. This social interaction with peers makes them more aware of interpersonal privacy risks, but less of other privacy risks \cite{digital2017}. So while they often care deeply about their personal information being shared with their peers, parents, and others online, they remain unaware of other actors, including platforms, app designers, malicious actors, and others operating in digital ecosystems~\cite{livingstone2018childrenc,Pangrazio:2017:MDM:3097286.3097338}. 

Vygotsky's Zone of Proximal Development (ZPD) is a theory that relates the difference between what learners can do independently to what can be achieved by through guidance by a skilled partner~\cite{chaiklin2003zone}. It has been applied to assess the effectiveness of teaching and learning~\cite{miller2010theories}, to identify key barriers in learning by understanding knowledge gaps~\cite{daniels2005introduction}, and in the design of better intelligent tutoring systems (ITS)~\cite{quintana2004scaffolding}. \comm{Inspired by ZPD, we focus on identifying children's misconceptions of privacy risks and highlighting areas of knowledge that could be developed through future active scaffolding. } 
\section{Key Findings}

Through careful data analysis of 20-hour interview recordings, we have achieved some positive findings with respect to children's level of awareness. However, at the same time, we have also identified critical gaps in children's online privacy knowledge. Children sometimes may be able to capture certain risks with the right language, however, they may not have fully comprehended the risks. These are critical insights that should draw further attention to families and schools when applying risk mediation to children. The terms used by the participant children and their abilities to apply these terms to different types of risk context has provided valuable inputs to our future design of technologies to facilitate children's risk coping abilities. 

\subsection{Children care about online privacy risks}
Our results reinforced that, like teenagers and younger children, our participants valued the positive experiences of being online and keeping their personal space online~\cite{lwin2008protecting}. They enjoyed going online so that they could keep in touch with friends, learn new things or just have fun.

At the same time, children felt `annoyed', `surprised' or `angry' when they felt coerced or not in control. As a result, children cared about, and were sensitive to, who might access their sensitive information (e.g. real names, age, location etc), and applied a range of techniques to safeguard this space, such as by verifying identities through face-to-face interactions or avoiding using real names as usernames. 

\subsection{Children still need help with certain privacy risks} 

Children demonstrated a strong consciousness of their online identity and the importance of avoiding sharing their real identity or over-sharing their personal information. In these cases, children applied various effective strategies to protect their sensitive personal information, such as `making up a new name', and asking their friend to verify the online invitation in person. However, in contrast to explicit privacy risks, children struggled to associated ``online promotions'' with losing control of personal information. 

Several children correctly discussed their interpretation of how YouTubers may try to `persuade' them to watch their videos in order to gain `money' or `more subscribers'. However, none of these children expressed resistance to these video promotions. They treated it as if this is how the Internet works. 

Only a few children (3/12 focus groups) recognised how new videos might be related to personalised recommendations. However, they struggled to understand who was performing these recommendations, and as a result, they were less sure about the consequences of their privacy. As a result, the children usually applied the play-and-see strategy to assess the content or apps they came across, without any more complex reasoning.




\subsection{Children's ability to describe risks depends on their actual understanding of risks} 

Our study is partially inspired by Vygotsky's Zone of Proximal Development (ZPD), and therefore, we really focused on understanding children's current knowledge by observing the language they used to describe risks. We observed that some terms were repeatedly used by children across different situations --- children were able to describe risks accurately when they could recognise the actual risks. However, when they had only a vague understanding of the risks, they struggled to describe things consistently or to provide a good explanation of what they meant.

When children only made a partial sense of certain types of risks, their use of terms can be inconsistent. For example, although some children were able to describe the scenario of new videos being presented to Bertie using words like `people trying to make money' or `them trying to make you watch more', they struggled to explain who these `people' are and how information might be transmitted to these `[YouTube] channel people' in this context. 

Another example is the term `hacking', which has been used by children across different focus groups, but in fact with very different meanings. For example, when trying to explain why a new video was shown following a previous video, children used `hacking' to mean `someone stole my data', `take your account', or `steal from house'. `Hacking' has been used by the same group of children to make sense of other scenarios, including in-app pop-ups or data tracking.


\subsection{Children struggle to comprehend the implications of risks}

Children could miss risks due to a lack of knowledge, or due to their past experiences, which did not lead to any direct consequences related to the risks. For those children who interacted with certain technologies and experienced no implications before, they would tend to be more (over-) confident with technologies.

Children from 7 focus groups treated online video promotions as part of an `autoplay' function of the platform, without questioning how the new content might be presented to them. 12/29 children demonstrated trust in the content provided by their familiar YouTubers. As a result, children reported having been exposed to unexpected content and online baiting. These same children reported that they often saw upsetting content online (e.g. `sometimes in autoplay, it comes up with these really freaky ones like pictures of dead people').

When children not fully comprehending the risks, we also observed that a child's or their peers' personal experiences had a strong influence upon their decision making, even though they didn't always understand what may pose threats to them.

\newpage

\frame{
\textbf{Take-Home Messages For Parents \& Caretakers}
\begin{enumerate}
    \item Continue to talk to children about being careful online, because they are indeed facing challenges in their use of digital devices every day and they really care about protecting their personal space online!
    
    \item Pay attention to how children describe risks or things that made them uncomfortable online. This is a good start to get to know what they actually mean, and it's always good to follow this up with a question like ``what do you mean?''.
    
    \item Do not be afraid of talking about digital technologies with children. They would enjoy learning with you and sharing their knowledge with you.
    
    \item If you are indeed struggling, please take a look at the resources we recommend below. It's never too late to get to know what your child knows about online privacy and how you may help them or how you may help each other!
\end{enumerate}
}

\section{Resources}
\begin{itemize}
    \item \href{commonsensemedia.org}{commonsensemedia.org} would be a good place for finding out additional information about which mobile app would be good for your child, such as Minecraft, Roblox, etc. 
    
    \item \href{https://www.net-aware.org.uk}{NSPCC Net Aware} provides good evidence-based advice on apps that are most popular among children.
    
    \item \href{childnet.com}{childnet.com} has good resources for talking about online privacy with younger children, such as the story of \href{https://www.childnet.com/resources/smartie-the-penguin}{Smartie the Penguin} or the story book of  \href{https://www.childnet.com/resources/digiduck-stories}{Digiduck}.
    
    \item Finally, our KOALA Hero is always here to help, either through the \href{https://sites.google.com/view/koala-project-ox/toolkits/cards}{story cards} or regularly updated \href{https://sites.google.com/view/koala-project-ox/reports-and-publications}{public reports}. 
\end{itemize}


\section{Introduction}
Today, children are spending more time online than with other media sources, such as televisions or offline video games~\cite{ofcom2017,livingstone2017children}. Among the many kinds of devices now connected to the Internet, mobile devices (such as tablet computers or smartphones) have become the primary means by which children go online~\cite{livingstone2017children}. In the UK, 35\% of children aged five to seven and 52\% of children aged eight to eleven have been reported to have their own tablets  (see Figure 1)~\cite{ofcom2017}; while in the US, ownership of tablets by children in this age group grew five-fold between 2011 and 2013~\cite{commonsense2013}. Children under five are also using smartphones and tablets more often, as the category of apps designed for younger kids continues to expand rapidly~\cite{ofcom2017, lifewire}. 

\begin{figure}[h!]
\centering
\begin{bchart}[step=20,max=100,unit=\%]
        \bcbar[label=3-4]{21}
            \smallskip
        \bcbar[label=5-7]{35}
            \smallskip
        \bcbar[label=8-11]{52}
            \smallskip
        \bcbar[label=5-11]{49}
            \smallskip
        \bcbar[label=12-15]{55}
        
\end{bchart}
\caption{Percentage of Children Owning Their Own Tablets In Proportion to the Total Number of Tablet Accessible to the Household by age~\cite{ofcom2017}}
\end{figure}\label{fig:devices}

However, for children aged between 6 and 10, their awareness of online privacy has been seldom studied in existing literature. It's probably due to the expectation that children of this age cannot comprehend the concept of privacy, and therefore their online safety and privacy should be safeguarded by the trusted grown-ups. However, our previous reports~\cite{DBLP:journals/corr/abs-1809-10841,DBLP:journals/corr/abs-1809-10944} have shown that although parents largely take a protective approach to restrict and monitor what their children can or cannot access on their children's devices, parents do not always fully comprehend all risks. Furthermore, a more recent study led by the LSE has shown that parents struggle to seek for help to mediate their children's use of digital devices, and they largely rely on themselves to figure things out~\cite{livingstone2018parentsb}. 

Therefore, the goal of the KOALA project is to, through a series of studies directly interacting with children and their families, to achieve a deeper understanding of the following questions:
\begin{itemize}
    \item How well can children aged 6-10 understand online privacy risks related to their use of tablet computers?
    \item What are the barriers for their parents to mediate children's use of technologies and recognition of risks?
\end{itemize}

Between June and August 2018 we conducted 12 focus group interviews with 29 children, aged 6-10, from UK schools. We found that children in our study had a good understanding of risks related to \textit{inappropriate content, the approach of strangers}, and \textit{oversharing of personal information online}. However, they struggled to fully understand and describe risks related to \textit{online game/video promotions and personal data tracking}. Moreover, children's risk coping strategies depended on their understanding of the risks and their previous experiences: effective risk strategies were applied only if children recognised certain risks or when they \textit{felt} something untoward. These findings demonstrate the importance of learning about potential risks through a multitude of channels, such as school, parents, friends, siblings, and personal experiences~\cite{zhang2017cyberheroes,hashish2014involving}.
\section{Acknowledgement} 

We thank all the schools, families and children who have contributed their time and knowledge to our study. Without their support, this study would not have been successful!

The report is based on a full academic publication \href{https://arxiv.org/abs/1901.10245}{`I make up a silly name': Understanding Children's Perception of Privacy Risks Online}, as part of the project of KOALA (https://sites.google.com/view/koala-project-ox/): Kids Online Anonymity \& Lifelong Autonomy, funded by EPSRC Impact Acceleration Account Award, under the grant number of EP/R511742/1.

\section{Contact} 
\begin{itemize}
    \item Jun Zhao: jun.zhao@cs.ox.ac.uk
    \item Nigel Shadbolt: nigel.shadbolt@cs.ox.ac.uk
\end{itemize}


\section{Methodology}
We chose the focus group method to elicit children's responses to a collection of hypothetical scenarios that reflect different types of explicit and implicit threats to children's online personal data privacy. 

Each focus group study contained four parts, including a warm-up and introduction session, a sharing of favourite apps, a walk-through of three hypothetical risk scenarios, and finally an open-ended session about issues not so far discussed. 

Children were encouraged to share their personal experience related to the scenarios, by responding to questions like \textit{whether this has happened to them} and \textit{what they did}. This enabled us to listen to children's descriptions of their experiences not covered in the scenarios. 

The hypothetical scenarios used in the study have been created into story cards that can be reused and shared in the classrooms or at home. They can be found in the Appendix towards the end of the report, accompanied by descriptions about the privacy risks related to each card. Parents and teachers can also download our story cards from \href{https://sites.google.com/view/koala-project-ox/toolkits}{our project web site} and use these cards to inspire conversations with children about online privacy issues.




