\section{Introduction}
The notion of conditional probability is one of the main idea in probability theory.
In order to define conditional probabilities rigorously, Kolmogorov introduced measure theory into probability theory \cite{Kol33}.
The notion of randomness is  another important subject in probability and statistics. 

In statistics, the set of random points is defined as the complement of given statistical tests.
In practice, data is finite and a statistical test is a set of small probability, say 3\%, with respect to a null hypothesis.
In order to discuss whether a point is random or not rigorously, we study the randomness of  sequences (infinite data) and 
null sets as statistical tests.
The random set depends on the class of statistical tests.
Kolmogorov brought an idea of recursion theory into statistics and 
 proposed to define the random set as the complement of the effective null sets \cite{{Kol63},{Kol65},{Kol68},{martin-lof66}}.
 An effective null set is defined as the limit of recursively enumerable sets  whose probability goes to zero effectively.
Many standard statistical tests are shown to be effective null sets.
The random set defined in this manner is also called Martin-L\"of random (ML-random) set \cite{martin-lof66}.
 One of the advantage of this definition is that the universal (nonparametric) character of the class of tests, which leads to universal (nonparametric)  theory   of statistics, for example
 see \cite{LV2008}.
Another advantage of the definition is that the random set is characterized with entropy, i.e., Kolmogorov complexity.
It is impossible to define random finite strings in a clear manner, however it is possible to argue asymptotic theory of random sequences with the help of  complexity theory. 

Kolmogorov showed that conditional probabilities exist with probability one, however it was not known whether conditional probability exists for each given parameter.
Therefore only a few research have been made about conditional randomness with very restricted conditions \cite{vovkandvyugin93}.
For survey on conditional randomness, see \cite{BST2016}.



In order to study conditional randomness, we start with probability on product space, i.e., sample and parameter space.
Then conditional probabilities are defined as the limit of the conditional probability when the probability of the conditional event goes to zero, i.e., the limit of martingale convergence sequences, see \cite{Kol33}.
We study ML-random sets with respect to the computable probability \(P\) on product space, which is denoted by \({\cal R}^P\),  
and show that the conditional probabilities exist for random parameters with respect to marginal distribution \cite{{takahashiISIT2006},{takahashiIandC}}.
For each random parameter \(y^\infty\), we consider
(i) \(\bigcap_{y\to y^\infty}{\cal R}^{P(\cdot\vert y)}\), the limit of ML-random set with respect to \(P(\cdot\vert y)\) as \(y\to y^\infty\),
(ii)  \({\cal R}^P_{y^\infty}\), the section of global ML-random set  \({\cal R}^P\) at  \(y^\infty\),
(iii) \({\cal R}^{P(\cdot\vert y^\infty)}\), ML-random set with respect to \(P(\cdot\vert y^\infty)\), and
(iv) \({\cal R}^{P(\cdot\vert y^\infty),y^\infty}\), ML-random set with respect to \(P(\cdot\vert y^\infty)\) with oracle \(y^\infty\).
We call four sets above {\it variants of conditional ML-random set}.
The relations between these sets are shown in  Fig~\ref{figA} below.




From Fig.~\ref{figA},  \({\cal R}^{P(\cdot\vert y^\infty),y^\infty}\) is the smallest one among the variants  of conditional ML-random set.
One may think it is suitable for conditional ML-random set,
because if other variants of  conditional ML-random set satisfies some probability laws with respect to conditional probabilities for random parameters
then \({\cal R}^{P(\cdot\vert y^\infty),y^\infty}\) satisfies the same laws as well. 
However the only \({\cal R}^P_{y^\infty}\) satisfies the following three  statistical requirements  on global \(P\).
At moment we call variants of conditional ML-random set that satisfy the following statistical properties 1-3 {\it Bayesian ML-random set}.

1.  Property of pair of random points. By property of pair of random points (generalized Lambalgen theorem), we require that if \(x^\infty\) is Bayesian ML-random with respect to conditional probability \(P(\cdot\vert y^\infty)\)
at random parameter \(y^\infty\) then the pair \((x^\infty, y^\infty)\) is ML-random with respect to \(P\).
By definition  \({\cal R}^P_{y^\infty}\) and hence \({\cal R}^{P(\cdot\vert y^\infty),y^\infty}\) satisfies the property of pair of random points (Theorem~\ref{col-ifpart}).
Since \({\cal R}^P_{y^\infty}\subseteq \bigcap_{y\to y^\infty}{\cal R}^{P(\cdot\vert y)}\) and  they are not equal in general (Remark~\ref{remarkA} below),
we see that \(\bigcap_{y\to y^\infty}{\cal R}^{P(\cdot\vert y)}\) does not satisfy this property in general. 


2.  Bayesian mixture property. For two  probabilities \(Q_1, Q_2\), \(P=p_1Q_1+p_2Q_2\) is called Bayesian mixture distribution where \(\{p_1,p_2\}\) is prior with \(0\leq p_1,p_2\leq 1, p_1+p_2=1\), 
and if \(Q_1\) and \(Q_2\) are computable then we have \({\cal R}^P={\cal R}^{Q_1}\cup{\cal R}^{Q_2}\).
Bayesian mixture property requires  the continuous version of this property, i.e., for \(P_X=\int P(\cdot\vert y^\infty)dP_Y\)
we require that \({\cal R}^{P_X}\) equals to the union of Bayesian  ML-random set with respect to conditional probabilities for random parameters.
From (\ref{bayes-mixture}) Theorem~\ref{bayes-mix} below, we see that the conditional ML-randomness \({\cal R}^P_{y^\infty}\) satisfies Bayesian mixture property but from Theorem~\ref{counter-example}, there is an example that 
other variants of conditional ML-randomness \({\cal R}^{P(\cdot\vert y^\infty)}\) and \({\cal R}^{P(\cdot\vert y^\infty),y^\infty}\) do not satisfy this property. 


3. Bayesian consistency property. Theorem~\ref{th-consis-pos} shows equivalent conditions for consistency of posterior distribution. 
Informally speaking, this theorem says that the equivalent condition of consistency of posterior distribution is mutual singularity of conditional probabilities on disjoint cylinder sets. 
By consistency property, we require that under one of the statements in Theorem~\ref{th-consis-pos},
if \(x^\infty\) is ML-random with respect to marginal distribution on sample space then there is a unique random parameter \(y^\infty\)  
and \(x^\infty\) is Bayesian  ML-random with respect to conditional probability \(P(\cdot\vert y^\infty)\).
By definition, the conditional ML-randomness \({\cal R}^P_{y^\infty}\) satisfies this property 
but from Theorem~\ref{counter-example}, there is an example that other variants of  conditional ML-randomness \({\cal R}^{P(\cdot\vert y^\infty)}\) and \({\cal R}^{P(\cdot\vert y^\infty),y^\infty}\) do not 
satisfy this  property. 



Among the variants of conditional ML-randomness,  only  \({\cal R}^P_{y^\infty}\) satisfies the three properties above, i.e., \({\cal R}^P_{y^\infty}\) is Bayesian ML-random set. 
In order to understand this, we need to understand the difference between Bayesian statistical laws and probability laws with respect to conditional probabilities. 
Roughly speaking, in statistics we consider  a class of  models simultaneously and infer the statistically reasonable model  from samples. 
In other words, statistical laws are the probability laws of multiple models but not  those of a single model. 
Bayesian statistics is the law of global probability that consists of conditional probabilities and prior.  
This is why the section of global ML-random set satisfies laws of Bayesian statistics. 
Note that other variations of ML-randomness for conditional probabilities, i.e.,
ML-random set defined from conditional probabilities and ML-random set defined from conditional probabilities with oracle of the random parameters,
are the properties of randomness for each  local conditional probabilities.

As in \cite{{takahashiISIT2006},{takahashiIandC}} I propose that 
 \({\cal R}^P_{y^\infty}\)  {\it is the Bayesian definition of random sequences with respect to conditional probabilities for} \(y^\infty\in{\cal R}^{P_Y}\).



In the following sections, in order to discuss the relation of these sets, 
 we review a generalization of  van Lambalgen theorem \cite{{takahashiISIT2006},{takahashiRIMS2008},{takahashiIandC},{takahashiIandC2}} and show a  generalized form of Kjos Hanssen's theorem.

Lambalgen theorem \cite{lambalgen87} states that a pair of random sequences \((x^\infty, y^\infty)\) is ML-random with respect to the product of uniform probability if and only if
\(y^\infty\) is ML-random with respect to uniform probability and \(x^\infty\) is ML-random with respect to uniform probability with oracle \(y^\infty\).
In other words Lambalgen theorem states  that the section of random set at the parameter and the random set with oracle of the parameter are equal if the parameter is random. 
A generalized form of Lambalgen theorem extends the theorem to computable joint probabilities and states the relation between the section of random set at the random parameter and
the random set with respect to conditional probability with oracle of the random parameter.

Kjos Hanssen theorem \cite{kjoshanssen} shows that the random set with respect to Bernoulli model \(P_\theta\) is equivalent to the random set with respect to \(P_\theta\) with oracle \(\theta\), i.e.,
\[{\cal R}^{P_\theta}={\cal R}^{P_\theta,\theta}\text{ for all }\theta.\]
We generalize Kjos Hanssen theorem  when the consistency of the posterior distributions holds.
Equivalent conditions of consistency of posterior distributions are shown in Theorem~\ref{th-consis-pos}.
In Theorem~\ref{mutual-singular} we show that under one of the statements in Theorem~\ref{th-consis-pos},
\[
\bigcap_{y\to y^\infty}{\cal R}^{P(\cdot\vert y)}={\cal R}^P_{y^\infty}\supseteq{\cal R}^{P(\cdot\vert y^\infty)}\supseteq{\cal R}^{P(\cdot\vert y),y^\infty} \text{ for all }y^\infty\in{\cal R}^{P_Y}.
\]
In addition under the condition of consistency of posterior distribution, 
if conditional probabilities  are computable with oracle of the random parameter  then the four variants of conditional ML-random set are equal,
which is a generalization of Kjos Hanssen theorem. 
In the proof of  \cite{kjoshanssen} computability of parameter from random sample sequence is required. 
In our proof,  we do not need the computability of parameter from random sample sequence.
In Theorem~\ref{counter-example} we also show an example that under the same condition,
\[{\cal R}^{P_X}=\bigcup_{y^\infty\in{\cal R}^{P_Y}}{\cal R}^P_{y^\infty} \ne\bigcup_{y^\infty\in{\cal R}^{P_Y}}{\cal R}^{P(\cdot\vert y^\infty)},\]
where \(P_X\) is the mixture distribution. 



\section{Conditional probabilities and generalized Lambalgen theorem}

Since conditional probabilities are defined through a joint probability of product space, we study the randomness of a joint probability of product space.
We consider the space \(\Omega:=\{0,1\}^\infty\) and computable probability on \((\Omega\times\Omega, {\cal B}_2)\).
\({\cal B}_2\) is the \(\sigma\)-algebra generated from cylinder sets \(\{\Delta(x)\times\Delta(y) \vert x, y\in S\}\). 
 \(S\) is the set of the finite binary strings and \(\Delta(x):=\{ x \omega \vert \omega\in\Omega\}\), where \(x\omega\) is the concatenation of \(x\) and \(\omega\).
 Let \(\lambda\) be the empty word.  We write \(x\sqsubseteq y\) if \(x\) is a prefix of \(y\). 
 \({\mathbb{N}}\) is the set of natural numbers and \({\mathbb{Q}}\) is the set of rational numbers.
 For a set \(A\subseteq \Omega\), the complement of \(A\) is denoted by \(A^c\).
 In order to clarify  the difference between strings and sequences, we use symbols such as  \(x, y\) for strings \(S\) and  \(x^\infty, y^\infty\) for sequences \(\Omega\).
 
 \(P\) on \((\Omega\times\Omega, {\cal B}_2)\) is called computable if there is a computable function \(A: S\times S\times {\mathbb{N}}\to {\mathbb{Q}}\) such that
\(\forall  x,y\in S,  k\in{\mathbb{N}}\  \vert P(x,y) - A(x, y, k)\vert < \frac{1}{k}\).
In other words \(P\) is computable if it is approximated with arbitrary precision with Turing machine.
Computable probabilities on \((\Omega, {\cal B})\) are defined similar manner, where 
\({\cal B}\) is the \(\sigma\)-algebra generated from  \(\{\Delta(x) \vert x\in S\}\).

Let \(U\subseteq S\times {\mathbb{N}}\). 
 \(U\) is called test (effective null set) with respect to \((\Omega, {\cal B}, P)\) if 
 \begin{eqnarray}
 U \text{ is a recursively enumerable set (r.e.~set)},\label{def-testA}\\
  \forall n\ \tilde{U}_n\supseteq \tilde{U}_{n+1},\text{ and } P(\tilde{U}_n)< 2^{-n},\nonumber
 \end{eqnarray}
 where \(U_n=\{ x \vert (x, n)\in U\}\text{ and } \tilde{U}_n=\bigcup_{x\in U_n}\Delta(x)\).
 In the following we write \(\tilde{A}:=\bigcup_{x\in A}\Delta(x)\) for \(A\subseteq S\).
 A set \(A\) is called recursively enumerable if there is a computable \(f:{\mathbb{N}}\to A\) such that \(f({\mathbb{N}})=A\).
 
 
The set of  ML-random sequences w.r.t.~\(P\) is defined as the complement of the effective null sets w.r.t.~\(P\). We denote it by \({\cal R}^P\), i.e.,
 \({\cal R}^P:= (\bigcup_{U: test}\bigcap_n \tilde{U}_n)^c\).
Note that if \(P\) is not computable, it is also called Hippocratic randomness \cite{kjoshanssen}.
For simplicity we call \({\cal R}^P\) as ML-random set  and if computability assumption of  \(P\) is necessary, we always state it.
Tests and ML-random set \({\cal R}^P\) with respect to  \((\Omega\times\Omega, {\cal B}_2, P)\) are defined in similar manner. 
If we consider the class of tests that is r.e.~with oracle \(y^\infty\) in (\ref{def-testA}), 
the random set defined with the extended class of tests is called  ML-random with oracle \(y^\infty\) and denote it with \({\cal R}^{P, y^\infty}\).

Lambalgen's theorem \cite{lambalgen87} says that a pair of sequences \((x^\infty,y^\infty)\in\Omega^2\) is ML-random w.r.t.~the product  of uniform measures iff \(x^\infty\) is ML-random and \(y^\infty\) is ML-random with oracle  \(x^\infty\).
In \cite{{vovkandvyugin93},{takahashiISIT2006},{takahashiRIMS2008},{takahashiIandC},{takahashiIandC2}} Lambalgen's theorem is generalized for computable correlated probabilities. 



Let \(X=Y=\Omega\) and  \(P\) be a computable probability on \(X\times Y\).  \(P_X\) and \( P_Y\) are marginal distributions on \(X\) and \(Y\), respectively. 
In the following we write \(P(x,y):=P(\Delta(x)\times\Delta(y))\) and \(P(x\vert y):=P(\Delta(x)\vert\Delta(y))\) for \(x,y\in S\).

Let \({\cal R}^P\) be the set of ML-random points and \({\cal R}^P_{y^\infty}:=\{ x^\infty\mid (x^\infty,y^\infty)\in{\cal R}^P\}\).
In \cite{{takahashiISIT2006},{takahashiIandC}} it is shown that  ML-random sequences satisfy the martingale convergence theorem and from this fact we can show the existence of conditional probabilities as follows.
\begin{theorem}[\cite{{takahashiISIT2006},{takahashiRIMS2008},{takahashiIandC}}]\label{conditional-prob}
Let  \(P\) be a computable probability on \(X\times Y\) and
\[
\forall x\in S,\ y^\infty\in{\cal R}^{P_Y}\ \ P(x\vert y^\infty):=\lim_{y\to y^\infty}P(x\vert y),
\]
if the right-hand-side exist.
Then \(P(\cdot\vert y^\infty)\) is a probability on \((\Omega, {\cal B})\) for each \(y^\infty\in{\cal R}^{P_Y}\).
\end{theorem}


Let \({\cal R}^{P(\cdot\vert y^\infty),y^\infty}\) be the set of ML-random set w.r.t.~\(P(\cdot\vert y^\infty)\) with oracle \(y^\infty\).


\begin{theorem}[\cite{{takahashiISIT2006},{takahashiRIMS2008},{takahashiIandC},{takahashiIandC2}}]\label{col-ifpart}
Let \(P\) be a computable probability on \(X\times Y\). Then 
\begin{equation}
{\cal R}^P_{y^\infty}\supseteq{\cal R}^{P(\cdot\vert y^\infty),y^\infty} \text{ for all }y^\infty\in{\cal R}^{P_Y}.
\end{equation}
Fix \(y^\infty\in{\cal R}^{P_Y}\) and suppose that \(P(\cdot\vert y^\infty)\) is computable with oracle \(y^\infty\).
Then 
\begin{equation}\label{generalLambal}
{\cal R}^P_{y^\infty}={\cal R}^{P(\cdot\vert y^\infty),y^\infty}.
\end{equation}
\end{theorem}



In Vovk and V'yugin \cite{vovkandvyugin93} the equation (\ref{generalLambal}) is shown under the condition that conditional probabilities  exist for all parameters 
and  conditional probabilities  are uniformly computable with oracle of parameters. 
Bayesian theory consists of prior and parametric model and many  models satisfy the assumption of the theorem in \cite{vovkandvyugin93}, e.g., Bernoulli and finite order Markov processes satisfy the model.
However in our model we start with computable global \(P\), and prior and parametric model are derived from \(P\)  as marginal distribution and  conditional probabilities, respectively.
In particular we can argue randomness even if  conditional probability is not computable with oracle of given parameter.
It is possible that conditional probabilities and parametric model may differ at a null set of parameters \cite{takahashiISIT2006}. 

It is known that there are  non-computable conditional probabilities \cite{roy2011} and in \cite{Bauwens2015} Bauwens showed an example that violates the equality in (\ref{generalLambal})
when the conditional probability is not computable with oracle \(y^\infty\). In \cite{takahashi2014} an example that  for all \(y^\infty\), the  conditional probabilities are not computable with oracle \(y^\infty\) and (\ref{generalLambal}) holds.

\section{Bayesian statistics}
We apply randomness theory to Bayesian statistics and show a point wise theory for Bayesian statistics.
In Bayesian statistics, we suppose a probability on parameter space and it is called prior.
Let   \(X\) and \(Y\) be sample and parameter spaces, respectively. 
Let \(P\) be a probability on \(X\times Y\) then the marginal distributions 
\begin{equation}\label{marginalX}
P_X=\int_Y P(\cdot\vert y^\infty)dP_Y
\end{equation}
 and \(P_Y\) are called mixture and prior distributions, respectively.
\begin{theorem}[\cite{{takahashiISIT2006},{takahashiRIMS2008},{takahashiIandC}}]\label{bayes-mix}
Let  \(P\) be a computable probability on \(X\times Y\).
\[
P({\cal R}^P_{y^\infty}\vert y^\infty)=1\text{ if }y^\infty\in{\cal R}^{P_Y}\text{ and }{\cal R}^P_{y^\infty}=\emptyset\text{ else.}
\]
\begin{equation}\label{bayes-mixture}
{\cal R}^{P_X}=\bigcup_{y^\infty\in{\cal R}^{P_Y}}{\cal R}^P_{y^\infty}.
\end{equation}
\end{theorem}
(\ref{bayes-mixture}) shows the natural properties of Bayesian mixture.

Next we study the case when consistency of posterior of distribution holds.
Kjos Hanssen showed the following theorem.
\begin{theorem}[Kjos Hanssen \cite{kjoshanssen}]\label{hanssen}
For Bernoulli model \(P(\cdot\vert \theta)\),
\begin{equation}
{\cal R}^{P(\cdot\vert\theta)}={\cal R}^{P(\cdot\vert\theta),\theta}\text{ for all }\theta.\nonumber
\end{equation}
\end{theorem}
Note that consistency of posterior distribution holds for Bernoulli model.


We generalize  Theorem~\ref{hanssen} when consistency of posterior of distribution holds.
Strictly speaking, prior space of Bernoulli model is \([0,1]\), however we consider \(\Omega\) as prior space.
The following theorem shows equivalent conditions for the consistency of posterior distributions.
In particular it is equivalent to the condition that conditional probabilities are mutually singular for disjoint cylinder sets. 
We write \(P \perp Q\) if  \(P\) and \(Q\) are mutually singular.
\begin{theorem}[\cite{{takahashiISIT2006},{takahashiRIMS2008},{takahashiIandC2}}]\label{th-consis-pos}
Let \(P\) be a computable probability on \(X\times Y\), where \(X=Y=\Omega\).
The following six statements are equivalent:\\
(1) \( P({\cdot\,}{\vert\,}  y)\perp  P({\cdot\,}{\vert\,}  z)\) if \(\Delta( y)\cap\Delta( z)=\emptyset\) for \( y, z\in S\).\\
(2) \({\cal R}^{ P({\cdot\,}{\vert\,}  y)}\cap{\cal R}^{ P({\cdot\,}{\vert\,}  z)}=\emptyset\) if \(\Delta( y)\cap\Delta( z)=\emptyset\) for \( y, z\in S\).\\
(3) For \((x^\infty,y^\infty)\in{\cal R}^P\), the posterior distribution \(P(\cdot\vert x)\)  converges weakly to \(P(\cdot\vert x^\infty)\) as \(x\to x^\infty\),  where \(P(y^\infty\vert x^\infty)=1\).\\
(4) \({\cal R}^{ P}_{ y^\infty}\cap{\cal R}^{ P}_ {z^\infty}=\emptyset\) if \( y^\infty\ne z^\infty\) for all \(y^\infty, z^\infty\).\\
(5) \(x^\infty\in{\cal R}^{P_X} \Rightarrow \text{ there is a unique }y^\infty\in{\cal R}^{P_Y}\text{  s.t. }x^\infty\in{\cal R}^P_{y^\infty}\).\\
(6) There exists \(f:X\to Y\) such that  \(f(x^\infty)= y^\infty\) for \((x^\infty, y^\infty)\in{\cal R}^{ P}\).\\
(7) There exists \(f:X\to Y\) and \( Y'\subseteq Y\) such that \(P_Y( Y')=1\) and 
 for each  \( y^\infty\in Y'\), 
\(f(x^\infty)= y^\infty\) for almost all \(x^\infty\) w.r.t.~\(P({\cdot\,}\vert  y^\infty)\).
\end{theorem}

It is interesting to see that  the statements (1) and (7) do not have algorithmic notion. 
A proof for the equivalence of the statements  except for  (5) is shown in \cite{{takahashiISIT2006},{takahashiRIMS2008},{takahashiIandC2}}.
The equivalence of the statements (5) and (6) is obvious by definition.
For Theorem~\ref{bayes-mix} and \ref{th-consis-pos}, see also Figure~\ref{figB}.

Before we prove Theorem~\ref{mutual-singular}, we need a lemma.

\begin{lemma}\label{lemma-A}
Let \(P\) be a computable probability on \(X\times Y\), where \(X=Y=\Omega\).
Fix \(y^\infty\in{\cal R}^{P_Y}\) then
\[ \forall y\sqsubset y^\infty\  {\cal R}^{P(\cdot\vert y^\infty)}\subseteq{\cal R}^{P(\cdot\vert y)}.\]
\end{lemma}
Proof)
In \cite{takahashiIandC} Corollary~4.1, it is shown that there is an integer \(M\) such that 
\begin{equation}\label{fund-inequatlity}
y^\infty\in{\cal R}^{P_Y}\Rightarrow \sum_n P((\tilde{U}_n)_{y^\infty}\vert y^\infty)< M <\infty,
\end{equation}
where \(U\) is a test with respect to \(P\) and \((\tilde{U}_n)_{y^\infty}=\{ x^\infty\vert (x^\infty, y^\infty)\in\tilde{U}_n\}\).

From (\ref{fund-inequatlity}), we have for all \(k\)
\begin{equation}\label{fund-ineqA}
P(\sum_n I_{(\tilde{U}_n)_{y^\infty}}> k\vert y^\infty) < \frac{M}{k},
\end{equation}
where \(I\) is the characteristic function, i.e., \(I_{(\tilde{U}_n)_{y^\infty}}(x^\infty)=1\) if \(x^\infty\in (\tilde{U}_n)_{y^\infty}\) else \(0\).

Let \(y=\lambda\) then we have \(P(\cdot\vert \lambda)=P_X\).
Let \(U^X\) be a test with respect to \(P_X\) and  \(U^{X\times \lambda}:=\{(x,\lambda, n)\vert (x, n)\in U^X\}\).
We see that \(U^{X\times \lambda}\) is a test with respect to \(P\).
Since \((\tilde{U}^{X\times \lambda}_n)_{y^\infty}=\tilde{U}^X_n\), from (\ref{fund-ineqA})
we have
\[\forall k\ P(\sum_n I_{\tilde{U}^X_n}> k\vert y^\infty) < \frac{M}{k}.\]
Thus we have a test \(V^X\) with respect to \({\cal R}^{P(\cdot\vert y^\infty)}\) such that \(\forall k\ \tilde{V}^X_k=\{\sum_n I_{\tilde{U}^X_n} > M2^k\}\) and the lemma proved for \(y=\lambda\).

We can show the lemma for any finite prefix \(y\sqsubset y^\infty\) in the similar way.
\hfill\hbox{\rule{6pt}{6pt}}



A generalized form of Kjos Hanssen's theorem~\ref{hanssen} is as follows.
\begin{theorem}\label{mutual-singular}
Let \(P\) be a computable probability on \(X\times Y\), where \(X=Y=\Omega\).
\[\bigcap_{y\to y^\infty}{\cal R}^{P(\cdot\vert y)}\supseteq {\cal R}^{P(\cdot\vert y^\infty)}\text{ and }  \bigcap_{y\to y^\infty}{\cal R}^{P(\cdot\vert y)}\supseteq {\cal R}^P_{y^\infty} \text{ for all }y^\infty\in{\cal R}^{P_Y}.\]
Under one of the statements of Theorem~\ref{th-consis-pos}, we have 
\begin{equation}\label{eq-singular-case}
\bigcap_{y\to y^\infty}{\cal R}^{P(\cdot\vert y)}={\cal R}^P_{y^\infty}\supseteq{\cal R}^{P(\cdot\vert y^\infty)}\supseteq{\cal R}^{P(\cdot\vert y),y^\infty} \text{ for all }y^\infty\in{\cal R}^{P_Y}.
\end{equation}
Fix \(y^\infty\in{\cal R}^{P_Y}\) and suppose that \(P(\cdot\vert y^\infty)\) is computable with oracle \(y^\infty\).
Under one of the statements of Theorem~\ref{th-consis-pos}, we have 
\[\bigcap_{y\to y^\infty}{\cal R}^{P(\cdot\vert y)}={\cal R}^P_{y^\infty}={\cal R}^{P(\cdot\vert y^\infty)} ={\cal R}^{P(\cdot\vert y^\infty),y^\infty}.\]
\end{theorem}



Proof of Theorem~\ref{mutual-singular})
As with the same way of Theorem~\ref{bayes-mix}, we have
\begin{equation}\label{bayes-mixtureB}
\forall y\ {\cal R}^{P(\cdot\vert y)}=\bigcup_{y^\infty\in{\cal R}^{P_Y}\cap\Delta(y)}{\cal R}^P_{y^\infty}.
\end{equation}
For example consider  \(P(\cdot\vert X\times\Delta(y))\) and its ML-random set and observe that
\({\cal R}^{P(\cdot\vert X\times\Delta(y))}={\cal R}^P\cap (X\times \Delta(y))\).
From Lemma~\ref{lemma-A}, we have the first statement.

Suppose that  \({\cal R}^{ P}_{ y^\infty}\cap{\cal R}^{ P}_ {z^\infty}=\emptyset\) if \( y^\infty\ne z^\infty\).
Then from (\ref{bayes-mixtureB}), we have
\[\bigcap_{y\to y^\infty} {\cal R}^{P(\cdot\vert y)}= {\cal R}^P_{y^\infty}.\]
From Lemma~\ref{lemma-A} and Theorem~\ref{col-ifpart}, we have the first statement of the theorem.
The latter part follows from Theorem~\ref{col-ifpart} and the first statement. 
\hfill\hbox{\rule{6pt}{6pt}}

\begin{remark}\label{remarkA}
Without the consistency conditions, the second statement of Theorem~\ref{mutual-singular} does not hold.
For example let \(P=U\times U\) then
\(P(\cdot\vert y^\infty)=U\), where \(U\) is the uniform probability on \(\Omega\).
We see that the diagonal set is covered by a test, i.e., \(\{(x^\infty, x^\infty)\vert x^\infty\in\Omega\}\subseteq ({\cal R}^P)^c\).
From Lambalgen theorem \cite{lambalgen87} we have \(y^\infty\in {\cal R}^U=\bigcap_{y\to y^\infty}{\cal R}^{P(\cdot\vert y)}\) and \(y^\infty\notin{\cal R}^{U,y^\infty}={\cal R}^P_{y^\infty}\subseteq {\cal R}^U=\bigcap_{y\to y^\infty}{\cal R}^{P(\cdot\vert y)}\).
\end{remark}


There is an example that the equality in (\ref{eq-singular-case}) does not hold.
\begin{theorem}\label{counter-example}
There is a probability on \(X\times Y\) with the properties that \\
(1) \(P\) is computable and \(\{1^\infty\}\in{\cal R}^{P_Y}\), \\
(2) \(P(\cdot\vert 1^\infty)\) is not computable, \\
(3) \(P(\cdot\vert y)\perp P(\cdot\vert z)\) for all \(\Delta(y)\cap\Delta(z)=\emptyset\), \\
(4) \({\cal R}^P_{1^\infty}\supseteq {\cal R}^{P(\cdot\vert 1^\infty)}\) and  \({\cal R}^P_{1^\infty}\ne {\cal R}^{P(\cdot\vert 1^\infty)}\), and \\
(5) \({\cal R}^{P_X}\supseteq \bigcup_{y^\infty\in{\cal R}^{P_Y}}{\cal R}^{P(\cdot\vert y^\infty)}\) and  \({\cal R}^{P_X}\ne \bigcup_{y^\infty\in{\cal R}^{P_Y}}{\cal R}^{P(\cdot\vert y^\infty)}\),\\
(6) there is \(x^\infty\in{\cal R}^{P_X}\) such that there is no \(y^\infty\in{\cal R}^{P_Y}\) with \(x^\infty\in{\cal R}^{P(\cdot\vert y^\infty)}\), i.e.,
the statement (5) in Theorem~\ref{th-consis-pos} does not hold for \({\cal R}^{P(\cdot\vert y^\infty)}\).
\end{theorem}
Here \(1^\infty\) is the sequence consists of 1s.
Except for clearly stated cases, 
do not confuse symbols such as  \(x^\infty\) with the repetition of a string.
We have the counter-example above by modifying the construction of the counter-example in \cite{BST2016}.
Note that the counter-example in \cite{BST2016} neither implies (5) nor (6) in Theorem~\ref{counter-example}.

Proof of Theorem~\ref{counter-example})
Let \(X\times Y=\Omega^2\).
Fix \(P_X\) is the uniform distribution on \(X\).
For \(s\in S\), set \(r(s)=\sum_{1\leq i\leq k}2^{-i}s_i\) and \(l(s)=k\) where \(s=s_1\cdots s_k, s_i=0\text{ or }1\text{ for all i}\).
Fix a non-computable sequence \(\alpha\in{\cal R}^{P_X}\) with computable increasing sequence of strings \(r(a_1)<r(a_2)<\cdots < r(\alpha)\) and \(\lim_i r(a_i)=r(\alpha)\).
Set for all \(i\in{\mathbb{N}}\)
\begin{align*}
&A_i=\{ s\in S\mid  l(s)=l(a_i), r(s)<r(a_i)\},\\
&B_i=\tilde{A}_i\setminus \tilde{A}_{i-1}\text{ for }i\geq 2,\text{ and }B_1=\tilde{A}=1.
\end{align*}
We define the measure \(P\) on \(X\times Y\) as follows.
For all \(x\in S\) and \(i\in {\mathbb{N}}\) define
\begin{align*}
&P((B_i\cap \Delta(x))\times \{1^i0^\infty\}):=P_X(B_i\cap\Delta(x)),\\
&P(((\tilde{A}_i)^c\cap \Delta(x))\times \Delta(1^{i-1}0)):=0,\text{ and } \\
&P(((\tilde{A}_i)^c\cap \Delta(x))\times \Delta(1^i)):=P_X((\tilde{A}_i)^c\cap \Delta(x)),
\end{align*}
where \(1^i\) is the string consists of 1s with length \(i\) and \(0^\infty\) is the sequence consists of 0s.
For the construction of the measure, see Fig.~\ref{figC}.
We show that \(P\) is a computable probability on \((X\times Y), {\cal B}_2)\).
For all \(x,y\) if \(y\ne 1^{l(y)}\) then
\[\Delta(x)\times \Delta(y)\subseteq \tilde{A}_{l(y)}\times \Omega \cup (\tilde{A}_{l(y)})^c\times \bigcup_{1\leq i\leq l(y)}\Delta(1^{i-1}0),\]
otherwise
\[ \Delta(x)\times \Delta(y)\subseteq \cup_i \tilde{A}_i\times \Delta(1^i)\cup (\tilde{A}_i)^c\times \Delta(1^i).\]
Since \(P_X\) is computable, we see that \(P\) is computable. 

Next we show that \(\{1^\infty\}\in {\cal R}^{P_Y}\). 
Since \(P(\Omega\times \Delta(1^k))=P_X((\tilde{A}_{k-1})^c)\), we have \(P_Y(\{1^\infty\})=P(\Omega\times \Delta(1^\infty))=P_X(\cap_k (\tilde{A}_k)^c)=1-\alpha>0\) and \(\{1^\infty\}\in {\cal R}^{P_Y}\). 
We have the statement (1) of this theorem. 

We have
\(P(x\vert 1^\infty)=0\) if \(\Delta(x)\subseteq \cup_i \tilde{A}_i\) and 
 \(P(x\vert 1^\infty)=P_X(x)/(1-\alpha)\) if \(\Delta(x)\subseteq \cap_i (\tilde{A}_i)^c\).
 Since \(\alpha\) is not computable, \(P(\cdot\vert 1^\infty)\) is not computable. 
 As in the same way with \cite{{BST2016},{Bauwens2015}}, we see that \(\alpha\notin{\cal R}^{P(\cdot\vert 1^\infty)}\).
 For example, set \(U_n:=\{s\mid \exists i\ r(s)<r(a^i)+\frac{1}{n}\}\) then \(U=\{ (s,n)\mid \exists n\ s\in U_n\}\) is r.e. and \(P(\tilde{U}_n\vert 1^\infty)\leq 1/n\) for all \(n\). We have \(U\) is a test and 
cover \(\alpha\).

Next we show that \(P(\cdot\vert \alpha) =1\) with \(\alpha\in X\).
Let \(b_1\sqsubseteq b_2\cdots\) be an increasing sequence of prefix of \(\alpha\) such that \(b_i\to \alpha\) as \(i\to\infty\).
From the construction we have
\(\forall k\exists i\  P(1^k\vert b_i)=1\) and hence \(\forall k\ P(1^k\vert \alpha)=1\) and \(P(\{1^\infty\}\vert\alpha)=1\).
Since \(\{1^\infty\}\in{\cal R}^{P(\cdot\vert \alpha),\alpha}\) and 
\(P(\cdot\vert \alpha)\) is computable with oracle  \(\alpha\in{\cal R}^{P_X}\),
from Theorem~\ref{col-ifpart}, we have \((\alpha, 1^\infty)\in{\cal R}^P\) and \(\alpha\in{\cal R}^P_{1^\infty}\).
Since \(\alpha\notin{\cal R}^{P(\cdot\vert 1^\infty)}\), from Theorem~\ref{mutual-singular}, we have  
the statement (4) in this theorem. 

By construction we have \(P_Y(\{1^i0^\infty\})>0\) for all \(i\) and \(P_Y(\{1^\infty\})>0\).
Since \(P_Y((\bigcup_i \{1^i0^\infty\} \cup \{1^\infty\})^c)=0\), it is covered by a test and we have \({\cal R}^{P_Y}=\bigcup_i \{1^i0^\infty\} \cup \{1^\infty\}\).
By construction, \(P(\cdot\vert y^\infty)\) are mutually singular for different \(y^\infty\in{\cal R}^{P_Y}\), we have the statement (3).
Since (i) \(\alpha\in{\cal R}^P_{1^\infty}\) and \(\alpha\notin{\cal R}^{P(\cdot\vert 1^\infty)}\), (ii) \({\cal R}^P_{y^\infty}\) are disjoint for different \(y^\infty\in{\cal R}^{P_Y}\) (Theorem~\ref{th-consis-pos}), and (iii)
\({\cal R}^P_{y^\infty}\supseteq {\cal R}^{P(\cdot\vert y^\infty)}\) for all \(y^\infty\in{\cal R}^{P_Y}\) (Theorem~\ref{mutual-singular}), we have the statement (5) and (6) in this theorem. 
\hfill\hbox{\rule{6pt}{6pt}}

\begin{figure}[ht]
\centering
\begin{picture}(200,170)(0,0)
\put(20,20){\framebox(140,140)}
\put(175,20){\(Y\)}
\put(0,160){\(X\)}
\put(20,50){\line(1,0){140}}


\put(20,70){\line(1,0){140}}


\put(20,100){\line(1,0){140}}

\multiput(143,80)(0,4){3}{\circle*{2}}

\put(0,30){\(B_1\)}
\put(0,58){\(B_2\)}
\put(0,100){\(\alpha\)}
{\small
\put(86,3){\(10^\infty\)}
\put(118,3){\(110^\infty\)}
\put(160,3){\(1^\infty\)}
}
\linethickness{1.5\unitlength}
\put(125,50){\line(0,1){20}}
\put(90,20){\line(0,1){30}}
\put(160,100){\line(0,1){60}}

\end{picture}
\caption{The construction of counter example.
The measure concentrates on the thick lines and conditional probabilities are mutually singular.}\label{figC}
\end{figure}


 
We  summarize Theorem~\ref{mutual-singular}, \ref{counter-example} and Remark~\ref{remarkA}  in Figure~\ref{figA}.
\begin{figure}[ht]
\centering
\subfloat[General Case]{
\begin{picture}(100,140)(20,0)
\put(40, 130){\(\bigcap_{y\to y^\infty} {\cal R}^{P(\cdot\vert y)}\)}
\put(35, 100){\rotatebox{50}{\(\subseteq\)}}
\put(10, 100){{\footnotesize (i)}}
\put(70, 100){\rotatebox{130}{\(\subseteq\)}}
\put(95, 100){{\footnotesize (ii)}}
\put(20,  70){\({\cal R}^P_{y^\infty}\)}
\put(80,70){\({\cal R}^{P(\cdot\vert y^\infty)}\)}
\put(35, 40){\rotatebox{130}{\(\subseteq\)}}
\put(10, 40){{\footnotesize (iii)}}
\put(70, 40){\rotatebox{50}{\(\subseteq\)}}
\put(95, 40){{\footnotesize (iv)}}
\put(50, 10){\({\cal R}^{P(\cdot\vert y^\infty),y^\infty}\)}
\end{picture}
}
\subfloat[Consistent Case]{
\begin{picture}(100,140)(20,0)
\put(45,  130){\({\cal R}^P_{y^\infty}\)}
\put(45, 100){\rotatebox{90}{\(\subseteq\)}}
\put(70, 100){{\footnotesize (v)}}
\put(45,70){\({\cal R}^{P(\cdot\vert y^\infty)}\)}
\put(45, 40){\rotatebox{90}{\(\subseteq\)}}
\put(45, 10){\({\cal R}^{P(\cdot\vert y^\infty),y^\infty}\)}
\end{picture}
}
\caption{Relations between conditional random sets. Figure (a) shows relations between conditional ML-random sets.  Figure (b) shows
those relations when consistency of posterior distribution holds.
In addition if the conditional probability is computable with oracle of random \(y^\infty\) and consistency condition holds then the four sets in figure (a) are equal. 
Remark~\ref{remarkA} shows a counter-example of equality in (i) and (iv). 
The counter-example of equality in (iii) due to Bauwens \cite{{BST2016},{Bauwens2015}}.
Theorem~\ref{counter-example} shows a counter-example of equality in (ii) and (v).
}\label{figA}
\end{figure}

\begin{figure}[ht]
\centering

\begin{picture}(200,170)(0,0)
\put(20,20){\framebox(140,140)}
\put(175,20){\(Y\)}
\put(0,160){\(X\)}
\put(170,160){\({\cal R}^P\)}

\put(70,20){\line(0,1){140}}
\multiput(70,45)(0,25){5}{\circle*{2}}
\put(70, 10){\(y^\infty\)}

\put(110,20){\line(0,1){140}}
\multiput(110,35)(0,25){5}{\circle*{2}}
\put(110, 10){\(z^\infty\)}

\put(20,120){\line(1,0){140}}
\put(20,85){\line(1,0){140}}
\end{picture}

\caption{The vertical lines show the section of \({\cal R}^P\) at parameters \(y^\infty\) and \(z^\infty\), i.e., \({\cal R}^P_{y^\infty}\) and \({\cal R}^P_{z^\infty}\), respectively. 
\(P(R^P_{y^\infty}\vert y^\infty)=1\) if \(y^\infty\in{\cal R}^{P_Y}\) and \({\cal R}^P_{y^\infty}=\emptyset\) else.
We see that ML-random set w.r.t. marginal distributions are projection of \(R^P\) to \(X\) and \(Y\), respectively.
In particular we have \({\cal R}^{P_X}=\bigcup_{y^\infty\in{\cal R}^{P_Y}}{\cal R}^P_{y^\infty}\).
If \({\cal R}^P_{y^\infty}\cap{\cal R}^P_{z^\infty}=\emptyset\) for all  \(y^\infty, z^\infty\in{\cal R}^{P_Y}, y^\infty\ne z^\infty\), then we see that there is a function \(f: X\to Y\) such that
\(f(x^\infty)=y^\infty\) for all \((x^\infty,y^\infty)\in{\cal R}^P\).
}\label{figB}
\end{figure}



\section{Concluding remark}


In this paper we discussed four variations of ML-random sets with respect to conditional probabilities, i.e., 
\(\bigcap_{y\to y^\infty}{\cal R}^{P(\cdot\vert y)}\), \({\cal R}^P_{y^\infty}\), \({\cal R}^{P(\cdot\vert y^\infty)}\), and \({\cal R}^{P(\cdot\vert y^\infty),y^\infty}\).
These relation are shown in Fig.~\ref{figA}.
The former two sets are defined from global probability and the latter sets are defined from local conditional probability. 


One might think that a sequence is a {\it good} random one if it is in the complement of a large class of statistical tests and correspondingly it satisfies a large class of probability laws.
Theorem~\ref{mutual-singular} shows that for mutually singular models, the class of ML-tests with respect to 
the local conditional probability at a fixed random parameter is larger than the class of ML-tests with respect to the  global probability at the section of the random parameter.
Theorem~\ref{counter-example} shows that they are not equal in general. 
Note that many conventional statistical models consist of mutually singular models. 

However these fine versions of conditional randomness, i.e., \({\cal R}^{P(\cdot\vert y^\infty)}\) and \({\cal R}^{P(\cdot\vert y^\infty),y^\infty}\), do not satisfy theorems in Bayesian statistics, which could be against to intuition. 
This is a new discovery in randomness theory. 
Informally speaking, statistical laws is the law of global probability and not a law of a local conditional probability model.
Local conditional randomness does not govern  global  statistical laws in general.

This point of view might originate in the work of von Mises. 
In \cite{mises} von Mises proposed several axioms (probability laws) that specify random sequences.
Though his theory  have some technically controversial  problem and did not mathematically matured, 
he insisted  that the model of a given random sequence is not defined a priori
but it should be determined from each random sequence, which is a unique and significant point in his theory.
If we consider statistical problem and its relevant randomness, e.g., Bayesian consistency theorems, 
we need to consider multiple models simultaneously and cannot fix a model a priori. 
We can find a close idea to von Mises theory in the proof of Kjos Hanssen theorem \cite{kjoshanssen}.
In fact  a computable function from random samples to parameters plays important role in his proof. 

On the other hand, Kolmogorov developed a general theory of randomness.
However randomness theory for conditional probabilities were not well developed.
In fact even  the existence of conditional probabilities for random parameters was not known.

With this respect, our theory for conditional randomness might be a fine and modern version of von Mises theory and Kolmogorov theory. 

\({\cal R}^P_{y^\infty}\) satisfies many probability laws related to Bayesian statistics and information theory, for more details see \cite{{takahashiISIT2006},{takahashiIandC},{takahashiRIMS2008},{takahashiIandC2}}.
For example we can show a point-wise optimality for Bayes coding with the help of complexity theory \cite{takahashiIandC}, which cannot be obtained without randomness theory. 

As stated above, \({\cal R}^{P(\cdot\vert y^\infty)}\) and \({\cal R}^{P(\cdot\vert y^\infty),y^\infty}\) are defined from local conditional probability \(P(\cdot\vert y^\infty)\) and \(y^\infty\), and 
\(\bigcap_{y\to y^\infty}{\cal R}^{P(\cdot\vert y)}\) and \({\cal R}^P_{y^\infty}\) are defined from global \(P\) and \(y^\infty\). 
It is interesting to see that these four sets are equal if the conditional probabilities are mutually singular on different cylinder sets of parameters and computable with oracle \(y^\infty\).

Note that conditional probabilities are derived from global probability and defined on the set of parameter space with probability one, for example see \cite{Kol33}.
Apart from global probability, a conditional probability with a particular parameter is not well defined. 



\begin{center}
{\bf Acknowledgment}
\end{center}
A part of work was done when the author was with Gifu University and supported by KAKENHI (24540153).


{\small
