If $S$ is a set consisting of $12$ elements, what is the maximum number of pairs $(a, b)$ such that $a, b \in S$ and $\frac{b}{a}$ is a prime number?

Let's think step by step and output the final answer within \boxed{}.
user