A cubic block with dimensions $n$ by $n$ by $n$ is made up of a collection of $1$ by $1$ by $1$ unit cubes. What is the smallest value of $n$ so that if the outer layer of unit cubes are removed from the block, more than half the original unit cubes will still remain?

Let's think step by step and output the final answer within \boxed{}.
user