to be on the safe side

to be on the safe side  {adv. phr.}

To take extra precautions; reduce or eliminate the possibility of a mistake, an error, or even danger.

Dad always keeps his valuables in a bank's safe deposit box, just to be on the safe side.

Compare: JUST IN CASE.