on the sly
on the sly {adv. phr.}
So that other people won't know; secretly.
The boys smoked on the sly.
Mary's mother did not approve of lipstick, but Mary used it on the sly.
Categories:
Source: A Dictionary of American Idioms