go to pot
go to pot {v. phr.}, {informal}
To be ruined; become bad; be destroyed.
Mr. Jones' health has gone to pot.
The motel business went to pot when the new highway was built.
Categories:
Source: A Dictionary of American Idioms