Is there any way to get rid of the 'infinity' button in Cookie Clicker?

You can either:

Fully wipe your save, achieved by going to Options > Wipe save, shown below:

Or, assuming the only thing they changed was the amount of cookies you had, press F12 (Chrome) or Ctrl+Shift+K (Firefox) to enter the console, and type Game.cookies = 0 to set your game's cookies to 0. You can change that number to whatever you want if you want to reverse the changes.

Keep in mind that modifying Game.cookies will grant you the Shadow Achievement "Cheated cookies taste awful", so if you don't want that, you'll have to wipe your save.