Disable 'OK Google' recognition without completely disabling Google app

I believe this website has a guide to the solution that you're looking for.

Edit: I was asked to summarize the site (Yes, I did just copy and paste the instructions for the most part):

Google Now has a neat ability to listen for the “OK, Google” command to automatically launch the app and get you into Search as fast as possible. But, you may not want Google Now to listen to that prompt or hotword for some reason or another. Good news–it’s super easy to turn off!

Turning off the “OK, Google” hotword: To turn off the hotword detection, open the Search app. You can find this in your App Drawer, and you can usually access it by holding down the home button as well.

Next, at the top left there’s a hamburger menu. Tap it and head into Settings > Voice.

Next, head into OK, Google Detection. From here, you can toggle it on or off.

Keep in mind that, even if you have it off, you can still head into Google Now and tap the microphone on the right side of the Search bar to use your voice. It’s certainly not as convenient as using the hotword, which gets you into Search right away, but still enables you to use your voice to search.