Agree with/on/about/to the book - In what scenarios are these correct?

Is my current use of these correct:

We agreed on that book. – Eg.: When in a book club you are choosing the book for the next session.

We agreed about that book. -Eg.: When two friends discussed a book and they had the same opinion about it.

We agreed with that book. – Eg.:When two friends discussed a book and they both agreed with the points that were made in the book.

We agreed to that book. – This is not correct. You can only agree to something that you will do.

We agreed the book. This is not correct. You can only agree a price or other detail for something you are negotiating about.

Thank you