Is Ultegra 6800 GS compatible with an HG800 11 speed 11-34 cassette?

As for clearing a 34, there is no universal answer because it's slightly dependent on where the dropout puts the RD mounting bolt relative to the axle center, which despite Shimano having a standard spec for does have some variance bike to bike. So it's a little bit of an experiment every time. Generally speaking this can be made to work on RDs, including the one in question, with a max cog spec of 32t by running the B-tension screw in reverse or subbing in a longer one. It's also common for it to barely work and/or be a little grindier than desirable in the low gear but basically work. And remember that when you do this, something is lost in shifting performance on the rest of the cassette, so it's not all upside.

Exceeding the printed large cog spec by two whole teeth without any B-tension hacks usually doesn't work. I've tried it with various of the 11-speed road RDs, probably including this one but I can't be sure.

As for exceeding the total capacity, there are many answers here on this topic but basically the bike will be rideable but will have chain slack in some number of the small/small combinations. Total capacity numbers can basically be taken at face value (in other words they're reasonably honest), so you can do the math to see which ones will be slack, again with a very slight bit of wiggle room due to variance in dropout geometry. Here 18t of the total capacity is being used up front, leaving 19t in back. 34-19=15, so once the chain is long enough to wrap around the 52t ring, 34t cog combo, it can be tensioned on a 15t or above, and below that it will be sagging/floppy. Riding in the floppy gears carries some risk of derailment/jamming.