Shimano 5700 group set with 5800 crank

The compatibility of a crankset with the rest of the drivetrain is determined by the chain thickness for which the crankset is intended for. This can be adjusted by swapping the chainrings.

If you have a crank that's for 10-speed systems and you have a 10-speed chain, it works perfectly.

If you have a crank that's for 10-speed systems and you have an 11-speed chain, if you are on the big ring, and first upshift on the rear to the smallest sprocket, and only later shift on the front to the little ring, it may be possible the chain "skates" for some time over the small ring teeth. It isn't reasonable to shift that way, so with reasonable shifting the "problem" never materializes.

If you have a crank that's for 11-speed systems and you have a 10-speed chain, if you are in the extreme criss-cross gear (little ring front, small sprocket rear), the chain may rub on the big ring. It isn't reasonable to use that gear combination.

So, the result? It generally doesn't matter if your crankset is compatible with the chain. The problems a mismatch can cause are very limited. Of course you may try this to the extreme (8-speed crankset, 11-speed chain) and then the problems could be so large that you start to notice them.

In any case, you can always swap the chainrings if you want perfect compatibility. For example for 11-speed chain you would choose 11-speed chainrings. Swapping the chainrings may be advisable anyway as I don't think the Shimano 105 chainrings are made of 7075T6 (or 7075T651) aluminum. You can find plenty of third-party manufacturers selling machined 7075T6 (or 7075T651) aluminum chainrings. Those will be far more durable than the Shimano 105 chainrings.

You may also want to ensure compatibility with the front derailleur, but that's rarely a problem if switching from road bike component to another road bike component. Then the style of the crankset (double vs triple) and the chainline should match, and the big ring size is very similar.

Also you need compatibility with the bottom bracket. That's a cheap part to change if incompatible. Both the 5700 and 5800 cranksets use Hollowtech II bottom bracket, the best bottom bracket technology available today. So BB compatibility will not be an issue.

11-speed cranks on 10-speed bikes almost always work without issue. The reason is that the width jump in that generational change is the smallest because it's one of the speed plus-oneings that was done primarily by making the cassette longer rather than the spacing a bunch narrower. See chart from bikegremlin.com

10 speed cranks in 9 speed bikes often make you need to drop in chainring shims to avoid the chain rubbing in certain gears. 11 to 10 rarely has that problem. It's still possible, and the reason is short chainstays where even the right crank would barely not rub. It's usually tri/TT where you see that most. So, it is possible you need to drop in one or two sets of 0.1mm microshims on each chainring bolt, which several companies make. You could do one set premptively because 0.1mm is basically half the width difference, and the shifting will only improve, but it's not usually necessary.