Grinder guard question - can I use a mix sizes?

Some angle grinders, such as the DeWalt model that I have, come with three different sizes of guards. I have not measured them but I believe they support 4", 5" and 6" grinding wheels.

The down side I see using a bigger guard than the size of the wheel is that it will be like working with a wheel that is nearly worn out compared to the larger size. Depending upon the specific design of the grinder and its guard mount scheme it is possible that the larger size guard can limit the amount of freedom that you have in getting the grinding wheel up to the work piece.

Another thing to consider even though it is a little off topic for this question is the advantage of a larger diameter wheel. For a given rotational speed of the grinder (i.e. RPM) the larger wheel will have a much higher circumferential velocity of the cutting edge as it engages with the work piece. This can have a dramatic effect on how quickly the grinder can remove material.