Is it illegal to field/pitch with a batting glove inside a regular glove?

Generally in both baseball and softball, a pitcher can be asked to remove a white or ball colored piece of clothing that interferes with the batter's ability to see the ball coming in.

This is both a safety issue and a competitive advantage issue. If a hitter develops confusion about where the ball is coming from due to the pitcher's clothing they may not be able to get out of the way of the ball if it were to be about to hit them and they would also be at a severe disadvantage at the plate.

However, without knowing the exact league you are dealing with I cannot quote a rules citation.

Yes it is illegal in all higher levels of baseball. For college games we referred to the PBUC:

PBUC manual 1.22 (page 9.) No pitcher shall be allowed to wear a batting glove while pitching.

From the mlb website:

While not specifically referred to in the Official Rules, MLB regulations prohibit a pitcher from wearing a golf glove under his pitching glove. To do so could be distracting to the batter as well as a violation of the spirit of Rule 1.15(a).

Most high school associations and little leagues prohibit them too. I think LL allows them though. It is a no brainer to make the pitcher take it off. I really don't see it as a distraction or safety issue, it is just making a clear line on fairness. The batting glove is simply a foreign object in the glove. It doesn't help the player and can only be used for harm. Players could use it to scuff ball or whatever.

Also I want to note that there is a rule that explicitly states that a pitcher may only rub the ball with his bare hands. As an umpire I probably would wait to let the other coach complain if the kids were under 16 - but it has been 15 years since I umped that age. If over I would ask pitcher to remove during warm-ups and do it all the time. Most of the time they just left it on from hitting and don't care at all.

Note: Your title indicates the question being around fielders too. They can wear batting gloves. Batting gloves cannot contain foreign substances on them. What is a foreign substance? Something that you wouldn't innately find on a baseball diamond.