Is passing traffic on the cars' passenger side ("Undertaking") with a bicycle frowned upon in the UK?

This isn't the example situation described in the question, but it is what immediately what came to mind when I read the title, which is:

Undertaking another cyclist whilst on a bicycle is in my experience extremely frowned upon, and to me (when I am the one being undertaken) feels very dangerous. My rationale for why it feels like this to me is that as cyclists we have a constant sense of threat from motor traffic, and our "emergency escape route" is to move away from the traffic towards the side of the road. Furthermore unless we have moved in to a separate lane away from the side of the road (for example if we are going straight on at a roundabout with a left-turn lane) we only expect (and therefore only check for) traffic coming from behind to appear on our right side. Then the appearance or presence of an obstacle inside this emergency channel and in a position we would not normally check for before executing a leftward manouevre feels extremely sketchy.

As we talk about the UK, yes ride on to the stop stripe (or the lights) unless that gets you in an unsafe situation.

As many of the bike lanes in the UK are too narrow for cycling when cars are stopped close, you may have to get off if you want to move forward, even when the car is in the proper position on the road.
And many of the bigger vehicles warn you not to pass while it is stationary, like stopped for a light. I have seen those stickers on buses as well as on heavy goods vehicles.
If you need to pass one of those vehicles with such a sticker, consider staying behind it, (as the drivers may not check well for cyclists on the inside even though law tells them to do so.) Too many lives have been lost in accidents where the driver did not see a bike next to the lorry.

Where the bike lane is safe to continue, wide enough and with enough separation cars do not cross it while turning, you should continue on.

As the previous answer states, you're perfectly within your rights to continue along the cycle lane in this situation, regardless of any cars to your right.

Some junctions even have advanced stop lines (see rule 178), which are explicitly designed for exactly this purpose: allowing and encouraging bicycles to pass beyond the stopped cars and position themselves in front of other traffic (this is generally preferable, as it makes the cyclists much more visible to drivers).

You should of course still maintain awareness of the traffic (especially if there are large vehicles such as HGVs or buses, which often have significant blind spots), and be prepared to make a quick stop if it appears that a vehicle is about to cross the cycle lane.

But not only is this allowed for bikes, in the specific situation you're asking about - passing traffic which is stopped at a red light - it's also perfectly fine (and indeed expected) for cars to undertake.

See Rule 163 of the highway code:

Rule 163
Overtake only when it is safe and legal to do so. You should:
[...]
stay in your lane if traffic is moving slowly in queues. If the queue on your right is moving more slowly than you are, you may pass on the left
[...]

(my emphasis)

Note also (admittedly now getting some way from the original question) that the points in rule 163 are governed by "You should", not "You must". As explained in the Wording of the Highway Code, that indicates advice rather than legal requirements per se (failing to follow the advice can lead to liability in an accident, but isn't prohibited in itself).

Take the example where a bicycle lane in on the road, rather than the pavement. There is a queue of cars waiting at the lights, and the bike lane is clear right up to the lights. If you were to cycle right up to the lights, you would be undertaking all the queuing cars on the way. Is this an example of "dangerous driving", or otherwise frowned upon (UK?

Absolutely not, you have a cycle lane allocated to you. There’s no point to slow down because some cars next to you slow down. You have a free path ahead of you so you can move forward. I’ve seen people do this all the time. It’s one of the main reasons cycling is better and quicker.

What if the cars were moving?

Same rules apply, you have a cycle lane, use it.

What if there wasn't a bike lane, but you were staying to the side of the road so that other cars could overtake you if they wanted?

That’s also fine, in fact you should stay at the side of the road (especially a busy one). Ducking in the middle of the road could be “dangerous driving”.