Blocks = Streets

There is no ambiguity because a block does not refer to distance, but instead to the actual number of streets you cross.

If you’re giving directions, go 5 blocks means you go to the fifth intersection afterwards, no matter whether those blocks are long or short, or some mix of long and short, or even variable. But using 'blocks' on curving and variable length streets like in suburbs is apt to be difficult and you'l be more likely to say 'turn on your fifth street on your right'.

Of course people sometimes use words loosely, vaguely, without really exactly knowing things, and also blocks has a loose correlation with some concept of distance that may well be used (though not often) as a substitute for distance. But in general, a block is not a particular distance; it just counts up the number of streets you cross to get there.

In Manhattan, the distance between avenues (1st Ave, 2nd Ave, etc) from east to west is very reliably ∼3½ (or ⁷⁄₂) times the distance between streets (42nd St to 43rd St) from north to south (note that both streets and avenues run at a slight angle compared with exact latitude or longitude). So 5 blocks going north–south is a much, much shorter distance than 5 blocks east–west is: just ²⁄₇ the distance. But that's just Manhattan. Other cities will be different.

There is no ambiguity because you usually have some context of which direction you’re going:

A: "How do I get to Carnegie Hall from Ed Sullivan Theater?"

B: "Go 1 block east on 54th and then 2 blocks and some on 7th Ave"

As Mitch’s answer says, in specific contexts like giving directions for a route, blocks are used to specify a route, regardless of distance.

But in informal contexts, blocks are widely used as a rough measure of distance, as you say. The key words here are “informal” and “rough”. Yes, it’s quite ambiguous since block lengths vary in many ways. But that’s nothing unusual; lots of widely used informal measures are similarly imprecise. A place may be described as two hours’ drive away (even though this depends on the traffic conditions and the car/driver), or fifteen minutes’ walk (though this depends on the individual’s walking speed); a slightly old-fashioned recipe might call for a handful of chopped parsley or a glass of wine (though hands and glasses differ widely). All of these vary and don’t have precise values, but they’re useful as informal measures because they’re more intuitively tangible from day-to-day experience than miles or millilitres.

Similarly, when used as a unit of distance, a block isn’t precisely or explicitly defined; it’s based on a consensus intuition of what a typical block feels like. My intuition/experience (from a decade living in northeastern North America, mostly Pittsburgh) is that usage of block as a generic unit of distance means something more like a long block (typically East–West) than to a short block (N–S). But I guess this may well vary between regions/communities; and since it’s an informal measurement, it may be hard to find more authoritative sources on the usage.

It's true that people usually don't specify what type of block they mean because it often isn't important to be that precise, as the other answers say. In the case where more precision is desired, you differentiate the two by calling them "short blocks" or "long blocks".

Blocks equals

local streets !

Once I was in Manhattan, New York and I asked directions. I was told the destination was about 8 blocks. No problem I thought. 15 minutes later I realized how big some of those blocks were!

At a later time I was in a Colorado town. My destination was said, by a local, to be 8 blocks away. Oh-oh I thought, might need a cab (thinking of manhattan) as my time was very limited and 15 mins each way would be too much time. Turned out 8 blocks was 4 minutes away via a short walk cos the blocks were tiny.

So when considering blocks for a walk, local facts will count a lot.

Another conclusion (the thing that draw me to answer) is that, from the above, it will be clear that there is a large ('infinite') number of possibilities depending on many factors, not limited to but including, the one I mentioned above.

So five blocks can mean a short dis­tance to walk or it can mean a long dis­tance to walk, de­pend­ing on which side of the block one takes.

That only matters if you don't care where you are going. For instance, you receive someone's opinion that a five block walk is good exercise, and doesn't clarify which blocks, in what town, what part of that town and in which direction.

If you're getting directions to some specific location, like "oh, the art museum is five blocks that way", then that settles it. It's five of those big blocks in that direction, and not five of those little blocks in the perpendicular direction that don't bring you any closer to the art museum.