Why do liquids exert pressure on the sides of a container?

The reason is because liquids don't have preferred directions.

It's true that if you squeeze a regular solid by pushing down on it, it'll push back up on your hand but it won't exert any force to the sides (though it might bulge out a bit). If you model a solid as a cubic lattice of masses connected by springs, this makes sense, because only the vertical springs get compressed. A solid has enough order to 'remember' which way you pushed on it.

A liquid doesn't have this long-range order: pushing down on a block of water just makes the water shear to the side. On a microscopic level, you can't push the atoms together only in one direction because directional correlation decays fast; all you can do is push them together in general. Then the liquid responds by pushing out in all directions too.

Your example with a bin of baseballs is in between the two cases, but I think it's closer to a solid. Most of the balls are locked in place by the weight of the balls above, making a lattice. If you pop a small hole in the side of the bin, the balls won't flow out, they're jammed.

In a liquid (or any fluid), the molecules are in random motion (best to say is vibration). So each molecule is vibrating and hence collides with each other. Likewise the molecules in contact with the container also collides with the container walls. Assuming perfect elastic collision, the collided molecules are pushed backwards as insisted by Newton's third law. Under static condition (i.e., no external force acting on the liquid) the liquid exerts force on the wall perpendicular to the surface area. The amount of how much force molecules exert on a unit area is what we call pressure. The force exerted by molecules under no external force will be equal to the reaction force exerted by the container walls. So pressure is actually a consequence reaction force.

Additional : The pressure exerted by solids is a little different from that of fluids as in the case of solids, due to tight interactions of atoms/ molecules, the solid body feels a tension that causes it to deform. But in the case of fluids, the pressure on them sets them in motion, as they have no definite shape to be deformed.

Unless the balls were perfectly stacked they would be rolling off one another and the walls of the bin would stop them.

Even in zero gravity fluids will exert an equal pressure on all walls. In zero gravity the balls would continue to remain stationary without exerting any force on the surroundings. So just thinking in terms of basketballs isn't enough.

The best answer I can give is extends on what @knzhou said, liquids do not have a preferred direction. If you press down slowly on a piston containing a fluid*, the particles will continue to move in all directions at the same velocity. However, the distance the particles move between the top and bottom surfaces decreases, hence the frequency of collisions with the sidewalls will increase and the force exerted, $F$ will increase. At the same time the area of the side-walls, $A$, decreases. Pressure = $F/A$ so the pressure on the side walls increases.

*(so that the temperature does not change)