Amine vs Amide Solubility

The mixing of two compounds is a process which requires consideration of three types of interactions: solute-solvent, solvent-solvent and solute-solute. You make a good argument for the solute-solvent interaction being stronger in mixtures of butanamide and water, compared to butylamine and water. However, as you say, this argument alone would predict a higher solubility for butanamide, which goes against the observed data. Also, in both cases the solvent-solvent interaction is the same (water with water), so this can't really be the source of an explanation.

By exclusion, we're guided towards considering the solute-solute interactions. At a first glance, both butylamine and butanamide display a combination of dipole-dipole and hydrogen bonding interactions, making it tricky to compare how their relative strengths change between compounds.

To make things easier, we can refer to additional experimental data. A nice way to gauge the strength of intermolecular interactions in a substance is to look at its melting and boiling points. According to Wikipedia, the melting and boiling points of butylamine are approximately -49 °C and 78 °C respectively. Meanwhile, for butanamide, the values are a whopping 115 °C and 216 °C! Clearly the mutual interactions between butanamide molecules are much stronger than those between butylamine molecules.

And there is our answer. You are correct that, compared to a butylamine molecule, a butanamide molecule should interact more strongly with water molecules. However, it turns out that butanamide molecules also interact very strongly with themselves, far more so than butylamine molecules. Taking everything into account, the effect of the solute-solute interactions is more prevalent; the water molecules have more difficulty keeping butanamide molecules apart, to the point that after a critical value of 163 grams of butanamide in a liter of aqueous solution (at 15 °C), the water molecules just can't stop the butanamide molecules from meeting and packing into a solid. Even if you add more butanamide, it doesn't visibly dissolve. Some butanamide molecules in the solid continuously manage to break loose from each other and get pulled into the aqueous solution, but they are replaced just as fast by molecules of butanamide already in solution which get too close and pack into the solid state.

You will find that this can be made into a very rough rule of thumb: when comparing the solubility of two roughly similar substances in a same solvent, if one of the substances has significantly higher melting and boiling points, they are likely to be less soluble.

Note that I have discussed only the influence of the enthalpic contributions to mixing. An entropic contribution also exists (favouring dissolution for both compounds), but in this case it can be safely neglected due to its comparatively small effect.