Engine won't turn over without key fob

I have a 2006 Honda Accord. The key fob broke off and I tried starting my car without it, but the engine does not turn over. It works fine with my spare key.

The obvious explanation is that I’m not applying enough torque, because there’s only a little stub of a key sticking out. But I really am cranking it all the way, so it seems hard to believe. Is there any other reason my key wouldn’t turn over the engine without the fob?