Most efficient cheap heating system for small apartment in the US midwest?

I'll assume for now the earlier answer on indoor heating systems that you linked remains correct, and address two other points.

---

Heat pump efficiency and temperature

Engineering toolbox provides a thorough explanation of heat pump performance, which includes this chart:

A few definitions:

• COP stands for coefficient of performance, and is a measure of heat pump efficiency (higher is better). It is the ratio of energy input to useful heat output. For home heating, COP at rated conditions is typically around 3, meaning that for each unit of electricity used, three units of heat are provided. By contrast, baseboard electric heat has a COP of exactly 1.
• Cold temperature is the outdoor/ambient air temperature -- this is the the source where the heat pump will extract heat
• Hot temperature is the temperature to which the heat pump needs to heat. This is typically higher than your thermostat setting, because heat is coming in at one or a few places, but needs to propagate throughout the living space

The highest COP occurs when the hot temperature is lowest, and the cold temperature is highest. To use a pumping analogy, it's like moving water up a short hill. The colder it gets outside, or the hotter we set the system inside, the higher the hill that the water needs to be pumped up.

In your case, since you plan to keep the indoor temperature lower than the "standard" 65 degF (18 degC), a heat pump will do better than for the average home. (However, I'd recommend taking a look at this question: Potential health risks with keeping temperature in a house low?.)

Cold weather performance

The COP/temperature relationship results in one of the biggest challenges with heat pumps -- at low outdoor temperatures, they start to perform poorly. In the US Midwest, the outdoor temperature can drop to 5 degF / -15 degC on a regular basis.

NEEP (an energy efficiency organization in the U.S.) provides a directory of air-source heat pumps (ASHPs) and their performance. Here's one example for a smallish system:

At 5 degF (-15 degC) the capacity (amount of heat the system can provide) is about half of what it is at 47 degF (8 degC), and the COP drops from 3.77 to 2.29. When it gets really cold outside, this system will have high electric use, and depending on the size and insulation of the home, it might not be able to keep it warm all winter long.

There are a few solutions to this:

• Buy a heat pump that has enough capacity to heat your home regardless of how cold it gets.
• Add a back-up heat source that only runs when the outdoor temperature is really cold.

In your case, since cost is an important factor, I'd recommend leaving your baseboard heating in place to use as a back-up source.

Typically, heating systems are sized to be sufficient for 99% of the hours in an "average" winter (see this Energy Star guide which tells you the 99% temperature for selected cities in the U.S.). With an existing back-up system, you could design for 95% or even 90%, and use your baseboard heat to cover the remaining 5-10% of winter hours. This would save significantly on the up-front costs since you could use a much smaller heat pump.

Setting up optimal controls might be a challenge depending on the ASHP you choose, but in a pinch you could just set the baseboard heater thermostat a few degrees lower than the heat pump thermostat -- once it gets too cold for the ASHP to keep up, the indoor temperature will drop and the baseboard heaters will come on. (This isn't ideal as the ASHP will keep running and you might get into a feedback loop. A better system would monitor outdoor air temperature and switch heat source accordingly.)