Is there any principle of contract law regarding failure to take a step towards fulfilling a contract?

It depends on the nature of the deposit

The deposit can either be:

1. A condition of the contract, or
2. A condition precedent to the contract.

If 1., then the contract has been formed and the offer cannot be withdrawn because it has been accepted.

The tenderer is clearly in breach and the government can seek damages, likely payment of the deposit and interest if the contract is silent about the consequences of such a breach.

Whether the breach is “material” allowing the government to lawfully terminate the contract or whether the government has unlawfully repudiated the contract (allowing the tenderer to sue and/or terminate) depends on the circumstances (and the skill of each parties lawyers).

If 2., then paying the deposit is a required part of acceptance and must happen for the contract to come into existence. If so, then the offer can be withdrawn because it hasn’t been accepted.

While this is called a “condition precedent”, such a condition can temporary occur antecedent, simultaneous with or subsequent the formation of the contract e.g. a requirement that the deposit be paid within 10 days of notification is still a condition precedent. Contracts can time travel.

Whether it’s 1. or 2. depends on the wording of the invitation and the contract documents and arguing about that is also something that makes lawyers richer.

If the party to whom the offer letter is issued fails to take any step towards fulfilling its end, can it be argued that there was no meaningful acceptance as for all intents and purposes, no part of the offer that was 'accepted' has been performed?

There was acceptance — the successful bidder communicated its acceptance to the government body and therefore made a promise to pay the deposit. All the elements of a legally binding contract are met:

1. Offer

The offer letter from the government body.

2. Acceptance

Self-evident in your question. The successful bidder communicated that it had accepted the offer from the government body.

3. Consideration

Ordinarily, this would be the deposit. In this case, since the deposit was never sent, the consideration is both the promise (from the bidder) to submit the deposit and the counter promise (from the government party) to give whatever the contract stipulates the bidder will get from the government party.

4. Intent to create legal relations

Not evidenced in the question, but we can generally assume this based on the evidence that both sides are commercial parties.

Therefore, can the government body 'rescind' the offer and argue that there was no contract to begin with because there was no 'acceptance'?

No. The offer was accepted by the bidder so there was 'acceptance', and all the other elements of the contract stand too (there was consideration, etc.).

Conclusion

The bidder likely breached the terms of the contract. This could potentially be a repudiatory breach allowing the government body to choose to end the contract if they wanted (by "withdrawing"). At the very least, the government body would be entitled to sue the bidder for breaching the terms, but there is no legal position of "the contract doesn't exist" from my perspective.