Does the Chinese Consulate in Houston have to comply with the closure order?

https://www.justice.gov/file/22361/download

The President has inherent constitutional power to declare foreign diplomatic personnel persona non grata and to expel them forcibly from the United States; the exercise of this power is consistent with international law, including specifically the Vienna Convention on Diplomatic Relations.

Inherent in the President’s power to recognize foreign countries and their ministers is implied power over the physical premises of diplomatic properties, including power to take actions necessary to protect embassies from damage, and to deny possession to or to eject those not recognized as diplomatic personnel of the sending state.

Ultimately the territory of a consulate is still the territory of the host country. From relevant research, it seems that the extraterritoriality is an extension of its recognition as an diplomatic mission. It stands to reason that, if it ceases to be recognized as a diplomatic mission, it would thus lose its extraterritoriality.

It is too early to speculate on how this incident will play out but we don't have to look very far for a historical precedent. In 2018, the Russian consulate in Seattle was closed under rather similar circumstances. Sixty diplomats were accused of spying and given 7 days to leave the country. We don't know their identities but they presumably complied. Had they not complied, I imagine they may have have potentially faced arrest and deportation.

Doesn't China have to comply with this closure order?

I'd say so. Per Vienna Convention on Consular Relations, of which China is a signatory:

TERMINATION OF THE FUNCTIONS OF A MEMBER OF A CONSULAR POST

The functions of a member of a consular post shall come to an end inter alia:

(a) on notification by the sending State to the receiving State that his functions have come to an end;

(b) on withdrawal of the exequatur;

(c) on notification by the receiving State to the sending State that the receiving State has ceased to consider him as a member of the consular staff.

If the United States has formally notified China of any of these conditions, China no longer has any right to run a consulate in Houston.

Consulates, in general, exist at the forbearance of the host country.

Per Wikipedia:

Contrary to popular belief, many of the staff of consulates may be career diplomats, but they do not generally have diplomatic immunity unless they are also accredited as such. Immunities and privileges for consuls and accredited staff of consulates (consular immunity) are generally limited to actions undertaken in their official capacity and, with respect to the consulate itself, to those required for official duties.

If the host country has withdrawn permission for the consulate to exist, there are no longer any "official duties" for anyone working there to perform.

If China were to try to continue running the consulate, the US would have multiple options, likely up to and including physically detaining any Chinese officials performing such acts and summarily putting them on a plane to China. Or perhaps even revoking diplomatic immunity (as they would be operating outside defined bounds) and arresting them.

Houston is literally over 10,000 km from China. If the United States wants the Chinese consulate in Houston closed, there really isn't anything China can practically do to stop it.

What can China do? This:

China Orders U.S. to Close Chengdu Consulate as Payback for Houston Move

Now, the United States can't really do anything about keeping its Chengdu consulate open. All the things the US can do to force the Chinese consulate in Houston to close, the Chinese can also do to the US with respect to the US Chengdu consulate.

The Vienna Convention on Diplomatic Relations govern the rules surrounding embassies and consulates around the world.

For your question, Article 12 is the most relevant:

The sending State may not, without the prior express consent of the receiving State, establish offices forming part of the mission in localities other than those in which the mission itself is established.

Most, if not all, countries with whom the US has diplomatic relations have their missions (embassies) located in Washington. Where a country has consulates elsewhere, they exist because the US government has allowed them to exist.

When a receiving country orders a consulate to be closed, they will be given a reasonable period of time (generally a few days) to take the necessary actions to do so, including gathering documents, terminating lease agreements, and arranging for staff to leave the country. While the treaty is silent on consequences, it would not be unreasonable to believe that federal police could raid the building if they no longer recognise it as being a consulate.

In addition to this, Article 9 gives the US the right to expel any foreign diplomat from the US, for any reason or no reason at all. If those diplomats do not leave within a reasonable period of time, then they can be subjected to arrest as any other foreign national.