From a business point of view, what are some possible reasons that the Knicks didn't do more to keep Jeremy Lin through the Linsanity era?

As you note, your premise is flawed. The Knicks did maximize their revenue relative to Jeremy Lin, at least as far as I can tell - not knowing what would've happened had he stayed in New York, anyway.

In 2012, Lin was a Restricted Free Agent, and Houston, who was an extremely popular team in China due to fellow Chinese player¹ Yao Ming, chose to sign Lin to a "poison pill" contract. The contract they offered was $25 million for 3 years, heavily backloaded - $5M for the first two years, and $15M for the third. New York already had an extremely high payroll at that time, and so would have paid a huge amount in luxury tax as a result of signing him. Further, he was a player who had success over a very small sample size - 26 games over one year.

As it turns out, New York made a (surprisingly, for them) good move in not matching the contract Houston offered. Lin proceeded to become a role player at best, only showing periodically small bursts of his 2011-2012 ability, during the Houston contract. His popularity waned with his failure to succeed, and his jersey sales plummeted after the initial burst after signing with Houston.

The Brooklyn Nets seemingly tried what you suggested; after a barely over replacement level year in Charlotte, for some reason they offered him $35m over 3 years. I didn't see him on the 2016-2017 jersey sales top lists, and he didn't produce much of anything.

I'm also not so sure why you think the Knicks could have, or should have, traded Carmelo Anthony. He was making $20m/year then, and was one of the most explosive players in the game - except that he had a history of burning coaches, and he was coming off his worst year in a while, making him harder to trade than usual. His jersey sales were incredible every year - go back to that 2016 list, he came in at #15, and that was typical if maybe even low for him for most years in that era. Anthony's popularity, and skill, far exceeded Lin's, except in a very short period of time.

Lin signing in Houston was probably the Pareto optimal result, to borrow an economics term; he signed with a team very popular in China, and so his 2012 Houston jersey sold like hotcakes. Houston made a lot of money, and the Knicks avoided sinking $25m (plus more from luxury taxes) into a player that was worth maybe half of that, and would quickly dwindle in popularity once it was clear he was a flash in the pan, and no more than a role player going forward. (And, all in all, he did okay for himself ultimately - winning a championship with Toronto, and banking $65MM.)

¹_{Yes, Lin is Taiwanese, but that did not particularly limit his popularity in mainland China, which considers Taiwain to be a part of China in any event.}