What are the arguments against abolishing the Confederate flag?

First of all, you seem to be missing that the Confederate flag meant and continues to mean different things to different people for different reasons.

Like most things in history, the reasons for the American Civil War were more complicated than the version taught in primary or secondary school textbooks. Slavery was certainly the most central issue, but it was far from the only one. Four states (Virginia, Tennessee, North Carolina, and Arkansas) had rejected secession purely on the basis of preserving slavery, but then voted to secede after Lincoln called for them to send tens of thousands of troops to the Union army for an invasion of the Southern states which had seceded, for example. In a Feb 9, 1861 special election on the issue, Tennesseans rejected secession by a factor of almost 4 to 1. Sentiment turned dramatically in response to the April 15 order to send troops and another vote was held on June 8 with the state voting in favor of secession.

Thus, the meaning of the war and, therefore, the flag would be quite different to those in border states who primarily just didn't want to be invaded or go invade their neighbor states vs. those whose primary goal was actually the preservation of slavery.

From the other side, even President Lincoln himself had this to say about his reasons for the war in his letter to Horace Greeley, editor of the New York Tribune (emphasis in original):

I would save the Union. I would save it the shortest way under the Constitution. The sooner the national authority can be restored; the nearer the Union will be "the Union as it was." If there be those who would not save the Union, unless they could at the same time save slavery, I do not agree with them. If there be those who would not save the Union unless they could at the same time destroy slavery, I do not agree with them. My paramount object in this struggle is to save the Union, and is not either to save or to destroy slavery. If I could save the Union without freeing any slave I would do it, and if I could save it by freeing all the slaves I would do it; and if I could save it by freeing some and leaving others alone I would also do that. What I do about slavery, and the colored race, I do because I believe it helps to save the Union; and what I forbear, I forbear because I do not believe it would help to save the Union.

-- President Lincoln, excerpt from Letter to Horace Greeley, Aug 22, 1862

It's completely understandable why abolitionists and, especially, slaves and former slaves would have seen the Confederate flag as a symbol of racism and slavery. That was, after all, the primary reason for the secession of the Deep South states and the central issue of the division that ultimately led to the war. However, it's also understandable that not everyone saw it that way. A Tennessean or Virginian who had voted to remain in the Union after the Deep South states seceded, but then supported secession after Lincoln's order for troops would understandably have a rather different view of it, for example.

Nowadays, as Brian Z's answer points out, there is also a significant group of people in the South who view the Confederate battle flag as a symbol of the region, its culture, and its history moreso than of the actual Confederacy itself. At least in my experience as someone in my 30s who lives in the South, this is (by far) the primary reason I've heard from those who have flown a Confederate battle flag in my lifetime for why they've done so. For those who fly it, it's more a symbol of regional pride than anything else, not dissimilar to cheering for SEC (American) football teams.

---

Second, free speech really is the real reason that there isn't and hasn't been any sort of ban on the display of the flag. The Constitution of the United States explicitly protects freedom of speech in the First Amendment:

Congress shall make no law respecting an establishment of religion, or prohibiting the free exercise thereof; or abridging the freedom of speech, or of the press; or the right of the people peaceably to assemble, and to petition the Government for a redress of grievances.

There's not much grey area there. This is by design. Allowing private display, but not public display, of a flag or political symbol would most certainly not be within the bounds of the Constitution, nor would almost any limitations on such public display be legal. Whether you want to fly an American flag, a Confederate battle flag, a Union Jack, a Nazi flag, a hammer and sickle, or any other flag, it is perfectly legal to publicly do so in the United States and any law whose purpose is to the contrary would be unconstitutional.

I caveat that in regards to purpose because laws with a purpose unrelated to restricting speech might be permissible if they are the least restrictive means to accomplish the permissible goal. For example, a law preventing you from flying any sort of flag from a 1,000 foot pole right off the end of an airport runway would be permissible because its purpose is a permissible one (aviation safety) and there isn't a less restrictive means of accomplishing that purpose.

Two caveats before I answer... First, contrary to popular opinion, it would be possible for the Federal government to ban display of the Confederate flag, completely within the bounds of the constitution. Free Speech is not unlimited; speech that is deemed to be seditious or is meant to incite harm or violence can be restricted, as various court cases have demonstrated. Since the Confederate flag — technically, the main Confederate battle flag, since the Confederacy never established its own sovereign flag — was raised against the Union violently during the Civil War, it could be viewed as both seditious and inciting. I'm not certain why it wasn't banned at the end of the Civil War: perhaps because it was merely a battle flag, and no one thought it would be anything more than a historical curiosity? At any rate, the point stands, even if it is an unlikely outcome at this point in time.

Second, I personally feel that the Confederate flag ought to banned, both for that reason and because of its strong association with colonial white supremacist ideology. As I see it, that flag is too intractably linked with the history of slavery to ever be 'clean'. Just so my biases are clear.

As to why and in what contexts the Confederate flag might be displayed... It's worth bearing in mind that a tremendous number of soldiers died defending the Confederacy, and for all that we can condemn the ideals and actions of the separationists, the soldiers themselves were Americans, and deserve to be honored for their sacrifice. Flying the Confederate battle flag over a Civil War military graveyard, or over monuments dedicated to the fallen, seems a perfectly appropriate gesture of respect. The important point here is to distinguish the heroism of those who fought for their side without simultaneously lionizing the cause they fought for.

Further, the deep South does reflect a culturally distinct group that has as much right to rally itself as any other culturally distinct group. Even before the nation was formed, the Southern seaboard was a significantly different demographic than the Northern. The Northern colonies were largely people by emigrant groups like the Puritans, who were particularly entrenched in Liberal ideology and carried a deep dislike and distrust of Old World culture and values. The Southern colonies kept a much closer connection to the Old World, maintaining a number of 'aristocratic' values (e.g., cotillion, duels of honor...). Further, the Southern colonies were primarily agricultural — from the big plantations of the flatlands to the 'homesteaders' of the Appalachians — while the Northern seaboard leaned more heavily towards commercial, industrial, and maritime activities. This made the North generally more cosmopolitan than the South, with all the respective impacts on outlook that entails, but left the South more genteel. There is a 'Southern culture' that's worthy of respect, to the extent that it can be separated from the ugliness of slavery, and a flag honoring the South is no more (and no less) objectionable than a Pride flag that celebrates LGBTQ, or the flag of some other nation used to celebrate cultural heritage.

The problem, of course, is that a certain group of Southerners uses the flag that was specifically associated with insurrection and the defense of slavery. There are historical/political reasons why that flag was chosen, obviously. The flag is seen as a symbolic rejection of 'Northern' culture; I suspect most Southerners are being honest when they say they don't see the flag as a symbol of racism per se, because they are seeing it more as a middle finger to Northern busybodies who think they know better how things should be run. But sooner or later they will have to find another symbolic outlet for that frustrated aggression.

The chief problem with the question appears to be that it omits the actor. Who should abolish the Confederate flag?

As the question already notes, the US government is not the answer to this question. It does not have the right to limit speech in this way.

The US government itself does not fly the Confederate flag. It has also banned the use of the Confederate flag by its soldiers (who, being soldiers, do not have have the same rights to free speech).

US states cannot abolish it either. The Privileges or Immunities Clause of the 14th amendment restricts them in much the same way as the Federal government.

Now the question specifically asks for other arguments. Other arguments do not enter into the equation for the governments. That's logical: before you answer the question "Should we do X", you must first affirm that you in fact can do X. Governments can't get past that first test because of the freedom of speech.

Other actors can ban the Confederate flag. Many do. A recent example is NASCAR. So it does appears the other arguments are indeed good enough for many private parties. These other parties may still consider freedom of speech to be valuable, but for them it factors into the "should we?" question, not in "can we?".