Is using the plural form (in place of the original singular) of these Latin/Greek loan words acceptable?

We all make mistakes. Mistakes should be forgiven.

But I'm sure most style books do not consider these acceptable. A basic rule of style is as follows: do not use a 'sophisticated' word unless you know how to use it.

Certainly many pedants still cringe at seeing such mistakes, which is reason enough to say it is not acceptable, even if a large number of people in academia should fail to notice.

Many words borrowed from another language do not retain their prior language's number or case structure in English. Singular or non-count bacteria and criteria are frequently used with verbs in a singular form, and are at least somewhat assimilated into standard forms of English. Singular or non-count millennia and phenomena rarely appear with verbs in a singular form, and thus appear to still be anomalies.

Bacteria

It is possible to refer to a single strain as a bacterium (the Oxford English Dictionary lists bacteria under bacterium). Note, however, that bacteria is far more common in use than bacterium (Ngram):

The Corpus of Contemporary American English also has several results for "bacteria _VVZ" (bacteria with a verb ending in -s). The official status of some of these examples leads me to believe bacteria is an acceptable form of the singular in many uses:

The bacteria grows under fluorescent light. (Cain Burdeau. "LA scientist's oysters safe from oil, but pricey." Associated Press, 20 Aug. 2010.)

Bacteria grows, you ingest it, you become ill. It's no secret. ("Behind the Counter - Restaurant Filth Exposed." ABC Primetime, 27 Sept. 1995.)

What's frightening is that the vibrio bacteria occurs naturally in warm seawater, unrelated to pollution. (ABC 20/20, 9 February 1990.)

But if we do know that bacteria causes, you know, gum disease and cavities, why not just do something right there? ("Potential links between chronic disease and bacterial infections." NPR Science, 26 February 1999.)

AP, ABC, NPR - a lot of news organizations use bacteria with a singular verb, especially in spoken contexts. That leads me to conclude it is widely accepted in standard language, even if some prescriptivists still wince upon hearing it.

Criteria

Criteria has become more common through the 20th century.

It has also been used with -s verbs in publication and speech, with several significant results from COCA:

For example, the first criteria includes the use of creatine and artificial low-oxygen living environments. (Matthew Mitten. "Is drug testing of athletes necessary?" USA Today Magazine, vol. 134, iss. 2726, Nov 2005. )

The selection criteria includes academic achievement, leadership and community involvement, and students must graduate from one of the participating senior high schools in Brazoria, Chambers, Fort Bend, Galveston, Harris, Liberty, Montgomery or Waller counties. ("Katy American Little League^s Astros take first place in 8-year-old play." Houston Chronicle, 12 July 2000.)

The same criteria applies to next month's debates. (PBS NewsHour, 28 August 2019.)

So at least in some official uses, criteria used with a singular verb is acceptable.

Millennia

The singular form remains more common, especially near the turn of the millennium.

No significant results for millennia with -s verbs in COCA. In my experience, millennium is still overwhelmingly used for the singular.

Phenomena

The Ngram indicates more recent growth in the use of phenomenon compared to phenomena.

In COCA, there are a couple of examples of publications using phenomena with an -s verb, but far fewer than for bacteria or criteria. I suggest that this remains uncommon, and phenomenon remains generally recognized as the singular form of the word:

I think it's fair to say that the phenomena exists from just off the coast of China all the way to a few hundred miles from the coast of California. (PBS News Hour, 13 November 2008.)

The tracking phenomena proposes that these V0 2 max levels at age 20 project the individual to be in the low-risk CVD category at age 45. (Bradley Cardinal. Cardiorespiratory fitness and physical activity behavior ..." Physical Educator, vol. 52, issue 4, winter 1995.)

I agree with your opinion. The question of frequency of usage, although informative, is not the main issue. What matters is the creeping corruption of meanings. This not to do with pedantry but with the retention of useful words and the avoidance of unnecessary neologisms. If we name a singular thing by its plural we lose the capacity to talk of the single thing and we lose the capacity to discriminate between the plural and the singular. This, in the case of the Italian panino/panini misuse leads on to the gruesome plural paninis. Similarly, how long before a group of some of the professor's singular phenomena becomes his phenomenas?

Here are some interesting results (I found a new toy)

We live in uncertain times:

The American-English stats, however, show that the bacterium seemed to die out, but was somehow revived:

Not all hope is lost for Italian, too:

The criteria also applies much less now than in the 1980s