I need help in neighborhood dispute about feeding a stray cat

Encouraging stray cats like this is not a good thing. Feral cat populations can do masses of damage to the local wildlife. The food you leave out will also influence other animals, e.g. the racoon. I understand your neighbours' perspective and desire that you stop.

On the other hand, what decent person wants to see any animal suffer and starve? Not you or I!

1. In the immediate future, try feeding the cat in the manner suggested by lila in the comments.
2. Unless you are absolutely certain the cat is a stray, attach a paper collar reaching out the owner, e.g. https://www.spca.nz/images/assets/31718/1/spca2018_stray_cat_paper_collar_d3.pdf.
3. If there is no owner, the cat needs to be taken in. A vet or rescue shelter should be able to give the wee fellow a health check, neuter him if necessary, and then give him a chance at finding a secure home where he can be safe, warm and well fed. Go down this route and you may find yourself with a furry little addition to your household, or the satisfaction of knowing that you have converted a nuisance on the streets into a fellow creature having a vastly improved quality of life with a family of his own.

Letting a living animal starve to death is cruel, but once again I agree that you cannot choose easily. One way is to adopt him and take him into the vet’s, which costs a lot of money and you must be able to take him in. In the vet’s you must check his health and you must get him a collar. Doing this is expensive. In some countries, they neuter wild cats so they don’t reproduce.

You could take him to a rescue centre, but if he is unwanted, he may be euthanized, depending on the country, so he might die unless someone wants him.

You could offer it to a good friend of yours, but that would require responsibility of your friend.

Overall, I don’t think you should let these ladies boss you around or let the cat starve.