Why did Dumbledore say that even his most protective spells weren't invincible against Voldemort at full power?

Dumbledore wasn't talking about the protecting a place or an object. He was talking about protection spells for Harry, in the prospect that Voldemort would return.

‘My answer is that my priority was to keep you alive. You were in more danger than perhaps anyone but I realised. Voldemort had been vanquished hours before, but his supporters – and many of them are almost as terrible as he – were still at large, angry, desperate and violent. And I had to make my decision, too, with regard to the years ahead. Did I believe that Voldemort was gone for ever? No. I knew not whether it would be ten, twenty or fifty years before he returned, but I was sure he would do so, and I was sure, too, knowing him as I have done, that he would not rest until he killed you.

I knew that Voldemort’s knowledge of magic is perhaps more extensive than any wizard alive. I knew that even my most complex and powerful protective spells and charms were unlikely to be invincible if he ever returned to full power.

_{Harry Potter and the Order of the Phoenix. Chapter Thirty-Seven: The Lost Prophecy.}

You hardly can use the Fidelius Charm in a person. Even if it was magically possible, which I don't know, you would've condemning that person to a life of isolation where no one could interact with him. Dumbedore might have compromised with the Dursleys, but he wouldn't turn Harry invisible and unhearable for everyone else.

As for the rest of Dumbledore's protection spells... As he says in the quoted text, Voldemort wouldn't rest until Harry was dead. So, Dumbledore could've tried to protect Harry with his best charms, but Voldemort would've eventually break through them.

That's why Dumbledore decided to use the "love magic" which was put in place by Lily. Because it was the only magic that Voldemort didn't fully understand, and therefore the only magic that he wouldn't be able to break. The Durlseys were the only safe option for Harry in the long run. That's the sole reason why Dumbledore let Harry to grow miserable in a home with those horrible people, because Harry wouldn't be safe in any other place.

---

As for your second part of the question...

The Fidelius Charm works nicely if you want to keep the secret safe and only share it with a few people.

But let's imagine that Dumbledore would've tried to protect Hogwarts with the Fidelius Charm, being him the Secret Keeper.

1. Voldemort (and pretty much everyone in the United Kingdom) already knew where Hogwarts was. The Fidelius Charm wouldn't have erased that knowledge. It would merely hide Hogwarts, but it would be pretty simple for Voldemort to keep a bunch of Death Eaters in the door waiting for the students each 1st September. Not pretty good.
2. It would be hard to share the secret only and only with the students and the staff. Let's suppose that they're just 280. Dumbledore would have to personally share the secret with each of them, making sure that no one else overhears it or somehow discovers it from him. It would be harder as the number goes higher (J.K. Rowling said there were around 1,000 students).