Meaning of 'Make things as simple as possible, but not simpler'

It means that one should find the simplest way of explaining or expressing something, that's just short of oversimplifying.

There. I hope I have managed to follow his dictum.

This is a variant of Occam’s Razor.

If you cannot understand it in English, perhaps the Latin will help you:

Entia non sunt multiplicanda, praeter necessitatem

I think he was saying two things:

1. Make sure your statements are true (science).
2. Find the simplest human language to describe the thing completely.

If 0 is complicated, 10 is the simplest description, 11 might be the step you are removing information/adding false information to make it "simpler", thereby departing from the underlying truth of what you are trying to describe (and moving from science to fiction).

I think he is saying "10 is as simple as possible, but to go to step 11 you need to make the underlying thing simpler (and in the process factually incorrect)".

Started as a comment but would not fit the last 10 chars;

Though it can provide some insight it is more of a Buddhist koan than advice or a dictum. It looks just silly if you apply it to anything else; You should make as much money as you need but no more. You must walk as far as you should but no further. It is like the phrase "Form follows function" [Wright]. It is obvious to engineers but since it flows trippingly from the tongue it sounds like genius to anyone else. I'm Very! pro-Einstein but this quote gets a bit more play than I would expect. What he failed to say was "Always have a picture to use for your theories." Had he one in the end he might have got further on the unified field theory.

It is useful to remember that once he was found studying a sharp pencil. When asked why he said, "So simple!" There is the lesson I think.