What was the earliest example of a riddle or puzzle being used as a security system?

An early fictional example is Jules Verne, Voyage au centre de la Terre (Journey to the Center of the Earth) from 1864.

In this book, the (fictional) 18th century alchemist Arne Saknussemm makes an important discovery, wants to hide it for a while but preserve it. Saknussem was killed by the Inquisition, and his books were destroyed. Professor Lidenbrock has founds a message from Saknussem, protected by weak cryptography, on a slip of paper that was hidden in a book that wasn't written by Saknussem. Lidenbrock presumes that Saknussem did this to preserve his discovery from the Inquisition destroying it.

I believe this qualifies to your question. The protagonists are generally educated, but they didn't have any special knowledge or connection from Saknussem. They found and decoded the message despite that. They had to break cryptography for this, which counts as a puzzle. The Inquisition did not find the hidden message. Saknussem and Lidenbrock both believe that the information about the discovery is valuable and should be protected: Lidenbrock specifically orders Axel to keep the information secret until they can profit from it.

I believe that this story was influenced by similar real-world examples of 17th century scientists, most famously Galileo in 1610, who published their discoveries hid behind a riddle, with the intention of later proving that they discovered something first, while hiding the actual discovery so that they can profit from it first.

The later novel La Jangada (1881) by Jules Verne also use weak cryptography, and may count as an example. According to the novel the criminal Ortega writes a confession of the diamond robbery that they organized, because, when he is deathly ill, he feels regret about how the protagonist Noam Garral got falsely accused and committed for the robbery. He encrypts the confession for reasons unclear to me. He entrusts the writing to a friend Torrès, gives him the key for the encryption, and asks him to deliver the writing to Garral after Ortega's death. Torrès instead blackmails Garral (or so Garral claims) for the writing that would save them from being a wanted criminal. Eventually Garral's friend Fragoso retrieves the document, and the puzzle lover judge Jarriquez breaks the encryption with his help, thus proving Noam Garral's innocence.

I will mention for completeness that a third Jules Verne novel, Mathias Sandorf also has information hidden in an encrypted document as a key plot element. This, however, does not seem to be intended as an example in universe, because the only way that the antagonists Zirone and Sarcany manage to get the information is to steal the physical form of the encryption key from the house of the intended recipient.

I'm going to add this as an answer, following on from my comment earlier. I'll go with Lord of the Rings - Part 1 The Fellowship of the Ring, originally published in in July 1954.

The particular part that relates to OP's question is chapter called "Into the Dark". In this chapter, Gandalf has reluctantly assented to guiding his companions in the Fellowship through Khazad-dûm (later called Moria).

To enter Khazad-dûm, the party ventures to the West Gate (also called the Doors of Durin). To open the gates, the party needs to answer the riddle carved into them by the builders back in the second age. The riddle is as follows:

The inscription on the archivolt over the Doors read:

"Ennyn Durin Aran Moria. Pedo Mellon a Minno. Im Narvi hain echant. Celebrimbor o Eregion teithant i thiw hin."

As Gandalf first translated it to the other Walkers:

"The Doors of Durin, Lord of Moria. Speak, friend, and enter. I, Narvi, made them. Celebrimbor of Hollin drew these signs."

Initially Gandalf could not find out the password to open them. Merry Brandybuck unknowingly gave Gandalf the answer by asking, "What does it mean by speak, friend, and enter?" When Gandalf realized that the correct translation was "Say friend and enter" he sprang up, laughed, and said "Mellon", which means "friend" in Sindarin, and the Doors opened.

There may be earlier literature that has similar circumstances or situations, (I haven't found any yet) but to my mind the Dwarfs wanted to protect themselves, their riches under the mountain and their trade routes and the inscription over the Doors provides the password reminder to anyone that may of forgotten, this seems to meet the requirements of OP's question.