What stories are connected to Lord of the Rings and The Hobbit?

There are a lot of books about that, but you can start with The Silmarillion, a compilation of stories from the creation of the world to the age of TLOTR.

This Wikipedia entry will direct you to the source of most of the info. They are books that include Tolkien's wider works in the universe within which the Lord of the Rings and the Hobbit are set, alternative versions, works in progress, and academic discussions on his writings. There's also Tolkien's letters. And the Silmarillion, as mentioned in another answer. If you're really keen, Marquette University in Milwaukee hosts an archive of Tolkien's manuscripts amongst other related material.

Tolkien's Legendarium was a lifelong work, and indeed the work has continued long after his death. The Hobbit, written in 1937, was the world's first public look at his mythos, and he followed soon after by starting the writing of The Lord of the Rings (published 1954-1955). The only other piece dealing with Middle-earth published in his lifetime was The Adventures of Tom Bombadil (1966), a collection of poems dealing with Tom, some of which appear in The Lord of the Rings.

Tolkien died on September 2nd, 1973, and control of his writings passed to his son, Christopher Tolkien, who he appointed his literary executor. A number of unfinished writings have been published posthumously, being edited and arranged by Christopher Tolkien. Most notably, these include The Silmarillion (1977); The Children of Húrin (2007), being an expansion of a major story contained in the Silmarillion; and Unfinished Tales of Númenor and Middle-earth (1980), a collection of stories in various stages of completion.

Much like Unfinished Tales, The History of Middle-earth (often stylized HoMe), a 12-volume set published between 1983 and 1996, contains many of Tolkien's notes, drafts, and ideas for the formation of the world of Middle-earth. Edited by Christopher Tolkien, they provide insight into some of the thoughts behind the writing of the Middle-earth Legendarium.

Additionally, Tolkien's Letters deal with The Lord of the Rings in many places.

Your question is sufficiently vague, so I will also add Edda, Parsifal, Beowulf, and Nibelung saga to a list of works that are connected to the Lord of the Rings :).

More practically, apart from Silmarillion, recommended above, I would suggest preferentially reading Unfinished Tales and Lays of Beleriand. But you have to accept that those are books rather different from the Hobbit and LoTR.

The Silmarillion is both the history and the mythology of the world of LoTR and it's essential for enjoying the vastness and beauty of Tolkien's creation. If, reading LoTR, you had a feeling that the stories are much deeper, that there are undercurrents to the story of four hobbits, you were right. And that background is in the Silmarillion.

Stylistically the Silmarillion resembles a chronicle or the Scripture - there's a great amount of names, dates, events and interpersonal relations there, together with a creation mythos, gods and demigods, overall story spanning thousands of years, hatred between elves and dwarves and among elves themselves, epic battles, vain victories and bitter defeats, love and betrayal, and several beautiful and moving stories. It's a different experience than LoTR, and much more difficult, but very rewarding.

The Lays of Beleriand is the third volume of HoME and it mostly comprises of two long poems, thousands of verses each, telling the two principal stories from the Silmarillion, about the children of Hurin and about Beren and Luthien. Aragorn singing to the hobbits at the Weathertop in LoTR might have been quoting from the second one.

Unfortunately, both poems are unfinished, just like the Unfinished Tales.