Why is the geometry package messing my margins up?

I want to write a document making use of as much of the page as I can. The code I am using is this (disclaimer: I took the sample text from here):

documentclass[12pt, oneside, a4paper]{memoir}

usepackage{layout}
usepackage{showframe}

%usepackage[a4paper]{geometry}

usepackage{amssymb, amsfonts, amsthm}
newtheorem{problem}{Problem}

begin{document}
title{Sample Document}
author{John Doe}
maketitle

layout{}

Here is some sample text to show you what LaTeX does. 

To start a new paragraph, you need to leave a line of white space in your tex file.

To include math, you have two options. The first is called ``in line,'' and you do this by putting your math between two dollar signs ($). For example, Fermat's Little Theorem tells us that if $p$ is a prime and $a$ is an integer such that $p nmid a$, then $a^{p-1} equiv 1 pmod{p}$. Note that the exponent on $a$ has to go in braces (in the tex file). 

But if you really want an equation (or congruence) to stand out, then you ``display'' the math. It looks like this:
[
    a^{p-1} equiv 1 pmod{p}.
]
Nice, right?

Finally, we can make things look really nice for homeworks as follows:

begin{problem}
    Let $X$ and $Y$ be blah blah blah ldots
end{problem}

begin{problem}
    Let $X$ and $Y$ be as in the previous problem. What is $Z$?
end{problem}

Notice that LaTeX automatically numbers the problems for us. 

end{document}

It looks nice, but with a lots of unused paper on the sides:

However as soon as I uncomment the line invoking the geometry package the margins get messed up, even though I haven’t even issued any command to alter them.

I tried looking at some sources but they do not mention anything like this happening, so I would like to understand why geometry is behaving that way.

Also, if possible, some help getting the code right to leave as little unused paper space as possible.

% memgeomprob.tex  SE 562072

documentclass[12pt, oneside, a4paper]{memoir}

setlrmarginsandblock{0.8in}{*}{1.0} % set left/right margins to 0.8in
setulmarginsandblock{1in}{*}{1.0}   % set upper/lower margins to 1in
checkandfixthelayout

usepackage{layout}
usepackage{showframe}

%usepackage[a4paper]{geometry}
%usepackage{geometry}

usepackage{amssymb, amsfonts, amsthm}
newtheorem{problem}{Problem}

begin{document}
title{Sample Document}
author{John Doe}
maketitle

layout{}

Here is some sample text to show you what LaTeX does. 

To start a new paragraph, you need to leave a line of white space in your tex file.

To include math, you have two options. The first is called ``in line,'' and you do this by putting your math between two dollar signs ($). For example, Fermat's Little Theorem tells us that if $p$ is a prime and $a$ is an integer such that $p nmid a$, then $a^{p-1} equiv 1 pmod{p}$. Note that the exponent on $a$ has to go in braces (in the tex file). 

But if you really want an equation (or congruence) to stand out, then you ``display'' the math. It looks like this:
[
    a^{p-1} equiv 1 pmod{p}.
]
Nice, right?

Finally, we can make things look really nice for homeworks as follows:

begin{problem}
    Let $X$ and $Y$ be blah blah blah ldots
end{problem}

begin{problem}
    Let $X$ and $Y$ be as in the previous problem. What is $Z$?
end{problem}

Notice that LaTeX automatically numbers the problems for us. 

end{document}