tfm files in local ~/texmf not found

I am trying to add the gray font to ~/texmf so I can run gftodvi, but I get the following error:

$ gftodvi beta.2602gf
gftodvi: fatal: tfm file `gray.tfm' not found.

I am using TeX Live 2018 installed on Debian like so:

apt-get install texlive-font-utils texlive-metapost texlive-xetex

According to kpsewhich, the local texmf home really is ~/texmf:

$ kpsewhich -var-value TEXMFHOME
/home/<my name>/texmf

I have tried adding the fonts to ~/texmf as follows (including various permutations thereof):

$ tlmgr init-usertree
$ mkdir -p ~/texmf/fonts/pk/ljfour/public/misc/dpi600
$ mv gray.tfm gray.pk ~/texmf/fonts/pk/ljfour/public/misc/dpi600/
$ texhash ~/texmf

When I inspect ~/texmf/ls-R, I can see that the gray font is listed:

$ cat ~/texmf/ls-R | grep gray
gray.pk
gray.tfm

What else should I do so the files in the local ~/texmf are found? The only way I have been able to run gftodvi was by adding the gray font to the current working directory, but that is contrary to the purpose of having a local ~/texmf.