How to use input inside setlength?

The LaTeX input command is not expandable: this is partly due to the fact it does 'safe' check for the existence of a file, and partly due to supporting both TeX primitive (input <file> ) and LaTeX (input{<file>}) syntaxes. This means that if you want to work by expansion, as you do here, you need to use the primitive

begin{filecontents*}[overwrite]{size.txt}
5%
end{filecontents*}
documentclass{article}
usepackage{calc}
newlength{foo}
makeatletter
letprimitiveinput@@input
makeatother
newcommand{zzz}{primitiveinput size.txt }
setlength{foo}{2in * zzz}
usepackage[paperwidth=foo,paperheight=foo]{geometry}
begin{document}
end{document}

You'll need to make sure your input file doesn't have a par token in it: I've added an example using a comment char.