TeX capacity exceeded even though I externalize

Question

here is my MWE:
documentclass[10pt, a4paper, twoside, ngerman]{scrartcl} 
usepackage[utf8]{inputenc}
usepackage[ngerman]{babel}
usepackage{siunitx}    
usepackage{geometry}
usepackage{tikz}       
usepackage{pgfplots}

usepgfplotslibrary{colormaps, groupplots,external}
usetikzlibrary{pgfplots.colormaps}
tikzexternalize
tikzsetexternalprefix{external_figs/}

begin{document}
    
begin{figure}
    centering
    input{fig.tex}
end{figure}

end{document}

The figure is a surface plot with a lot of datapoints which I cannot sample down further. I'm using TeXstudio as TeX-environment and Pdflatex with the following arguments
pdflatex -synctex=1 -interaction=nonstopmode -shell-escape %.tex

I still get the error
TeX capacity exceeded, sorry [main memory size=3000000].

I'm aware of the possibility to use LuaLaTeX (which I already used in a prior work) but the problem with it is (was), that it always took ages to build the file if the file contains more such memory intense images (apparently all images were recompiled everytime I built the document even though they didn't change).
Can anyone think of a good, stable and sustainable solution?

Simon Dispa · Answer

Once I had a project with many (30+), very large, some more than 10k as  exported from R, tikz graphs, maybe similar to yours, (lualatex MikTex 2.9, win10 x64, 8GB RAM) and I ran into a a similar problem.
I edited the file
texmfapp.ini
C:Program Files MiKTeX miktexconfig
Finally everything worked with these values
main_memory = 79999999

save_size = 79999

extra_mem_bot = 79999999

extra_mem_top = 79999999

font_mem_size = 799999

pool_size = 7999999

buf_size = 7999999

stack size = 200000

max-strings = 79999999

It was trial and error. I don't know exactly the impact of each parameter.
This answer it a reproduction of my previous
Again: ! TeX capacity exceeded, sorry [save size=500000] (LuaLaTeX, MikTeX)
Then I compiled the figures one by one (up to 15 min the largest!). At the end, monitoring the CPUs and memory while compiling the large figures, I found all 4 CPUs at mostly 100% and memory used at 80-90%.
Then inserted them into the LaTeX document using this configuration
tikzexternalize[%
up to date check={simple},
prefix=./GRAPHS/md5/]% Folder needs to be created before compiling

tikzset{external/system call={%
        lualatex tikzexternalcheckshellescape
        -halt-on-error -shell-escape -interaction=batchmode
        -jobname "image" "texsource"}}

in order to avoid a second compiling of the figures. There are several options of checks for changes to avoid the remaking.
Still some  checks took  up to 20-30, very long, seconds each time!
Now, with a little more wisdom afterwards, I guess I could have inserted the pdf generated files instead of the original tikz's, once each final version was reached and approved.
Take a look to
http://www.bakoma-tex.com/doc/latex/pgfplots/pgfplots.pdf
section 6 for more detail instructions.

TeX capacity exceeded even though I externalize

One Answer

Add your own answers!

Ask a Question