TransWikia.com

Is there a list of expandable TeX primitives? LaTeX? e-TeX? others?

TeX - LaTeX Asked on May 22, 2021

According to Christian Hupfer’s comment I understood that calculations are possible in a keyval list when the macro is expandable. The macro numexpr being expandable, calculations are possible.

Hence the interest in knowing which macros are and which are not expandable from TeX, LaTeX, ε-TeX, etc.

I am not asking if there is a test, but if there is a list already made.

3 Answers

Joseph has already answered, interpreting the "expandable` in your question as "fully expandable" (or what is perhaps better named as "safe in an expansion only context" see

Advantages and disadvantages of fully expandable macros

However to answer the question as actually asked, note that numexpr is neither a macro nor expandable.

All macros are by definition expandable, a macro works by expanding to its replacement text. There is no list of these as it is any command defined via def or its variants edef, gdef, xdef.

The TeXBook lists all the TeX primitivies, these are marked by a * in the index, although you need to check the description of each individually to see if they are expandable.


In classic TeX, the expandable primitives are (I think:-)

endinput
expandafter
csname
the
number
romannumeral
if
ifx
ifcat
ifcase
ifnum
ifodd
ifdim
ifeof
iftrue
iffalse
ifhbox
ifvbox
ifvoid
ifinner
ifvnode
ifhmode
ifmmode
else
or
fi
input % (@@input in LaTeX)
jobname
meaning
noexpand
string
topmark
firstmark
botmark
fontname

in e-TeX the additional expandable primitives are

topmarks
firstmarks
botmarks
ifdefined
ifcsname
iffontchar
unless
eTeXrevision
unexpanded
detokenize
scantokens

in addition to all the above pdfTeX adds the following expandable primitives

pdfescapestring
pdfescapename
pdfescapehex
pdfstrcmp
pdfmatch
ifpdfabsnum
ifpdfabsdim
pdfuniformdeviate
pdfnormaldeviate
pdffilemoddate
pdffilesize
pdfmdffivesum
pdffiledump
pdfcolorstackinit
ifincsname
ifpdfprimitive
pdfcreationdate
pdfinsertht
pdftexbanner
pdftexrevision
expanded

XeTeX has the primitives of etex, and adds the following expandable primitives

ifincsname
ifprimitive
normaldeviate
uniformdeviate
filedump
filemoddate
filesize
mdfivesum
expanded
XeTeXrevision
Uchar
Ucharcat
strcmp

LuaTeX has the expandable primitives of etex plus (at least)

luatexbanner
luatexrevision
formatname
Uchar
directlua
luaescapestring
scantextokens
csstring
expanded
ifincsname
pdfvariable
pdffeedback

pTeX (Japanese TeX engine) adds the following expandable primitives, which are available in ptex, uptex, eptex and euptex:

euc
ifdbox
ifddir
ifjfont    % for TL2020
ifmbox
ifmdir
iftbox
iftdir
iftfont    % for TL2020
ifybox
ifydir
jis
kansuji
kuten
ptexrevision
sjis

upTeX (Unicode-aware pTeX) adds the following expandable primitives, which are available in uptex and euptex:

ucs
uptexrevision

e-pTeX (pTeX + e-TeX) adds the following expandable primitives, which are available in eptex and euptex:

expanded
ifincsname    % for TL2020
ifpdfprimitive
pdfcreationdate
pdffiledump
pdffilemoddate
pdffilesize
pdfmdfivesum
pdfnormaldeviate
pdfstrcmp
pdfuniformdeviate
Uchar         % for TL2020
Ucharcat      % for TL2020

If you find missing ones, feel free to edit this answer here....

Correct answer by David Carlisle on May 22, 2021

A list of all expandable macros is entirely impossible to provide: there are an open-ended number of cases. What one can do is provide a list of primitives which work by expansion, and general rules for macros.

Essentially, any primitives which carry out assignment or typesetting are not expandable, whilst other primitives are. Thus for example def, let, etc. are all non-expandable, as are hbox, raise, setbox, whereas expandafter, the and ifx are all expandable. Note that some primitives will be expandable in the right context: numexpr is not expandable in itself, but is a valid token coming after the or number:

edeftesta{numexpr 1+2relax}showtesta
edeftestb{numbernumexpr 1+2relax}showtestb

In terms of macros, we have to be clear what we mean by 'expandable'. TeX is a macro expansion language, so if we have for example

defbaz{}
deffoo{defbaz{bar}}

then we can do

edeftest{foo}showtest

although the result is not what is likely wanted/useful. As such, when we talk about expandable macros we normally mean macros which contain only expandable primitives and which thus can be used 'safely' inside an edef or similar. With e-TeX, it' possible to ensure that macros which don't meet this criterion don't 'blow up':

defbaz{}
protecteddeffoo{defbaz{bar}}
edeftest{foo}showtest

Where does that take us? Any macro which contains:

  • Any assignment primitive
  • Any typesetting primitive
  • Any protected macro
  • Any macro which itself meets one of the three criteria above

is not expandable. That is the majority of 'useful' macros, so as mentioned in comments, if you are not sure, assume non-expandable.


Note that the list of non-expandable primitives is quite long, and we do have to remember about the context. For example, something like tracingcommands acts like a count register, so is non-expandable unless if follows the. Thus unless we implement a token-by-token processor, just seeing tracingcommands doesn't tell us that a macro containing this token is not expandable.

deffoo{tracingcommands=1 }% Not expandable
defbaz{thetracingcommands}% Expandable

In LaTeX3/expl3, all macros are either fully expandable or are protected. Moreover, the expandable ones are all marked as such (with a star) in the documentation. The reason is that it requires some knowledge to see if something is expandable, in particular checking all macro 'dependencies'. As such, one has to work carefully to track expandable macros.

Answered by Joseph Wright on May 22, 2021

Although it is explicitly not asked for a test it may be worth mentioning that a test for the expandability of a token can be based on the circumstance that with expandable tokens and undefined tokens the test

expandafterifxnoexpand⟨token⟩⟨token⟩⟨token not expandable⟩else⟨token expandable or undefined⟩fi

delivers the tokens of the else-branch/delivers the tokens of the ⟨token expandable or undefined⟩-branch. When the ⟨token expandable or undefined⟩-branch is taken, you can evaluate the meaning of the ⟨token⟩:

meaning in any case will deliver characters of category code 12(other). The only exception is the space character. meaning will deliver spaces always with category code 10 (space) (and character code 32).

If and only if the ⟨token⟩ is undefined, then meaning will deliver the leading catcode-12-token sequence undefined.

If and only if the ⟨token⟩ is a macro, then the tokens delivered by meaning will contain the sequence ->.

Some expandable primitives may require special treatment/attention in some situations, e.g., input (renamed to @@input in LaTeX), endinput, noexpand. Probably these, too, can be cranked out by evaluating meaning.

Besides this in LaTeX there are tokens which sometimes are expandable/unexpandable primitives and sometimes are expandable macros. While writing this sentence I think about the token protect. Sometimes protect is let equal to the unexpandable primitive relax. Sometimes protect is let equal to the expandable primitive noexpand. Sometimes protect is let equal to the macro @unexpandable@protect which expands to noexpandprotectnoexpand.


Here is a LaTeX-implementation of a test whether a macro argument does have a first token which is an expandable primitive:

(First you need to know whether the argument is empty. If it is not, you need to know whether its first token can be extracted as an undelimited argument. If and only if it is an explicit space of catcode 10 and charcode 32 or an opening brace/an explicit character token of catcode 1, then it cannot be extracted as an undelimited argument. But then it isn't an expandable primitive either. If it can be extracted, then have TeX extract it and check via expandafterifxnoexpand#1#1 whether it is expandable/undefined. If it is not expandable, it is not an expandable primitive. If it is expandable or undefined check its meaning for cranking out the case of it being a macro or undefined.)

errorcontextlines=10000
documentclass{article}

makeatletter
%%=============================================================================
%% Paraphernalia:
%%    UD@firstoftwo, UD@secondoftwo, UD@Exchange,  UD@CheckWhetherNull, 
%%    UD@CheckWhetherBrace, UD@CheckWhetherLeadingSpace, UD@ExtractFirstArg
%%=============================================================================
newcommandUD@firstoftwo[2]{#1}%
newcommandUD@secondoftwo[2]{#2}%
newcommandUD@Exchange[2]{#2#1}%
%%-----------------------------------------------------------------------------
%% Check whether argument is empty:
%%.............................................................................
%% UD@CheckWhetherNull{<Argument which is to be checked>}%
%%                     {<Tokens to be delivered in case that argument
%%                       which is to be checked is empty>}%
%%                     {<Tokens to be delivered in case that argument
%%                       which is to be checked is not empty>}%
%%
%% The gist of this macro comes from Robert R. Schneck's ifempty-macro:
%% <https://groups.google.com/forum/#!original/comp.text.tex/kuOEIQIrElc/lUg37FmhA74J>

newcommandUD@CheckWhetherNull[1]{%
  romannumeral0expandafterUD@secondoftwostring{expandafter
  UD@secondoftwoexpandafter{expandafter{string#1}expandafter
  UD@secondoftwostring}expandafterUD@firstoftwoexpandafter{expandafter
  UD@secondoftwostring}UD@firstoftwoexpandafter{} UD@secondoftwo}%
  {UD@firstoftwoexpandafter{} UD@firstoftwo}%
}%
%% With eTeX-engines you can also do:
%%newcommandUD@CheckWhetherNull[1]{%
%%  romannumeral0ifcat$detokenize{#1}$%
%%  expandafterUD@firstoftwoelseexpandafterUD@secondoftwofi
%%  {UD@firstoftwoexpandafter{} UD@firstoftwo}%
%%  {UD@firstoftwoexpandafter{} UD@secondoftwo}%
%%}%
%%-----------------------------------------------------------------------------
%% Check whether argument's first token is a catcode-1-character
%%.............................................................................
%% UD@CheckWhetherBrace{<Argument which is to be checked>}%
%%                      {<Tokens to be delivered in case that argument
%%                        which is to be checked has leading
%%                        catcode-1-token>}%
%%                      {<Tokens to be delivered in case that argument
%%                        which is to be checked has no leading
%%                        catcode-1-token>}%
newcommandUD@CheckWhetherBrace[1]{%
  romannumeral0expandafterUD@secondoftwoexpandafter{expandafter{%
  string#1.}expandafterUD@firstoftwoexpandafter{expandafter
  UD@secondoftwostring}UD@firstoftwoexpandafter{} UD@firstoftwo}%
  {UD@firstoftwoexpandafter{} UD@secondoftwo}%
}%
%%-----------------------------------------------------------------------------
%% Check whether brace-balanced argument starts with a space-token
%%.............................................................................
%% UD@CheckWhetherLeadingSpace{<Argument which is to be checked>}%
%%                             {<Tokens to be delivered in case <argument
%%                               which is to be checked>'s 1st token is a
%%                               space-token>}%
%%                             {<Tokens to be delivered in case <argument
%%                               which is to be checked>'s 1st token is not
%%                               a space-token>}%
newcommandUD@CheckWhetherLeadingSpace[1]{%
  romannumeral0UD@CheckWhetherNull{#1}%
  {UD@firstoftwoexpandafter{} UD@secondoftwo}%
  {expandafterUD@secondoftwostring{UD@CheckWhetherLeadingSpaceB.#1 }{}}%
}%
newcommandUD@CheckWhetherLeadingSpaceB{}%
longdefUD@CheckWhetherLeadingSpaceB#1 {%
  expandafterUD@CheckWhetherNullexpandafter{UD@firstoftwo{}#1}%
  {UD@Exchange{UD@firstoftwo}}{UD@Exchange{UD@secondoftwo}}%
  {UD@Exchange{ }{expandafterexpandafterexpandafterexpandafter
   expandafterexpandafterexpandafter}expandafterexpandafter
   expandafter}expandafterUD@secondoftwoexpandafter{string}%
}%
%%-----------------------------------------------------------------------------
%% Extract first inner undelimited argument:
%%   romannumeral0UD@ExtractFirstArgLoop{<argument>UD@SelDOm}%
%%   yields <argument>'s 1st undlimited argument.
%%   <argument> must not be blank, i.e., must not be empty or consist of
%%   explicit character tokens of catcode 10 and charcode 32 only.
%%.............................................................................
@ifdefinableUD@RemoveTillUD@SelDOm{%
  longdefUD@RemoveTillUD@SelDOm#1#2UD@SelDOm{{#1}}%
}%
newcommandUD@ExtractFirstArgLoop[1]{%
  expandafterUD@CheckWhetherNullexpandafter{UD@firstoftwo{}#1}%
  { #1}%
  {expandafterUD@ExtractFirstArgLoopexpandafter{UD@RemoveTillUD@SelDOm#1}}%
}%
%%=============================================================================
%% Check whether token is macro (only with macros the meaning contains the
%% sequence ->) - the argument of UD@CheckWhetherMacro must be a single token;
%% the argument of UD@CheckWhetherMacro must not be empty:
%%=============================================================================
newcommandUD@CheckWhetherMacro[1]{%
  expandafterUD@@CheckWhetherMacromeaning#1->X%
}%
@ifdefinableUD@@CheckWhetherMacro{%
  defUD@@CheckWhetherMacro#1->#2X{%
    UD@CheckWhetherNull{#2}{UD@secondoftwo}{UD@firstoftwo}%
  }%
}%
%%=============================================================================
%% Check whether token is undefined control sequence (only with undefined 
%% control sequences the meaning has the leading phrase "undefined"; besides
%% this meaning never delivers no tokens at all) - the argument of 
%% UD@CheckWhetherUndefined must be a single token; the argument of 
%% UD@CheckWhetherUndefined must not be empty:
%%=============================================================================
begingroup
defUD@CheckWhetherUndefined#1{%
  endgroup
  newcommandUD@CheckWhetherUndefined[1]{%
    expandafterUD@@CheckWhetherUndefinedmeaning##1#1X%
  }%
  @ifdefinableUD@@CheckWhetherUndefined{%
    defUD@@CheckWhetherUndefined##1#1##2X{%
      UD@CheckWhetherNull{##1}{UD@firstoftwo}{UD@secondoftwo}%
    }%
  }%
}%
escapechar=-1relax
expandafterUD@CheckWhetherUndefinedexpandafter{stringundefined}%
%%=============================================================================
%% Check whether argument has first token which is an expandable primitive
%%=============================================================================
newcommandCheckWhetherArgHasFirstTokenWhichIsExpandablePrimitive[1]{%
  romannumeral0%
  UD@CheckWhetherNull{#1}{UD@firstoftwoexpandafter{} UD@secondoftwo}{%
    UD@CheckWhetherBrace{#1}{UD@firstoftwoexpandafter{} UD@secondoftwo}{%
      UD@CheckWhetherLeadingSpace{#1}{UD@firstoftwoexpandafter{} UD@secondoftwo}{%
        expandafterUD@CheckWhetherArgExamineArgsFirstToken
        romannumeral0UD@ExtractFirstArgLoop{#1UD@SelDOm}%
      }%
    }%
  }%
}%
newcommandUD@CheckWhetherArgExamineArgsFirstToken[1]{%
  expandafterifxnoexpand#1#1%
  expandafterUD@firstoftwoelseexpandafterUD@secondoftwofi{%
    UD@firstoftwoexpandafter{} UD@secondoftwo
  }{%
    UD@CheckWhetherMacro{#1}{UD@firstoftwoexpandafter{} UD@secondoftwo}{%
      UD@CheckWhetherUndefined{#1}{UD@firstoftwoexpandafter{} UD@secondoftwo}{%
        UD@firstoftwoexpandafter{} UD@firstoftwo
      }%
    }%
  }%
}%
makeatother
begin{document}

These tests are "negative"

CheckWhetherArgHasFirstTokenWhichIsExpandablePrimitive{}%
                                                       {Argument does have a first token which is an expandble primitive}%
                                                       {Argument does not have a first token which is an expandble primitive}

CheckWhetherArgHasFirstTokenWhichIsExpandablePrimitive{{ifx}relax}%
                                                       {Argument does have a first token which is an expandble primitive}%
                                                       {Argument does not have a first token which is an expandble primitive}

CheckWhetherArgHasFirstTokenWhichIsExpandablePrimitive{relaxnumber}%
                                                       {Argument does have a first token which is an expandble primitive}%
                                                       {Argument does not have a first token which is an expandble primitive}

CheckWhetherArgHasFirstTokenWhichIsExpandablePrimitive{TeX}%
                                                       {Argument does have a first token which is an expandble primitive}%
                                                       {Argument does not have a first token which is an expandble primitive}

CheckWhetherArgHasFirstTokenWhichIsExpandablePrimitive{ something}%
                                                       {Argument does have a first token which is an expandble primitive}%
                                                       {Argument does not have a first token which is an expandble primitive}

CheckWhetherArgHasFirstTokenWhichIsExpandablePrimitive{UnDeFInEdSelDOMWeiRDStrangeToken}%
                                                       {Argument does have a first token which is an expandble primitive}%
                                                       {Argument does not have a first token which is an expandble primitive}


These tests are "positive"

makeatletter

CheckWhetherArgHasFirstTokenWhichIsExpandablePrimitive{@@input myfilewith weirdcode.tex }%
                                                       {Argument does have a first token which is an expandble primitive}%
                                                       {Argument does not have a first token which is an expandble primitive}

makeatother

CheckWhetherArgHasFirstTokenWhichIsExpandablePrimitive{numberrelax}%
                                                       {Argument does have a first token which is an expandble primitive}%
                                                       {Argument does not have a first token which is an expandble primitive}

CheckWhetherArgHasFirstTokenWhichIsExpandablePrimitive{ifx aa aa else aa fi}%
                                                       {Argument does have a first token which is an expandble primitive}%
                                                       {Argument does not have a first token which is an expandble primitive}

CheckWhetherArgHasFirstTokenWhichIsExpandablePrimitive{noexpandrelax}%
                                                       {Argument does have a first token which is an expandble primitive}%
                                                       {Argument does not have a first token which is an expandble primitive}

CheckWhetherArgHasFirstTokenWhichIsExpandablePrimitive{fontname}%
                                                       {Argument does have a first token which is an expandble primitive}%
                                                       {Argument does not have a first token which is an expandble primitive}

end{document}

enter image description here

Answered by Ulrich Diez on May 22, 2021

Add your own answers!

Ask a Question

Get help from others!

© 2024 TransWikia.com. All rights reserved. Sites we Love: PCI Database, UKBizDB, Menu Kuliner, Sharing RPP