Thoughts about taking LaTeX's protect-mechanism into account?

Question

In this question I am not asking for code.
I'm asking before I start implementing: Which direction do you think makes the most sense?
I am about to define a fully expandable macro to be used with LaTeX 2ε which in some sort of tail-recursive loop examines its argument token-wise.
One requirement is: In case an expandable token is found, it shall be expanded during examination unless it is preceded by noexpand or protect and/or is defined in terms of protected.
The case of the token being defined in terms of protected can be handled by examining the meaning. The tail-recursive loop can be implemented with an argument holding a "flag"-token indicating whether the token processed in the previous iteration either was the control-word-token protect or had the same meaning as the noexpand-primitive.
Checking for noexpand-meaning and "setting the flag for the next iteration" can be handled by an ifx-comparison. (You may probably "copy" the noexpand-primitive at a moment in time where it is definitely not redefined yet.)
The question that remains is about handling edge cases:

Somebody might temporarily have redefined the control-word-token protect so that temporarily it is not a component of LaTeX 2ε's protection-mechanism.
Somebody might very temporarily use a token protectcopy (which was let equal to protect at some stage of processing).

How would you check whether the token protect is currently used as a component of LaTeX 2ε's protection-mechanism?
How would you check whether some other token currently is used in the same way in which in LaTeX 2ε usually the token protect is used?
I tend to do two things:

Check for the control-word-token protect itself, e.g., via delimited arguments.
Check for one of the meanings which protect can have when used as component of LaTeX 2ε's protection-mechanism.

Would this be sufficient?
Is the condition of the token in question both being the control-word-token protect and having one of the meanings which protect can have (relax /  string / noexpand / noexpandprotectnoexpand ) sufficient for assuming that the token in question is used as a component of LaTeX 2ε's protection-mechanism?

Btw: What is expl3's equivalent to protect ?

Joseph Wright · Accepted Answer

In the code for text_expand:n, which aims to take 'broadly text' and expand any hidden content, the approach taken is as follows.
First, note that there is an importance to the order one works. In particular, expansion of macros or primitives has to be considered after various 'filtering' tests.
The protect token in LaTeX2e has to have exactly that name in order to work with protected@edef and so on. As such, text_purify:n tests for it by name not definition: this is done using pdfstrmcp as this makes string-based expandable tests easy. Other tokens with equivalent meaning are thus not picked up by this process.
Rather than use a flag, if one finds a protect, one can grab the next token and handle it in one loop pass. This ability to 'look ahead' is needed for any expansion anyway. In text_purify:n, I choose to re-combine protectfoo[space] into foo if that construct exists - a little effort but 'undoes' an otherwise-confusing expansion.
One needs to expand not only macros but also expandable primitives in such a scheme. That deals with 'raw' noexpand, which will do it's job such that the next token will be treated as relax.
In expl3 terms, there is no direct equivalent of protect as e-TeX is required. Thus non-expandable macros may be protected, and the availability of more machine cycles means this type of token-ise analysis is feasible. This means that expl3 tokens should always be 'safe' inside an edef or similar. (Internal macros as well as the documented API follows the same rule: either protected or safe to use in an expansion context.)

Thoughts about taking LaTeX's protect-mechanism into account?

One Answer

Add your own answers!

Ask a Question