Iterating through a list and modifying elements one at a time to construct new list

Question

Long time reader, first time poster.
I have a question about creating a macro that iterates through a list. I've read through some questions posted by other users, but all of those have something to do with using LaTeX in a complicated way. My question's about using plain TeX to iterate through a list and construct a new list by modifying each element in the old list. I'm currently self-learning TeX and I think programming some basic, but versatile and robust macros will help me get a better sense of how the guts of TeX work. Anyways, enough background, onto my question.
This is my code so far:
I defined the iterator recursively. It reads one argument at a time, determines if the argument is a comma or a stop marker (of my own definition, called myStop), moves on if it's a comma, stops if it's the stop marker, and otherwise replaces the element by itself and the string (or token list) "+ 1".
edefmyStop{)}
defIterator#1{
        ifxmyStop#1 %do nothing {end of list}
            else %
                ifx,#1 expandafterIteratorIntermediateOne % currently 2 if-statements deep
                    else #1 + 1, expandafterIteratorIntermediateOne %currently 2 if-statements deep
            fi%
        fi%
}

I've had to define an intermediate Iterator called IteratorIntermediateOne to house the command expandafterIterator because I'm not currently aware of a way to group terms after an expandafter command in a way that would be equivalent to something like expandafter{expandafterIterator}fifi. So I guess that's my first question: Is there a way to define nested expandafter commands?
Now that everything's in context, here's all my code:
edefMyList{1,2,3}
edefmyStop{)}
defIteratorIntermediateOne{expandafterIterator}
defIterator#1{%
        ifxmyStop#1 %do nothing {end of list}
            else %
                ifx,#1 expandafterIteratorIntermediateOne % currently 2 if-statements deep
                    else #1 + 1, expandafterIteratorIntermediateOne %currently 2 if-statements deep
            fi%
        fi%
}

edefnewList{expandafterIteratorMyListmyStop}

Disclaimer: I'm aware of the extra comma this places after the last element. I am also aware there aren't enough cases to detect poorly constructed lists. I don't know enough about TeX to even begin to imagine how to safeguard the macro against improperly constructed token lists, so apologies if the more experienced of you out can't help but laugh at this code.
Okay, my other question is this: Is there a more efficient way to define a macro that calls itself? Is it better practice to define an iterator macro separately and call it using TeX's built-in loop command? If so, can someone walk me through how I would do that, because I'm having trouble making sense of the loop call in the context of TeX's eye's, mouth, gullet, and stomach processes. Does the loop expand the macro, reach repeat pass it on to the stomach and return back to the macro? I can't find a good explanation anywhere.
Thanks for all your help!

David Carlisle · Answer

The intention seems to be to add 1 to each item I would code it more like this (assuming etex)
edefMyList{1,2,3,25,456,2}

defIterator#1{expandafterxiterator#1stopiteration,}

defxiterator#1,{thenumexpr#1+1relax,xiterator}
defstopiteration#1relax#2xiterator{#1relax}

message{IteratorMyList}

bye

which makes the message
2,3,4,26,457,3

Steven B. Segletes · Answer

input listofitems
defMyList{1,2,3,25,456,2}
defprocesslist#1{%
  readlistmytermsMyList
  foreachitemzinmyterms{%
    ifnumzcnt=1else,fi
    thenumexprz+1relax
  }%
}

processlistMylist
bye

If you actually need to save the updated list, we can do so in the mytoks token list:
input listofitems
newtoksmytoks
defMyList{1,2,3,25,456,2}
defprocesslist#1{%
  mytoks{}%
  readlistmytermsMyList
  foreachitemzinmyterms{%
    ifnumzcnt=1elsemytoksexpandafter{themytoks,}fi
    mytoksexpandafterexpandafterexpandafter{%
      expandaftertheexpandaftermytoksthenumexprz+1relax}
  }%
}

processlistMylist

List is themytoks
bye

Ulrich Diez · Answer

You said: ...so apologies if the more experienced of you out can't help but laugh at this code. When I started learning TeX, I had the feeling that it was a very steep learning curve. From time to time I was frustrated. ;-) I do not believe that people like you, who are taking on this learning curve, are in a situation where it is appropriate to laugh at their attempts at macro programming/TeX-programming. I think any attempt to achieve or learn something good in a way which both is also based on reason and in itself is no misdeed, basically deserves respect. If you have questions about how the code from my examples below works, don't hesitate to ask them. It is then useful to describe how you think the code works and where you get stuck with the understanding. In my experience, this makes it easier for the respondents to find out exactly what information (e.g. about how TeX primitives work and which of the "side effects" briefly hinted at in the back chapters of the TeXbook are used for programming tricks) is still missing for understanding. Assuming that the entries in your comma-list are not surrounded by spaces, and that the relax-primitive does not occur within the comma-list, and that numexpr from the ε-TeX-extensions is available, you can probably do something like this: longdefgobble#1{}% longdeffirstofone#1{#1}% defIterator#1#2,{% % #1 - element-separator to prepend; empty in the 1st iteration; % comma in consecutive iterations % #2 - either current element of old list or the relax that was % appended for denoting the end of the list ifxrelax#2expandaftergobbleelseexpandafterfirstofonefi {% #1numbernumexpr#2+1relaxIterator{,}% }% }% defMyList{1,2,3} edefnewList{expandafterIteratorexpandafter{expandafter}MyList,relax,} begingroup tt stringMyList: meaningMyList stringnewList: meaningnewList endgroup bye The gist with this example is: Within the edef-definition-text of newList the TeX-primitive expandafter is used for expanding MyList. Also the sequence ,relax, is appended. This way at the time of defining newList edef-driven expansion of the definition-text of newList at some stage yields a sequence Iterator{}Comma,sparated,items,from,MyList,relax,. So relax, marks the end of the list. Now—still driven by edef-expansion—Iterator (recursively) picks a non-delimited-argument #1 (which in the first iteration is empty and in consecutive iterations holds a comma, i.e., which holds the separator to prepend to the item of the new list) and a comma-delimited-argument #2 which either holds the next item coming from myList's comma-list or holds the end-marker relax , and in any case places—nested in curly braces— a token-sequence denoting the next iteration, formed by the undelimited-argument #1, i.e., the separator that must precede the next item of the new list, the expression numbernumexpr#2+1relax for adding 1 to the value represented by the comma-delimited-argument and this way forming the next item of the new list, a call to itself for processing the next item remaining from the expansion of myList, this time providing a comma within the undelimited argument, denoting that next time the next item of the new list is to be preceded by a comma. Via ifxrelax#2 it is checked whether the end of the comma-list/the relax appended to the list in the beginning of edef-expansion is reached. If so, the token-sequence nested in curly braces denoting the next iteration is "gobbled/removed" via gobble and thus not carried out, which terminates iteration/tail-recursion. If not so, the surrounding curly braces are removed from that sequence by applying firstofone whereafter that sequence gets processed. The undelimited argument #1 of Iterator, which holds the separator to prepend to the item of the new list, only in the first iteration is empty. In each consecutive iteration it holds a comma which in that consecutive iteration's previous iteration was provided by the Iterator-macro itself as a part of the token-sequence which then formed the next iteration. This way (only) the first item of the new list is not preceded by a comma. If you don't have ε-TeX-extensions' numexpr available I can offer a routine for incrementing non-negative integer numbers. (In "real life" you might be interested in the packages intcalc and bigintcalc.) %------------------------------------------------------------------------------ % Expandable incrementing of non-negative integer number formed by a sequence % of explicit catcode-12-character-tokens from the set {0,1,2,3,4,5,6,7,8,9} %.............................................................................. % Increment{} % -> % % In expansion-contexts the result is delivered after two expansion-steps/is % obtained by "hitting" Increment with expandafter twice. %------------------------------------------------------------------------------ defIncrement#1{% romannumeral0% IncrementReverse{IncrementFork{}}{relax}{}#1relax }% defIncrementReverse#1#2#3#4{% % #1 - tokens to prepend to reversed list % #2 - tokens to append to reversed list % #3 - reversed list constructed so far % #4 - current element of not-reversed list ifxrelax#4% expandafterfirstoftwo else expandaftersecondoftwo fi {#1#3#2}{IncrementReverse{#1}{#2}{#4#3}}% }% defIncrementSelect#10123456789relax#2#3!!{#2}% defIncrementFork#1#2{% % #1 - digits incremented so far % #2 - current digit to increment or end-marker relax IncrementSelect #2123456789relax{IncrementReverse{ }{}{}#11}% 0#223456789relax{IncrementReverse{ }{}{}#12}% 01#23456789relax{IncrementReverse{ }{}{}#13}% 012#2456789relax{IncrementReverse{ }{}{}#14}% 0123#256789relax{IncrementReverse{ }{}{}#15}% 01234#26789relax{IncrementReverse{ }{}{}#16}% 012345#2789relax{IncrementReverse{ }{}{}#17}% 0123456#289relax{IncrementReverse{ }{}{}#18}% 01234567#29relax{IncrementReverse{ }{}{}#19}% 012345678#2relax{IncrementFork{#10}}% 0123456789#2{IncrementReverse{ }{}{}#11relax}% 0123456789relax{IncrementReverse{ }{}{}#11#2}% !!% }% %%----------------------------------------------------------------------------- longdeffirstoftwo#1#2{#1}% longdefsecondoftwo#1#2{#2}% defIterator#1#2,{% % #1 - element-separator to prepend % #2 - current element of old list ifxrelax#2expandafterfirstoftwoelseexpandaftersecondoftwofi {}{% #1Increment{#2}Iterator{,}% }% }% defMyList{1,2,3} edefnewList{expandafterIteratorexpandafter{expandafter}MyList,relax,} begingroup tt stringMyList: meaningMyList stringnewList: meaningnewList endgroup bye If you wish a routine which does without edef, you can, e.g., use the romannumeral0-expansion- and argument-exchanging-technique—the gist of romannumeral0-expansion is: TeX expands expandable tokens while gathering tokens that belong to the ⟨number⟩-quantity that is to be represented in roman numerals. If the first token which TeX finds while gathering the ⟨number⟩-quantity is a digit, e.g., 0, then the process of gathering tokens that belong to the ⟨number⟩-quantity turns into a process of gathering more digits or something that is not a digit and therefore terminates the process of gathering. Expandable tokens get expanded while gathering digits. A space-token terminating a digit-sequence terminates the process of gathering more digits and gets silently discarded. If the number gathered is not positive, TeX will silently swallow the tokens forming the ⟨number⟩-quantity without delivering any token in return. This implies that romannumeral can be used for tricking TeX into doing a lot of expansion- and argument-exchanging-work as long as it is ensured that in the end a non-positive number is found. %------------------------------------------------------------------------------ % Expandable incrementing of non-negative integer number formed by a sequence % of explicit catcode-12-character-tokens from the set {0,1,2,3,4,5,6,7,8,9} %.............................................................................. % Increment{} % -> % % In expansion-contexts the result is delivered after two expansion-steps/is % obtained by "hitting" Increment with expandafter twice. %------------------------------------------------------------------------------ defIncrement#1{% romannumeral0% IncrementReverse{IncrementFork{}}{relax}{}#1relax }% defIncrementReverse#1#2#3#4{% % #1 - tokens to prepend to reversed list % #2 - tokens to append to reversed list % #3 - reversed list constructed so far % #4 - current element of not-reversed list ifxrelax#4% expandafterfirstoftwo else expandaftersecondoftwo fi {#1#3#2}{IncrementReverse{#1}{#2}{#4#3}}% }% defIncrementSelect#10123456789relax#2#3!!{#2}% defIncrementFork#1#2{% % #1 - digits incremented so far % #2 - current digit to increment or end-marker relax IncrementSelect #2123456789relax{IncrementReverse{ }{}{}#11}% 0#223456789relax{IncrementReverse{ }{}{}#12}% 01#23456789relax{IncrementReverse{ }{}{}#13}% 012#2456789relax{IncrementReverse{ }{}{}#14}% 0123#256789relax{IncrementReverse{ }{}{}#15}% 01234#26789relax{IncrementReverse{ }{}{}#16}% 012345#2789relax{IncrementReverse{ }{}{}#17}% 0123456#289relax{IncrementReverse{ }{}{}#18}% 01234567#29relax{IncrementReverse{ }{}{}#19}% 012345678#2relax{IncrementFork{#10}}% 0123456789#2{IncrementReverse{ }{}{}#11relax}% 0123456789relax{IncrementReverse{ }{}{}#11#2}% !!% }% %%----------------------------------------------------------------------------- longdeffirstoftwo#1#2{#1}% longdefsecondoftwo#1#2{#2}% longdefexchange#1#2{#2#1}% defIterator#1,#2relax#3#4{% % #1 - current element of old list % #2 - remaining elements of old list % #3 - element-separator to prepend % #4 - new list constructed so far ifxrelax#1expandafterfirstoftwoelseexpandaftersecondoftwofi { #4}{% expandafterexchange expandafter{% expandafter{% romannumeral0% expandafterexpandafterexpandafterexchange expandafterexpandafterexpandafter{% Increment{#1}}{ #4#3}}}{Iterator#2relax{,}}% }% }% defMyList{0,1,2,3} expandafterdef expandafternewList expandafter{% romannumeral0expandafterIteratorMyList,{relax},relax{}{}}% begingroup tt stringMyList: meaningMyList stringnewList: meaningnewList endgroup bye

egreg · Answer

Since you're new, you can start with expl3.
documentclass{article}
usepackage{xparse,xfp}

ExplSyntaxOn

NewDocumentCommand{generatelist}{mmm}
 {% #1=output, #2=input, #3=iterator
  harry_list_generate:nnn { #1 } { #2 } { #3 }
 }

% variables
clist_new:N l__harry_list_input_clist
clist_new:N l__harry_list_output_clist

% the main function
cs_new_protected:Nn harry_list_generate:nnn
 {
  % if the input is a single token, assume it is a control sequence
  tl_if_single:nTF { #2 }
   { clist_set_eq:NN l__harry_list_input_clist #2 }
   { clist_set:Nn l__harry_list_input_clist { #2 } }
  % now l__harry_list_input_clist contains the input

% clear the output list
  clist_clear:N l__harry_list_output_clist

% map the input list applying the iterator to each item
  clist_map_inline:Nn l__harry_list_input_clist
   {
    clist_put_right:Nx l__harry_list_output_clist { #3 { ##1 } }
   }

% make the output list
  clist_set_eq:NN #1 l__harry_list_output_clist
 }

ExplSyntaxOff

% two example iterators
newcommand{addone}[1]{inteval{#1+1}}
newcommand{addhyphens}[1]{-#1-}

% a control sequence expanding to a list
newcommand{List}{1,2,3,41}

generatelist{ListA}{List}{addone}

generatelist{ListB}{1,2,3}{addhyphens}

showListA
showListB

This outputs
> ListA=macro:
->2,3,4,42.
l.50 showListA

?
> ListB=macro:
->-1-,-2-,-3-.
l.51 showListB

Iterating through a list and modifying elements one at a time to construct new list

4 Answers

Add your own answers!

Ask a Question