Error with unicode characters in cseappto

[bbalrog]

\documentclass[a4paper,10pt]{book}
\usepackage[utf8]{inputenc}
\usepackage{etoolbox}

\begin{document}

     \cseappto{myList}{entry1}
     \cseappto{myList}{entry2}
     \cseappto{myList}{{\'e}ntry3}
     \cseappto{myList}{entry4}

     \csuse{myList}

\end{document}

I'm using Ubuntu 20.04. The code above compiles fine with the default texlive from apt (which is kinda outdated, from 2019).

The code above fails to compile with texlive 2021 installed with install-tl perl script (full installation). The error is:

! Missing } inserted.
<inserted text>
            }
l.12 \csuse{myList}

Is this behaviour known? Has it been fixed in any recent commit? How can I workaround this?

[josephwright]

You are trying to \edef a UTF-8 char: in an 8-bit engine, these are made up of a series of commands which can't be used safely in an \edef. I'm not sure why you've chosen \cseappto, as \csappto would work for your example.

[bbalrog]

Hi @josephwright , thanks for your reply and clarification.

This is just a snippet to showcase the bug (I'm assuming it is a bug, since it used to work before?). The actual code is huge and requires cseappto. The entries in the list are later used to build a table and everything breaks if I use csappto.

[josephwright]

@bbalrog It's not a bug: user input was never regarded as safe in \edef contexts, which is why LaTeX provides \protected@edef. As such, \cseappto and similar are really for 'programmer' content. Could you show a fuller example and I'll be able to suggest the best way to approach it? My immediate thought is to apply \unexpanded in a controlled manner.

[bbalrog]

Sorry, took my a while to make a smaller example. It outputs the wrong results, but it doesn't matter.
The point I'm trying to illustrate: the column "Name" must admit unicode characters for this usage.

\documentclass[a4paper,10pt]{book}
\usepackage[utf8]{inputenc}
\usepackage{etoolbox}
\usepackage{longtable}

\newcommand*{\ifstartswith}[2]{%
\if\@car#1.\@nil\@car#2.\@nil
  \expandafter\@firstoftwo
\else
  \expandafter\@secondoftwo
\fi}
\newcommand*{\eifstartswith}{\@expandtwoargs\ifstartswith}

\newcommand{\addCost}[3]{
\csdef{costList#1}{#1}%
\csdef{costList#1Ref}{#2}%
\csdef{costList#1Cost}{#3}%
}

\newcount\costCount
\newcommand{\computeCost}[1]{\ifnumless{\csuse{costList#1Cost}}{0}%
                                {\the\numexpr-\the\costCount*\csuse{costList#1Cost}\relax}%
                                {\the\numexpr \the\costCount*\csuse{costList#1Cost}\relax}}

                                
\newcommand{\newTable}[1]{
  \expandafter\newcount\csname TableCost#1\endcsname
  \csname TableCost#1\endcsname=0
}

\newcommand{\addTableLine}[3][]{
  \ifnumless{\csuse{costList#3Cost}}{0}{\costCount=-1}{\costCount=1}%
  \ifstartswith{#1}{+}{\nullfont\costCount=#1\normalfont}{}%
  \ifstartswith{#1}{-}{\nullfont\costCount=#1\normalfont}{}%
  \csappto{Table#2}{& \csuse{costList#3} #1 & \csuse{costList#3Ref} & \computeCost{#3}\unexpanded{\\\cline{2-4}}}%
  \advance\csname TableCost#2\endcsname by \computeCost{#3}%
}

\newcommand{\printTable}[1]{
    \begin{longtable}{c p{7cm}|c|c}
    \multicolumn{4}{c}{  }\\
        & \multicolumn{1}{ c|}{{\bf Name}} & {\bf Ref} & {\bf Cost}\\\cline{2-4}
      \csuse{Table#1}
  \end{longtable}
}

\begin{document}

\addCost{First Cost}{CI68}{25}
\addCost{Second Cost}{CD30}{-15}
\addCost{Third Cost}{BS25}{8}
\addCost{What Cost}{AB30}{-10}
\addCost{Strange Cost}{AB30}{1}

\newTable{CompA}
\newTable{CompB}

\addTableLine[+1]{CompB}{First Cost}
\addTableLine[-2]{CompB}{What Cost}
\addTableLine[+5]{CompA}{Third Cost}
\addTableLine[-1]{CompA}{Second Cost}
\addTableLine[-5]{CompB}{Third Cost}

\printTable{CompA}
Total Cost: \the\csname TableCostCompA\endcsname

\printTable{CompB}
Total Cost: \the\csname TableCostCompB\endcsname

\end{document}

And no, using Lua Scripting is not on the Table. It is sadly not my choice.