Construct math super/sub-script from reference (`ref`) with hyperref and cleveref being loaded

I finally figured out the solution. It is probably not the best solution in the sense that it is robust, because it relies on the format how hyperref writes references to the aux-file.

The macro hyb needs to be defined as

newcommand{hyb}[1]{%
  expandafterifxcsname r@#1endcsnamerelax%
    @latex@warning{Reference `#1' on page thepage space undefined}%
    mathbf{??}%
  else%
    @expand@hyb{expandafterexpandafterexpandafter@firstoffivecsname r@#1endcsname}%
  fi%
}

instead of

newcommand{hyb}[1]{%
  @expand@hyb{ref*{#1}}%
}

I first explain, why the original approach failed and then explain how the final solution works. @expand@hyb expects that its argument (here: ref*{#1}) can be expanded into two tokens {roman value of c@hybridseries}{arabic value of c@hybrid} before these two tokens are consumed by @hyb. However, ref*{#1} is unexpandable, because it internally uses protect.

The proposed solution is more or less an inline version of ref* (without the hyperref-magic) and thereby circumvents the problematic protect. Let me start with the else-branch of the solution which is the "good" case. When a new label is created by label{my-label} the following example line is inserted into the aux-file

newlabel{my-label}{{{i}{1}}{1}{}{hybrid.1.1}{}}

(N.b. in this example, the series counter equals i and the hybrid counter equals 1.) When the aux-file is re-read again, newlabel defines the macro r@my-label to be {{i}{1}}{1}{}{hybrid.1.1}{}. On the outer level these are five tokens with the first token being a group of two tokens {i}{1}. (N.b., the hyperref causes these five tokens, without hyperref there would only be two. Although note that r@my-label is not a normally valid macro name and can only be constructed by csname.)

Now, hyb works as follows. It constructs the macro r:my-label through csname r@#1endcsname. Then @firstoffive (an internal macro from the hyperref package) drops all but the first token, i.e. {i}{1} remains. And these are passed to @expand@hyb which eventually expands to @hyb{i}{1}.

The sequence of three expandafter's is required to get the right order of expansion. Before @firstoffive is executed, the macro r@my-label must be constructed, i.e. csname must be executed first. This yields the right most expandafter. However, @firstoffive must not be applied to r@my-label. The latter would only be a single token not five. Instead r@my-label must also be expanded once. This yields the other two expandafter.

The ifx-else-fi construction asserts that #1 actually denotes an existing label or a warning is printed.

Redefining hyb to use getrefnumber, as in

newcommand{hyb}[1]{%
  @expand@hyb{getrefnumber{#1}}%
}

seems to allow the code, including the equation, to compile. This was alluded to by J Kormylo in his comment from last year. The equation seems, to my limited understanding, to produce the desired result.

The only catch I've been able to determine is that newhybridseries must define a series before it can be used in a hyb reference. Thus, for example, hyb{hyb:priv-foo} could only be used in Equation 1 if newhybridseries{mathrm{privacy}} has been issued prior to equation 1.

documentclass{article}

usepackage[T1]{fontenc}
usepackage{lmodern}
usepackage[utf8]{inputenc}
usepackage[hyperref]{ntheorem}
usepackage[unicode,final]{hyperref}
usepackage{cleveref}
input{glyphtounicode}
pdfgentounicode=1


%
% How the hybrid mechanism works
%
% Hybrids are defined by a theorem-like environment call "hybrid" using the counter c@hybrid.
% Hybrids can be grouped into series.
% To make the reference unique, series are also counted using c@hybridseries.
% Each time c@hybridseries is advanced, c@hybrid is reset.
% Series have a human-readable name.
% 
% A hybrid should be printed as "H^{human-readable series name}_{hybrid counter}".
% To make this work correctly we need a little bit of trickery, because LaTeX writes the printable
% form of a label to the aux-file and not the plain values of the counters.
% Also, the "public" API of ntheorem and cref use the printable form of the counters.
%
% Under the hood the counter on the level of hybrid is internally formatted as
%
%    "{roman number of series}{arabic numer of hybrid}"
%
% The series counter is formatted as a roman number, because this allows to construct TeX macros that include the counter in their name.
% For example, @hybridseriesnameiii expands to the human-readable name of the 3rd series.
% For convinience, there is the helper macro @hybridseriesname{#1}.
% It takes the roman counter of a series as its only argument and constructs the makro @hybridseriesname<#1>, i.e. eventually it is expanded to the human-readable name.
% 

makeatletter

%
% The counter for the series formatted as a roman number
%
newcounter{hybridseries}
renewcommand{thehybridseries}{romannumeralc@hybridseries}

%
% A helper macro that expands to hybridseriesname#1, e.g. hybridseriesname{iii} expands to hybridseriesnameiii
%
% #1   The series formatted as a roman number
%
newcommand@hybridseriesname[1]{%
  csname @hybridseriesname#1endcsname%
}

%
% Advances the series counter c@hybridseries and defines a new command hybridseriesname<counter>, e.g. hybridseriesnameiii, that stores the human-readable name.
%
% #1 The human-readable name of the series; n.b., #1 is printed in math mode, hence, if it is a textual name, #1 should most probably something like text{...} or mathrm{...}
%
newcommand{newhybridseries}[1]{%
  stepcounter{hybridseries}%
  setcounter{hybrid}{-1}%
  expandafterdefcsname @hybridseriesnamethehybridseriesendcsname{#1}%
}


%
% @hyb takes an internally formatted counter value
%     {roman number of series}{arabic numer of hybrid}
% and expand to "H^{human-readable series name}_{hybrid counter}".
% The first part {roman number of series} is passed to @hybridseriesname to convert it into the human-readable name.
% The second part {arabic number of hybrid} is used as is.
%
newcommand{@hyb}[2]{%
  H^{@hybridseriesname{#1}}_{#2}%
}

%
% hyb takes a the designation of a label and expands to "H^{human-readable series name}_{hybrid counter}.
% hyb is supposed to be used in math mode.
%
newcommand{hyb}[1]{%
  @expand@hyb{getrefnumber{#1}}%
}

%
% A helper macro in the spirit of @expandtwoargs from the LaTeX2e sources
% 
% #1 is a token that ultimately expands into two tokens
%     {roman number of series}{arabic numer of hybrid}
%
% This macro first expands #1 and then lets @hyb process the expanded #1
%
def@expand@hyb#1{%
  edefreserved@hyb{noexpand@hyb #1}%
  reserved@hyb%
}

%
% This theorem style is mostly equivalent to the plain style.
% But the counter (##2) is passed to @hyb such that is nicely printed as
%     "H^{human-readable series name}_{hybrid counter}"
% and not as
%     {roman number of series}{arabic numer of hybrid}
%
% ##1   Printable name (e.g. Theorem, Lemma, etc.); here this always equals "Hybrid"
% ##2   Printable and internally formatted counter value, i.e. {roman number of series}{arabic numer of hybrid}
% ##3   Optional decription
newtheoremstyle{@hybridstyle}{%
  item[hskiplabelsep theorem@headerfont ##1 $@expand@hyb{##2}$theorem@separator]%
}{%
  item[hskiplabelsep theorem@headerfont ##1 $@expand@hyb{##2}$ (##3)theorem@separator]%
}

%
% Defines the hybrid-enviroment
% Uses the special @hybridstyle, make the counter c@hybrid subordinate to c@hybridseries and formats c@hybrid as
%     {roman number of series}{arabic numer of hybrid}
%
theoremstyle{@hybridstyle}
theoremheaderfont{normalfontbfseries}
theoremseparator{}
theorembodyfont{normalfontitshape}
theoremsymbol{}
newtheorem{hybrid}{Hybrid}[hybridseries]
renewcommand{thehybrid}{{thehybridseries}{numberc@hybrid}}

%
% Defines name and label format for cleveref.
% Again, the formatted counter #1 is passed to @hyb such that is nicely printed as
%     "H^{human-readable series name}_{hybrid counter}"
% and not as
%     {roman number of series}{arabic numer of hybrid}
%
crefname{hybrid}{hybrid}{hybrids}
Crefname{hybrid}{Hybrid}{Hybrids}
creflabelformat{hybrid}{#2$@expand@hyb{#1}$#3}

makeatother

begin{document}

To prove security, we define a sequence of hybrids first.

newhybridseries{mathrm{security}}

begin{hybrid}[Foo]label{hyb:sec-foo}
This namecref{hyb:sec-foo} modifies ldots
end{hybrid}

begin{hybrid}[Bar]label{hyb:sec-bar}
Didel di dum
end{hybrid}

We now show the indistinguishability of these hybrids, i.e. we show that
% TODO: Does not work.
begin{equation}
  hyb{hyb:sec-foo} equiv hyb{hyb:sec-bar}
end{equation}
holds.

Privacy is also proven by a sequence of hybrds.

newhybridseries{mathrm{privacy}}

begin{hybrid}[Foobar]label{hyb:priv-foobar}
As in cref{hyb:sec-foo}, this namecref{hyb:priv-foobar} also ldots
end{hybrid}

begin{hybrid}[Foo]label{hyb:priv-foo}
Didel di dum
end{hybrid}

begin{table}[hbtp]centering%
  begin{tabular}{l c c c}
    Label                     &  texttt{ref}           &  texttt{labelcref}           &      texttt{cref} 
    texttt{hyb:sec-foo}      &  ref{hyb:sec-foo}      &  labelcref{hyb:sec-foo}      &  cref{hyb:sec-foo}
    texttt{hyb:sec-bar}      &  ref{hyb:sec-bar}      &  labelcref{hyb:sec-bar}      &  cref{hyb:sec-bar}
    texttt{hyb:priv-foobar}  &  ref{hyb:priv-foobar}  &  labelcref{hyb:priv-foobar}  &  cref{hyb:priv-foobar}
    texttt{hyb:priv-foo}     &  ref{hyb:priv-foo}     &  labelcref{hyb:priv-foo}     &  cref{hyb:priv-foo}
  end{tabular}%
  caption{Test of References and Labels}%
end{table}

end{document}