In cryptography, XTR is an algorithm for public-key encryption. XTR stands for 'ECSTR', which is an abbreviation for Efficient and Compact Subgroup Trace Representation. It is a method to represent elements of a subgroup of a multiplicative group of a finite field. To do so, it uses the trace over ${\displaystyle GF(p^{2})}$ to represent elements of a subgroup of ${\displaystyle GF(p^{6})^{*))$.

From a security point of view, XTR relies on the difficulty of solving Discrete Logarithm related problems in the full multiplicative group of a finite field. Unlike many cryptographic protocols that are based on the generator of the full multiplicative group of a finite field, XTR uses the generator ${\displaystyle g}$ of a relatively small subgroup of some prime order ${\displaystyle q}$ of a subgroup of ${\displaystyle GF(p^{6})^{*))$. With the right choice of ${\displaystyle q}$, computing Discrete Logarithms in the group, generated by ${\displaystyle g}$, is, in general, as hard as it is in ${\displaystyle GF(p^{6})^{*))$ and thus cryptographic applications of XTR use ${\displaystyle GF(p^{2})}$ arithmetics while achieving full ${\displaystyle GF(p^{6})}$ security leading to substantial savings both in communication and computational overhead without compromising security. Some other advantages of XTR are its fast key generation, small key sizes and speed.

Fundamentals of XTR

XTR uses a subgroup, commonly referred to as XTR subgroup or just XTR group, of a subgroup called XTR supergroup, of the multiplicative group of a finite field ${\displaystyle GF(p^{6})}$ with ${\displaystyle p^{6))$ elements. The XTR supergroup is of order ${\displaystyle p^{2}-p+1}$, where p is a prime such that a sufficiently large prime q divides ${\displaystyle p^{2}-p+1}$. The XTR subgroup has now order q and is, as a subgroup of ${\displaystyle GF(p^{6})^{*))$, a cyclic group ${\displaystyle \langle g\rangle }$ with generator g. The following three paragraphs will describe how elements of the XTR supergroup can be represented using an element of ${\displaystyle GF(p^{2})}$ instead of an element of ${\displaystyle GF(p^{6})}$ and how arithmetic operations take place in ${\displaystyle GF(p^{2})}$ instead of in ${\displaystyle GF(p^{6})}$.

Arithmetic operations in ${\displaystyle GF(p^{2})}$

Let p be a prime such that p ≡ 2 mod 3 and p² - p + 1 has a sufficiently large prime factor q. Since p² ≡ 1 mod 3 we see that p generates ${\displaystyle (\mathbb {Z} /3\mathbb {Z} )^{*))$ and thus the third cyclotomic polynomial ${\displaystyle \Phi _{3}(x)=x^{2}+x+1}$ is irreducible over ${\displaystyle GF(p)}$. It follows that the roots ${\displaystyle \alpha }$ and ${\displaystyle \alpha ^{p))$ form an optimal normal basis for ${\displaystyle GF(p^{2})}$ over ${\displaystyle GF(p)}$ and

${\displaystyle GF(p^{2})\cong \{x_{1}\alpha +x_{2}\alpha ^{p}:x_{1},x_{2}\in GF(p)\}.}$

Considering that p ≡ 2 mod 3 we can reduce the exponents modulo 3 to get

${\displaystyle GF(p^{2})\cong \{y_{1}\alpha +y_{2}\alpha ^{2}:\alpha ^{2}+\alpha +1=0,y_{1},y_{2}\in GF(p)\}.}$

The cost of arithmetic operations is now given in the following Lemma labeled Lemma 2.21 in "An overview of the XTR public key system":^[1]

Lemma

• Computing x^p is done without using multiplication
• Computing x² takes two multiplications in ${\displaystyle GF(p)}$
• Computing xy takes three multiplications in ${\displaystyle GF(p)}$
• Computing xz-yz^p takes four multiplications in ${\displaystyle GF(p)}$.

Traces over ${\displaystyle GF(p^{2})}$

The trace in XTR is always considered over ${\displaystyle GF(p^{2})}$. In other words, the conjugates of ${\displaystyle h\in GF(p^{6})}$ over ${\displaystyle GF(p^{2})}$ are ${\displaystyle h,h^{p^{2))}$ and ${\displaystyle h^{p^{4))}$ and the trace of ${\displaystyle h}$ is their sum:

${\displaystyle Tr(h)=h+h^{p^{2))+h^{p^{4)).}$

Note that ${\displaystyle Tr(h)\in GF(p^{2})}$ since

${\displaystyle {\begin{aligned}Tr(h)^{p^{2))&=h^{p^{2))+h^{p^{4))+h^{p^{6))\\&=h+h^{p^{2))+h^{p^{4))\\&=Tr(h)\end{aligned))}$

Consider now the generator ${\displaystyle g}$ of the XTR subgroup of a prime order ${\displaystyle q}$. Remember that ${\displaystyle \langle g\rangle }$ is a subgroup of the XTR supergroup of order ${\displaystyle p^{2}-p+1}$, so ${\displaystyle q\mid p^{2}-p+1}$. In the following section we will see how to choose ${\displaystyle p}$ and ${\displaystyle q}$, but for now it is sufficient to assume that ${\displaystyle q>3}$. To compute the trace of ${\displaystyle g}$ note that modulo ${\displaystyle p^{2}-p+1}$ we have

${\displaystyle p^{2}=p-1}$ and

${\displaystyle p^{4}=(p-1)^{2}=p^{2}-2p+1=-p}$

and thus

${\displaystyle {\begin{aligned}Tr(g)&=g+g^{p^{2))+g^{p^{4))\\&=g+g^{p-1}+g^{-p}.\end{aligned))}$

The product of the conjugates of ${\displaystyle g}$ equals ${\displaystyle 1}$, i.e., that ${\displaystyle g}$ has norm 1.

The crucial observation in XTR is that the minimal polynomial of ${\displaystyle g}$ over ${\displaystyle GF(p^{2})}$

${\displaystyle (x-g)\!\ (x-g^{p-1})(x-g^{-p})}$

simplifies to

${\displaystyle x^{3}-Tr(g)\!\ x^{2}+Tr(g)^{p}x-1}$

which is fully determined by ${\displaystyle Tr(g)}$. Consequently, conjugates of ${\displaystyle g}$, as roots of the minimal polynomial of ${\displaystyle g}$ over ${\displaystyle GF(p^{2})}$, are completely determined by the trace of ${\displaystyle g}$. The same is true for any power of ${\displaystyle g}$: conjugates of ${\displaystyle g^{n))$ are roots of polynomial

${\displaystyle x^{3}-Tr(g^{n})\!\ x^{2}+Tr(g^{n})^{p}x-1}$

and this polynomial is completely determined by ${\displaystyle Tr(g^{n})}$.

The idea behind using traces is to replace ${\displaystyle g^{n}\in GF(p^{6})}$ in cryptographic protocols, e.g. the Diffie–Hellman key exchange by ${\displaystyle Tr(g^{n})\in GF(p^{2})}$ and thus obtaining a factor of 3 reduction in representation size. This is, however, only useful if there is a quick way to obtain ${\displaystyle Tr(g^{n})}$ given ${\displaystyle Tr(g)}$. The next paragraph gives an algorithm for the efficient computation of ${\displaystyle Tr(g^{n})}$. In addition, computing ${\displaystyle Tr(g^{n})}$ given ${\displaystyle Tr(g)}$ turns out to be quicker than computing ${\displaystyle g^{n))$ given ${\displaystyle g}$.^[1]

Algorithm for the quick computation of ${\displaystyle Tr(g^{n})}$ given ${\displaystyle Tr(g)}$

A. Lenstra and E. Verheul give this algorithm in their paper titled The XTR public key system in.^[2] All the definitions and lemmas necessary for the algorithm and the algorithm itself presented here, are taken from that paper.

Definition For c in ${\displaystyle GF(p^{2})}$ define

${\displaystyle F(c,X)=X^{3}-cX^{2}+c^{p}X-1\in GF(p^{2})[X].}$

Definition Let ${\displaystyle h_{0},\!\ h_{1},h_{2))$ denote the, not necessarily distinct, roots of ${\displaystyle F(c,X)}$ in ${\displaystyle GF(p^{6})}$ and let ${\displaystyle n}$ be in ${\displaystyle \mathbb {Z} }$. Define

${\displaystyle c_{n}=h_{0}^{n}+h_{1}^{n}+h_{2}^{n}.}$

Properties of ${\displaystyle c_{n))$ and ${\displaystyle F(c,X)}$

1. ${\displaystyle c=c_{1))$
2. ${\displaystyle c_{-n}=c_{np}=c_{n}^{p))$
3. ${\displaystyle c_{n}\in GF(p^{2}){\text{ for ))n\in \mathbb {Z} }$
4. ${\displaystyle c_{u+v}=c_{u}c_{v}-c_{v}^{p}c_{u-v}+c_{u-2v}{\text{ for ))u,v\in \mathbb {Z} }$
5. Either all ${\displaystyle h_{j))$ have order dividing ${\displaystyle p^{2}-p+1}$ and ${\displaystyle >3}$ or all ${\displaystyle h_{j))$ are in ${\displaystyle GF(p^{2})}$. In particular, ${\displaystyle F(c,X)}$ is irreducible if and only if its roots have order diving ${\displaystyle p^{2}-p+1}$ and ${\displaystyle >3}$.
6. ${\displaystyle F(c,X)}$ is reducible over ${\displaystyle GF(p^{2})}$ if and only if ${\displaystyle c_{p+1}\in GF(p)}$

Lemma Let ${\displaystyle c,\!\ c_{n-1},c_{n},c_{n+1))$ be given.

1. Computing ${\displaystyle c_{2n}=c_{n}^{2}-2c_{n}^{p))$ takes two multiplication in ${\displaystyle GF(p)}$.
2. Computing ${\displaystyle c_{n+2}=c_{n+1}\cdot c-c^{p}\cdot c_{n}+c_{n-1))$ takes four multiplication in ${\displaystyle GF(p)}$.
3. Computing ${\displaystyle c_{2n-1}=c_{n-1}\cdot c_{n}-c^{p}\cdot c_{n}^{p}+c_{n+1}^{p))$ takes four multiplication in ${\displaystyle GF(p)}$.
4. Computing ${\displaystyle c_{2n+1}=c_{n+1}\cdot c_{n}-c\cdot c_{n}^{p}+c_{n-1}^{p))$ takes four multiplication in ${\displaystyle GF(p)}$.

Definition Let ${\displaystyle S_{n}(c)=(c_{n-1},c_{n},c_{n+1})\in GF(p^{2})^{3))$.

Algorithm 1 for computation of ${\displaystyle S_{n}(c)}$ given ${\displaystyle n}$ and ${\displaystyle c}$

• If ${\displaystyle n<0}$ apply this algorithm to ${\displaystyle -n}$ and ${\displaystyle c}$, and apply Property 2 to the resulting value.
• If ${\displaystyle n=0}$, then ${\displaystyle S_{0}(c)\!\ =(c^{p},3,c)}$.
• If ${\displaystyle n=1}$, then ${\displaystyle S_{1}(c)\!\ =(3,c,c^{2}-2c^{p})}$.
• If ${\displaystyle n=2}$, use the computation of ${\displaystyle c_{n+2}=c_{n+1}\cdot c-c^{p}\cdot c_{n}+c_{n-1))$ and ${\displaystyle S_{1}(c)}$ to find ${\displaystyle c_{3))$ and thereby ${\displaystyle S_{2}(c)}$.
• If ${\displaystyle n>2}$, to compute ${\displaystyle S_{n}(c)}$ define

${\displaystyle {\bar {S))_{i}(c)=S_{2i+1}(c)}$

and ${\displaystyle {\bar {m))=n}$ if n is odd and ${\displaystyle {\bar {m))=n-1}$ otherwise. Let ${\displaystyle {\bar {m))=2m+1,k=1}$ and compute ${\displaystyle {\bar {S))_{k}(c)=S_{3}(c)}$ using the Lemma above and ${\displaystyle S_{2}(c)}$. Let further

${\displaystyle m=\sum _{j=0}^{r}m_{j}2^{j))$

with ${\displaystyle m_{j}\in {0,1))$ and ${\displaystyle m_{r}=1}$. For ${\displaystyle j=r-1,r-2,...,0}$ in succession, do the following:

• If ${\displaystyle m_{j}=0}$, use ${\displaystyle {\bar {S))_{k}(c)}$ to compute ${\displaystyle {\bar {S))_{2k}(c)}$.
• If ${\displaystyle m_{j}=1}$, use ${\displaystyle {\bar {S))_{k}(c)}$ to compute ${\displaystyle {\bar {S))_{2k+1}(c)}$.
• Replace ${\displaystyle k}$ by ${\displaystyle 2k+m_{j))$.

When these iterations finish, ${\displaystyle k=m}$ and ${\displaystyle S_{\bar {m))(c)={\bar {S))_{m}(c)}$. If n is even use ${\displaystyle S_{\bar {m))(c)}$ to compute ${\displaystyle {\bar {S))_{m+1}(c)}$.

Parameter selection

Finite field and subgroup size selection

In order to take advantage of the above described representations of elements with their traces and furthermore ensure sufficient security, that will be discussed below, we need to find primes ${\displaystyle p}$ and ${\displaystyle q}$, where ${\displaystyle p}$ denotes the characteristic of the field ${\displaystyle GF(p^{6})}$ with ${\displaystyle p\equiv 2\ {\text{mod))\ 3}$ and ${\displaystyle q}$ is the size of the subgroup, such that ${\displaystyle q}$ divides ${\displaystyle p^{2}-p+1}$.

We denote with ${\displaystyle P}$ and ${\displaystyle Q}$ the sizes of ${\displaystyle p}$ and ${\displaystyle q}$ in bits. To achieve security comparable to 1024-bit RSA, we should choose ${\displaystyle 6P}$ about 1024, i.e. ${\displaystyle P\approx 170}$ and ${\displaystyle Q}$ can be around 160.

A first easy algorithm to compute such primes ${\displaystyle p}$ and ${\displaystyle q}$ is the next Algorithm A:

Algorithm A

1. Find ${\displaystyle r\in \mathbb {Z} }$ such that ${\displaystyle q=r^{2}-r+1}$ is a ${\displaystyle Q}$-bit prime.
2. Find ${\displaystyle k\in \mathbb {Z} }$ such that ${\displaystyle p=r+k\cdot q}$ is a ${\displaystyle P}$-bit prime with ${\displaystyle p\equiv 2\ {\text{mod))\ 3}$.

Correctness of Algorithm A:

It remains to check that ${\displaystyle q\mid p^{2}-p+1}$ because all the other necessary properties are obviously satisfied per definition of ${\displaystyle p}$ and ${\displaystyle q}$. We easily see that ${\displaystyle p^{2}-p+1=r^{2}+2rkq+k^{2}q^{2}-r-kq+1=r^{2}-r+1+q(2rk+k^{2}q-k)=q(1+2rk+k^{2}q-k)}$ which implies that ${\displaystyle q\mid p^{2}-p+1}$.

Algorithm A is very fast and can be used to find primes ${\displaystyle p}$ that satisfy a degree-two polynomial with small coefficients. Such ${\displaystyle p}$ lead to fast arithmetic operations in ${\displaystyle GF(p)}$. In particular if the search for ${\displaystyle k}$ is restricted to ${\displaystyle k=1}$, which means looking for an ${\displaystyle r}$ such that both ${\displaystyle r^{2}-r+1{\text{ and ))r^{2}+1}$ are prime and such that ${\displaystyle r^{2}+1\equiv 2{\text{ mod ))3}$, the primes ${\displaystyle p}$ have this nice form. Note that in this case ${\displaystyle r}$ must be even and ${\displaystyle r\equiv 1{\text{ mod ))4}$.

On the other hand, such ${\displaystyle p}$ may be undesirable from a security point of view because they may make an attack with the Discrete Logarithm variant of the Number Field Sieve easier.

The following Algorithm B doesn't have this disadvantage, but it also doesn't have the fast arithmetic modulo ${\displaystyle p}$ Algorithm A has in that case.

Algorithm B

1. Select a ${\displaystyle Q}$-bit prime ${\displaystyle q}$ so that ${\displaystyle q\equiv 7\ {\text{mod))\ 12}$.
2. Find the roots ${\displaystyle r_{1))$ and ${\displaystyle r_{2))$ of ${\displaystyle X^{2}-X+1\ {\text{mod))\ q}$.
3. Find a ${\displaystyle k\in \mathbb {Z} }$ such that ${\displaystyle p=r_{i}+k\cdot q}$ is a ${\displaystyle P}$-bit prime with ${\displaystyle p\equiv 2\ {\text{mod))\ 3}$ for ${\displaystyle i\in \{1,2\))$

Correctness of Algorithm B:

Since we chose ${\displaystyle q\equiv 7\ {\text{mod))\ 12}$ it follows immediately that ${\displaystyle q\equiv 1\ {\text{mod))\ 3}$ (because ${\displaystyle 7\equiv 1\ {\text{mod))\ 3}$ and ${\displaystyle 3\mid 12}$). From that and quadratic reciprocity we can deduce that ${\displaystyle r_{1))$ and ${\displaystyle r_{2))$ exist.

To check that ${\displaystyle q\mid p^{2}-p+1}$ we consider again ${\displaystyle p^{2}-p+1}$ for ${\displaystyle r_{i}\in \{1,2\))$ and get that ${\displaystyle p^{2}-p+1=r_{i}^{2}+2r_{i}kq+k^{2}q^{2}-r_{i}-kq+1=r_{i}^{2}-r_{i}+1+q(2rk+k^{2}q-k)=q(2rk+k^{2}q-k)}$, since ${\displaystyle r_{1))$ and ${\displaystyle r_{2))$ are roots of ${\displaystyle X^{2}-X+1}$ and hence ${\displaystyle q\mid p^{2}-p+1}$.

Subgroup selection

In the last paragraph we have chosen the sizes ${\displaystyle p}$ and ${\displaystyle q}$ of the finite field ${\displaystyle GF(p^{6})}$ and the multiplicative subgroup of ${\displaystyle GF(p^{6})^{*))$, now we have to find a subgroup ${\displaystyle \langle g\rangle }$ of ${\displaystyle GF\!\ (p^{6})^{*))$ for some ${\displaystyle g\in GF(p^{6})}$ such that ${\displaystyle \mid \!\!\langle g\rangle \!\!\mid =q}$.

However, we do not need to find an explicit ${\displaystyle g\in GF(p^{6})}$, it suffices to find an element ${\displaystyle c\in GF(p^{2})}$ such that ${\displaystyle c=Tr(g)}$ for an element ${\displaystyle g\in GF(p^{6})}$ of order ${\displaystyle q}$. But, given ${\displaystyle Tr(g)}$, a generator ${\displaystyle g}$ of the XTR (sub)group can be found by determining any root of ${\displaystyle F(Tr(g),\ X)}$ which has been defined above. To find such a ${\displaystyle c}$ we can take a look at property 5 of ${\displaystyle F(c,\ X)}$ here stating that the roots of ${\displaystyle F(c,\ X)}$ have an order dividing ${\displaystyle p^{2}-p+1}$ if and only if ${\displaystyle F(c,\ X)}$ is irreducible. After finding such ${\displaystyle c}$ we need to check if it really is of order ${\displaystyle q}$, but first we focus on how to select ${\displaystyle c\in GF(p^{2})}$ such that ${\displaystyle F(c,\ X)}$ is irreducible.

An initial approach is to select ${\displaystyle c\in GF(p^{2})\backslash GF(p)}$ randomly which is justified by the next lemma.

Lemma: For a randomly selected ${\displaystyle c\in GF(p^{2})}$ the probability that ${\displaystyle F(c,\ X)=X^{3}-cX^{2}+c^{p}X-1\in GF(p^{2})[X]}$ is irreducible is about one third.

Now the basic algorithm to find a suitable ${\displaystyle Tr(g)}$ is as follows:

Outline of the algorithm

1. Pick a random ${\displaystyle c\in GF(p^{2})\backslash GF(p)}$.
2. If ${\displaystyle F(c,\ X)}$ is reducible, then return to Step 1.
3. Use Algorithm 1 to compute ${\displaystyle d=c_{(p^{2}-p+1)/q))$.
4. If ${\displaystyle d}$ is not of order ${\displaystyle q}$, return to Step 1.
5. Let ${\displaystyle Tr(g)=d}$.

It turns out that this algorithm indeed computes an element of ${\displaystyle GF(p^{2})}$ that equals ${\displaystyle Tr(g)}$ for some ${\displaystyle g\in GF(p^{6})}$ of order ${\displaystyle q}$.

More details to the algorithm, its correctness, runtime and the proof of the Lemma can be found in "An overview of the XTR public key system" in.^[1]

Cryptographic schemes

In this section it is explained how the concepts above using traces of elements can be applied to cryptography. In general, XTR can be used in any cryptosystem that relies on the (subgroup) Discrete Logarithm problem. Two important applications of XTR are the Diffie–Hellman key agreement and the ElGamal encryption. We will start first with Diffie–Hellman.

XTR-DH key agreement

We suppose that both Alice and Bob have access to the XTR public key data ${\displaystyle \left(p,q,Tr(g)\right)}$ and intend to agree on a shared secret key ${\displaystyle K}$. They can do this by using the following XTR version of the Diffie–Hellman key exchange:

1. Alice picks ${\displaystyle a\in \mathbb {Z} }$ randomly with ${\displaystyle 1<a<q-2}$, computes with Algorithm 1 ${\displaystyle S_{a}(Tr(g))=\left(Tr(g^{a-1}),Tr(g^{a}),Tr(g^{a+1})\right)\in GF(p^{2})^{3))$ and sends ${\displaystyle Tr(g^{a})\in GF(p^{2})}$ to Bob.
2. Bob receives ${\displaystyle Tr(g^{a})}$ from Alice, selects at random ${\displaystyle b\in \mathbb {Z} }$ with ${\displaystyle 1<b<q-2}$, applies Algorithm 1 to compute ${\displaystyle S_{b}(Tr(g))=\left(Tr(g^{b-1}),Tr(g^{b}),Tr(g^{b+1})\right)\in GF(p^{2})^{3))$ and sends ${\displaystyle Tr(g^{b})\in GF(p^{2})}$ to Alice.
3. Alice receives ${\displaystyle Tr(g^{b})}$ from Bob, computes with Algorithm 1 ${\displaystyle S_{a}(Tr(g^{b}))=\left(Tr(g^{(a-1)b}),Tr(g^{ab}),Tr(g^{(a+1)b})\right)\in GF(p^{2})^{3))$ and determines ${\displaystyle K}$ based on ${\displaystyle Tr(g^{ab})\in GF(p^{2})}$.
4. Bob analogously applies Algorithm 1 to compute ${\displaystyle S_{b}(Tr(g^{a}))=\left(Tr(g^{a(b-1)}),Tr(g^{ab}),Tr(g^{a(b+1)})\right)\in GF(p^{2})^{3))$ and also determines ${\displaystyle K}$ based on ${\displaystyle Tr(g^{ab})\in GF(p^{2})}$.

XTR ElGamal encryption

For the ElGamal encryption we suppose now that Alice is the owner of the XTR public key data ${\displaystyle (p,q,Tr(g))}$ and that she has selected a secret integer ${\displaystyle k}$, computed ${\displaystyle Tr(g^{k})}$ and published the result. Given Alice's XTR public key data ${\displaystyle \left(p,q,Tr(g),Tr(g^{k})\right)}$, Bob can encrypt a message ${\displaystyle M}$, intended for Alice, using the following XTR version of the ElGamal encryption:

1. Bob selects randomly a ${\displaystyle b\in \mathbb {Z} }$ with ${\displaystyle 1<b<q-2}$ and computes with Algorithm 1 ${\displaystyle S_{b}(Tr(g))=\left(Tr(g^{b-1}),Tr(g^{b}),Tr(g^{b+1})\right)\in GF(p^{2})^{3))$.
2. Bob next applies Algorithm 1 to compute ${\displaystyle S_{b}(Tr(g^{k}))=\left(Tr(g^{(b-1)k}),Tr(g^{bk}),Tr(g^{(b+1)k})\right)\in GF(p^{2})^{3))$.
3. Bob determines a symmetric encryption key ${\displaystyle K}$ based on ${\displaystyle Tr(g^{bk})\in GF(p^{2})}$.
4. Bob uses an agreed upon symmetric encryption method with key ${\displaystyle K}$ to encrypt his message ${\displaystyle M}$, resulting in the encryption ${\displaystyle E}$.
5. Bob sends ${\displaystyle (Tr(g^{b}),\ E)}$ to Alice.

Upon receipt of ${\displaystyle (Tr(g^{b}),\ E)}$, Alice decrypts the message in the following way:

1. Alice computes ${\displaystyle S_{k}(Tr(g^{b}))=\left(Tr(g^{b(k-1)}),Tr(g^{bk}),Tr(g^{b(k+1)})\right)\in GF(p^{2})^{3))$.
2. Alice determines the symmetric key ${\displaystyle K}$ based on ${\displaystyle Tr(g^{bk})\in GF(p^{2})}$.
3. Alice uses the agreed upon symmetric encryption method with key ${\displaystyle K}$ to decrypt ${\displaystyle E}$, resulting in the original message ${\displaystyle M}$.

The here described encryption scheme is based on a common hybrid version of the ElGamal encryption, where the secret key ${\displaystyle K}$ is obtained by an asymmetric public key system and then the message is encrypted with a symmetric key encryption method Alice and Bob agreed to.

In the more traditional ElGamal encryption the message is restricted to the key space, which would here be ${\displaystyle GF(p^{2})}$, because ${\displaystyle Tr(g)\in GF(p^{2})\ \forall p\in GF(p^{6})^{*))$. The encryption in this case is the multiplication of the message with the key, which is an invertible operation in the key space ${\displaystyle GF(p^{2})}$.

Concretely this means if Bob wants to encrypt a message ${\displaystyle M\!\ '}$, first he has to convert it into an element ${\displaystyle M}$ of ${\displaystyle GF(p^{2})}$ and then compute the encrypted message ${\displaystyle E}$ as ${\displaystyle E=K\cdot M\in GF(p^{2})}$. Upon receipt of the encrypted message ${\displaystyle E}$ Alice can recover the original message ${\displaystyle M}$ by computing ${\displaystyle M=E\cdot K^{-1))$, where ${\displaystyle K^{-1))$ is the inverse of ${\displaystyle K}$ in ${\displaystyle GF(p^{2})}$.

Security

In order to say something about the security properties of the above explained XTR encryption scheme, first it is important to check the security of the XTR group, which means how hard it is to solve the Discrete Logarithm problem there. The next part will then state the equivalency between the Discrete Logarithm problem in the XTR group and the XTR version of the discrete logarithm problem, using only the traces of elements.

Discrete logarithms in a general ${\displaystyle GF\left(p^{t}\right)}$

Let now ${\displaystyle \langle \gamma \rangle }$ be a multiplicative group of order ${\displaystyle \omega }$. The security of the Diffie–Hellman protocol in ${\displaystyle \langle \gamma \rangle }$ relies on the Diffie–Hellman (DH) problem of computing ${\displaystyle \gamma ^{xy}{\text{ given ))\gamma ,\gamma ^{x}{\text{ and ))\gamma ^{y))$. We write ${\displaystyle DH(\gamma ^{x},\ \gamma ^{y})=\gamma ^{xy))$. There are two other problems related to the DH problem. The first one is the Diffie–Hellman Decision (DHD) problem to determine if ${\displaystyle c=DH(a,b)}$ for given ${\displaystyle a,b,c\in \langle \gamma \rangle }$ and the second one is the Discrete Logarithm (DL) problem to find ${\displaystyle x=DL(a)}$ for a given ${\displaystyle a=\gamma ^{x}\in \langle \gamma \rangle {\text{ with ))0\leq x<\omega }$.

The DL problem is at least as difficult as the DH problem and it is generally assumed that if the DL problem in ${\displaystyle \langle \gamma \rangle }$ is intractable, then so are the other two.

Given the prime factorization of ${\displaystyle \omega }$ the DL problem in ${\displaystyle \langle \gamma \rangle }$ can be reduced to the DL problem in all subgroups of ${\displaystyle \langle \gamma \rangle }$ with prime order due to the Pohlig–Hellman algorithm. Hence ${\displaystyle \omega }$ can safely be assumed to be prime.

For a subgroup ${\displaystyle \langle \gamma \rangle }$ of prime order ${\displaystyle \omega }$ of the multiplicative group ${\displaystyle GF\left(p^{t}\right)^{*))$ of an extension field ${\displaystyle GF(p^{t})}$ of ${\displaystyle GF(p)}$ for some ${\displaystyle t}$, there are now two possible ways to attack the system. One can either focus on the whole multiplicative group or on the subgroup. To attack the multiplicative group the best known method is the Discrete Logarithm variant of the Number Field Sieve or alternatively in the subgroup one can use one of several methods that take ${\displaystyle {\mathcal {O))({\sqrt {\omega )))}$ operations in ${\displaystyle \langle \gamma \rangle }$, such as Pollard's rho method.

For both approaches the difficulty of the DL problem in ${\displaystyle \langle \gamma \rangle }$ depends on the size of the minimal surrounding subfield of ${\displaystyle \langle \gamma \rangle }$ and on the size of its prime order ${\displaystyle \omega }$. If ${\displaystyle GF\left(p^{t}\right)}$ itself is the minimal surrounding subfield of ${\displaystyle \langle \gamma \rangle }$ and ${\displaystyle \omega }$ is sufficiently large, then the DL problem in ${\displaystyle \langle \gamma \rangle }$ is as hard as the general DL problem in ${\displaystyle GF\left(p^{t}\right)}$.

The XTR parameters are now chosen in such a way that ${\displaystyle p}$ is not small, ${\displaystyle q}$ is sufficiently large and ${\displaystyle \langle g\rangle }$ cannot be embedded in a true subfield of ${\displaystyle GF(p^{6})}$, since ${\displaystyle q\mid p^{2}-p+1}$ and ${\displaystyle p^{2}-p+1}$ is a divisor of ${\displaystyle \mid \!GF(p^{6})^{*}\!\mid =p^{6}-1}$, but it does not divide ${\displaystyle p^{s}-1{\text{ for ))s\in \{1,2,3\))$ and thus ${\displaystyle \langle g\rangle }$ cannot be a subgroup of ${\displaystyle GF\!\ (p^{s})^{*))$ for ${\displaystyle s\in \{1,2,3\))$. It follows that the DL problem in the XTR group may be assumed as hard as the DL problem in ${\displaystyle GF(p^{6})}$.

Security of XTR

Cryptographic protocols that are based on Discrete Logarithms can use many different types of subgroups like groups of points of elliptic curves or subgroups of the multiplicative group of a finite field like the XTR group. As we have seen above the XTR versions of the Diffie–Hellman and ElGamal encryption protocol replace using elements of the XTR group by using their traces. This means that the security of the XTR versions of these encryption schemes is no longer based on the original DH, DHD or DL problems. Therefore, the XTR versions of those problems need to be defined and we will see that they are equivalent (in the sense of the next definition) to the original problems.

Definitions:

• We define the XTR-DH problem as the problem of computing ${\displaystyle Tr(g^{xy})}$ given ${\displaystyle Tr(g^{x})}$ and ${\displaystyle Tr(g^{y})}$ and we write ${\displaystyle XDH(g^{x},\ g^{y})=g^{xy))$.
• The XTR-DHD problem is the problem of determining whether ${\displaystyle XDH(a,b)=c}$ for ${\displaystyle a,b,c\in Tr(\langle g\rangle )}$.
• Given ${\displaystyle a\in Tr(\langle g\rangle )}$, the XTR-DL problem is to find ${\displaystyle x=XDL(a)}$, i.e. ${\displaystyle 0\leq x<q}$ such that ${\displaystyle a=Tr(g^{x})}$.
• We say that problem ${\displaystyle {\mathcal {A))}$ is (a,b)-equivalent to problem ${\displaystyle {\mathcal {B))}$, if any instance of problem ${\displaystyle {\mathcal {A))}$ (or ${\displaystyle {\mathcal {B))}$) can be solved by at most a (or b) calls to an algorithm solving problem ${\displaystyle {\mathcal {B))}$ (or ${\displaystyle {\mathcal {A))}$).

After introducing the XTR versions of these problems the next theorem is an important result telling us the connection between the XTR and the non-XTR problems, which are in fact equivalent. This implies that the XTR representation of elements with their traces is, as can be seen above, faster by a factor of 3 than the usual representation without compromising security.

Theorem The following equivalencies hold:

i. The XTR-DL problem is (1,1)-equivalent to the DL problem in ${\displaystyle \langle g\rangle }$.

ii. The XTR-DH problem is (1,2)-equivalent to the DH problem in ${\displaystyle \langle g\rangle }$.

iii. The XTR-DHD problem is (3,2)-equivalent to the DHD problem in ${\displaystyle \langle g\rangle }$.

This means that an algorithm solving either XTR-DL, XTR-DH or XTR-DHD with non-negligible probability can be transformed into an algorithm solving the corresponding non-XTR problem DL, DH or DHD with non-negligible probability and vice versa. In particular part ii. implies that determining the small XTR-DH key (being an element of ${\displaystyle GF(p^{2})}$) is as hard as determining the whole DH key (being an element of ${\displaystyle GF(p^{6})}$ ) in the representation group ${\displaystyle \langle g\rangle }$.