In cryptography, the McEliece cryptosystem is an asymmetric encryption algorithm developed in 1978 by Robert McEliece.^[1] It was the first such scheme to use randomization in the encryption process. The algorithm has never gained much acceptance in the cryptographic community, but is a candidate for "post-quantum cryptography", as it is immune to attacks using Shor's algorithm and – more generally – measuring coset states using Fourier sampling.^[2]

The algorithm is based on the hardness of decoding a general linear code (which is known to be NP-hard^[3]). For a description of the private key, an error-correcting code is selected for which an efficient decoding algorithm is known, and which is able to correct ${\displaystyle t}$ errors. The original algorithm uses binary Goppa codes (subfield codes of algebraic geometry codes of a genus-0 curve over finite fields of characteristic 2); these codes can be efficiently decoded, thanks to an algorithm due to Patterson.^[4] The public key is derived from the private key by disguising the selected code as a general linear code. For this, the code's generator matrix ${\displaystyle G}$ is perturbated by two randomly selected invertible matrices ${\displaystyle S}$ and ${\displaystyle P}$ (see below).

Variants of this cryptosystem exist, using different types of codes. Most of them were proven less secure; they were broken by structural decoding.

McEliece with Goppa codes has resisted cryptanalysis so far. The most effective attacks known use information-set decoding algorithms. A 2008 paper describes both an attack and a fix.^[5] Another paper shows that for quantum computing, key sizes must be increased by a factor of four due to improvements in information set decoding.^[6]

The McEliece cryptosystem has some advantages over, for example, RSA. The encryption and decryption are faster.^[7] For a long time, it was thought that McEliece could not be used to produce signatures. However, a signature scheme can be constructed based on the Niederreiter scheme, the dual variant of the McEliece scheme. One of the main disadvantages of McEliece is that the private and public keys are large matrices. For a standard selection of parameters, the public key is 512 kilobits long.

Scheme definition

McEliece consists of three algorithms: a probabilistic key generation algorithm which produces a public and a private key, a probabilistic encryption algorithm, and a deterministic decryption algorithm.

All users in a McEliece deployment share a set of common security parameters: ${\displaystyle n,k,t}$.

Key generation

The principle is that Alice chooses a linear code ${\displaystyle C}$ from some family of codes for which she knows an efficient decoding algorithm, and to make ${\displaystyle C}$ public knowledge but keep the decoding algorithm secret. Such a decoding algorithm requires not just knowing ${\displaystyle C}$, in the sense of knowing an arbitrary generator matrix, but requires one to know the parameters used when specifying ${\displaystyle C}$ in the chosen family of codes. For instance, for binary Goppa codes, this information would be the Goppa polynomial and the code locators. Therefore, Alice may publish a suitably obfuscated generator matrix of ${\displaystyle C}$.

More specifically, the steps are as follows:

1. Alice selects a binary ${\displaystyle (n,k)}$-linear code ${\displaystyle C}$ capable of (efficiently) correcting ${\displaystyle t}$ errors from some large family of codes, e.g. binary Goppa codes. This choice should give rise to an efficient decoding algorithm ${\displaystyle A}$. Let also ${\displaystyle G}$ be any generator matrix for ${\displaystyle C}$. Any linear code has many generator matrices, but often there is a natural choice for this family of codes. Knowing this would reveal ${\displaystyle A}$ so it should be kept secret.
2. Alice selects a random ${\displaystyle k\times k}$ binary non-singular matrix ${\displaystyle S}$.
3. Alice selects a random ${\displaystyle n\times n}$ permutation matrix ${\displaystyle P}$.
4. Alice computes the ${\displaystyle k\times n}$ matrix ${\displaystyle {\hat {G))=SGP}$.
5. Alice's public key is ${\displaystyle ({\hat {G)),t)}$; her private key is ${\displaystyle (S,P,A)}$. Note that ${\displaystyle A}$ could be encoded and stored as the parameters used for selecting ${\displaystyle C}$.

Message encryption

Suppose Bob wishes to send a message m to Alice whose public key is ${\displaystyle ({\hat {G)),t)}$:

1. Bob encodes the message ${\displaystyle m}$ as a binary string of length ${\displaystyle k}$.
2. Bob computes the vector ${\displaystyle c^{\prime }=m{\hat {G))}$.
3. Bob generates a random ${\displaystyle n}$-bit vector ${\displaystyle z}$ containing exactly ${\displaystyle t}$ ones (a vector of length ${\displaystyle n}$ and weight ${\displaystyle t}$)^[1]
4. Bob computes the ciphertext as ${\displaystyle c=c^{\prime }+z}$.

Message decryption

Upon receipt of ${\displaystyle c}$, Alice performs the following steps to decrypt the message:

1. Alice computes the inverse of ${\displaystyle P}$ (i.e. ${\displaystyle P^{-1))$).
2. Alice computes ${\displaystyle {\hat {c))=cP^{-1))$.
3. Alice uses the decoding algorithm ${\displaystyle A}$ to decode ${\displaystyle {\hat {c))}$ to ${\displaystyle {\hat {m))}$.
4. Alice computes ${\displaystyle m={\hat {m))S^{-1))$.

Proof of message decryption

Note that ${\displaystyle {\hat {c))=cP^{-1}=m{\hat {G))P^{-1}+zP^{-1}=mSG+zP^{-1))$, and that ${\displaystyle P}$ is a permutation matrix, thus ${\displaystyle zP^{-1))$ has weight ${\displaystyle t}$.

The Goppa code ${\displaystyle G}$ can correct up to ${\displaystyle t}$ errors, and the word ${\displaystyle mSG}$ is at distance at most ${\displaystyle t}$ from ${\displaystyle cP^{-1))$. Therefore, the correct code word ${\displaystyle {\hat {m))=mS}$ is obtained.

Multiplying with the inverse of ${\displaystyle S}$ gives ${\displaystyle m={\hat {m))S^{-1}=mSS^{-1))$, which is the plain text message.

Key sizes

Because there is a free choice in the matrix ${\displaystyle S}$, it is common to express ${\displaystyle {\hat {G))}$ in "systematic form" so that the last ${\displaystyle k}$ columns correspond to the identity matrix ${\displaystyle {\hat {G))=({\tilde {G))|I)}$. This reduces the key size to ${\displaystyle (n-k)\times k}$.^[8][9] McEliece originally suggested security parameter sizes of ${\displaystyle n=1024,k=524,t=50}$,^[1] resulting in a public key size of 524*(1024−524) = 262,000 bits. Recent analysis suggests parameter sizes of ${\displaystyle n=2048,k=1751,t=27}$ for 80 bits of security when using standard algebraic decoding, or ${\displaystyle n=1632,k=1269,t=34}$ when using list decoding for the Goppa code, giving rise to public key sizes of 520,047 and 460,647 bits respectively.^[5] For resiliency against quantum computers, sizes of ${\displaystyle n=6960,k=5413,t=119}$ with Goppa code were proposed, giving the size of public key of 8,373,911 bits.^[10] In its round 3 submission to the NIST post quantum standardization the highest level of security, level 5 is given for parameter sets 6688128, 6960119, and 8192128. The parameters are ${\displaystyle n=6688,k=128,t=13}$, ${\displaystyle n=6960,k=119,t=13}$, ${\displaystyle n=8192,k=128,t=13}$ respectively.

Attacks

An attack consists of an adversary, who knows the public key ${\displaystyle ({\hat {G)),t)}$ but not the private key, deducing the plaintext from some intercepted ciphertext ${\displaystyle y\in \mathbb {F} _{2}^{n))$. Such attempts should be infeasible.

There are two main branches of attacks for McEliece:

Brute-force / unstructured attacks

The attacker knows ${\displaystyle {\hat {G))}$ which is the generator matrix of an ${\displaystyle (n,k)}$ code ${\displaystyle {\hat {C))}$ which is combinatorially able to correct ${\displaystyle t}$ errors. The attacker may ignore the fact that ${\displaystyle {\hat {C))}$ is really the obfuscation of a structured code chosen from a specific family, and instead just use an algorithm for decoding with any linear code. Several such algorithms exist, such as going through each codeword of the code, syndrome decoding, or information set decoding.

Decoding a general linear code, however, is known to be NP-hard,^[3] however, and all of the above-mentioned methods have exponential running time.

In 2008, Bernstein, Lange, and Peters^[5] described a practical attack on the original McEliece cryptosystem, using the information set decoding method by Stern.^[11] Using the parameters originally suggested by McEliece, the attack could be carried out in 2^60.55 bit operations. Since the attack is embarrassingly parallel (no communication between nodes is necessary), it can be carried out in days on modest computer clusters.

Structural attacks

The attacker may instead attempt to recover the "structure" of ${\displaystyle C}$, thereby recovering the efficient decoding algorithm ${\displaystyle A}$ or another sufficiently strong, efficient decoding algorithm.

The family of codes from which ${\displaystyle C}$ is chosen completely determines whether this is possible for the attacker. Many code families have been proposed for McEliece, and most of them have been completely "broken" in the sense that attacks which recover an efficient decoding algorithm has been found, such as Reed-Solomon codes.

The originally proposed binary Goppa codes remain one of the few suggested families of codes which have largely resisted attempts at devising structural attacks.

Post-quantum encryption candidate

A variant of this algorithm combined with NTS-KEM^[12] was entered into and selected during the third round of the NIST post-quantum encryption competition.^[13]