In mathematics and computer algebra the factorization of a polynomial consists of decomposing it into a product of irreducible factors. This decomposition is theoretically possible and is unique for polynomials with coefficients in any field, but rather strong restrictions on the field of the coefficients are needed to allow the computation of the factorization by means of an algorithm. In practice, algorithms have been designed only for polynomials with coefficients in a finite field, in the field of rationals or in a finitely generated field extension of one of them.

All factorization algorithms, including the case of multivariate polynomials over the rational numbers, reduce the problem to this case; see polynomial factorization. It is also used for various applications of finite fields, such as coding theory (cyclic redundancy codes and BCH codes), cryptography (public key cryptography by the means of elliptic curves), and computational number theory.

As the reduction of the factorization of multivariate polynomials to that of univariate polynomials does not have any specificity in the case of coefficients in a finite field, only polynomials with one variable are considered in this article.

Background

Finite field

The theory of finite fields, whose origins can be traced back to the works of Gauss and Galois, has played a part in various branches of mathematics. Due to the applicability of the concept in other topics of mathematics and sciences like computer science there has been a resurgence of interest in finite fields and this is partly due to important applications in coding theory and cryptography. Applications of finite fields introduce some of these developments in cryptography, computer algebra and coding theory.

A finite field or Galois field is a field with a finite order (number of elements). The order of a finite field is always a prime or a power of prime. For each prime power q = p^r, there exists exactly one finite field with q elements, up to isomorphism. This field is denoted GF(q) or F_q. If p is prime, GF(p) is the prime field of order p; it is the field of residue classes modulo p, and its p elements are denoted 0, 1, ..., p−1. Thus a = b in GF(p) means the same as a ≡ b (mod p).

Irreducible polynomials

Let F be a finite field. As for general fields, a non-constant polynomial f in F[x] is said to be irreducible over F if it is not the product of two polynomials of positive degree. A polynomial of positive degree that is not irreducible over F is called reducible over F.

Irreducible polynomials allow us to construct the finite fields of non-prime order. In fact, for a prime power q, let F_q be the finite field with q elements, unique up to isomorphism. A polynomial f of degree n greater than one, which is irreducible over F_q, defines a field extension of degree n which is isomorphic to the field with qⁿ elements: the elements of this extension are the polynomials of degree lower than n; addition, subtraction and multiplication by an element of F_q are those of the polynomials; the product of two elements is the remainder of the division by f of their product as polynomials; the inverse of an element may be computed by the extended GCD algorithm (see Arithmetic of algebraic extensions).

It follows that, to compute in a finite field of non prime order, one needs to generate an irreducible polynomial. For this, the common method is to take a polynomial at random and test it for irreducibility. For sake of efficiency of the multiplication in the field, it is usual to search for polynomials of the shape xⁿ + ax + b.^{[citation needed][1]}

Irreducible polynomials over finite fields are also useful for pseudorandom number generators using feedback shift registers and discrete logarithm over F_2ⁿ.

The number of irreducible monic polynomials of degree n over F_q is the number of aperiodic necklaces, given by Moreau's necklace-counting function M_q(n). The closely related necklace function N_q(n) counts monic polynomials of degree n which are primary (a power of an irreducible); or alternatively irreducible polynomials of all degrees d which divide n.^[2]

Example

The polynomial P = x⁴ + 1 is irreducible over Q but not over any finite field.

• On any field extension of F₂, P = (x + 1)⁴.
• On every other finite field, at least one of −1, 2 and −2 is a square, because the product of two non-squares is a square and so we have

1. If ${\displaystyle -1=a^{2},}$ then ${\displaystyle P=(x^{2}+a)(x^{2}-a).}$
2. If ${\displaystyle 2=b^{2},}$ then ${\displaystyle P=(x^{2}+bx+1)(x^{2}-bx+1).}$
3. If ${\displaystyle -2=c^{2},}$ then ${\displaystyle P=(x^{2}+cx-1)(x^{2}-cx-1).}$

Complexity

Polynomial factoring algorithms use basic polynomial operations such as products, divisions, gcd, powers of one polynomial modulo another, etc. A multiplication of two polynomials of degree at most n can be done in O(n²) operations in F_q using "classical" arithmetic, or in O(nlog(n) log(log(n)) ) operations in F_q using "fast" arithmetic. A Euclidean division (division with remainder) can be performed within the same time bounds. The cost of a polynomial greatest common divisor between two polynomials of degree at most n can be taken as O(n²) operations in F_q using classical methods, or as O(nlog²(n) log(log(n)) ) operations in F_q using fast methods. For polynomials h, g of degree at most n, the exponentiation h^q mod g can be done with O(log(q)) polynomial products, using exponentiation by squaring method, that is O(n²log(q)) operations in F_q using classical methods, or O(nlog(q)log(n) log(log(n))) operations in F_q using fast methods.

In the algorithms that follow, the complexities are expressed in terms of number of arithmetic operations in F_q, using classical algorithms for the arithmetic of polynomials.

Factoring algorithms

Many algorithms for factoring polynomials over finite fields include the following three stages:

1. Square-free factorization
2. Distinct-degree factorization
3. Equal-degree factorization

An important exception is Berlekamp's algorithm, which combines stages 2 and 3.

Berlekamp's algorithm

Berlekamp's algorithm is historically important as being the first factorization algorithm which works well in practice. However, it contains a loop on the elements of the ground field, which implies that it is practicable only over small finite fields. For a fixed ground field, its time complexity is polynomial, but, for general ground fields, the complexity is exponential in the size of the ground field.

Square-free factorization

The algorithm determines a square-free factorization for polynomials whose coefficients come from the finite field F_q of order q = p^m with p a prime. This algorithm firstly determines the derivative and then computes the gcd of the polynomial and its derivative. If it is not one then the gcd is again divided into the original polynomial, provided that the derivative is not zero (a case that exists for non-constant polynomials defined over finite fields).

This algorithm uses the fact that, if the derivative of a polynomial is zero, then it is a polynomial in x^p, which is, if the coefficients belong to F_p, the pth power of the polynomial obtained by substituting x by x^1/p. If the coefficients do not belong to F_p, the pth root of a polynomial with zero derivative is obtained by the same substitution on x, completed by applying the inverse of the Frobenius automorphism to the coefficients.

This algorithm works also over a field of characteristic zero, with the only difference that it never enters in the blocks of instructions where pth roots are computed. However, in this case, Yun's algorithm is much more efficient because it computes the greatest common divisors of polynomials of lower degrees. A consequence is that, when factoring a polynomial over the integers, the algorithm which follows is not used: one first computes the square-free factorization over the integers, and to factor the resulting polynomials, one chooses a p such that they remain square-free modulo p.

Algorithm: SFF (Square-Free Factorization)
    Input: A monic polynomial f in Fq[x] where q = pm
    Output: Square-free factorization of f
    R ← 1
   
    # Make w be the product (without multiplicity) of all factors of f that have 
    # multiplicity not divisible by p
    c ← gcd(f, f′)
    w ← f/c 
   
    # Step 1: Identify all factors in w
    i ← 1 
    while w ≠ 1 do
        y ← gcd(w, c)
        fac ← w / y
        R ← R · faci
        w ← y; c ← c / y; i ← i + 1 
    end while
    # c is now the product (with multiplicity) of the remaining factors of f
   
    # Step 2: Identify all remaining factors using recursion
    # Note that these are the factors of f that have multiplicity divisible by p
    if c ≠ 1 then
        c ← c1/p
        R ← R·SFF(c)p
    end if 
   
    Output(R)

The idea is to identify the product of all irreducible factors of f with the same multiplicity. This is done in two steps. The first step uses the formal derivative of f to find all the factors with multiplicity not divisible by p. The second step identifies the remaining factors. As all of the remaining factors have multiplicity divisible by p, meaning they are powers of p, one can simply take the pth square root and apply recursion.

Example of a square-free factorization

Let

${\displaystyle f=x^{11}+2x^{9}+2x^{8}+x^{6}+x^{5}+2x^{3}+2x^{2}+1\in \mathbf {F} _{3}[x],}$

to be factored over the field with three elements.

The algorithm computes first

${\displaystyle c=\gcd(f,f')=x^{9}+2x^{6}+x^{3}+2.}$

Since the derivative is non-zero we have w = f/c = x² + 2 and we enter the while loop. After one loop we have y = x + 2, z = x + 1 and R = x + 1 with updates i = 2, w = x + 2 and c = x⁸ + x⁷ + x⁶ + x²+x+1. The second time through the loop gives y = x + 2, z = 1, R = x + 1, with updates i = 3, w = x + 2 and c = x⁷ + 2x⁶ + x + 2. The third time through the loop also does not change R. For the fourth time through the loop we get y = 1, z = x + 2, R = (x + 1)(x + 2)⁴, with updates i = 5, w = 1 and c = x⁶ + 1. Since w = 1, we exit the while loop. Since c ≠ 1, it must be a perfect cube. The cube root of c, obtained by replacing x³ by x is x² + 1, and calling the square-free procedure recursively determines that it is square-free. Therefore, cubing it and combining it with the value of R to that point gives the square-free decomposition

${\displaystyle f=(x+1)(x^{2}+1)^{3}(x+2)^{4}.}$

Distinct-degree factorization

This algorithm splits a square-free polynomial into a product of polynomials whose irreducible factors all have the same degree. Let f ∈ F_q[x] of degree n be the polynomial to be factored.

Algorithm Distinct-degree factorization(DDF)
    Input: A monic square-free polynomial  f ∈ Fq[x]
    Output: The set of all pairs (g, d), such that 
             f has an irreducible factor of degree d and
             g is the product of all monic irreducible factors of f of degree d.
    Begin
        ${\displaystyle i:=1;\qquad S:=\emptyset ,\qquad f^{*}:=f;}$
        while ${\displaystyle \deg f^{*}\geq 2i}$ do 
            ${\displaystyle g=\gcd(f^{*},x^{q^{i))-x)}$
            if g ≠ 1, then 
                ${\displaystyle S:=S\cup \{(g,i)\))$;
                ${\displaystyle f^{*}:=f^{*}/g}$
            end if
            i := i + 1;
        end while;
        if ${\displaystyle f^{*}\neq 1}$, then ${\displaystyle S:=S\cup \{(f^{*},\deg f^{*})\))$;
        if ${\displaystyle S=\emptyset }$, then return {(f, 1)},
            else return S
    End

The correctness of the algorithm is based on the following:

Lemma. For i ≥ 1 the polynomial

${\displaystyle x^{q^{i))-x\in \mathbf {F} _{q}[x]}$

is the product of all monic irreducible polynomials in F_q[x] whose degree divides i.

At first glance, this is not efficient since it involves computing the GCD of polynomials of a degree which is exponential in the degree of the input polynomial. However,

${\displaystyle g=\gcd \left(f^{*},x^{q^{i))-x\right)}$

may be replaced by

${\displaystyle g=\gcd \left(f^{*},\left(x^{q^{i))-x\mod f^{*}\right)\right).}$

Therefore, we have to compute:

${\displaystyle x^{q^{i))-x\mod f^{*},}$

there are two methods:

Method I. Start from the value of

${\displaystyle x^{q^{i-1))\mod f^{*))$

computed at the preceding step and to compute its qth power modulo the new f*, using exponentiation by squaring method. This needs

${\displaystyle O\left(\log(q)\deg(f)^{2}\right)}$

arithmetic operations in F_q at each step, and thus

${\displaystyle O\left(\log(q)\deg(f)^{3}\right)}$

arithmetic operations for the whole algorithm.

Method II. Using the fact that the qth power is a linear map over F_q we may compute its matrix with

${\displaystyle O\left(\deg(f)^{2}(\log(q)+\deg(f))\right)}$

operations. Then at each iteration of the loop, compute the product of a matrix by a vector (with O(deg(f)²) operations). This induces a total number of operations in F_q which is

${\displaystyle O\left(\deg(f)^{2}(\log(q)+\deg(f))\right).}$

Thus this second method is more efficient and is usually preferred. Moreover, the matrix that is computed in this method is used, by most algorithms, for equal-degree factorization (see below); thus using it for the distinct-degree factorization saves further computing time.

Equal-degree factorization

Cantor–Zassenhaus algorithm

In this section, we consider the factorization of a monic squarefree univariate polynomial f, of degree n, over a finite field F_q, which has r ≥ 2 pairwise distinct irreducible factors ${\displaystyle f_{1},\ldots ,f_{r))$ each of degree d.

We first describe an algorithm by Cantor and Zassenhaus (1981) and then a variant that has a slightly better complexity. Both are probabilistic algorithms whose running time depends on random choices (Las Vegas algorithms), and have a good average running time. In next section we describe an algorithm by Shoup (1990), which is also an equal-degree factorization algorithm, but is deterministic. All these algorithms require an odd order q for the field of coefficients. For more factorization algorithms see e.g. Knuth's book The Art of Computer Programming volume 2.

Algorithm Cantor–Zassenhaus algorithm.
    Input: A finite field Fq of odd order q.
           A monic square free polynomial f in Fq[x] of degree n = rd, 
                which has r ≥ 2 irreducible factors each of degree d
    Output: The set of monic irreducible factors of f.

    Factors := {f};
    while Size(Factors) < r do,
        Choose h in Fq[x] with deg(h) < n at random;
        ${\displaystyle g:=h^{\frac {q^{d}-1}{2))-1{\pmod {f))}$
        for each u in Factors with deg(u) > d do
            if gcd(g, u) ≠ 1 and gcd(g, u) ≠ u, then
                Factors:= Factors${\displaystyle \,\setminus \,\{u\}\cup \{(\gcd(g,u),u/\gcd(g,u))\))$;
            endif
    endwhile

    return Factors

The correctness of this algorithm relies on the fact that the ring F_q[x]/f is a direct product of the fields F_q[x]/f_i where f_i runs on the irreducible factors of f. As all these fields have q^d elements, the component of g in any of these fields is zero with probability

${\displaystyle {\frac {q^{d}-1}{2q^{d))}\sim {\tfrac {1}{2)).}$

This implies that the polynomial gcd(g, u) is the product of the factors of g for which the component of g is zero.

It has been shown that the average number of iterations of the while loop of the algorithm is less than ${\displaystyle 2.5\log _{2}r}$, giving an average number of arithmetic operations in F_q which is ${\displaystyle O(dn^{2}\log(r)\log(q))}$.^[3]

In the typical case where dlog(q) > n, this complexity may be reduced to

${\displaystyle O(n^{2}(\log(r)\log(q)+n))}$

by choosing h in the kernel of the linear map

${\displaystyle v\to v^{q}-v{\pmod {f))}$

and replacing the instruction

${\displaystyle g:=h^{\frac {q^{d}-1}{2))-1{\pmod {f))}$

by

${\displaystyle g:=h^{\frac {q-1}{2))-1{\pmod {f)).}$

The proof of validity is the same as above, replacing the direct product of the fields F_q[x]/f_i by the direct product of their subfields with q elements. The complexity is decomposed in ${\displaystyle O(n^{2}\log(r)\log(q))}$ for the algorithm itself, ${\displaystyle O(n^{2}(\log(q)+n))}$ for the computation of the matrix of the linear map (which may be already computed in the square-free factorization) and O(n³) for computing its kernel. It may be noted that this algorithm works also if the factors have not the same degree (in this case the number r of factors, needed for stopping the while loop, is found as the dimension of the kernel). Nevertheless, the complexity is slightly better if square-free factorization is done before using this algorithm (as n may decrease with square-free factorization, this reduces the complexity of the critical steps).

Victor Shoup's algorithm

Like the algorithms of the preceding section, Victor Shoup's algorithm is an equal-degree factorization algorithm.^[4] Unlike them, it is a deterministic algorithm. However, it is less efficient, in practice, than the algorithms of preceding section. For Shoup's algorithm, the input is restricted to polynomials over prime fields F_p.

The worst case time complexity of Shoup's algorithm has a factor ${\displaystyle {\sqrt {p)).}$ Although exponential, this complexity is much better than previous deterministic algorithms (Berlekamp's algorithm) which have p as a factor. However, there are very few polynomials for which the computing time is exponential, and the average time complexity of the algorithm is polynomial in ${\displaystyle d\log(p),}$ where d is the degree of the polynomial, and p is the number of elements of the ground field.

Let g = g₁ ... g_k be the desired factorization, where the g_i are distinct monic irreducible polynomials of degree d. Let n = deg(g) = kd. We consider the ring R = F_q[x]/g and denote also by x the image of x in R. The ring R is the direct product of the fields R_i = F_q[x]/g_i, and we denote by p_i the natural homomorphism from the R onto R_i. The Galois group of R_i over F_q is cyclic of order d, generated by the field automorphism u → u^p. It follows that the roots of g_i in R_i are

${\displaystyle p_{i}(x),p_{i}(x^{q}),p_{i}\left(x^{q^{2))\right),\ldots ,p_{i}\left(x^{q^{d-1))\right).}$

Like in the preceding algorithm, this algorithm uses the same subalgebra B of R as the Berlekamp's algorithm, sometimes called the "Berlekamp subagebra" and defined as

${\displaystyle {\begin{aligned}B&=\left\{\alpha \in R\ :\ p_{1}(\alpha ),\cdots ,p_{k}(\alpha )\in \mathbf {F} _{q}\right\}\\&=\{u\in R\ :\ u^{q}=u\}\end{aligned))}$

A subset S of B is said a separating set if, for every 1 ≤ i < j ≤ k there exists s ∈ S such that ${\displaystyle p_{i}(s)\neq p_{j}(s)}$. In the preceding algorithm, a separating set is constructed by choosing at random the elements of S. In Shoup's algorithm, the separating set is constructed in the following way. Let s in R[Y] be such that

${\displaystyle {\begin{aligned}s&=(Y-x)\left(Y-x^{q}\right)\cdots \left(Y-x^{q^{d-1))\right)\\&=s_{0}+\cdots +s_{d-1}Y^{d-1}+Y^{d}\end{aligned))}$

Then ${\displaystyle \{s_{0},\dots ,s_{d-1}\))$ is a separating set because ${\displaystyle p_{i}(s)=g_{i))$ for i =1, ..., k (the two monic polynomials have the same roots). As the g_i are pairwise distinct, for every pair of distinct indexes (i, j), at least one of the coefficients s_h will satisfy ${\displaystyle p_{i}(s_{h})\neq p_{j}(s_{h}).}$

Having a separating set, Shoup's algorithm proceeds as the last algorithm of the preceding section, simply by replacing the instruction "choose at random h in the kernel of the linear map ${\displaystyle v\to v^{q}-v{\pmod {f))}$" by "choose h + i with h in S and i in {1, ..., k−1}".

Time complexity

As described in previous sections, for the factorization over finite fields, there are randomized algorithms of polynomial time complexity (for example Cantor–Zassenhaus algorithm). There are also deterministic algorithms with a polynomial average complexity (for example Shoup's algorithm).

The existence of a deterministic algorithm with a polynomial worst-case complexity is still an open problem.

Rabin's test of irreducibility

Like distinct-degree factorization algorithm, Rabin's algorithm^[5] is based on the Lemma stated above. Distinct-degree factorization algorithm tests every d not greater than half the degree of the input polynomial. Rabin's algorithm takes advantage that the factors are not needed for considering fewer d. Otherwise, it is similar to distinct-degree factorization algorithm. It is based on the following fact.

Let p₁, ..., p_k, be all the prime divisors of n, and denote ${\displaystyle n/p_{i}=n_{i))$, for 1 ≤ i ≤ k polynomial f in F_q[x] of degree n is irreducible in F_q[x] if and only if ${\displaystyle \gcd \left(f,x^{q^{n_{i))}-x\right)=1}$, for 1 ≤ i ≤ k, and f divides ${\displaystyle x^{q^{n))-x}$. In fact, if f has a factor of degree not dividing n, then f does not divide ${\displaystyle x^{q^{n))-x}$; if f has a factor of degree dividing n, then this factor divides at least one of the ${\displaystyle x^{q^{n_{i))}-x.}$

Algorithm Rabin Irreducibility Test
    Input: A monic polynomial f in Fq[x] of degree n, 
        p1, ..., pk all distinct prime divisors of n.
    Output: Either "f is irreducible" or "f is reducible".

    for j = 1 to k do 
        ${\displaystyle n_{j}=n/p_{j))$;
    for i = 1 to k do 
        ${\displaystyle h:=x^{q^{n_{i))}-x{\bmod {f))}$;
        g := gcd(f, h);
        if g ≠ 1, then return "f is reducible" and STOP;
    end for;
    ${\displaystyle g:=x^{q^{n))-x{\bmod {f))}$;
    if g = 0, then return "f is irreducible", 
        else return "f is reducible"

The basic idea of this algorithm is to compute ${\displaystyle x^{q^{n_{i))}{\bmod {f))}$ starting from the smallest ${\displaystyle n_{1},\ldots ,n_{k))$ by repeated squaring or using the Frobenius automorphism, and then to take the correspondent gcd. Using the elementary polynomial arithmetic, the computation of the matrix of the Frobenius automorphism needs ${\displaystyle O(n^{2}(n+\log q))}$ operations in F_q, the computation of

${\displaystyle x^{q^{n_{i))}-x{\pmod {f))}$

needs O(n³) further operations, and the algorithm itself needs O(kn²) operations, giving a total of ${\displaystyle O(n^{2}(n+\log q))}$ operations in F_q. Using fast arithmetic (complexity ${\displaystyle O(n\log n)}$ for multiplication and division, and ${\displaystyle O(n(\log n)^{2})}$ for GCD computation), the computation of the ${\displaystyle x^{q^{n_{i))}-x{\bmod {f))}$ by repeated squaring is ${\displaystyle O(n^{2}\log n\log q)}$, and the algorithm itself is ${\displaystyle O(kn(\log n)^{2})}$, giving a total of ${\displaystyle O(n^{2}\log n\log q)}$ operations in F_q.