In mathematics, the automorphism group of an object X is the group consisting of automorphisms of X under composition of morphisms. For example, if X is a finite-dimensional vector space, then the automorphism group of X is the group of invertible linear transformations from X to itself (the general linear group of X). If instead X is a group, then its automorphism group ${\displaystyle \operatorname {Aut} (X)}$ is the group consisting of all group automorphisms of X.

Especially in geometric contexts, an automorphism group is also called a symmetry group. A subgroup of an automorphism group is sometimes called a transformation group.

Automorphism groups are studied in a general way in the field of category theory.

Examples

If X is a set with no additional structure, then any bijection from X to itself is an automorphism, and hence the automorphism group of X in this case is precisely the symmetric group of X. If the set X has additional structure, then it may be the case that not all bijections on the set preserve this structure, in which case the automorphism group will be a subgroup of the symmetric group on X. Some examples of this include the following:

• The automorphism group of a field extension ${\displaystyle L/K}$ is the group consisting of field automorphisms of L that fix K. If the field extension is Galois, the automorphism group is called the Galois group of the field extension.
• The automorphism group of the projective n-space over a field k is the projective linear group ${\displaystyle \operatorname {PGL} _{n}(k).}$^[1]
• The automorphism group ${\displaystyle G}$ of a finite cyclic group of order n is isomorphic to ${\displaystyle (\mathbb {Z} /n\mathbb {Z} )^{\times ))$, the multiplicative group of integers modulo n, with the isomorphism given by ${\displaystyle {\overline {a))\mapsto \sigma _{a}\in G,\,\sigma _{a}(x)=x^{a))$.^[2] In particular, ${\displaystyle G}$ is an abelian group.
• The automorphism group of a finite-dimensional real Lie algebra ${\displaystyle {\mathfrak {g))}$ has the structure of a (real) Lie group (in fact, it is even a linear algebraic group: see below). If G is a Lie group with Lie algebra ${\displaystyle {\mathfrak {g))}$, then the automorphism group of G has a structure of a Lie group induced from that on the automorphism group of ${\displaystyle {\mathfrak {g))}$.^[3][4][a]

If G is a group acting on a set X, the action amounts to a group homomorphism from G to the automorphism group of X and conversely. Indeed, each left G-action on a set X determines ${\displaystyle G\to \operatorname {Aut} (X),\,g\mapsto \sigma _{g},\,\sigma _{g}(x)=g\cdot x}$, and, conversely, each homomorphism ${\displaystyle \varphi :G\to \operatorname {Aut} (X)}$ defines an action by ${\displaystyle g\cdot x=\varphi (g)x}$. This extends to the case when the set X has more structure than just a set. For example, if X is a vector space, then a group action of G on X is a group representation of the group G, representing G as a group of linear transformations (automorphisms) of X; these representations are the main object of study in the field of representation theory.

Here are some other facts about automorphism groups:

• Let ${\displaystyle A,B}$ be two finite sets of the same cardinality and ${\displaystyle \operatorname {Iso} (A,B)}$ the set of all bijections ${\displaystyle A\mathrel {\overset {\sim }{\to )) B}$. Then ${\displaystyle \operatorname {Aut} (B)}$, which is a symmetric group (see above), acts on ${\displaystyle \operatorname {Iso} (A,B)}$ from the left freely and transitively; that is to say, ${\displaystyle \operatorname {Iso} (A,B)}$ is a torsor for ${\displaystyle \operatorname {Aut} (B)}$ (cf. #In category theory).
• Let P be a finitely generated projective module over a ring R. Then there is an embedding ${\displaystyle \operatorname {Aut} (P)\hookrightarrow \operatorname {GL} _{n}(R)}$, unique up to inner automorphisms.^[5]

In category theory

Automorphism groups appear very naturally in category theory.

If X is an object in a category, then the automorphism group of X is the group consisting of all the invertible morphisms from X to itself. It is the unit group of the endomorphism monoid of X. (For some examples, see PROP.)

If ${\displaystyle A,B}$ are objects in some category, then the set ${\displaystyle \operatorname {Iso} (A,B)}$ of all ${\displaystyle A\mathrel {\overset {\sim }{\to )) B}$ is a left ${\displaystyle \operatorname {Aut} (B)}$-torsor. In practical terms, this says that a different choice of a base point of ${\displaystyle \operatorname {Iso} (A,B)}$ differs unambiguously by an element of ${\displaystyle \operatorname {Aut} (B)}$, or that each choice of a base point is precisely a choice of a trivialization of the torsor.

If ${\displaystyle X_{1))$ and ${\displaystyle X_{2))$ are objects in categories ${\displaystyle C_{1))$ and ${\displaystyle C_{2))$, and if ${\displaystyle F:C_{1}\to C_{2))$ is a functor mapping ${\displaystyle X_{1))$ to ${\displaystyle X_{2))$, then ${\displaystyle F}$ induces a group homomorphism ${\displaystyle \operatorname {Aut} (X_{1})\to \operatorname {Aut} (X_{2})}$, as it maps invertible morphisms to invertible morphisms.

In particular, if G is a group viewed as a category with a single object * or, more generally, if G is a groupoid, then each functor ${\displaystyle F:G\to C}$, C a category, is called an action or a representation of G on the object ${\displaystyle F(*)}$, or the objects ${\displaystyle F(\operatorname {Obj} (G))}$. Those objects are then said to be ${\displaystyle G}$-objects (as they are acted by ${\displaystyle G}$); cf. ${\displaystyle \mathbb {S} }$-object. If ${\displaystyle C}$ is a module category like the category of finite-dimensional vector spaces, then ${\displaystyle G}$-objects are also called ${\displaystyle G}$-modules.

Automorphism group functor

Let ${\displaystyle M}$ be a finite-dimensional vector space over a field k that is equipped with some algebraic structure (that is, M is a finite-dimensional algebra over k). It can be, for example, an associative algebra or a Lie algebra.

Now, consider k-linear maps ${\displaystyle M\to M}$ that preserve the algebraic structure: they form a vector subspace ${\displaystyle \operatorname {End} _{\text{alg))(M)}$ of ${\displaystyle \operatorname {End} (M)}$. The unit group of ${\displaystyle \operatorname {End} _{\text{alg))(M)}$ is the automorphism group ${\displaystyle \operatorname {Aut} (M)}$. When a basis on M is chosen, ${\displaystyle \operatorname {End} (M)}$ is the space of square matrices and ${\displaystyle \operatorname {End} _{\text{alg))(M)}$ is the zero set of some polynomial equations, and the invertibility is again described by polynomials. Hence, ${\displaystyle \operatorname {Aut} (M)}$ is a linear algebraic group over k.

Now base extensions applied to the above discussion determines a functor:^[6] namely, for each commutative ring R over k, consider the R-linear maps ${\displaystyle M\otimes R\to M\otimes R}$ preserving the algebraic structure: denote it by ${\displaystyle \operatorname {End} _{\text{alg))(M\otimes R)}$. Then the unit group of the matrix ring ${\displaystyle \operatorname {End} _{\text{alg))(M\otimes R)}$ over R is the automorphism group ${\displaystyle \operatorname {Aut} (M\otimes R)}$ and ${\displaystyle R\mapsto \operatorname {Aut} (M\otimes R)}$ is a group functor: a functor from the category of commutative rings over k to the category of groups. Even better, it is represented by a scheme (since the automorphism groups are defined by polynomials): this scheme is called the automorphism group scheme and is denoted by ${\displaystyle \operatorname {Aut} (M)}$.

In general, however, an automorphism group functor may not be represented by a scheme.