In population genetics, F-statistics (also known as fixation indices) describe the statistically expected level of heterozygosity in a population; more specifically the expected degree of (usually) a reduction in heterozygosity when compared to Hardy–Weinberg expectation.

F-statistics can also be thought of as a measure of the correlation between genes drawn at different levels of a (hierarchically) subdivided population. This correlation is influenced by several evolutionary processes, such as genetic drift, founder effect, bottleneck, genetic hitchhiking, meiotic drive, mutation, gene flow, inbreeding, natural selection, or the Wahlund effect, but it was originally designed to measure the amount of allelic fixation owing to genetic drift.

The concept of F-statistics was developed during the 1920s by the American geneticist Sewall Wright,^[1][2] who was interested in inbreeding in cattle. However, because complete dominance causes the phenotypes of homozygote dominants and heterozygotes to be the same, it was not until the advent of molecular genetics from the 1960s onwards that heterozygosity in populations could be measured.

F can be used to define effective population size.^{[further explanation needed]}

Definitions and equations

The measures F_IS, F_ST, and F_IT are related to the amounts of heterozygosity at various levels of population structure. Together, they are called F-statistics, and are derived from F, the inbreeding coefficient. In a simple two-allele system with inbreeding, the genotypic frequencies are:

${\displaystyle p^{2}(1-F)+pF{\text{ for ))\mathbf {AA} ;\ 2pq(1-F){\text{ for ))\mathbf {Aa} ;{\text{ and ))q^{2}(1-F)+qF{\text{ for ))\mathbf {aa} .}$

The value for ${\displaystyle F}$ is found by solving the equation for ${\displaystyle F}$ using heterozygotes in the above inbred population. This becomes one minus the observed frequency of heterozygotes in a population divided by the expected frequency of heterozygotes at Hardy–Weinberg equilibrium:

${\displaystyle F=1-{\frac {\operatorname {O} (f(\mathbf {Aa} ))}{\operatorname {E} (f(\mathbf {Aa} ))))=1-{\frac {\operatorname {ObservedFrequency} (\mathbf {Aa} )}{\operatorname {ExpectedFrequency} (\mathbf {Aa} ))),\!}$

where the expected frequency at Hardy–Weinberg equilibrium is given by

${\displaystyle \operatorname {E} (f(\mathbf {Aa} ))=2pq,\!}$

where ${\displaystyle p}$ and ${\displaystyle q}$ are the allele frequencies of ${\displaystyle \mathbf {A} }$ and ${\displaystyle \mathbf {a} }$, respectively. It is also the probability that at any locus, two alleles from a random individual of the population are identical by descent.

For example, consider the data from E.B. Ford (1971) on a single population of the scarlet tiger moth:

Table 1:

Genotype	White-spotted (${\displaystyle \mathbf {AA} }$)	Intermediate (${\displaystyle \mathbf {Aa} }$)	Little spotting (${\displaystyle \mathbf {aa} }$)	Total
Number	1469	138	5	1612

From this, the allele frequencies can be calculated, and the expectation of ${\displaystyle f\left(\mathbf {Aa} \right)}$ derived :

${\displaystyle p={2\times \mathrm {obs} (AA)+\mathrm {obs} (Aa) \over 2\times (\mathrm {obs} (AA)+\mathrm {obs} (Aa)+\mathrm {obs} (aa))}=0.954}$

${\displaystyle q=1-p=0.046\,}$

${\displaystyle F=1-{\frac {\mathrm {obs} (Aa)/n}{2pq))=1-{138/1612 \over 2(0.954)(0.046)}=0.023}$

The different F-statistics look at different levels of population structure. F_IT is the inbreeding coefficient of an individual (I) relative to the total (T) population, as above; F_IS is the inbreeding coefficient of an individual (I) relative to the subpopulation (S), using the above for subpopulations and averaging them; and F_ST is the effect of subpopulations (S) compared to the total population (T), and is calculated by solving the equation:

${\displaystyle (1-F_{IS})(1-F_{ST})=1-F_{IT},\,}$

as shown in the next section.

Partition due to population structure

Consider a population that has a population structure of two levels; one from the individual (I) to the subpopulation (S) and one from the subpopulation to the total (T). Then the total ${\displaystyle F}$, known here as ${\displaystyle F_{IT))$, can be partitioned into ${\displaystyle F_{IS))$ and ${\displaystyle F_{ST))$:

${\displaystyle 1-F_{IT}=(1-F_{IS})\,(1-F_{ST}).\!}$

This may be further partitioned for population substructure, and it expands according to the rules of binomial expansion, so that for I partitions:

${\displaystyle 1-F=\prod _{i=0}^{i=I}(1-F_{i,i+1})\!}$

Fixation index

A reformulation of the definition of ${\displaystyle F}$ would be the ratio of the average number of differences between pairs of chromosomes sampled within diploid individuals with the average number obtained when sampling chromosomes randomly from the population (excluding the grouping per individual). One can modify this definition and consider a grouping per sub-population instead of per individual. Population geneticists have used that idea to measure the degree of structure in a population.

Unfortunately, there is a large number of definitions for ${\displaystyle F_{ST))$, causing some confusion in the scientific literature. A common definition is the following:

${\displaystyle F_{ST}={\frac {\operatorname {var} (\mathbf {p} )}{p\,(1-p)))\!}$

where the variance of ${\displaystyle \mathbf {p} }$ is computed across sub-populations and ${\displaystyle p\,(1-p)}$ is the expected frequency of heterozygotes.

Fixation index in human populations

It is well established that the genetic diversity among human populations is low,^[3] although the distribution of the genetic diversity was only roughly estimated. Early studies argued that 85–90% of the genetic variation is found within individuals residing in the same populations within continents (intra-continental populations) and only an additional 10–15% is found between populations of different continents (continental populations).^{[4][5][6][7][8]} Later studies based on hundreds of thousands single-nucleotide polymorphism (SNPs) suggested that the genetic diversity between continental populations is even smaller and accounts for 3 to 7%^{[9][10][11][12][13][14]} A later study based on three million SNPs found that 12% of the genetic variation is found between continental populations and only 1% within them.^[15] Most of these studies have used the F_ST statistics ^[16] or closely related statistics.^[17][18]