In particle physics, the family symmetries or horizontal symmetries are various discrete, global, or local symmetries between quark-lepton families or generations. In contrast to the intrafamily or vertical symmetries (collected in the conventional Standard Model and Grand Unified Theories) which operate inside each family, these symmetries presumably underlie physics of the family flavors. They may be treated as a new set of quantum charges assigned to different families of quarks and leptons.

Spontaneous symmetry breaking of these symmetries is believed to lead to an adequate description of the flavor mixing of quarks and leptons of different families.  This is certainly one of the major problems that presently confront particle physics. Despite its great success in explaining the basic interactions of nature, the Standard Model still suffers from an absence of such a unique ability to explain the flavor mixing angles or weak mixing angles (as they are conventionally referred to) whose observed values are collected in the corresponding Cabibbo–Kobayashi–Maskawa matrices.

While being conceptually useful and leading in some cases to the physically valuable patterns of the flavor mixing, the family symmetries are not yet observationally confirmed.

Introduction

The Standard Model is based on the internal symmetries of the unitary product group ${\displaystyle SU(3)_{C}\times SU(2)_{W}\times U(1)_{Y))$ the members of which have a quite different nature. The color symmetry ${\displaystyle SU(3)_{C))$ has the vectorlike structure due to which the lefthanded and righthanded quarks are transformed identically as its fundamental triplets. At the same time, the electroweak symmetry consisting of the weak isospin ${\displaystyle SU(2)_{W))$and hypercharge ${\displaystyle U(1)_{Y))$ is chiral. So, the lefthanded components of all quarks and leptons are the ${\displaystyle SU(2)_{W))$ doublets,

${\displaystyle \quad \quad \quad \quad \quad \quad \quad \quad Q_{L}^{(i)}={\begin{pmatrix}u&c&t\\d&s&b\end{pmatrix))_{L}\quad \quad \quad }$  ${\displaystyle L_{L}^{(i)}={\begin{pmatrix}\nu _{e}&\nu _{\mu }&\nu _{\tau }\\e&\mu &\tau \end{pmatrix))_{L))$  

whereas  their righthanded components are its singlets

${\displaystyle \quad \quad \quad \quad \quad \quad \quad \quad U_{R}^{(i)}=(u,c,t)_{R))$   ${\displaystyle \quad \quad \quad \quad \quad \quad D_{R}^{(i)}=(d,s,b)_{R))$

${\displaystyle \quad \quad \quad \quad \quad \quad \quad \quad N_{R}^{(i)}=(N_{e},N_{\mu },N_{\tau })_{R))$ ${\displaystyle \quad \quad \quad \quad E_{R}^{(i)}=(e,\mu ,\tau )_{R))$

Here, the quark-lepton families are numbered by the index ${\displaystyle i\ (i=1,2,3)}$ both for the quark and lepton ones. The up and down righthanded quarks and leptons are written separately and for completeness the righthanded neutrinos ${\displaystyle N_{R}^{(i)))$ are also included.

Many attempts have been made to interpret the existence of the quark-lepton families and the pattern of their mixing in terms of various family symmetries – discrete or continuous, global or local. Among them, the abelian ${\displaystyle U(1)_{F))$ and non-abelian ${\displaystyle SU(2)_{F))$ and ${\displaystyle SU(3)_{F))$ family symmetries seem to be most interesting. They provide some guidance to the mass matrices for families of quarks and leptons, leading to relationships between their masses and mixing parameters. In the framework of the supersymmetric Standard Model, such a family symmetry should at the same time provide an almost uniform mass spectrum for superpartners, with a high degree of the family flavor conservation, that makes its existence even more necessary in the SUSY case.

The U(1) symmetry case

This class of the family symmetry models was first studied by Froggatt and Nielsen in 1979 ^[1] and extended later on in.^[2][3][4] In this mechanism, one introduces a new complex scalar field called flavon ${\displaystyle \phi }$ whose vacuum expectation value (VEV) ${\displaystyle \langle \phi \rangle }$ presumably breaks a global family symmetry ${\displaystyle U(1)_{F))$ imposed. Under this symmetry different quark-lepton families carry different charges ${\displaystyle I_{i}\quad (i=1,2,3)}$. Aсcordingly, the connection between families is provided by inclusion into play (via the relevant see-saw mechanism) some intermediate heavy fermion(s) being properly charged under the family symmetry ${\displaystyle U(1)_{F))$. So, the effective Yukawa coupling constants ${\displaystyle Y_{ij}\quad (i,j=1,2,3)}$ for quark-lepton families are arranged in a way that they may only appear through the primary couplings of these families with the messenger fermion(s) and the flavon field ${\displaystyle \phi }$. The hierarchy of these couplings is determined by some small parameter ${\displaystyle \epsilon }$, which is given by ratio of the flavon VEV ${\displaystyle \langle \phi \rangle }$ to the mass ${\displaystyle M_{F))$ of the intermediate heavy fermion, ${\displaystyle \epsilon =\langle \phi \rangle /M_{F))$  (or ${\displaystyle \epsilon =\langle \phi \rangle /\Lambda _{F))$, if the messenger fermions have been integrated out at some high-energy cut-off scale${\displaystyle \Lambda _{F))$). Since different quark-lepton families carry different charges the various coupling constants ${\displaystyle Y_{ij))$ are suppressed by different powers of ${\displaystyle \epsilon }$ being primarily controlled by the postulated fermion charge assignment.

Specially, for quarks these couplings acquire the form

${\displaystyle \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad Y_{ij}^{(f)}\thicksim \epsilon ^{I(f_{L})_{i}-I({f_{R})}_{j))\quad \quad \quad f_{L,R}=(U,D)_{L,R))$

where the index ${\displaystyle f}$ stands for the particular family of the up quarks (${\displaystyle U=u,c,t}$) and down quarks (${\displaystyle D=d,s,b}$) including their lefthanded and righthanded components, respectively. This hierarchy is then transferred to their mass matrices once the conventional Standard Model Higgs boson ${\displaystyle H}$ develops its own VEV, ${\displaystyle m_{ij}^{(f)}=Y_{ij}^{(f)}\langle H\rangle }$. So, the mass matrices being proportional to the matrices of Yukawa coupling constants can generally produce (by an appropriate choice of the family ${\displaystyle U(1)_{F))$ charges) the required patterns for the weak mixing angles which are in basic conformity with the corresponding Cabibbo–Kobayashi–Maskawa matrices observed. In the same way the appropriate  mass matrices can also be arranged for the lepton families.

Among some other applications the family ${\displaystyle U(1)_{F))$ symmetry, the most interesting one could stem from its possible relation to (or even identification) with the Peccei–Quinn symmetry. This may point out some deep connection between the fermion mixing problem and the strong CP problem of the Standard Model that was also discussed in the literature.^[2][5]

The SU(2) family symmetry

The ${\displaystyle SU(2)_{F))$ family symmetry models were first addressed by Wilczek and Zee in 1979 ^[6] and then the interest in them was renewed in the 1990s ^[7][8] especially in connection with the Supersymmetric Standard Model.

In the original model ^[6] the quark-lepton families fall into the horizontal triplets of the local ${\displaystyle SU(2)_{F))$ symmetry taken. Fortunately, this symmetry is generically free from the gauge anomaly problem which may appear for other local family symmetry candidates. Generally, the model contains the set of the Higgs boson multiplets being scalar, vector and tensor of ${\displaystyle SU(2)_{F))$, apart from they all are the doublets of the conventional electroweak symmetry ${\displaystyle SU(2)_{W}\times U(1)_{Y))$. These scalar multiplets provide the mass matrices for quarks and leptons giving eventually the reasonable weak mixing angles in terms of the fermion mass ratios. In principle, one could hope to reach it in a more economic way when the heavy family masses appears at the tree-level, while the light families acquire their masses from radiative corrections at the one–loop level and higher ones.^[7]

Another and presumably more realistic way ^[7][8] of using of the family ${\displaystyle SU(2)_{F))$ symmetry is based on the picture that, in the absence of flavor mixing, only the particles belonging to the third generation ( ${\displaystyle t,b,\tau }$ ) have non-zero masses. The masses and the mixing angles of the light first and second families being doublets of the ${\displaystyle SU(2)_{F))$ symmetry appear then as a result of the tree-level mixings of families, related to spontaneous breaking of this symmetry. The VEV hierarchy of the horizontal scalars are then enhanced by the effective cut-off scale involved. Again, as in the above ${\displaystyle U(1)_{F))$ symmetry case, the family mixings are eventually turned out to be proportional to powers of some small parameter, which are determined by the dimensions of the ${\displaystyle SU(2)_{F))$ family symmetry allowed operators. This finally generate the effective (diagonal and off-diagonal Yukawa couplings for the light families in the framework of the (ordinary or supersymmetric) Standard Model.

In supersymmetric theories there are mass and interaction matrices for the squarks and sleptons, leading to a rich flavor structure. In particular, if fermions and scalars of a given charge have mass matrices which are not diagonalized by the same rotation, new mixing matrices occur at gaugino vertices. This may lead in general to the dangerous light family flavor changing processes unless the breaking of ${\displaystyle SU(2)_{F))$ symmetry, which controls the light family sector, together with small fermion masses yields the small mass splittings of their scalar superpartners.^[8]

Apart from with all that, there is also the dynamical aspect of the local ${\displaystyle SU(2)_{F))$ symmetry, related to its horizontal gauge bosons. The point is, however, that these bosons (as well as various Higgs bosons involved) have to be several orders of magnitude more massive than the Standard Model W and Z bosons  in order to avoid forbidden quark-flavor- and lepton-flavor-changing transitions. Generally, this requires the introduction of additional Higgs bosons to give the large masses to the horizontal gauge bosons so as to not disturb the masses of the fermions involved.

The chiral SU(3) symmetry alternative

It can be generally argued that the presumably adequate family symmetry should be chiral rather than vectorlike, since the vectorlike family symmetries^[6][9] do not in general forbid the large invariant masses for quark-lepton families. This may lead (without some special fine tuning of parameters) to the almost uniform mass spectra for them that would be natural if the family symmetry were exact rather than broken. Rather intriguingly, both known examples of the local vectorlike symmetries, electromagnetic ${\displaystyle U(1)_{EM))$ and color ${\displaystyle SU(3)_{C))$, appear to be exact symmetries, while all chiral symmetries including the conventional electroweak symmetry ${\displaystyle SU(2)_{W}\times U(1)_{Y))$ and grand unifications SU(5), SO(10) and E(6) appear broken. In this connection, one of the most potentially relevant option considered in the literature may be associated with the local chiral ${\displaystyle SU(3)_{F))$ family symmetry introduced by Chkareuli in 1980^[10] in the framework of the family-unified ${\displaystyle SU(8)}$ symmetry and further developed by its own.^{[11][12][13][14][15][16][17][18]}

Motivation

The choice of the ${\displaystyle SU(3)_{F))$ as the underlying family symmetry beyond the Standard Model appears related to the following issues:

• (i) It provides a natural explanation of the number three of observed quark-lepton families correlated with three species of massless or light neutrinos contributing to the invisible Z boson  partial decay width;
• (ii) Its local nature conforms with the other local symmetries of the Standard Model, such as the weak isospin symmetry ${\displaystyle SU(2)_{W))$ or color symmetry ${\displaystyle SU(3)_{C))$. This actually leads to the family-unified Standard Model with a total symmetry ${\displaystyle SM\times SU(3)_{F))$ which then breaks at some high family scale ${\displaystyle M_{F))$ down to the conventional SM;
• (iii) Its chiral nature, according to which the left-handed and right-handed fermions are proposed to be, respectively, the fundamental triplets and antitriplets of the ${\displaystyle SU(3)_{F))$ symmetry. This means that their masses may only appear as a result of its spontaneous symmetry breaking of the ${\displaystyle SU(3)_{F))$ whose anisotropy in the family flavor space provides the hierarchical mass spectrum of quark-lepton families;
• (iv) The ${\displaystyle SU(3)_{F))$ invariant Yukawa couplings are always accompanied by an accidental global chiral ${\displaystyle U(1)}$ symmetry which can be identified with the Peccei–Quinn symmetry, thus giving a solution to the strong CP problem;
• (v) Due to its chiral structure, it admits a natural unification with conventional Grand unified theories in a direct product form, such as ${\displaystyle SU(5)\times SU(3)_{F))$, ${\displaystyle SO(10)\times SU(3)_{F))$ or ${\displaystyle E(6)\times SU(3)_{F))$, and also as a subgroup of the extended (family-unified) ${\displaystyle SU(8)}$ or ${\displaystyle E(8)}$ GUTs;
• (vi) It has a straightforward extension to the supersymmetric Standard Model and GUTs.

With these natural criteria accepted, other family symmetry candidates have turned out to be at least partially discriminated. Indeed, the ${\displaystyle U(1)_{F))$ family symmetry does not satisfy the criterion (i) and is in fact applicable to any number of quark-lepton families. Also, the ${\displaystyle SU(2)_{F))$ family symmetry can contain, besides two light families treated as its doublets, any number of additional (singlets or new doublets of ${\displaystyle SU(2)_{F))$ ) families. All global non-Abelian symmetries are excluded by the criterion (ii), while the vectorlike symmetries are excluded by the criteria (iii) and (v).

Basic applications

In the Standard Model and GUT extended by the local chiral ${\displaystyle SU(3)_{F))$ symmetry quarks and leptons are supposed to be ${\displaystyle SU(3)_{F))$ chiral triplets, so that their left-handed (weak-doublet) components – ${\displaystyle Q_{L}^{(i)))$ and ${\displaystyle L_{L}^{(i)))$ – are taken to be the triplets of ${\displaystyle SU(3)_{F))$, while their right-handed (weak-singlet) components – ${\displaystyle U_{R}^{(i)))$, ${\displaystyle D_{R}^{(i)))$,   ${\displaystyle N_{R}^{(i)))$ and ${\displaystyle E_{R}^{(i)))$ – are anti-triplets (or vice versa). Here ${\displaystyle i}$ is the ${\displaystyle SU(3)_{F))$ family symmetry index ( ${\displaystyle i=1,2,3}$ ), rather than the index ${\displaystyle (i)}$ introduced in Section ${\displaystyle 1}$ in order to simply number all the families involved. The spontaneous breaking of this symmetry gives some understanding to the observed hierarchy between elements of the quark-lepton mass matrices and presence of texture zeros in them. This breaking is normally provided by some set of the horizontal scalar multiplets being symmetrical and anti-symmetrical under the ${\displaystyle SU(3)_{F))$,  ${\displaystyle \chi _{ij}^{(a)))$ and ${\displaystyle \eta _{[ij]}^{(b)))$ (${\displaystyle a}$ = 1, 2, ..., ${\displaystyle b}$ = 1, 2, ...). When they develop their VEVs, the up and down quark families acquire their effective Yukawa coupling constants which generally have a form

${\displaystyle \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad Y_{ij}^{f}=A_{f}^{(a)}{\frac {<\chi _{ij}^{(a)}>}{M_{F))}\quad +\quad B_{f}^{(b)}{\frac {<\eta _{ij}^{(b)}>}{M_{F))}\quad \quad \quad \quad f=(U,D)}$

where again the index ${\displaystyle f}$ stands for the particular family of the up quarks ( ${\displaystyle U=u,c,t}$ ) and down quarks ( ${\displaystyle D=d,s,b}$ ), respectively (${\displaystyle A_{f}^{(a)))$ and ${\displaystyle B_{f}^{(b)))$ are some dimensionless proportionality constants of the ${\displaystyle O{(1)))$ order).  These coupling constants normally appear via the sort of the see-saw mechanism due to the exchange of a special set of heavy (of order the family symmetry scale ${\displaystyle M_{F))$ ) vectorlike fermions. The VEVs of the horizontal scalars taken in general as large as ${\displaystyle M_{F))$, are supposed to be hierarchically arranged along the different directions in family flavor space. This hierarchy is then transferred to their mass matrices ${\displaystyle m_{ij}^{(U)))$ and ${\displaystyle m_{ij}^{(D)))$, when the conventional Standard Model Higgs boson ${\displaystyle H}$ develops its own VEV in the corresponding Yukawa couplings

${\displaystyle \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad Y_{ij}^{U}({\bar {Q))_{L}^{i}U_{R}^{j})H\quad +\quad Y_{ij}^{D}({\bar {Q))_{L}^{i}D_{R}^{j}){\bar {H))}$                         

In the minimal case with one  ${\displaystyle SU(3)_{F))$ sextet ${\displaystyle \chi _{ij))$ and two triplets ${\displaystyle \eta _{[ij]}^{(1,2)))$ developing the basic VEV configuration

                         ${\displaystyle \quad \quad \quad \quad \quad <\chi _{33}>\quad ,\quad \eta _{[23]}^{(1)}\quad ,\quad \eta _{[12]}^{(2)))$

one comes the typical nearest-neighbor family mixing pattern in the mass matrices ${\displaystyle m_{ij}^{(U)))$ and ${\displaystyle m_{ij}^{(D)))$  that leads to the weak mixing angles being generally in approximate conformity with the corresponding Cabibbo–Kobayashi–Maskawa matrices. In the same way, the appropriate  mass matrices can also be arranged for the lepton families that leads to the realistic description – both in the Standard Model and  GUT – of the lepton masses and mixings, including neutrino masses and oscillations.

In the framework of supersymmetric theories, the family ${\displaystyle SU(3)_{F))$  symmetry hand in hand with hierarchical masses and mixings for quarks and leptons leads to an almost uniform mass spectrum for their superpartners with a high degree of flavor conservation. Due to the special relations between the fermion mass matrices and soft SUSY breaking terms, dangerous supersymmetric contributions to the flavor-changing processes can be naturally suppressed.^[19]

Among other applications of the ${\displaystyle SU(3)_{F))$ symmetry, the most interesting ones are those related to its gauge sector. Generally, the family scale ${\displaystyle M_{F))$ may be located in the range from ${\displaystyle 10^{5))$ GeV up to the grand unification scale ${\displaystyle M_{GUT))$ and even higher. For the relatively low family scale ${\displaystyle M_{F))$, the ${\displaystyle SU(3)_{F))$ gauge bosons will also enter into play so that there may become important many flavor-changing rare processes including some of their astrophysical consequences.^[16] In contrast to the vectorlike family symmetries the chiral ${\displaystyle SU(3)_{F))$ is not generically free from gauge anomalies. They, however, can be readily cancelled by introduction of the appropriate set of the pure horizontal fermion multiplets. Being sterile with respect to all the other Standard Model interactions, they may treated as one of possible candidates for a dark matter in the Universe.

The special sector of applications is related to a new type of topological defects – flavored cosmic strings and monopoles – which can appear during the spontaneous violation of the ${\displaystyle SU(3)_{F))$ which may be considered as possible candidates for the cold dark matter in the Universe.^[20]

Summary

Despite some progress in understanding the family flavor mixing problem, one still has the uneasy feeling that, in many cases, the problem seems just to be transferred from one place to another. The peculiar quark-lepton mass hierarchy is replaced by a peculiar set of ${\displaystyle U(1)_{F))$ flavor charges or a peculiar hierarchy of the horizontal Higgs field VEVs in the non-abelian symmetry case ${\displaystyle SU(2)_{F))$ or ${\displaystyle SU(3)_{F))$. As a result,  there are not so many distinctive and testable generic predictions relating the weak mixing angles to the quark-lepton masses that could distinctively differentiate the one family symmetry model from the other. This indeed related to the fact that Yukawa sector in the theory is somewhat arbitrary as compared with its gauge sector. Actually, one can always arrange the flavor charges of families or the VEVs of horizontal scalars in these models in a way to get the acceptable hierarchical mass matrices for quarks and relatively smooth ones for leptons.

As matter of fact, one of the possible ways for these models to have their own specific predictions might appear if nature would favor the local family symmetry case. This would then allow to completely exclude the global ${\displaystyle U(1)_{F))$ family symmetry case and properly differentiate the non-Abelian ${\displaystyle SU(2)_{F))$ and ${\displaystyle SU(3)_{F))$ symmetry cases. All that is possible, of course, provided that the breaking scale ${\displaystyle M_{F))$ of such a family symmetry is not as large as the GUT scale or Planck scale. Otherwise, all the flavor-changing processes caused by the exchanges of the horizontal gauge bosons will be, therefore, vanishingly suppressed.

Another way for these models to be distinguished might appear, if they were generically being included in some extended GUT. In contrast to many others, such a possibility appears for the chiral ${\displaystyle SU(3)_{F))$ family symmetry (considered in the previous section) which could be incorporated into the family-unified ${\displaystyle SU(8)}$ symmetry.^{[10][21][22][23][24][25][26]} Even if this ${\displaystyle SU(8)}$ GUT would not provide the comparatively low ${\displaystyle SU(3)_{F))$ family symmetry scale, the existence of several ${\displaystyle SU(5)\times SU(3)_{F))$ multiplets of extra heavy  fermions in the original SU(8) matter sector could help with a model verification. Some of them through a natural see-saw mechanism could provide the physical neutrino masses which, in contrast to conventional picture, may appear to follow both the direct or inverted family hierarchy. Others mix with ordinary quark-lepton families in a way that there may arise a marked violation of unitarity in the CKM matrix.

It is also worth pointing out some important aspect related to the family symmetries. As matter of fact, an existence of three identical quark-lepton families could mean that there might exist the truly elementary fermions, preons, being actual carriers of all the Standard Model fundamental quantum numbers involved and composing the observed quarks and leptons at larger distances. Generally, certain regularities in replications of particles may signal about their composite structure. Indeed, just regularities in the spectroscopy of hadrons observed in the nineteen-sixties made it possible to discover the constituent quark structure of hadrons. As to the quarks and leptons, it appears that an idea of their composite structure may distinguish the local chiral ${\displaystyle SU(3)_{F))$ family symmetry among other candidates.^[10][26] Namely, the preon model happens under certain natural conditions to determine a local “metaflavor” ${\displaystyle SU(8)_{MF))$ symmetry as a basic internal symmetry of the physical world at small distances. Being exact for preons, it gets then broken at large distances down to a conventional  SU(5) GUT with an extra local family symmetry ${\displaystyle SU(3)_{F))$ and three standard families of composite quarks and leptons.