Phylogenetic bracketing is a method of inference used in biological sciences. It is used to infer the likelihood of unknown traits in organisms based on their position in a phylogenetic tree. One of the main applications of phylogenetic bracketing is on extinct organisms, known only from fossils, going all the way back to the last universal common ancestor (LUCA). The method is often used for understanding traits that do not fossilize well, such as soft tissue anatomy, physiology and behaviour. By considering the closest and second-closest well-known (usually extant) organisms, traits can be asserted with a fair degree of certainty, though the method is extremely sensitive to problems from convergent evolution.

Method

Extant Phylogenetic Bracketing requires that the species forming the brackets be extant. More general forms of phylogenetic bracketing do not require this and may use a mix of extant and extinct taxa to form the bracket. These more generalized forms of phylogenetic bracketing have the advantage in that they can be applied to a wider array of phylogenetic cases. However, since these forms of bracketing are also more generalized and may rely on inferring traits in extinct animals, they also offer lower explanatory power compared to the EPB.

Extant phylogenetic bracketing (EPB)

This is a popular form of phylogenetic bracketing first introduced by Witmer in 1995.^[1] It works by comparing an extinct taxon to its nearest living relatives.^[1][2][3] For example, Tyrannosaurus, a theropod dinosaur, is bracketed by birds and crocodiles. A feature found in both birds and crocodiles would likely be present in Tyrannosaurus, such as the capability to lay an amniotic egg, whereas a feature both birds and crocodiles lack, such as hair, would probably not be present in Tyrannosaurus. Sometimes this approach is used for the reconstruction of ecological traits as well.^[4]

Levels of inference

The extant phylogenetic bracket approach allows researchers to infer traits in extinct animals with varying levels of confidence. This is referred to as the levels of inference.^[1] There are three levels of inference, with each higher level indicating less confidence for the inference.^[1]

Inferences based on osteological correlates

Level 1 — The inference of a character that leaves a bony signature on the skeleton in both members of the extant sister groups. Example: Saying that Tyrannosaurus rex had an eyeball is a level 1 inference because both extant members of the groups encompassing Tyrannosaurus rex have eyeballs, and eyeball sockets (orbital excavations) in the skull, the homology of which is well established, and the fossils of Tyrannosaurus rex skulls have similar morphology.^[1]

Level 2 — The inference of a character that leaves a signature on the skeleton of only one of the extant sister groups.^[1] For example, saying that Tyrannosaurus rex had air sacs running through its skeleton is a level 2 inference as birds are the only extant sister group to Tyrannosaurus rex to show such air sacs. However the underlying pneumatic fossae, air sacs, in the bones of extant birds are remarkably similar to the cavities seen in the fossil vertebrae of Tyrannosaurus rex. The high degree of similarity between the pneumatic fossae in Tyrannosaurus rex and extant birds makes this a fairly strong inference, yet not as strong as a level 1 inference.

Level 3 — The inference of a character that leaves a bony signature on the skeleton but is not present in either extant sister group to the taxon in question.^[1] For example, saying that ceratopsian dinosaurs such as Triceratops horridus had horns in life would be a level 3 inference. Neither extant crocodylians, nor extant birds have horns today, but the osteological evidence for horns in ceratopsians is without question. Thus a level 3 inference receives no support from the extant phylogenetic bracket, but can still be used with confidence based on the merits of the fossil data itself.

Inferences that lack osteological correlates

The Extant Phylogenetic Bracket can be used to infer the presence of soft tissues even when those tissues do not interact with the skeleton. As before, there are three different levels of inference. These levels are designated as prime levels. They descend in confidence as they move up a level.^[1]

Level 1′ — The inference of a character that is shared by both extant sister groups, but does not leave behind a bony signature.^[1] For example, saying that Tyrannosaurus rex had a four-chambered heart would be a level 1′ inference as both extant sister groups (Crocodylia and Aves) have four-chambered hearts, but this trait does not leave behind any bony evidence.

Level 2′ — The inference of a character that is found in only one sister group to the taxon in question and that does not leave behind any bony evidence.^[1] For instance saying that Tyrannosaurus rex was warm-blooded would be a level 2′ inference as extant birds are warm-blooded but extant crocodylians are not. Further, since warm-bloodedness is a physiological trait rather than an anatomical one, it does not leave behind any bony signatures to indicate its presence.

Level 3′ — The inference of a character that is found in neither sister group to the taxon in question and that does not leave behind any bony signatures.^[1] For example, saying that the large sauropod dinosaur Apatosaurus ajax gave birth to live young similar to mammals and many lizards^[5] would be a level 3′ inference as neither crocodylians nor birds give birth to live young and these traits do not leave impressions on the skeleton.

In general the primes are always less confident than their underlying levels; however, the confidence between levels is less clear cut. For instance it is unclear if a level 1′ would be less confident than a level 2. The same would go for a level 2′ versus a level 3.^[1]

Example of bracketing with one extinct and one extant group

The Late Cretaceous Kryptobaatar and the extant monotremes (family Tachyglossidae and Ornithorhynchidae) all sport extratarsal spurs on their hind feet. Greatly simplified, the phylogeny is as follows, with taxa known to have extratarsal spurs in bold:^[6]

Assuming that the Kryptobaatar and monotreme spurs are homologous, they were a feature of their mammalian last common ancestor, so we can tentatively conclude that they were present among the Early Cretaceous Eobaataridae—its descendants—as well.

Example of bracketing with only extinct groups

A fragmentary fossil with a known phylogeny can be compared to more complete fossil specimen to give an idea about general build and habit. The body of labyrinthodonts can usually be inferred to be broad and squat with a sideways compressed tail, although only the skull has been known for many taxa, based on the shape of more well-known labyrinthodont finds.

Example of failure using phylogenetic bracketing

Phylogenetic bracketing is based on the notion of anatomical conservationism. The general body shape of an animal can be fairly constant through large groups, but not always.

The large theropod dinosaur Spinosaurus was until 2014 only known from fragmentary remains, mainly of the skull and vertebrae. It was assumed that the remaining skeleton would look more or less like that of related animals like Baryonyx and Suchomimus, who sport a traditional theropod anatomy of long, strong hind legs and relatively small front legs. A 2014 find, however, included a set of hind legs.^[7] The new reconstruction indicate earlier Spinosaurus reconstructions were wrong, and the animal was mainly aquatic and had relatively weak hind legs. It is possible it walked on all fours when on land, the only theropod to do so.^[8]