In mathematical logic, monadic second-order logic (MSO) is the fragment of second-order logic where the second-order quantification is limited to quantification over sets. It is particularly important in the logic of graphs, because of Courcelle's theorem, which provides algorithms for evaluating monadic second-order formulas over graphs of bounded treewidth. It is also of fundamental importance in automata theory, where the Büchi–Elgot–Trakhtenbrot theorem gives a logical characterization of the regular languages.

Second-order logic allows quantification over predicates. However, MSO is the fragment in which second-order quantification is limited to monadic predicates (predicates having a single argument). This is often described as quantification over "sets" because monadic predicates are equivalent in expressive power to sets (the set of elements for which the predicate is true).

## Variants

Monadic second-order logic comes in two variants. In the variant considered over structures such as graphs and in Courcelle's theorem, the formula may involve non-monadic predicates (in this case the binary edge predicate $E(x,y)$ ), but quantification is restricted to be over monadic predicates only. In the variant considered in automata theory and the Büchi–Elgot–Trakhtenbrot theorem, all predicates, including those in the formula itself, must be monadic, with the exceptions of equality ($=$ ) and ordering ($<$ ) relations.

## Computational complexity of evaluation

Existential monadic second-order logic (EMSO) is the fragment of MSO in which all quantifiers over sets must be existential quantifiers, outside of any other part of the formula. The first-order quantifiers are not restricted. By analogy to Fagin's theorem, according to which existential (non-monadic) second-order logic captures precisely the descriptive complexity of the complexity class NP, the class of problems that may be expressed in existential monadic second-order logic has been called monadic NP. The restriction to monadic logic makes it possible to prove separations in this logic that remain unproven for non-monadic second-order logic. For instance, in the logic of graphs, testing whether a graph is disconnected belongs to monadic NP, as the test can be represented by a formula that describes the existence of a proper subset of vertices with no edges connecting them to the rest of the graph; however, the complementary problem, testing whether a graph is connected, does not belong to monadic NP. The existence of an analogous pair of complementary problems, only one of which has an existential second-order formula (without the restriction to monadic formulas) is equivalent to the inequality of NP and coNP, an open question in computational complexity.

By contrast, when we wish to check whether a Boolean MSO formula is satisfied by an input finite tree, this problem can be solved in linear time in the tree, by translating the Boolean MSO formula to a tree automaton and evaluating the automaton on the tree. In terms of the query, however, the complexity of this process is generally nonelementary. Thanks to Courcelle's theorem, we can also evaluate a Boolean MSO formula in linear time on an input graph if the treewidth of the graph is bounded by a constant.

For MSO formulas that have free variables, when the input data is a tree or has bounded treewidth, there are efficient enumeration algorithms to produce the set of all solutions, ensuring that the input data is preprocessed in linear time and that each solution is then produced in a delay linear in the size of each solution, i.e., constant-delay in the common case where all free variables of the query are first-order variables (i.e., they do not represent sets). There are also efficient algorithms for counting the number of solutions of the MSO formula in that case.

## Decidability and complexity of satisfiability

The satisfiability problem for monadic second-order logic is undecidable in general because this logic subsumes first-order logic.

The monadic second-order theory of the infinite complete binary tree, called S2S, is decidable. As a consequence of this result, the following theories are decidable:

• The monadic second-order theory of trees.
• The monadic second-order theory of $\mathbb {N}$ under successor (S1S).
• WS2S and WS1S, which restrict quantification to finite subsets (weak monadic second-order logic). Note that for binary numbers (represented by subsets), addition is definable even in WS1S.

For each of these theories (S2S, S1S, WS2S, WS1S), the complexity of the decision problem is nonelementary.

### Use of satisfiability of MSO on trees in verification

Monadic second-order logic of trees has applications in formal verification. Decision procedures for MSO satisfiability have been used to prove properties of programs manipulating linked data structures, as a form of shape analysis, and for symbolic reasoning in hardware verification.

1. ^ Courcelle, Bruno; Engelfriet, Joost (2012-01-01). Graph Structure and Monadic Second-Order Logic: A Language-Theoretic Approach. Cambridge University Press. ISBN 978-0521898331. Retrieved 2016-09-15.
2. ^ Fagin, Ronald (1975), "Monadic generalized spectra", Zeitschrift für Mathematische Logik und Grundlagen der Mathematik, 21: 89–96, doi:10.1002/malq.19750210112, MR 0371623.
3. ^ Fagin, R.; Stockmeyer, L.; Vardi, M. Y. (1993), "On monadic NP vs. monadic co-NP", Proceedings of the Eighth Annual Structure in Complexity Theory Conference, Institute of Electrical and Electronics Engineers, doi:10.1109/sct.1993.336544, S2CID 32740047.
4. ^ Thatcher, J. W.; Wright, J. B. (1968-03-01). "Generalized finite automata theory with an application to a decision problem of second-order logic". Mathematical Systems Theory. 2 (1): 57–81. doi:10.1007/BF01691346. ISSN 1433-0490. S2CID 31513761.
5. ^ a b Meyer, Albert R. (1975). Parikh, Rohit (ed.). "Weak monadic second order theory of succesor is not elementary-recursive". Logic Colloquium. Lecture Notes in Mathematics. Springer Berlin Heidelberg: 132–154. doi:10.1007/bfb0064872. ISBN 9783540374831.
6. ^ Bagan, Guillaume (2006). Ésik, Zoltán (ed.). "MSO Queries on Tree Decomposable Structures Are Computable with Linear Delay". Computer Science Logic. Lecture Notes in Computer Science. Springer Berlin Heidelberg. 4207: 167–181. doi:10.1007/11874683_11. ISBN 9783540454595.
7. ^ Arnborg, Stefan; Lagergren, Jens; Seese, Detlef (1991-06-01). "Easy problems for tree-decomposable graphs". Journal of Algorithms. 12 (2): 308–340. doi:10.1016/0196-6774(91)90006-K. ISSN 0196-6774.
8. ^ Rabin, Michael O. (1969). "Decidability of Second-Order Theories and Automata on Infinite Trees". Transactions of the American Mathematical Society. 141: 1–35. doi:10.2307/1995086. ISSN 0002-9947. JSTOR 1995086.
9. ^ Stockmeyer, Larry; Meyer, Albert R. (2002-11-01). "Cosmological lower bound on the circuit complexity of a small problem in logic". Journal of the ACM. 49 (6): 753–784. doi:10.1145/602220.602223. ISSN 0004-5411. S2CID 15515064.
10. ^ Henriksen, Jesper G.; Jensen, Jakob; Jørgensen, Michael; Klarlund, Nils; Paige, Robert; Rauhe, Theis; Sandholm, Anders (1995). Brinksma, E.; Cleaveland, W. R.; Larsen, K. G.; Margaria, T.; Steffen, B. (eds.). "Mona: Monadic second-order logic in practice". Tools and Algorithms for the Construction and Analysis of Systems. Lecture Notes in Computer Science. Berlin, Heidelberg: Springer. 1019: 89–110. doi:10.1007/3-540-60630-0_5. ISBN 978-3-540-48509-4.
11. ^ Fiedor, Tomáš; Holík, Lukáš; Lengál, Ondřej; Vojnar, Tomáš (2019-04-01). "Nested antichains for WS1S". Acta Informatica. 56 (3): 205–228. doi:10.1007/s00236-018-0331-z. ISSN 1432-0525. S2CID 57189727.
12. ^ Traytel, Dmitriy; Nipkow, Tobias (2013-09-25). "Verified decision procedures for MSO on words based on derivatives of regular expressions". ACM SIGPLAN Notices. 48 (9): 3–f12. doi:10.1145/2544174.2500612. hdl:20.500.11850/106053. ISSN 0362-1340.
13. ^ Møller, Anders; Schwartzbach, Michael I. (2001-05-01). "The pointer assertion logic engine". Proceedings of the ACM SIGPLAN 2001 conference on Programming language design and implementation. PLDI '01. Snowbird, Utah, USA: Association for Computing Machinery. pp. 221–231. doi:10.1145/378795.378851. ISBN 978-1-58113-414-8. S2CID 11476928.
14. ^ Basin, David; Klarlund, Nils (1998-11-01). "Automata based symbolic reasoning in hardware verification". Formal Methods in System Design. 13 (3): 255–288. doi:10.1023/A:1008644009416. ISSN 0925-9856.