This article has multiple issues. Please help improve it or discuss these issues on the talk page. (Learn how and when to remove these template messages)
This article may be confusing or unclear to readers. In particular, the title and the lead are about curves, and the body of the article is only about polygonal lines. Please help clarify the article. There might be a discussion about this on the talk page. (June 2020) (Learn how and when to remove this template message)
This article does not cite any sources. Please help improve this article by adding citations to reliable sources. Unsourced material may be challenged and removed.Find sources: "Curve orientation" – news · newspapers · books · scholar · JSTOR (September 2013) (Learn how and when to remove this template message)
(Learn how and when to remove this template message)

In mathematics, an **orientation** of a curve is the choice of one of the two possible directions for travelling on the curve. For example, for Cartesian coordinates, the x-axis is traditionally oriented toward the right, and the y-axis is upward oriented.

In the case of a planar simple closed curve (that is, a curve in the plane whose starting point is also the end point and which has no other self-intersections), the curve is said to be **positively oriented** or **counterclockwise oriented**, if one always has the curve interior to the left (and consequently, the curve exterior to the right), when traveling on it. Otherwise, that is if left and right are exchanged, the curve is **negatively oriented** or **clockwise oriented**. This definition relies on the fact that every simple closed curve admits a well-defined interior, which follows from the Jordan curve theorem.

The inner loop of a beltway road in a country where people drive on the right side of the road is an example of a negatively oriented (clockwise) curve. In trigonometry, the unit circle is traditionally oriented counterclockwise.

The concept of *orientation* of a curve is just a particular case of the notion of orientation of a manifold (that is, besides orientation of a curve one may also speak of orientation of a surface, hypersurface, etc.).

In two dimensions, given an ordered set of three or more connected vertices (points) (such as in connect-the-dots) which forms a simple polygon, the orientation of the resulting polygon is directly related to the sign of the angle at any vertex of the convex hull of the polygon, for example, of the angle ABC in the picture. In computations, the sign of the smaller angle formed by a pair of vectors is typically determined by the sign of the cross product of the vectors. The latter one may be calculated as the sign of the determinant of their orientation matrix. In the particular case when the two vectors are defined by two line segments with common endpoint, such as the sides BA and BC of the angle ABC in our example, the orientation matrix may be defined as follows:

A formula for its determinant may be obtained, e.g., using the method of cofactor expansion:

If the determinant is negative, then the polygon is oriented clockwise. If the determinant is positive, the polygon is oriented counterclockwise. The determinant is non-zero if points A, B, and C are non-collinear. In the above example, with points ordered A, B, C, etc., the determinant is negative, and therefore the polygon is clockwise.

In practical applications, the following considerations are commonly taken into account.

One does not need to construct the convex hull of a polygon to find a suitable vertex. A common choice is the vertex of the polygon with the smallest X-coordinate. If there are several of them, the one with the smallest Y-coordinate is picked. It is guaranteed to be a vertex of the convex hull of the polygon. Alternatively, the vertex with the smallest Y-coordinate among the ones with the largest X-coordinates or the vertex with the smallest X-coordinate among the ones with the largest Y-coordinates (or any other of 8 "smallest, largest" X/Y combinations) will do as well. Once a vertex of the convex hull is chosen, one can then apply the formula using the previous and next vertices, even if those are not on the convex hull, as there can be no local concavity on this vertex.

If the orientation of a convex polygon is sought, then, of course, any vertex may be picked.

For numerical reasons, the following equivalent formula for the determinant is commonly used:

The latter formula has four multiplications less. What is more important in computer computations involved in most practical applications, such as computer graphics or CAD, the absolute values of the multipliers are usually smaller (e.g., when A, B, C are within the same quadrant), thus giving a smaller numerical error or, in the extreme cases, avoiding the arithmetic overflow.

When it is not known in advance that the sequence of points defines a simple polygon, the following things must be kept in mind.

For a self-intersecting polygon (complex polygon) (or for any self-intersecting curve) there is no natural notion of the "interior", hence the orientation is not defined. At the same time, in geometry and computer graphics there are a number of concepts to replace the notion of the "interior" for closed non-simple curves; see, e.g., "flood fill" and "winding number".

In "mild" cases of self-intersection, with degenerate vertices when three consecutive points are allowed be on the same straight line and form a zero-degree angle, the concept of "interior" still makes sense, but an extra care must be taken in selection of the tested angle. In the given example, imagine point A to lie on segment BC. In this situation the angle ABC and its determinant will be 0, hence useless. A solution is to test consecutive corners along the polygon (BCD, DEF,...) until a non-zero determinant is found (unless all points lie on the same straight line). (Notice that the points C, D, E are on the same line and form a 180-degree angle with zero determinant.)

Once the orientation of a polygon formed from an ordered set of vertices is known, the concavity of a local region of the polygon can be determined using a second orientation matrix. This matrix is composed of three consecutive vertices which are being examined for concavity. For example, in the polygon pictured above, if we wanted to know whether the sequence of points F-G-H is concave, convex, or collinear (flat), we construct the matrix

If the determinant of this matrix is 0, then the sequence is collinear - neither concave nor convex. If the determinant has the same sign as that of the orientation matrix for the entire polygon, then the sequence is convex. If the signs differ, then the sequence is concave. In this example, the polygon is negatively oriented, but the determinant for the points F-G-H is positive, and so the sequence F-G-H is concave.

The following table illustrates rules for determining whether a sequence of points is convex, concave, or flat:

Negatively oriented polygon (clockwise) | Positively oriented polygon (counterclockwise) | |
---|---|---|

determinant of orientation matrix for local points is negative | convex sequence of points | concave sequence of points |

determinant of orientation matrix for local points is positive | concave sequence of points | convex sequence of points |

determinant of orientation matrix for local points is 0 | collinear sequence of points | collinear sequence of points |