Optical sorting (sometimes called digital sorting) is the automated process of sorting solid products using cameras and/or lasers.

Depending on the types of sensors used and the software-driven intelligence of the image processing system, optical sorters can recognize an object's color, size, shape, structural properties and chemical composition.^[1] The sorter compares objects to user-defined accept/reject criteria to identify and remove defective products and foreign material (FM) from the production line, or to separate product of different grades or types of materials.

Optical sorters are in widespread use in the food industry worldwide, with the highest adoption in processing harvested foods such as potatoes, fruits, vegetables and nuts where it achieves non-destructive, 100 percent inspection in-line at full production volumes.^{[citation needed]} The technology is also used in pharmaceutical manufacturing and nutraceutical manufacturing, tobacco processing, waste recycling and other industries. Compared to manual sorting, which is subjective and inconsistent, optical sorting helps improve product quality, maximize throughput and increase yields while reducing labor costs.^[2]

History

Optical sorting is an idea that first came out of the desire to automate industrial sorting of agricultural goods like fruits and vegetables.^[3] Before automated optical sorting technology was conceived in the 1930s, companies like Unitec were producing wooden machinery to assist in the mechanical sorting of fruit processing.^[3] In 1931, a company known as “the Electric Sorting Company” was incorporated and began the creation of the world’s first color sorters, which were being installed and used in Michigan’s bean industry by 1932.^[4] In 1937, optical sorting technology had advanced to allow for systems based on a two-color principle of selection.^[4] The next few decades saw the installation of new and improved sorting mechanisms, like gravity feed systems and the implementation of optical sorting in more agricultural industries.^[5]

In the late 1960s, optical sorting began to be implemented to new industries beyond agriculture, like the sorting of ferrous and non-ferrous metals.^[6] By the 1990s, optical sorting was being used heavily in the sorting of solid wastes.^[6]

With the large technological revolution happening in the late 1990s and early 2000s, optical sorters were being made more efficient via the implementation of new optical sensors, like CCD, UV, and IR cameras.^[5] Today, optical sorting is used in a wide variety of industries and, as such, is implemented with a varying selection of mechanisms to assist in that specific sorter’s task.

The sorting system

In general, optical sorters feature four major components: the feed system, the optical system, image processing software, and the separation system.^[7] The objective of the feed system is to spread products into a uniform monolayer so products are presented to the optical system evenly, without clumps, at a constant velocity. The optical system includes lights and sensors housed above and/or below the flow of the objects being inspected. The image processing system compares objects to user-defined accept/reject thresholds to classify objects and actuate the separation system. The separation system — usually compressed air for small products and mechanical devices for larger products, like whole potatoes — pinpoints objects while in-air and deflects the objects to remove into a reject chute while the good product continues along its normal trajectory.

The ideal sorter to use depends on the application. Therefore, the product's characteristics and the user's objectives determine the ideal sensors, software-driven capabilities and mechanical platform.

Sensors

Optical sorters require a combination of lights and sensors to illuminate and capture images of the objects so the images can be processed. The processed images will determine if the material should be accepted or rejected.

There are camera sorters, laser sorters and sorters that feature a combination of the two on one platform. Lights, cameras, lasers and laser sensors can be designed to function within visible light wavelengths as well as the infrared (IR) and ultraviolet (UV) spectrums. The optimal wavelengths for each application maximize the contrast between the objects to be separated. Cameras and laser sensors can differ in spatial resolution, with higher resolutions enabling the sorter to detect and remove smaller defects.

Cameras

Monochromatic cameras detect shades of gray from black to white and can be effective when sorting products with high-contrast defects.

Sophisticated color cameras with high color resolution are capable of detecting millions of colors to better distinguish more subtle color defects. Trichromatic color cameras (also called three-channel cameras) divide light into three bands, which can include red, green and/or blue within the visible spectrum as well as IR and UV.

Coupled with intelligent software, sorters that feature cameras are capable of recognizing each object's color, size and shape; as well as the color, size, shape and location of a defect on a product. Some intelligent sorters even allow the user to define a defective product based on the total defective surface area of any given object.

Lasers

While cameras capture product information based primarily on material reflectance, lasers and their sensors are able to distinguish a material's structural properties along with their color. This structural property inspection allows lasers to detect a wide range of organic and inorganic foreign material such as insects, glass, metal, sticks, rocks and plastic; even if they are the same color as the good product.

Lasers can be designed to operate within specific wavelengths of light; whether on the visible spectrum or beyond.^[8] For example, lasers can detect chlorophyll by stimulating fluorescence using specific wavelengths; which is a process that is very effective for removing foreign material from green vegetables.^[9]

Camera/laser combinations

Sorters equipped with cameras and lasers on one platform are generally capable of identifying the widest variety of attributes. Cameras are often better at recognizing color, size and shape while laser sensors identify differences in structural properties to maximize foreign material detection and removal.

Hyperspectral Imaging

Driven by the need to solve previously impossible sorting challenges, a new generation of sorters that feature multispectral and hyperspectral imaging systems are being developed.^[10]

Like trichromatic cameras, multispectral and hyperspectral cameras collect data from the electromagnetic spectrum. Unlike trichromatic cameras, which divide light into three bands, hyperspectral systems can divide light into hundreds of narrow bands over a continuous range that covers a vast portion of the electromagnetic spectrum. Compared to the three data points per pixel collected by trichromatic cameras, hyperspectral cameras can collect hundreds of data points per pixel, which are combined to create a unique spectral signature (also called a fingerprint) for each object. When complemented by capable software intelligence, a hyperspectral sorter processes those fingerprints to enable sorting on the chemical composition of the product. This is an emerging area of chemometrics.

Software-driven intelligence

Once the sensors capture the object's response to the energy source, image processing is used to manipulate the raw data. The image processing extracts and categorizes information about specific features. The user then defines accept/reject thresholds that are used to determine what is good and bad in the raw data flow. The art and science of image processing lies in developing algorithms that maximize the effectiveness of the sorter while presenting a simple user-interface to the operator.

Object-based recognition is a classic example of software-driven intelligence. It allows the user to define a defective product based on where a defect lies on the product and/or the total defective surface area of an object. It offers more control in defining a wider range of defective products. When used to control the sorter's ejection system, it can improve the accuracy of ejecting defective products. This improves product quality and increases yields.

New software-driven capabilities are constantly being developed to address the specific needs of various applications. As computing hardware becomes more powerful, new software-driven advancements become possible. Some of these advancements enhance the effectiveness of sorters to achieve better results while others enable completely new sorting decisions to be made.

Platforms

The considerations that determine the ideal platform for a specific application include the nature of the product – large or small, wet or dry, fragile or unbreakable, round or easy to stabilize – and the user's objectives. In general, products smaller than a grain of rice and as large as whole potatoes can be sorted. Throughputs range from less than 2 metric tons of product per hour on low-capacity sorters to more than 35 metric tons of product per hour on high-capacity sorters.

Channel sorters

The simplest optical sorters are channel sorters, a type of color sorter that can be effective for products that are small, hard, and dry with a consistent size and shape; such as rice and seeds. For these products, channel sorters offer an affordable solution and ease of use with a small footprint. Channel sorters feature monochromatic or color cameras and remove defects and foreign material based only on differences in color.

For products that cannot be handled by a channel sorter – such as soft, wet, or nonhomogeneous products – and for processors that want more control over the quality of their product, freefall sorters (also called waterfall or gravity-fed sorters), chute-fed, sorters or belt sorters are more ideal. These more sophisticated sorters often feature advanced cameras and/or lasers that, when complemented by capable software intelligence, detect objects' size, shape, color, structural properties, and chemical composition.

Freefall and chute-fed sorters

Freefall sorters inspect product in-air during the freefall and chute-fed sorters stabilize product on a chute prior to in-air inspection. The major advantages of freefall and chute-fed sorters, compared to belt sorters, are a lower price point and lower maintenance. These sorters are often most suitable for nuts and berries as well as frozen and dried fruits, vegetables, potato strips and seafood, in addition to waste recycling applications that require mid-volume throughputs.

Belt sorters

Belt sorting platforms are often preferred for higher capacity applications such as vegetable and potato products prior to canning, freezing or drying. The products are often stabilized on a conveyor belt prior to inspection. Some belt sorters inspect products from above the belt, while other sorters also send products off of the belt for an in-air inspection. These sorters can either be designed to achieve traditional two-way sorting or three-way sorting if two ejector systems with three outfeed streams are equipped.

ADR systems

A fifth type of sorting platform, called an automated defect removal (ADR) system, is specifically for potato strips (French fries). Unlike other sorters that eject products with defects from the production line, ADR systems identify defects and actually cut the defects from the strips. The combination of an ADR system followed by a mechanical nubbin grader is another type of optical sorting system because it uses optical sensors to identify and remove defects.

Single-file inspection systems

The platforms described above all operate with materials in bulk; meaning they do not need the materials to be in a single-file to be inspected. In contrast, a sixth type of platform, used in the pharmaceutical industry, is a single-file optical inspection system. These sorters are effective in removing foreign objects based on differences in size, shape and color. They are not as popular as the other platforms due to decreased efficiency.

Mechanical graders

For products that require sorting only by size, mechanical grading systems are used because sensors and image processing software is not necessary. These mechanical grading systems are sometimes referred to as sorting systems, but should not be confused with optical sorters that feature sensors and image processing systems.

Practical usage

Waste and recycling

Optical sorting machines can be used to identify and discard manufacturing waste, such as metals, drywall, cardboard, and various plastics.^[11] In the metal industry, optical sorting machines are used to discard plastics, glass, wood, and other non-needed metals.^[12] The plastic industry uses optical sorting machines to not only discard various materials like those listed, but also different types of plastics. Optical sorting machines discard different types of plastics by distinguishing resin types. Resin types that optical sorting machines can identify are: HDPE, PVC, PLA, PE, and others.^[12]

Optical sorting also aids in recycling since the discarded materials are stored in bins. Once a bin is full of a given material, it can be sent to the appropriate recycling facility.^[13] Optical sorting machines’ ability to distinguish between resin types also aids in the process of plastic recycling because there are different methods used for each plastic type.^[14]

Food and drink

In the coffee industry, optical sorting machines are used to identify and remove underdeveloped coffee beans called quakers; quakers are beans that contain mostly carbohydrates and sugars.^[15] A more accurate calibration offers a lower total number of defective products.^[15] Some coffee companies like Counter Culture use these machines in addition to pre-existing sorting methods in order to create a better tasting cup of coffee.^[15] One limitation is that someone has to program these machines by hand to identify defective products.^[15]

However, this science is not limited to coffee beans; food items such as mustard seeds, fruits, wheat, and hemp can all be processed through optical sorting machines.^[16]

In the wine manufacturing process, grapes and berries are sorted like coffee beans.^[17] Grape sorting is used to ensure no unripe/green parts to the plant are involved in the wine making process.^[17] In the past, manual sorting via sorting tables was used to separate the defective grapes from the more effective grapes.^[17] Now, mechanical harvesting provides a higher effectiveness rate compared to manual sorting.^[17] At different points in the line, materials are sorted out via several optical sorting machines.^[17] Each machine is looking for various materials of differing shapes and sizes.^[17]

The berries or grapes can then be sorted accordingly using a camera, a laser, or a form of LED technology with regard to the shape and form of the given fruit. The sorting machine then discards any unnecessary elements.^[18]

^[19]