Camera and smartphone benchmarking website
DXOMARK is a commercial website described as "an independent benchmark that scientifically assesses smartphones, lenses and cameras".[2][3] Founded in 2008, DxOMark was originally owned by DxO Labs,[4] a French engineering and consulting company, which is headquartered in Boulogne-Billancourt, Paris, France.[1][4][2][5] DXOMARK Image Labs was separated from DxO Labs in September 2017, and was later re-branded to DXOMARK in 2019.[6][7] DXOMARK is now a wholly independent privately-owned company.[6]
DXOMARK scores are used in many independent news publications and specialist media sites.[2]
DXOMARK rating systems
The score(s) awarded by DXOMARK are based upon a comprehensive and highly-documented, but commercially secret methodology. An overall 'headline' (v) score is awarded, but that is based upon an aggregated overview of a number of other very specific and detailed test attributes, although the actual weighting of those individual attributes is unknown.[4] DXOMARK themselves stress that "the overall score is not a weighted sum of the sub-scores. It is a proprietary and confidential mapping of sub-scores into a combined score". DXOMARK also highlight that they consult directly with the equipment manufacturers, with the objective of helping them make better cameras.[4]
In respect of specific score figures, it is not clear what headroom or ceiling level is available to the DXOMARK scoring system.[4] Some early high performers might have suggested an upper limit of 100 points,[4] yet later high performers are awarded scores in excess of 100. It is also important to note that the DXOMARK score is not based on any averages.[4]
Camera sensor rating
The DXOMARK Sensor Score measures several important image quality metrics of the RAW image[8] captured by the camera's sensor.[9] The overall score is a confidential combination of three sub-scores:[9][10]
Another metric, the Perceptual MegaPixel (P-MPix), defined as 'the unit of a sharpness measurement', is used to rate the resolution a camera produces when paired to a particular lens.[11] DXOMARK claims that P-MPix is a more accurate and relevant value for photographers to consider than alternate measures of sharpness when evaluating camera and lens image quality.[11][12][13][14][15][unreliable source] As of December 2015, the Canon EF 300mm f/2.8L IS II USM lens mounted on a Canon EOS 5DS R has the highest measured P-MPix (45 P-MPix), followed by the Carl Zeiss APO Sonnar T* 2/135 ZE (41 P-MPix on Canon EOS 5DS R and 36 P-MPix on Nikon D800E).[16]
Camera lens rating
The DXOMARK Lens Score provides ratings for camera lenses, as tested using its proprietary tool-set in combination with various camera models.[8][10][17] As with the DxOMark Sensor Score, the DXOMARK Lens Score is an aggregation of five separate sub-scores; namely: sharpness, distortion, vignetting, transmission, and chromatic aberration.[17]
Smartphone camera rating
As smartphones began to overtake point-and-shoot cameras,[18] DXOMARK began testing smartphones and other mobile devices in 2011, and introduced DXOMARK Mobile[7] in 2012.[19][20][21] A major update was made in September 2017, adding tests designed to stress the capabilities of current-model smartphones, including those with dual lenses; such as lower-light shooting, telephoto zoom, depth effect, and bokeh.[5][19][20][22] In September 2019, the DXOMARK Mobile score was renamed DXOMARK Camera.[7][19][23]
DXOMARK Camera Overall Score is the headline number reported for each tested device, and consists of a proprietary combination of DXOMARK Camera Photo, and DXOMARK Camera Video category scores.
DXOMARK's Camera Photo score is a proprietary combination of nine category sub-scores:[23]
- Exposure and contrast
- Colour
- Auto-focus
- Texture
- Noise
- Artifacts
- Night (enhanced from Flash, September 2019)[19][22][23][24]
- Zoom[22]
- Bokeh[22]
- Wide (added September 2019)[19][23][24]
- Preview (added October 2020)[25]
DXOMARK's Camera Video score includes six of the same sub-scores as DXOMARK's Mobile Photo score (Exposure, Colour, Auto-focus, Texture, Noise, and Artifacts), along with Stabilisation.
DXOMARK's tests are conducted by the company's technical staff under a variety of lighting conditions; ranging from low-light 1 Lux, to bright daylight outdoors.[22] Sub-scores are combined using a proprietary and confidential mapping into an overall score. Tests are also confined to default modes, except for Zoom and Bokeh, which has caused reviewers to be cautious when using them.[4]
Selfie-camera rating
On 22 January 2019, DXOMARK started to release the results of a new metric; testing the front-facing 'selfie' cameras on smartphones.[19][26][27] These new standalone DXOMARK Selfie tests are undertaken for both Photo and Video, though the 'headline' DXOMARK Selfie score will combine both.[27] For DXOMARK Selfie Photo, there are sub-scores for Exposure, Colour, Focus, Texture, Noise, Artifacts, Flash, and Bokeh. For DXOMARK Selfie Video, sub-scores include Exposure, Colour, Focus, Texture, Noise, Artifacts, and Stabilisation.[27]
Smartphone audio rating
On 10 October 2019, DXOMARK introduced a new Audio benchmark for smartphones.[28] Phones are tested for playback using their internal speakers, and for recording using their built-in microphones.[29] DXOMARK Audio tested categories include:[28][29]
- Timbre (frequency response, treble/mid-range/bass, total balance, volume dependency)
- Dynamics (attack, bass precision, punch, volume dependency)
- Spatial (wideness, balance, distance, localisation)
- Volume (maximum, minimum, user volume consistency)
- Artifacts (noise, pumping, clipping, user artifacts, other artifacts)
For recording only:[28][29]
- Background (directivity, noise profile, artifacts)
Smartphone display ratings
In October 2020, DXOMARK introduced a rating system for mobile device displays (screens). DXOMARK Display tests over 400 measurements, and more than 20 hours of laboratory evaluations and real-life scenarios. The result includes six sub-scores: Readability, Colour, Video, Motion, Touch, and Artifacts.[30]
Wireless speaker ratings
In November 2020, DXOMARK introduced a rating system, DXOMARK Sound, for wireless speakers, based on factors including both lab tests using sound-level meters and calibrated microphones, as well as 20 hours of perceptual assessment. Customized music clips, created by DXOMARK in collaboration with professional musicians and recording studios, include those in the styles of jazz, hip-hop, classical, pop, rock, Latin, electronic, and alternative genres.[31]
Battery ratings
DXOMARK started smartphone battery testing on 10 May 2021. DXOMARK's battery test protocol is based on 70 measurements and takes 150 hours to test.[32]
Analyzer
Analyzer is a suite of software tools published by DXOMARK, that includes test targets and test equipment. It is used by camera companies, as well as press publications and websites;[34] to test sensors, lenses, and standalone cameras, as well as mobile devices with cameras. Testing can be performed on both RAW and JPEG images, as well as video. Analyzer is also the analysis engine behind dxomark.com[8][34][35] Results can be displayed either numerically or graphically.[36] Originally introduced by DxO Labs, Analyzer is now a product of DxOMark, which has been separated from DxO.[6][37]
Analyzer includes modules for testing optics, sensors, stabilisation, video, timing, and 3D features.[38]
