Diary studies is a research method that collects qualitative information by having participants record entries about their everyday lives in a log, diary or journal about the activity or experience being studied. This collection of data uses a longitudinal technique, meaning participants are studied over a period of time. This research tool, although not being able to provide results as detailed as a true field study, can still offer a vast amount of contextual information without the costs of a true field study.[1][2] Diary studies are also known as experience sampling or ecological momentary assessment (EMA) methodology.
Traditionally diary studies involved participants keeping a written diary of events. However the emergence of smartphones now enables participants to diary with photos, videos and text using a variety of online or offline apps and tools. Since the diary studies are recorded sequentially over time, it can be used to investigate time-based phenomena, temporal dynamics, and fluctuating phenomena such as moods.[3]
Diary studies can also be employed together with other research techniques within a mixed method framework and is particularly useful in obtaining rich subjective data.[4] For instance, experience sampling method (ESM) combines it with questionnaires to gather data and examine people's experiences in daily life.[5]
An early example of a diary study was "How Workingmen Spend Their Time" (Bevans, 1913), which went unpublished by George Esdras Bevans.[6]
Diary studies originate from the fields of psychology and anthropology. In the field of human–computer interaction (HCI), diary studies have been adopted as one method of learning about user needs towards designing more appropriate technologies.[7][8]
A key characteristic of diary studies is their ability to track daily events over time. Researchers have begun conducting studies that allow them to explore the connection between a previous day's events and a current day's outcome, or to what extent prior events make people responsive to current events. Researchers Robert E. Wickham and C. Raymond Knee have concluded that future research studies would benefit from evaluating temporal processes, or processes related to time, in diary studies. This would serve as a way for researchers to test unique questions and hypotheses.[9]
Researchers have been able to use diary studies to evaluate how people can change over time. Traditional diary studies have evaluated change between individuals, but new studies have been conducted to evaluate within-person changes using diary studies. Through a framework of the generalizability theory, researchers have used a condensed version of the Profile of Mood States (POMS) to study within-person emotional changes via diaries.[10]
Feedback studies involve participants answering predefined questions about the phenomenon of interest in a natural setting, with the answers acting as a diary entry. This is usually at assigned times, frequencies, or occurrences of the phenomenon, stated by the researcher. The most common method is using ‘paper and pencil’, although there are some studies that both utilise and suggest other technologies such as mobile phones or Psion Organisers.[11] As such, feedback studies involve asynchronous communication between the participants and the researchers, as the participants’ data is recorded in their diary first, and then passed on to the researchers once complete.[7]
Feedback studies are scalable - that is, a large-scale sample can be used, since it is mainly the participants themselves who are responsible for collecting and recording data.[7]
In elicitation studies, participants capture media as soon as the phenomenon occurs - the media is usually in the form of a photograph, but can be in other different forms as well, and so the recording is generally quick and less effortful than feedback studies. These media are then used as prompts and memory cues - to elicit memories and discussion - in interviews that take place much later.[12] As such, elicitation studies involve synchronous communication between the participants and the researchers, usually through interviews.[7]
In these later interviews, the media and other memory cues (such as what activities were done before and after the event) can improve participants’ episodic memory. In particular, photos were found to elicit more specific recall than all other media types.[7][12][13]
There are two prominent trade-offs between each type of study. Feedback studies involve answering questions more frequently and in situ, therefore enabling more accurate recall but more effortful recording. In contrast, elicitation studies involve quickly capturing media in situ but answering questions much later, therefore enabling less effortful recording but potentially inaccurate recall.[7][12][13]
Diary studies are most often used when observing behavior over time in a natural environment. They can be beneficial when one is looking to find new qualitative and quantitative data.[14] Diary studies aim to measure people's behavior over an extended period. They provide the opportunity for exploratory research, collecting large quantities of precise data that are both in-depth and contextual. What makes diary studies particularly unique is that these substantial amounts of data are collected at the micro-level. When the subject of research undergoes a change, utilizing diary studies becomes interesting because they allow for the measurement of change over an extended period and the observation of its effects on the individual; this is referred to as within-subject analysis. Additionally, it is possible to conduct between-subjects analysis to observe the different effects among respondents.[15] A diary study offers the advantage over a traditional survey study in that it allows for the collection of data on a daily basis or even multiple times a day. In contrast, a survey study typically gathers data at a single point in time, or in the case of a longitudinal study, with time lags spanning months or years.[16]
Advantages of diary studies are numerous.[17][18][19] They allow:
There are some limitations of diary studies, mainly due to their characteristics of reliance on memory and self-report measures.
There is low control, low participation and there is a risk of disturbing the action. In feedback studies, it can be troubling and disturbing to write everything down.[20] This is called a respondent burden[21]
The validity of diary studies rests on the assumption that participants will accurately recall and record their experiences. This is somewhat more easily enabled by the fact that diaries are completed & media is captured in a natural environment and closer, in real-time, to any occurrences of the phenomenon of interest. However, there are multiple barriers to obtaining accurate data, such as:
Diary studies tend to dictate specific times or a frequency at which the participant should complete their diary entries or capture media. This is usually a time that is close to the event or phenomenon of interest to the researcher, and it is assumed that participants will comply with these instructions. However, researchers often find it difficult to verify the extent to which this assumption is met, and have observed various forms of non-compliance. One example of this is hoarding, where previously-missed entries are 'backfilled' or completed all at once, therefore being completed retrospectively rather than at the assigned time. A further example of this is where entries are 'forward-filled', i.e. recorded for a future date.[23]
Researchers have found significant rates of non-compliance and entries written retrospectively in feedback studies. A study by Hyland and colleagues (1993) estimated that the percentage of errors in paper and pencil diaries could be anywhere from 2-24%,[11] due to diaries written retrospectively and/or due to inaccuracies in recording. Another study by Stone and colleagues (2003) compared paper diaries versus compliance-enhancing electronic diaries. The paper diaries contained a hidden instrument which detected when the diary was opened - from this, actual compliance rate was found to be only 11%, while 79% of entries were faked, i.e. not written within 30 minutes of the assigned time. In contrast, the electronic diaries created timestamps for each entry, which prevented the possibility of fake timestamped entries, and therefore yielded a compliance rate of 94%.[23]
In 2015, a diary study was conducted at a Dutch University of Applied Sciences to evaluate how learning spaces affect students' learning activities. 52 business management students kept records of the learning activities that they worked on, where they worked on them, and why they worked there for a week. Through evaluating the diary entries of this study, researchers found a significant correlation between the spaces in which students chose to work and their learning activities.[28]
PACO[29] is an open source mobile platform for behavioral science.