Generational statistics keep disagreeing once age, cohort and period blur together

Two studies of young people and screens can reach opposite conclusions while both are competently done. That happens even more often when the studies compare generations. Age effects, cohort effects and period effects blur together in birth-year data, and one survey cannot pull them apart.

Age effects come with getting older: sleep, school, employment and parenting demands. Period effects land on every age at once, such as the 2020 shift to remote work and school. Cohort effects belong to a birth group, such as people who were teenagers when the iPhone arrived in 2007.

This page covers that blur, how the central terms get turned into measurements, and what to ask before treating a generational statistic as settled.

What to take away

Where the disagreement comes from

Any pair of studies that made different choices in any row is measuring a different thing. Reporting them as agreeing or disagreeing treats the label as the variable, which it is not. This is the construct validity problem in its most consequential form, and it is a design issue rather than an error.

Age, cohort and period in one number

Suppose a 2022 survey reports that 18-year-olds use social media more than 50-year-olds. Three stories fit the same gap. Age: 18-year-olds have more free hours and fewer dependents. Cohort: they were born in 2004 and met social media in childhood. Period: both groups were living through the pandemic's aftermath.

The three are tied arithmetically. Age plus birth year equals the survey year, so cohort is period minus age. Control for two, and the third is whatever remains.

That dependency is the identification problem, named in the 1970s by Norval Glenn and by Mason and colleagues. Yang Yang and Kenneth Land proposed an intrinsic estimator in 2008, which trades one assumption for another.

Disagreements between generational findings are therefore often not disputes about data. They are disputes about which of the three effects the model was allowed to see.

The best-known case is the argument over smartphones. Jean Twenge, in her 2017 book iGen and a 2018 paper, tied rising US teenage depression and suicide to smartphone adoption after 2012.

Amy Orben and Andrew Przybylski analyzed three large UK and US datasets, about 355,000 adolescents in total. They reported that digital technology use accounted for at most 0.4 percent of the variation in wellbeing.

Both sides have used some of the same public surveys, so the fight sits in modeling choices. The CDC's Youth Risk Behavior Survey found 42 percent of US high school students reporting persistent sadness or hopelessness in 2021, up from 28 percent in 2011.

Suicide rates also climbed across most adult age groups in those years. That points to a period effect rather than a teenager-only one.

Self-report, and its particular direction of error

People are poor at estimating their own time online, and the error is not random noise. Estimates drift with how someone feels about the activity. A mood measure and a self-reported use measure can therefore be correlated because both are downstream of the mood.

That makes a self-report study of use against wellbeing hard to interpret in exactly the place where the interesting claim lives. Device logs solve half of it and add their own problems.

Apple's Screen Time and Google's Digital Wellbeing, both launched in 2018, record what a device did rather than who held it. They cannot tell reading from scrolling past.

The wider problem is that people who agree to be measured differ from those who do not, which shifts every estimate in an unknown direction. That is participation bias, and it bites harder here because willingness to be surveyed about online life plausibly tracks online life.

The sampling frame problem

A study of online behavior delivered through an online panel samples people who are online enough to be in a panel. A study delivered by phone reaches people who answer phones. Neither frame is neutral with respect to the subject, and no frame is.

Three named frames show the gaps. Pew Research Center recruits its American Trends Panel from a random sample of US addresses. The General Social Survey has interviewed US adults since 1972 through NORC at the University of Chicago.

The CDC's Youth Risk Behavior Survey polls high school students in classrooms every two years, so teenagers who dropped out sit outside the frame.

For comparisons across age groups, this is acute: each frame's coverage differs by age. Those gaps are hard to correct and easy to forget.

Any later weighting assumes non-covered people resemble covered people, which is the assumption at issue. The general shape is sampling bias, and the honest version of a finding names the frame in the sentence.

Three readings of every correlation

Suppose a study finds that more use goes with worse wellbeing. At least three readings fit.

Separating them requires a randomized design, which is hard and often unethical with adolescents, or following the same people over time.

Two long-running studies show the cost. The Dunedin Study has followed 1,037 New Zealanders born in 1972 and 1973 since birth. The UK Millennium Cohort Study has followed roughly 19,000 people born in 2000 and 2001.

Both are expensive, both take decades, and neither was built to answer the screen question. Cross-sectional studies, which are most of them, cannot distinguish the three readings at all.

Why every number here comes with a publisher

Any number is an estimate produced by one of these designs, with one set of definitional choices, on one sampling frame. A generation label adds two choices: which birth years count, and which effects the model was allowed to estimate.

Reproducing a figure without that scaffolding leaves a bare number that reads as a fact about the world. The pattern by which a stripped figure travels further than its caveats, and outlives every correction, is the same one described under memes and virality.

The alternative is to name the publisher, so a reader sees the design beside the number. The general refusal, applied to counting attention, is set out under benefits of social media, and applied to standing under online status and identity.

Reading a study summary in six questions

If a summary cannot answer the first two, it is not a summary of a study; it is a rumor of one. What that does to the surrounding argument about young people and media is covered under generational digital culture.

Common questions

Does this mean the research is worthless?

No. It means single findings are weak and the useful unit is a body of work with varied designs. Where many designs point the same way, that is worth something. Where they do not, the disagreement is usually definitional.

Can one survey separate age from cohort from period?

No. The three are arithmetically linked, so one survey of different ages cannot say which effect produced a gap. Separating them takes the same ages measured in different periods, plus an outside assumption.

Why not just use device logs for everything?

Cost, consent, and the fact that a log measures a device rather than a person. Logs are better for quantity and no better for meaning.

Is a longitudinal study conclusive?

Better, not conclusive. Following people over time addresses reverse causation partly and confounding only for the things you thought to measure at the start.

What should I do with a striking statistic about a generation?

Find the design before repeating it. If the design is a one-time survey with self-reported use, the number is a description of what a sample said about itself, which is not what it will be quoted as meaning.