Three traps to avoid in data analysis
A social science perspective on technology and innovation
Last week I had the amazing experience of sharing the stage with leading innovators like Scott Harrison of Charity Water, Sallie Krawcheck of Ellevest, and Tyler Perry of All The Things. We spoke at the inaugural Masters of Scale Summit, and since I’ve spent my career primarily around the academic and DC policy communities, I saw the sessions through those lenses.
Here are some things I said, and some things I learned.
My talk in a nutshell:
We can use demography to read the world, but there’s a right and wrong way to do so. We have to know the right questions to ask, and the analytical traps to avoid. These apply for data analysis generally, so think about how they are useful in your own work.
Trap 1: Trends change, but our thinking doesn’t. I cited the example of falling fertility as a trend many people have missed. If your top concern is overpopulation and 2 of every 3 people on the planet lives somewhere with below replacement fertility, we might have to violate some human rights to meet population goals.
Trap 2: We have to put data in context. In particular, we need to remember that political structures—the rules of the game—amplify and dilute certain trends. While Japan has a greater proportion of the population above age 65 than does France, France has a greater proportion of retirees. This is a clear argument against using “dependency ratios” as shorthand for population aging.
Trap 3: Our biases cloud our judgement. I cited the example of hearing Pentagon officials dismiss Russia as a threat due to their dire demographics in the mid-2000s. One woman came up to me in the bathroom (do men have deep conversations in the loo as well?) to say she’d done exactly that during her military career. The US wanted to see itself as the world’s most powerful country, and that meant seeing its competitors as necessarily weak. The US national security community often looks for signs of weakness in its competitors still, without turning a critical eye to domestic trends.
When the video is posted, I’ll be sure to share it.
Other stuff I learned:
Conversing with some of the 700 esteemed participants and listening to the speakers also got me thinking.
My biggest takeaway: The tech/innovation communities could benefit from partnerships with social scientists, who would help to define the problem so companies can work towards solutions. From a demographic perspective, knowing how many of us there are, where we are, and who we are (including our wants and needs) is an essential part of the problem-solving process, but I got the sense that some businesses are innovating just to innovate. “Look at this cool thing I made!” Those innovations aren’t always developed in response to a clear problem, nor are they positioned for uptake. There’s a gap between many of these businesses and the policy community. I worry that lack of knowledge will lead to implementation problems—companies can’t see the roadblocks ahead. That’s one thing I’m passionate about helping companies with through my consulting practice (give me a call!). We need to recognize the value of including social scientists and humanists on these teams and actually do it.
Automation and elder care: In what ways can automation continue to help families care for aging loved ones? This article from the WaPo is a vivid and heartbreaking description of what care work actually looks like in a country with relatively developed care infrastructure. And, since women outlive men by an average of 6 years in developed countries, the work falls disproportionately on women. The low-hanging fruit is monitoring devices for health issues like blood sugar, but it’s the daily care activities, like feeding, dressing, and toileting, that put the biggest strain on carers. Meal prep is the most obvious space for outsourcing, but with seniors on a fixed income the challenge is scaling to a point where it’s affordable. An additional barrier is the difficulty of using technology, like meal delivery apps, for an age group that is less comfortable (including from a trust perspective) with technology. Given that 1 in 5 people in high-income countries is already aged 65 or older, it’s imperative that we accelerate innovation in this sphere.
Limits of a generational perspective: A lot of the businesses at the summit wanted to better understand “the human at the center” and they’re turning to research on generational differences for insight. I don’t want to put myself out of a job, but I’m worried that they’re placing too much emphasis on what are essentially arbitrary boundaries. There’s huge controversy among social scientists about the utility (and harm) of these generational labels and little to no scientific basis for them. Casting labels aside, it could be useful for employers to think about the characteristics and attitudes of various “generations” of workers if reframed as “ages”, so long as they don’t see them as either fixed or universal. As someone who spent a lot of years in a college classroom, let me tell you that there are very few blanket statements I could make about 18-22-year-olds. Some are great students, some are terrible. Some are proficient with technology, some are merely comfortable. I think companies would learn more from looking at trends among the workforce as a whole and how people’s preferences and skills are shifting no matter their ages.
Need for government capacity: Tech companies and other organizations might come up with promising solutions to problems like hunger, poverty, and disease, but their efficacy abuts the ability of governments in low- and low-middle-income countries to implement those solutions. Scott Harrison of Charity Water described the tremendous effort it has taken his team to figure out a way to monitor and repair broken water wells. Installing those wells was a false victory—the real problem has been keeping them working. When physical infrastructure, systems (like medical infrastructure), or bureaucracy are absent, innovations can die on the vine.