Who's an "Expert"?

With so many "experts" in the media and online, how can we know who to believe?

There are many people being heralded as “experts” who are peddling in fear and mis- or disinformation. We at Screenwise Parenting are even declaring ourselves experts who are cutting through all of this misinformation to give you the facts. In any argument like this we need to understand – what makes someone an expert?

black and white stories sell, but the truth is usually full of nuance

The first clue about whether someone has expertise in an area is whether they insist on nuance or give you general all-or-nothing decrees – the more someone specializing in an area of knowledge, the more likely they are to start their answers with “it depends…” and avoid statements about always or never. These answers don’t meet our desires for certainty, so they are harder to sell than black and white stories, but the truth is usually full of nuance.

The media also tend to make the same mistakes in identifying scientific experts:

1. People in clinical practice – clinical psychologists and therapists, medical doctors, nurses – are trained in how to treat medical issues. They are like very highly trained mechanics for humans. They are not trained as scientists, so they have not been trained in how to evaluate research and are not necessarily qualified to comment on the science.

2. Teachers are not scientific experts. They can report what they experience in their own classroom, which may or may not generalize to the other classrooms, but when they try to examine why these things happen, they are stepping into the scientific realm where they usually don’t have any formal training

3. Parents are not scientific experts. They are experts on their children and, like teachers, should be believed about their own experience, but again, understanding the why often requires bigger picture knowledge of kids generally and the scientific literature around how children (and humans more broadly) function.

Expert Investigation

The first steps for me, when trying to figure out if someone is really an expert, is to see what their credentials are. Doctorate degrees can be either research degrees or professional degrees. The most common research doctorate is the Ph.D. Research doctorates all require some sort of research, but the details of that research are specific to their area of study. M.D. or Medical Doctor is a professional doctorate indicating that you have been trained in the practice of medicine. Doctor of Education (Ed.D) and Doctor of Psychology (Psy.D.) are trickier to categorize. They are typically considered professional degrees, but some programs do require research as well.

As is always the case with humans, your mileage may vary

The next step is for me to see what their doctorate is in – Ph.D.s have impressively deep knowledge of very specific things and broad understanding of related topics. For instance, I have a Ph.D. in Brain and Cognitive Sciences and specifically studied how action video games can improve cognitive skills in college students and older adults. More generally, I’ve learned how humans process information through their brains. So if a topic involves humans, I feel at least some level of comfort discussing it, but I’m not an expert in medicine or even clinical psychology. I’m definitely not an expert in chemistry or physics.

As is always the case with humans, even when you find someone who looks like an expert, ymmv (your mileage may vary). Some Ph.D.s aren’t very good at conducting reliable, well-designed research or aren’t good at applying it to real life. Some clinicians have research experience and do know what they’re talking about when talking about science. Some teachers do turn to science to understand their students and can give informed explanations.

To further check if they do have research experience, you can search for what research they have published. To be clear, publishing a book does not count as research. Anyone can write a book, with or without a relevant degree and the contents of the book are not checked for accuracy by other scientists (peer-reviewed). Google Scholar is my go-to for both finding articles on a topic and specific researchers. Many researchers have profiles on Google Scholar and even if they don’t, you can search their name and see what research you find. You should also look at what topics they research. Many titles can be unreadable gobbledygook full of jargon, but sometimes the titles are clear. The names of the journals the articles are published in can also give you a clue.

When Experts Clash

Then there’s the matter that even when you have multiple appropriately experienced experts in relevant fields, they don’t agree. What do you do when this happens?

This is really hard, even for scientists. When I come across a piece of research that challenges what I know or expect, I have to ask myself whether it is:

1. Bad science

2. Good science, deceptively or unintentionally inaccurately reported

3. Good science that just counters my understanding

Any scientist worth their degree will pay close attention to the third possibility. Good scientists know how to consider how new data can change our understanding of the world. These changes may be slight or they can be paradigm shifting. We’re also taught a sense of how reliable new findings are – if there’s only one or two small studies, it could be a fluke or their could be something going on that isn’t being accounted for in the study but explains the results without changing what we know. When more evidence mounds up, we need to more closely examine what we know. However, another trap is taking the number of papers on a topic to be a reliable indicator of how clearly something is understood. For instance, most relevant to many of the topics we cover, there is no number of correlational studies that will ever add up to understanding how something causes another thing. It’s a cliche at this point, but correlation does not equal causation. It doesn’t matter how consistently we see that when A changes, B also changes, this does not mean that A causes B to change. It may be that B changes A or there are other factors that change both. Experimental studies are needed to untangle these different possibilities.

Our Process

To understand what this process can look like, here is a description of Maya’s process for our post about Cocomelon. We were both very fired up about the original NY Post article, but here’s what we do before we share our ire with you:

“Before I refute every single sentence in this article, I wanted to just double check that there was no research pointing to the perilous effects of Cocomelon. I did what I know best: I hopped onto Google Scholar and other databases to do some old school academic sleuthing. I wanted to find first if there was ANY research on the deleterious effects of Cocomelon specifically. What did I find? A myriad of articles and proceedings from the fields of linguistics, second language acquisition, and development all pointing to the effects of Cocomelon and other Youtube channels on English vocabulary learning (e.g., Sintawati et al., 2023; Rohman & Indah 2023). While most of these studies were qualitative, none of them showed any data on the harmful effects of Cocomelon. Full stop.

“Next I decided to broaden my search. Do fast-paced music based shows impair cognition or development long term? Quick answer…no. I found a few articles from Lillard and colleagues who sought to understand how fantastical (i.e., imaginary/fantasy shows) and fast-paced shows (e.g., Lillard & Peterson, 2011; Lillard et al., 2015) impact executive functioning (EF) among young children immediately after watching a show. They initially found that those who watched fast-paced and fantastical TV shows had lower EF immediately after watching than those who watched slower paced shows. Follow up studies found that there was not a difference between slow and fast-paced--only between fantastical and realistic shows. But, two issues clouded the results: 1) the initial study lacked objective pretests making it difficult to know if effects were due to pre-existing executive function skills and 2) they only looked at EF immediately after watching the TV show so we have no idea if EF would have been the same among the two groups later on.”

By the time we share information with you on Screenwise Parenting, we are completely confident in what we’re telling you. Even when we are sure we’re right, we check what the latest research shows and where claims you find in the news actually come from. We also know we’re human and can miss things, so we welcome challenges (ok, first we usually get a sense of panic and dread, BUT THEN we eagerly dive in to learn more and respond to you)!