Post | LinkedIn Justin Reich - I'm attending
I hope you'll join me on Tuesday, Oct. 20, for a conversation with Patrick O'Neill about his new working paper, Presumed Effective: The Manufacturing of an Evidence Base for AI-in-Education Through Flawed Meta-Analysis. (Link in Comments)
When Patrick sent me the paper this summer, I was quite taken aback. He was reviewing a group of 14 meta-analyses of AI in Education with thousands of citations between them; one of them had already been retracted and he argued that the other 13 had serious, fatal flaws.
How does a field that I've been part of for 20 years publish 14 faulty meta-analyses on one of the most important topics of our generation? What are the consequences for education research and policymaking if our scholarly review process cannot assure the highest possible standards for reviews and meta-analysis? How do school folks make evidence-based decisions if the self-correcting mechanisms of science are working much too slowly (and the self-policing mechanisms not working fast enough)?
I'm very grateful that Susanna Loeb and Elizabeth Tipton for joining us and helping provide analysis and context (which may be different from my own!).
If you see this, I hope you'll share it with a colleague who is an editor or reviewer for an education research or education technology journal or conference. I think if we can get a critical mass of scholars to look carefully at the evidence here, we can work together to ensure a better future for reviews and meta-analyses in this field.