Reform Reality Check: Just Because Something Works, Doesn’t Mean We Should Do It.

17 Jun

Research in education is a funny business.  You can find support for just about anything you want–whole language or phonics, back to basics or project-based learning, charter schools or public.

John Hattie, the author of Visible Learning and the Director of the Melbourne Education Research Institute at the University of Melbourne Graduate School of Education, shows us why.  In his research into what works in education, Hattie did a synthesis of over 800 meta-analyses related to educational achievement.  He looked at over 100 interventions and found that almost all of them produced positive results.  Class size reduction from 30 to 15 students, for example, showed a positive effect size of .2, or about 9 months of learning, which is statistically significant and sounds impressive. What Hattie points out, however, is that we shouldn’t be comparing an intervention’s results to the results if we do no intervention.  We should be comparing it to the results of doing something different.  Class size reduction might sound good, but it’s actually in the bottom half of interventions that produce results.  An intervention must produce an effect size of at least .4, in Hattie’s research, to be at the midline of effectiveness.*

Very few interventions, it turns out, produce negative results.  One that does is retention– a result of -.16.  Seeing that almost no interventions negatively impact a child’s learning, the idea that politicians are actually advocating for a policy that does is unfathomable.  It’s another example of people promoting ideas in education that have very little support in research.

So what does impact achievement the most?  Hattie ranked the interventions in order of effectiveness.  A few interesting ones?  Teacher-student relationship produced an effect size of .72.  

hattie-feedback
That’s incredible.  And it makes sense.  As Rita Pierson proclaimed in her Ted Talk that went viralkids don’t learn from people they don’t like.

Other interested findings?  Feedback and formative evaluation produced an achievement effect of .73 and .9 respectively.  This also makes sense.  Both are tools used to inform future teaching and learning.  Too bad we seem mostly obsessed with summative evaluation right now, in the form of standardized end-of-year tests.  Placing all of your emphasis on summative evaluation is a little like closing the barn door after the horse is out.  If your teaching wasn’t working, it’s too late to do anything about it.

Overall, Hattie found that only five items show a negative impact on student achievement in research:  mobility, television, retention, being on welfare, and summer vacation.  Of the 95 interventions showing positive results, it’s some of the weakest that are getting the most attention right now, such as charter schools, with an effect size of only .20. (I would add the caveat that this suffers from the problem of lumping all charter schools into one category.  It’s fair to say that some probably produce a far larger effect size, while others may be much more ineffective.  But with such weak overall results, it still begs the question of why there’s so much focus on this one intervention.)

We can’t do them all–we don’t have the money, time, or expertise to invest in every positive intervention.  So the question for educators, parents, and policymakers becomes not, “What can we change that positively impacts student achievement” but “What positively impacts student achievement more than all of the other interventions we can put in place?”

————

*I do want to throw in a plug for class-sized reduction.  When it was implemented, class sized reduction created a sudden and immediate need for significantly more teachers, more classrooms, more curriculum (teacher’s manuals) etc, which schools often didn’t have.  The result was thousands of teachers hired on emergency credential with very little back ground in teaching or their subject matter, classes taught in closets or other inappropriate rooms, and students and teachers with no learning materials.  With all of these negative competing factors, it’s a wonder that class sized reduction showed any improvement in learning at all.

See “Visible Learning: Part 1” and “Visible Learning: Part 2” to hear Hattie explain his research more in depth.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

CARDBOARD BOX OFFICE

A world of film, a house of stuff.

Literacy Changes Everything!

Teaching and Parenting as a Dedicated Reader and Writer

To Make a Prairie

A blog about reading, writing, teaching and the joys of a literate life

sunday cummins

Experience Nonfiction

Shanahan on Literacy

Literacy in Education

TWO WRITING TEACHERS

A meeting place for a world of reflective writers.

The Quick and the Ed

Literacy in Education

Shanker Blog

THE VOICE OF THE ALBERT SHANKER INSTITUTE

Free Technology for Teachers

Literacy in Education

chartchums

Smarter Charts from Marjorie Martinelli & Kristine Mraz

%d bloggers like this: