Archive | Research RSS feed for this section

What’s in a Stat? Using Data to Impact Small Choices

1 Aug
salutations-chart, from

salutations-chart, from

In a blog post titled, ‘We Experiment on Human Beings,” OKCupid founder Christian Rudder shares how the site manipulated user profiles to gain data on what led to interactions and meaningful conversations on the site.

His post is in reaction to the ruckus over the Facebook emotional contagion study, but I thought it was much more interesting how OKCupid uses it’s data.  Yes, they mine their data for information that will help them make the site more successful (and by extension, profitable) but they also share their information on their blog.  Oktrends is a veritable gold mine of information about our habits, preferences, and our often misguided assumption about what will appeal to a potential mate (or ourselves.)

What’s awesome about their data is that they interpret it for us–so when they share their thousands of data points about what first messages gained the most traction in their post, Exactly What to Say in a First Messagethey get very specific with advice–open with “how’s it goin” or “What’s up,” but not the more formal “Hi” or “Hello.”  Express interest by using the phrases, “I was curious,” or “You mentioned…”  Contrast that with the general advice we often get, like “be casual,” or “show specific interest in the other person.”

Imagine this transferred to other fields.  Let’s take the classroom.  We give feedback to students, but it’s often a general, not-easily-applied kind of message.  For example, “Johnny is not very engaged in his reading.  He needs to focus on his books for longer.”  We have a very general piece of data here–not engaged in reading–that’s sort of equivalent to OKCupid telling users, “you’re not successful at getting dates.”  It’s accurate, but it’s describing a problem, rather than being helpful.  In fact, it’s pretty discouraging to hear.

We could get more specific with our data.  “Johnny reads for an average of 5 minutes before he finds an alternate activity, like going to the drinking fountain or sharpening a pencil.”  But we’ve really just described the problem in more detail, like saying, “people look at your profile on OKCupid an average of 8 seconds before they click away.”

We need some data for when Johnny is successful at engaging in reading to see the difference–or barring that, some data about when other students similar to Johnny are successful.  “Johnny reads for an average of 15 minutes when his book is a series with characters he knows well,” or “Johnny focuses for more than ten minutes at a time when he’s sitting in a favorite spot, facing away from other students so he’s not distracted.”  Suddenly we have some strategies for how to help Johnny, like OKCupid telling us that pictures that show activities spark more meaningful conversations on average than selfies that just focus on a smiling face.

Now imagine we give this information to Johnny, instead of just sharing with parents at conferences or keeping the knowledge tucked in our head.

Johnny, I’ve been marking when you’re reading and when you’re doing a different activity, and I noticed something interesting.  Usually when you’re reading, you read for about 5 minutes before you get distracted.  But sometimes when you read, you can focus for ten or fifteen minutes at a time!  Usually that’s when you’re reading your series books, like Animorphs or The Lightening Thief.  What do you think of that?”

Johnny can make the cognitive leap.  And now he can devise some plans for how to stay engaged more in reading.

Data is a powerful tool for noticing trends, and what works and what doesn’t, but it’s often held by a those in charge.  OKCupid has opened up some of their data to benefit their subscribers, and they’ve made that data specific, comprehensible, and useful.  Too often in education the data is vague (such as “below standard in math” or “5 on the API”) or not shared with the ultimate actors–the students.  If we are really specific about the issue (struggles in reading because doesn’t notice when a vocabulary word is unknown) then we can be very specific about solutions (repeated lessons with short texts working on identifying and attacking unknown words.)

Data can help us to identify problems, but it can also help us to identify solutions.  We can share data with our students, in the form of grades, percentages, or smiley-faces, but the more specific we are with our observations, the more our students can respond with a positive solution to the problem.


Reform Reality Check: Just Because Something Works, Doesn’t Mean We Should Do It.

17 Jun

Research in education is a funny business.  You can find support for just about anything you want–whole language or phonics, back to basics or project-based learning, charter schools or public.

John Hattie, the author of Visible Learning and the Director of the Melbourne Education Research Institute at the University of Melbourne Graduate School of Education, shows us why.  In his research into what works in education, Hattie did a synthesis of over 800 meta-analyses related to educational achievement.  He looked at over 100 interventions and found that almost all of them produced positive results.  Class size reduction from 30 to 15 students, for example, showed a positive effect size of .2, or about 9 months of learning, which is statistically significant and sounds impressive. What Hattie points out, however, is that we shouldn’t be comparing an intervention’s results to the results if we do no intervention.  We should be comparing it to the results of doing something different.  Class size reduction might sound good, but it’s actually in the bottom half of interventions that produce results.  An intervention must produce an effect size of at least .4, in Hattie’s research, to be at the midline of effectiveness.*

Very few interventions, it turns out, produce negative results.  One that does is retention– a result of -.16.  Seeing that almost no interventions negatively impact a child’s learning, the idea that politicians are actually advocating for a policy that does is unfathomable.  It’s another example of people promoting ideas in education that have very little support in research.

So what does impact achievement the most?  Hattie ranked the interventions in order of effectiveness.  A few interesting ones?  Teacher-student relationship produced an effect size of .72.  

That’s incredible.  And it makes sense.  As Rita Pierson proclaimed in her Ted Talk that went viralkids don’t learn from people they don’t like.

Other interested findings?  Feedback and formative evaluation produced an achievement effect of .73 and .9 respectively.  This also makes sense.  Both are tools used to inform future teaching and learning.  Too bad we seem mostly obsessed with summative evaluation right now, in the form of standardized end-of-year tests.  Placing all of your emphasis on summative evaluation is a little like closing the barn door after the horse is out.  If your teaching wasn’t working, it’s too late to do anything about it.

Overall, Hattie found that only five items show a negative impact on student achievement in research:  mobility, television, retention, being on welfare, and summer vacation.  Of the 95 interventions showing positive results, it’s some of the weakest that are getting the most attention right now, such as charter schools, with an effect size of only .20. (I would add the caveat that this suffers from the problem of lumping all charter schools into one category.  It’s fair to say that some probably produce a far larger effect size, while others may be much more ineffective.  But with such weak overall results, it still begs the question of why there’s so much focus on this one intervention.)

We can’t do them all–we don’t have the money, time, or expertise to invest in every positive intervention.  So the question for educators, parents, and policymakers becomes not, “What can we change that positively impacts student achievement” but “What positively impacts student achievement more than all of the other interventions we can put in place?”


*I do want to throw in a plug for class-sized reduction.  When it was implemented, class sized reduction created a sudden and immediate need for significantly more teachers, more classrooms, more curriculum (teacher’s manuals) etc, which schools often didn’t have.  The result was thousands of teachers hired on emergency credential with very little back ground in teaching or their subject matter, classes taught in closets or other inappropriate rooms, and students and teachers with no learning materials.  With all of these negative competing factors, it’s a wonder that class sized reduction showed any improvement in learning at all.

See “Visible Learning: Part 1” and “Visible Learning: Part 2” to hear Hattie explain his research more in depth.

Preschool is too late: how families can alleviate the word gap

5 Apr

The news that children of low income families have often developed a significant word gap by age three when compared to their wealthier peers brings up the role of the family.  Parents and caregivers matter.  A lot.  Just like Geoffrey Canada’s Harlem Children’s Zone started with baby college–or school on how to be a parent–helping all children to reach their full potential needs to involve the family.

Let’s look at Hart and Risley’s three key findings.

1. The variation in children’s language abilities is relative to the amount parents speak to their children.

2. Children’s academic successes at ages nine and ten are attributable to the amount of talk they hear from birth to age three.

3. Parents of advanced children talk significantly more to their children than parents of children who are not as advanced.


Hart, B. & Risley, T.R. “The Early Catastrophe” (2004). Education Review, 77 (1), 100-118.


Hart, B. & Risley, T.R. “The Early Catastrophe” (2004). Education Review, 77 (1), 100-118.

There’s not a lot of room for interpretation here.  Oral language interaction impacts children’s language, literacy, and even their IQ scores.  Studies also point out that this language interaction begins from birth–parents respond to their infant’s noises by cooing and making nonsense noises back.  You don’t have to wait for a child to start talking to begin to develop their language centers.  This is surprising news for some, and as a new parent I can see why.  When a baby is barely tracking you with her eyes, it’s hard to imagine that she’s processing any part of what you’re saying!

Parent education and training is important.  While studying child-directed speech, Meredith Rowe, an assistant professor from the University of Maryland, found that “the relation between socioeconomic status and child-directed speech was mediated by parental knowledge of child development.” In other words, low-income mothers didn’t talk as much to their children because they didn’t know it was important, whereas middle and upper middle class mothers were more likely to be informed about the latest in child-development research and respond accordingly.

How to help inform parents?  It’s a tough challenge.  LENA (language environment analysis) provides a pocket recorder that sits in a child’s clothing and records their language interactions over the course of the day.

0000192_300The tool has proved to be a powerful motivator for families.  Just seeing the print-out each day creates an increase in language directed at a child, because parents of non-judgemental, visual data of what they are saying.  LENA functions a lot like a fitbit–just knowing how many steps you take (or words you say) motivates you to do more the next day.

Providence, Rhode Island is also beginning a new program about creating family conversations, where home visitors will work with families to create more verbal interactions with their children.  The will visit families once per month, review the LENA data, teach families strategies for increasing verbal interactions with their children, such as how to read a book together, or tell your child about your day, and then set goals with the family for the next month.

With children’s language centers developing from birth, early intervention with those who spend the most time with a child is key.



Preschool is too late: how school’s can address the word gap

4 Apr

Depending on if you come from a “the glass is half full” or “the glass is half empty” stance, the news that 3 year-old children from low-income families have acquired about 30 million fewer words that children from wealthier families could be enlightening or grim.  Does this mean we finally have a chance to address what might be one of the earliest and most significant causes of the achievement gap, or that schools and society are fighting an uphill battle they can’t seem to win?

I think the answer to that question is “yes”.

It’s still crazy to think that a 6 hour school day will alleviate all of the problems that poverty, racism, and general inequality create, but school’s can help.  But not by adding in long lists of required vocabulary that kids learn by rote memorization each day.  One of the biggest concerns about the new interest in vocabulary is that it will create some sort of required “canon” of words that each child will be expected to know.  But words are best learned in context, rather than in isolation with stark definitions.  A child who can connect a word to a story, an experience, other words or examples, is far more likely to use and own that word than one who can simply recite a definition.

Think about adults.  Many of the higher level or “academic” words that we read and/or write everyday fall into the nebulous category of “I know it when I hear it but I can’t define it.”  Despite the fact that we can’t give a technical definition of the word, we have full use of it because we’ve been exposed to it in a variety of contexts.  Contrast that with many technical words children learn in school, like denominator, erosion, or denouement.   Students often learn these words as parts of units of study in science, math, or English, complete with definitions.  But when it comes time for the test and you put these words in a question (or even worse, a form of the word in a question, like “erode” or “eroding”) and they fall to pieces–they don’t know what it means or how to use it.  This is probably why I have completely forgotten all of the words on my weekly 9th grade vocabulary tests except for quotidian.   I thought it was crazy that such a complicated words meant “everyday language,” and I therefore used it wherever I could out of a  teenage sense of irony.

Vocabulary acquisition is an important part of what it means to be literate, and conversely the more literate you are the more vocabulary you learn.  Reading in general, and read-alouds by adults (all the way through middle school and even high school) are some of the most effective ways to increase vocabulary.  Interactive read-alouds–those where the adult is thinking aloud and discussing the book with children–are even more effective.  Just as Hart and Risley found that talk had to be directed at a child, and not just surround them, to help improve their vocabulary, reading that involves conversation with children is more effective that just reading aloud.

I hope the new studies push for more real literature in school–more time to read aloud, read alone, and build rich experiences around reading for children.  The more a child can connect the words they hear to experiences and a wider net of knowledge, the more they will own that word.  There’s certainly a place for direct vocabulary instruction in schools (I talk about it here, here, and here) books are our greatest and most lifelong teachers.

Language Development in Infants

3 Apr

The push for universal preschool has focused a new spotlight on early childhood education.  Amidst the debates about how to fund and run pre-K for all comes the growing awareness that for many children, the preschool years come too late.  Research has shown that on average, a 3 year old living in poverty has heard 30 million fewer words than a child born into a wealthier family.  Are we just pushing down the age at which children will be deemed “below standard” in achievement?

I wanted to spend a little time looking at what all of this discussion and research is bringing up.  This post will focus on what these articles say meaningful language is.  Later on I’ll address what this might mean for the home and for schools. Articles exploring the word gap have been popping up everywhere, from research blogs to newspapers–even to blog posts by Hilary Clinton.  Children living in poverty show vocabulary delay as early as 18 months, which seems to come from the children hearing less language targeted specifically to them.

The studies point out that just listening to adults talk or watching the TV does not help children’s vocabularies grow noticeably.  The language must be directed to the child. Why?  It looks like it’s the interaction between the adult and child that develops language.  The adult is responding to the child (even something as subtle as the child’s gaze or expression.)  Later on, the child responds to the adult and they have a back-and-forth exchange.  It’s this responsive language that grows vocabulary.  This has some interesting implications for parents, caregivers, teachers, and the old maxim “children should be seen and not heard.”  🙂

Studies also found that a major difference in how much vocabulary children heard resulted in whether parents tended to respond to their children affirmatively or negatively.  Negative responses, such as “no!” “stop that!” or “not now!” were conversation closers.  They not only didn’t build vocabulary themselves, but they closed the door on continuing the conversation in a way that might help children to develop more words.  Affirmative responses, on the other hand, tended to grow the conversation and add words and syntax to children’s bank of words.

Chase-Lansdale and Takanishi (2009) provide an example of negative vs. positive interactions in a piece titled “Three Mothers and an Eggplant,” in their report titled How do families matter?:

The first mother wheels her shopping cart down the produce aisle, where her kindergartner spots an eggplant and asks what it is. The mother shushes her child, ignoring the question. A second mother, faced with the same question, responds curtly, ‘Oh, that’s an eggplant, but we don’t eat it.’  The third mother coos, ‘Oh, that’s an eggplant. It’s one of the few purple vegetables.’ She picks it up, hands it to her son, and encourages him to put it on the scale. ‘Oh, look, it’s about two pounds!’ she says. ‘And it’s $1.99 a pound, so that would cost just about $4. That’s a bit pricey, but you like veal parmesan, and eggplant parmesan is delicious too. You’ll love it. Let’s buy one, take it home, cut it open. We’ll make a dish together.’

We can see how the last mother not only builds vocabulary, but also knowledge about the world and engagement with new ideas.

Language development correlates strongly with children’s reading and writing skills.  The research of the past twenty years strongly suggests that focusing just on decoding, or spelling, or children’s education after age 5 (or even 3)  will fall short of what kids need to succeed.  On the other hand, responsive oral language and early, earlyintervention, could create ripple effects that we see in our school’s and society for years.

Doctors and Patients, Teachers and Students–Medicine and Education

12 Sep

I’ve been avoiding the Stanford Alumni magazine this issue, because the main article is about depression, and that seemed like such a…downer.  But I finally opened it, and the article, about Professor David Burns’s techniques for fighting depression, struck a chord.

It turns out that a lot of his techniques for successful therapy closely mirror what we know about successful teaching and coaching.  Not such a shock when you think about it–both fields are about working with people and moving them from where they are to a better (more advanced, more happy, more proficient) state.

In particular, Burns talks about his metholodogy, called TEAM–testing, empathy, agenda setting, and methods.  It bears a remarkable resemblance to best practices in teaching.

“Testing means requiring that patients complete a short mood survey before and after each therapy session…Therapists falsely believe that their impression or gut instinct about what the patient is feeling is accurate,” says May, when in fact their accuracy is very low. “  In teaching?  Assessment, particularly formative assessment, so we can track patient/student progress as we go, rather than just at the end.

“An error many therapists make, says Burns, is skipping empathy and agenda setting and jumping straight into methods. It’s the desire to fix patients instantly that drives this ultimately unproductive shortcut.”  In teaching?  Community building.  How many of us jump into curriculum and hard-core strategy work too early, because we feel like we need to be teaching SOMETHING.  It’s hard to slow down and do the foundational work of community building, norm setting, procedures and routines, but without them, the best lessons in the world have nothing to stick onto.

“…the key to agenda setting is specificity: focusing on an upsetting incident or moment around which different methods can be tried. Saying “I just haven’t made anything of my life” is unlikely to lead anywhere.”  In teaching?  Specific teaching points and goals.  It’s the difference between saying, “Proficient readers read fluently”  (true, but  broad) and, “Readers read smoothly and fluently.  One way they do this is by thinking about how the character is feeling and reading with that expression,”  (for your robot readers) OR “Readers read smoothly and fluently.  One way they do this is by scanning the punctuation beforehand and reading in chunks, up to commas and periods,”  (for readers who run through punctuation.)

Therapists need to forget that they are supposed to be disciples of this or that school and apply what has been proven and known to work. In teaching?  Instructional expertise in best practices.  I laughed when Burns wrote,

Can you imagine going to a doctor with a broken leg and he prescribes penicillin? You’d say, ‘Why are you giving me penicillin for a broken leg?’ And the doctor says, ‘Well I’m in the penicillin movement. Brain tumor, broken leg, you get penicillin.’ It would seem ridiculous.” Burns feels the same way about therapists who rigidly subscribe to a single therapeutic approach.

This feels so close to so many educational debates, like phonics vs. whole language (how about kids be able to decode words and comprehend them?).  It feels pretty safe to say that just like every patient doesn’t need the same treatment, every child doesn’t benefit from the same type of instruction.

It’s interesting that Burns’s methods are seen as revolutions in the field of psychiatry.  Education and business have long been thrown together by corporate America, but maybe education and medicine should take a closer look at one another.  It might be surprising what we can learn from each other.

Is Blended Learning the Magic Bullet?

14 Jun

The title is facetious, but I’m afraid the sentiment can be found throughout the US now.  Blended learning refers to using a mix of face-to-face instructional methods with computer-mediated learning, such as a mix of teacher instruction and computer learning programs, or the “flipped” classroom.  The primacy of technology in our everyday lives, combined with the rise of charter schools that use “individual learning modules” (i.e. children sitting at computers working on reading or math skills at different rates) has created a sentiment in some circles that if we just had KhanAcademy/Gigi math/RAZ-Kids etc in our classrooms (programs that target reading or math instruction to a student’s particular area of need and let him or her practice independently) we would see a meteoric rise in skills.

The problem is that this sentiment just isn’t true.  The New York Times published a review of a NAEP study that shows that not only is the case for blended learning questionable, but we see the same gap in instructional methods for low-income students vs. other students that we do in other areas of schooling.  Low-income students were more likely to use computers for basic drill activities, vs. more cognitively rich activities that other students engaged in.  Is blended learning just a new way to reinforce an old status quo?

Don’t get me wrong–I love computers, I love technology, and I love using both in the classroom.  I taught student to use animoto to create slideshows about themselves and then the Native Americans.  We use ALEKS to help remediate/reinforce/extend math skills for students at my school.  We introduced Tynker at the end of the year to help students learn basic coding skills.  My school recently received a class set of chromebooks through a grant that are in constant use (side plug for chromebooks: I have yet to experience a better bang for my buck than a $199 chromebook to help students learning typing, research skills, presentation skills, and use online apps.  They’re amazing.  If you have money, get them!)  My colleague, Jenny Maehara, has done some truly amazing things with google docs and google presentation, providing every student with an account and teaching them cloud-based skills and typing skills throughout the year.

My beef with blended learning is the idea that it’s an easy, people-proof way to improve student learning.  I don’t know who actually works with students who thinks we can plunk them in front of computers and have them magically overcome all obstacles.  We still need collaboration and instruction, especially to help students with something they are struggling with.  And we need to rethink how technology can help engage and illuminate, as opposed to just provide practice.  Every post by Dan Meyer is worth reading, but here’s one about What Silicon Valley Gets Wrong About Math Education Again and Again.  I’ll pair that with a way technology can engage and provide deeper levels of thinking: these 3-act problems (also by Meyers) such as this one on bubble wrap.



I hope we use technology more and more in the future.  Education needs a real change-up in the way we deliver instruction and experience learning, and technology can be part of that.  But I’m skeptical that individualized cubicles with students clicking answers to questions on their own is really the way we’re going to revolutionize the learning process.  Those programs have their part to play (practice is important!) but they’re far from the whole story.


A world of film, a house of stuff.

Literacy Changes Everything!

Teaching and Parenting as a Dedicated Reader and Writer

To Make a Prairie

A blog about reading, writing, teaching and the joys of a literate life

sunday cummins

Experience Nonfiction

Shanahan on Literacy

Literacy in Education


A meeting place for a world of reflective writers.

The Quick and the Ed

Literacy in Education

Shanker Blog


Free Technology for Teachers

Literacy in Education


Smarter Charts from Marjorie Martinelli & Kristine Mraz

%d bloggers like this: