LAB RATS AND CRAP SHOOTS

My friend the stand-up comic tells me that even the best comedians can't accurately predict an audience's reaction to their routine. Of course, comedians who become well-known are more likely to have success (i.e. get laughs) because their audiences come to see them. The audience is a self-selected group, not a random collection of people, and therefore more likely to be receptive to the comic's material.

A classroom isn't a whole lot different, really; however, students generally don't select teachers. And the results of one's teaching isn't based on a simple outcome (level and number of laughs). But administrators and politicians would use a random collection of students and "measure" the results to determine if the teacher is competent. If the results are equal to or better than some arbitrary score, the teacher keeps his or her job and maybe even gets a raise. Then they use the results of all of the teachers tested to determine if the school is or is not doing a good job and use those results to determine how much operating capital the school gets next year or how much in federal grants they get.

Some time ago, when I got my first administration job as the head of Humanities at a little community college in Virginia, I proposed something similar as a method for determining pay raises. I was 27 years old and my goal was to put teacher evaluation on an objective plane so I wouldn't have to deal with the salary decision. I told my faculty that to be eligible for raises they would have to devise a test based on their course objectives to measure the knowledge and skill level of their incoming students. Then they would have to create an alternative version of the same test to administer at the end of the semester and compare the two scores. Teachers whose test scores increased by at least five percent would get a raise. Simple, right?

Wrong. Even a scientist conducting experiments using lab rats will tell you that all rats are not equal, though efforts are made to breed lab rats to produce rats that are at least roughly equivalent. Even then, testing lab rats often is not reliable, since the very act of testing changes the circumstances of the study. That's not to say there isn't something worthwhile being identified if a test of a thousand rats gets the same results every time, especially if steps are taken to weed out the contradictory elements. But here's the bottom line. Kids aren't bred to produce equivalent students and classrooms aren't laboratories.

When I tried to set up my evaluation scheme, one of the faculty back at the community college came to me and said he was fine with the idea, provided he would be allowed to screen and select his students. He said that his teaching methods were not the same as others and that he was very effective with students who were 'on the same wave length.'

I told him he would have to work with whatever he got. And he said that turned it into a crap shoot.

Though I didn't then, I agree with him now. It is a crap shoot and those teachers who go along with it shouldn't be faulted if they do whatever they can to improve their odds (e.g. 'teaching to the test' and even modifying test results after the fact.)


But there are educators out there who would tell you that the so-called standardized tests being administered to kids across the country reflect the truth of the matter. If I was in their shoes, I would go with the better investment and buy stock in the testing companies. (According to Simba Information, a market-research firm, about $2.56 billion was spent on testing in U.S. schools for the 2014-15 school year. The previous year’s total was $2.46 billion.)

It seems to me that all that money would be much better spent on a serious evaluation, not of  student test scores, but rather on teaching methods matched to student populations. How about if we take schools with student populations that are equivalent on a socio-economic basis, but employ different teaching methods? And if we find that one method appears to produce better outcomes with a particular student population, let's promote that method, without penalizing the teachers who used a less successful method. In other words, match teachers to students. My friend the stand-up comedian would love that.