By Julie Hubbard, The Tennessean
In Murfreesboro City Schools, principals rated nearly half the teachers a five - the best score possible on the state's new evaluation.
But in Fayette County Schools in far West Tennessee, only 1 percent garnered that rating.
The first glimpse of how educators fared under the system, which ultimately will affect whether they earn and keep tenure, demonstrated how subjective the process can be. The Tennessee Department of Education released principal observation data in December after The Tennessean and Williamson County Schools Director Mike Looney filed separate open records requests for it.
Looney said he wanted the data for comparison after Williamson principals rated 97 percent of teachers a three or higher, and state education officials questioned those ratings. He said his county has a high level of teacher talent plus motivated students, and state officials shouldn't pressure districts to align scores with projections.
"To come to some conclusion that our scores are too high ... is preposterous," Looney said. "We are not going to feel compelled or pushed into making our teachers fit some bell curve."
State officials say it's premature to draw conclusions on midyear data, but they want principals and district directors to review it and ensure they are holding teachers to a high bar.
But some teachers say it shows what they've said all along - schools with stricter principals won't fare as well under the evaluation, where those observations count for 50 percent of the final score.
"I question how evaluators are evaluating if the scores vary greatly across the state," said Marshall Winkler, wellness teacher at Franklin High School. "I feel like the (state) jumped into this new plan too soon."
The broad range in scoring was apparent at the bottom of the scale, too. Only one Williamson County teacher received the lowest score. Humbolt City Schools, 140 miles west of Nashville, gave 7.6 percent of teachers a one, while no teachers in at least 16 other school districts received the bottom score.
The data is based on 47,000 teacher observations that districts uploaded onto a state portal from August through Dec. 13. Principals perform a minimum of four observations per teacher per year, so teachers had more than one observation score reflected in the data. Some districts didn't have enough observations uploaded to be included in the data, and districts using their own evaluation models, such as Memphis and Hamilton County, aren't reflected in it, either.
Tennessee and 16 other states redesigned teacher evaluation models in the past two years, tying ratings to student test scores, according to the National Council on Teacher Quality. Tennessee, further along than most, both designed and piloted its new system in the 2010-11 school year and put it into effect for all districts this school year.
Under the new system, 35 percent of the final score is on student learning gains and 15 percent on data the school chooses, such as ACT scores. Principals use a long list of measures for success to do their observations, which count for the other half.
The state predicted that districts would rate 3-5 percent of teachers as ones; 10-25 percent as twos; 40-50 percent as threes; 10-25 percent as fours and 5-10 percent as fives. No district that submitted data hit all those ranges.
The projections were based on value-added scores - which measure how much students learned in a year - how other districts using the same observation form distributed scores, and research from the National Institute for Effective Teaching, said Emily Barton, the state's assistant commissioner for curriculum and instruction.
Murfreesboro City Schools also got a visit from state officials after issuing more fives on teacher observations than any other district in the state but one - Clinton City Schools.
"When first seeing the score distributions, I questioned why they were different than the predicted distribution," Murfreesboro City Schools Director Linda Gilbert wrote in an email. "I have made the principals aware of the state average and expected distribution ... but I have not asked them to change what they are doing."
She said the higher evaluation scores are justified, since the district is among 18 statewide that showed the most student growth on 2011 test scores.
After Metro Nashville Schools Director Jesse Register reviewed the data, he said his principals may have graded teachers more strictly than other districts.
Metro gave 15 percent of teachers observed a top score, compared to the state average of 20 percent.
"The only conclusion you can draw at this time is that our principals and evaluators are taking the process very seriously and there is not grade inflation here," Register said.
Tennessee Organization of School Superintendents board member Dan Lawson, director of Tullahoma City Schools, said districts are more concerned about observation score variances within their own schools more than differences by districts.
"That's going to be a much more real issue than the concern about what happens between here and Memphis," he said.
Barton said the projections aren't a quota, rather a tool for districts to evaluate whether they are being fair, and that scores will change moving into next semester.
"(The data) is inconsistent with what the research would project, but we are midyear on this," Barton said. "None of us are jumping to conclusions, because what is really going to matter is not just at the district level, but school level - how it lines up with value added, and until we know that, it's hard to come to significant conclusions."
The state is holding three weeks of evaluation training in January to refresh evaluators and train new principals.
At the end of the school year, districts with observation scores that don't closely align with value-added growth scores could penalize principals by taking 15 percent off their own evaluations, Barton said. For now, the state focus is that educators continue to get used to the new evaluations and that teachers get constructive feedback.
The evaluations won't affect teacher tenure this year, Barton said, since it takes five years for consideration and teachers can't lose tenure unless they have two years of low evaluation scores.
Gov. Bill Haslam has asked SCORE, a Tennessee-based education nonprofit, to conduct an independent study on the new teacher evaluation system and report back on June 1. SCORE declined to comment on the observation data report this week.
The Tennessee Education Association, which has criticized the implementation of evaluations, said individual lawmakers are expected to file bills seeking to change the system this session.
"Teachers are going to look at (this report) and ask, 'How can this be?' It defies logic," TEA lobbyist Jerry Winters said. "You can't take the subjectivity out of the system completely."