FAQ

Frequently Asked Questions

This is a short-answer response page only. You should refer to the papers suggested to get more thorough understandings.


Note: If there is a question that you would like to see answered, please use the comment form at the bottom of the page.


  1. If kids don’t score well, it can only be the fault of their teachers, right?

    • We know from good research that this is simply not true for two primary reasons:
      1. The standardized tests overwhelmingly measure the living conditions and environment of the student, not academic achievement. Teachers have no control whatsoever over those non-school conditions; therefore, scores are not the fault of the teacher whether the scores are high, middle, or low. (See papers on Pseudo Accountability; Validity.)
      2. Research shows that the overall impact of a teacher on student achievement is a very small percent of the forces and factors that affect achievement. All the evidence from the studies I have read and the two research studies I have done indicates the direct teacher effect on student achievement is about 10 percent. The school environment, leadership, administration, facilities, and schoolmates add another 10 percent. Non-school forces and factors such as individuals, family, neighborhood, and socio-economic level determine about 80 percent. My own research suggests non-school effects at 80%. (See papers on Value Added; OTES; Hoover 2000; Hoover 2008.)
    • Implicit in all of the reform rhetoric at the state and federal levels is that there is no such thing as poorly performing students, only poorly performing educators. While obviously an absurd idea, the reform movement sells this ideas to the public in order to get the backing for what they intend to do. (See papers on Pseudo Accountability.)
  2. Shouldn’t kids who go to school be able to pass a test on what they have been taught?

    Of course they should, but the standardized test scores are more a function of non-school forces and factors far more influential and powerful than what they experience in a given teacher’s classroom. Standardized tests are not like the traditional teacher-made tests.

  3. What is a confidence interval?

    In terms of testing, it is the range of scores possible for any particular score given the measurement error of a test. All standardized tests have confidence intervals The confidence interval tells us the range in which a specific test score will likely fall. For example, a student earning a score of 82 on some standardized test may well have not scored 82, but scored somewhere in the confidence interval. It is a mathematical fact that we cannot know the precise score of any student on any standardized achievement tests. We can only approximate the probability of the score within a range of possible scores. (See papers on OTES; Value Added.)

  4. Teachers want to know if OTES is making teaching to the test permanent?

    • Since the accountability system is driven almost entirely by test scores, teaching to the test is inevitable. OTES explicitly locks in teaching to the test by essentially evaluating students and their teachers on how well teachers actually do teach to the test. However, there is no evidence to support that teaching to the test actually raises test scores significantly.
    • Since OTES uses test-score gains to judge teachers, it makes teaching to the test seem extremely important. This is a cruel paradox for the teacher because, in fact of research, teachers have an extremely minor role in affecting test scores. The methods used to determine gain scores are in no way valid with those doing SLO with pre and post assessments being totally at the mercy of random probability rather than with skilled methodology and professional intelligence.
    • Across the nation, including Ohio, teacher effectiveness and school success are defined by test scores; therefore, implicitly, the entire purpose for going to school now is to get high standardized test scores.
  5. Value-added is a factor in teacher evaluation, and it is here to stay per law. Why is there such a push to use only a small sample of 1-3 years to make a determination for teacher evaluation?  What materials could be used to show that using only a 1-3 year sample rigs the interpretation of the data?  OEA recommended 3 years when this law was first passed in 2012.

    • Value added is an estimate, therefore becoming more stable with multiple years of data. This is a mathematical reality. Battelle For Kids and EVAAS both say three years minimum in order to get more reliable results. This is based upon statistical principles, not the worthiness of value added itself. The more years used, the more stable the end result. Using fewer than 3 years does not “rig” the outcome, it just makes it more likely to be extremely inaccurate. Debating the number of years used begs the question that value added is not at all appropriate for evaluating teachers no matter the number of years of data that is used.
    • By using fewer than three years of data, ODE is adding a major violation of mathematical principles to an evaluation system that is already flawed. This is clearly voodoo metrics pushed by ODE and the anti-teacher faction in the state. Given a fair hearing, no court in the land would accept these results as in any way indicative of teacher effectiveness. Using less than three years contradicts the position of Battelle for Kids and EVAAS.
    • Debating whether to use a certain number of years is like debating what color shirt to wear to one’s own execution—it does not matter in the long run. (See paper on Value Added.)
  6. Why are all factors in the OTES pointed at the teacher without regard for students’ personal and socio-economic factors?  It seems that the achievement goals are arbitrary which in turn will affect evaluations.

    • This is the crux of the entire educational reform issue. The metrics are designed to place all non-school achievement variables on the teacher. This question goes to the heart of pseudo accountability and the metrics machine that perpetuates the system of false claims and lies about public schools and their teachers.
    • This also goes to the heart of the validity problem with standardized tests, which measure the historical effect of the living conditions of the student, not the teacher effects. The entire section of OTES dealing with student growth measures is 100 percent indefensible from a mathematical and psychometric perspective. (See papers on Validity Research; OTES; Value Added.)
  7. OTES provides limited professional development.  It is left up to districts, but it is being interpreted differently.  Ex.)  Some districts negotiate the system, what evaluation looks like and outcomes.  Other districts don't negotiate and simply implement whatever they want.

    • Teacher advocacy always supports negotiated professional development.
    • ODE fails miserably to provide appropriate guidance, so they pass the buck to local districts. (See paper on OTES.)
  8. How can administrators be prevented from creating schedules that facilitate teacher failure?

    Unless the assignment involves a policy issue in the contract, there is little recourse. If it does relate to a contract policy, then the grievance procedure may help resolve the problem.

  9. Is this a way for politicians to connect merit pay to teacher rating?  Can this be shown to be not valid?

    • A gradient of value-added performance could determine pay, or a combination of both sections of OTES could be used. Connecting pay in any way to OTES is taking pseudo accountability to its highest level.
    • Showing that this has no validity is the easy part. Getting Columbus to listen is the hard part. Our unions need to educate the members, the media, and the public. We need leadership to go to the wall on these issues.
  10. With state vendor assessments (PARCC) coming, will this be OGT/NCLB on steroids?

    Yes. PARCC & Common Core State Standards ramp up the role of testing like never before. (See paper on the PARCC & the Common Core.)

  11. Will test scores determine funding even if tests are designed for students to fail?

    Yes, it could happen. We would then likely have the wealthiest districts getting the most funding and the poverty districts the least.

  12. We teach with our target being the test because that is the law, but how can a message be crafted using evidence that the quality of teaching and learning is not measured at all in the test?

    I would hope that everything on The Teacher Advocate site will serve as an advance organizer for crafting the ultimate message. There is more than ample research available to prove the case. But until our unions are committed to using the research, nothing will happen. Remember, it was OEA that refused to use or even share my research showing the tests to be not valid. (See paper Reflections on My Union Experience.)

  13. We teach to the test because it is the law, but as an educator is it ethical to try to justify the quality of the education our students receive?

    I have been writing about schooling and ethics over the years, and I am firmly of the position that what teachers are being forced to do is highly unethical in the sense of what it denies students. To try to justify anything that is not justifiable is unethical by default. I have written about what I call “the pedagogical imperative,” a principle of teaching that says we must teach to empower, to give students the opportunity to learn how to use the concepts, principles, and ideas of our subject areas and of democracy. The immorality of what is happening to students is not the fault of teachers. Nonetheless, they are put in the middle and told to follow orders. I believe that it is morally incumbent on teachers to resist and subvert the system in order to do right by the students, an action much easier said than done. (See the first and last footnote references in A Personal Note to the Reader at the bottom of The Teacher Advocate homepage.)

  14. What can our unions do to help us?

    • The unions must embrace teacher advocacy fully before anything can be done to truly help teachers. The unions must then be proactive in educating memberships and the public at large as well.  The unions must aggressively take our case to the public with zeal and candor and must be willing to file court actions and legislative initiatives.
    • The unions must change their political function.  Don't shop our unions to politicians; make politicians shop us.  They need us; we don't need them.  The decisions to support candidates must be based on a clear litmus test as to their positions on the accountability system. Anyone with a track record of voting for any of it needs to be denied our support.
    • The unions must form active public coalitions with all other public and private unions as well as with civic groups that share root values with our cause.  It must be made absolutely clear that if you pick on one, you pick on all.
    • The unions must use the same principles of teacher advocacy nationally and at the state level. Fighting for teachers and public schools is always a war with two fronts, the national and the state front. NEA and OEA need to be organized and coordinated for fighting on the two fronts. We will never succeed if we only fight on one front. NEA’s resolution to get Arne Duncan to resign is a good sign for all of us. Likewise, the principle and values of teacher advocacy must be shared between NEA and AFT and between OEA and OFT if we are to succeed.

     

Comments

Randy this web site is marvelously done, and though I have just skimmed the surface, I do plan on reading the majority of the research and what not. This year I will be responsible for an additional tested area besides Reading and Math. Social Studies gets added this year in fourth grade as you well know. Can you or someone unveil the deep dark secret on the value-added formula in terms of rating the teachers? I can't get an answer from ODE that makes any sort of sense.

Great job Hoo, keep fighting the good fight!

Randy Hoover's picture

Thank you very much for your words. As for value added, it is a good question, but a hard one to answer specifically.  I have examined VA and even attended an all-day session where EVAAS explained it.  The complexity of it is beyond belief.  I am still not clear on what is being held back, if anything. However, VA as used in Ohio is invalid in at least two ways:  It is based upon a test that is demonstrably invalid and it assumes a level of teacher effects far beyond what research shows is even possible.  I suggest you hound Battelle for Kids about getting the calculus for it.  What does OEA say about it?  -rh

 

Congratulations on the start up of the The Teacher Advocate! This is a great forum for Ohio educators to visit and gather important information, as well as engage in important dialogue. Keep up the great work and best wishes for continued success.

What are some specific actions for parents to do in order to help?

Randy Hoover's picture

I think the first and most important thing is to understand the issues fully.  Ultimately all of this is up to the elected representatives in Columbus.  Talking to your representatives is important as is voting for a candidate for Governor who is supportive on public schools and understands the current accountability system is completely invalid.

Well done. Thank you.
I wrote about my high stakes testing experience in Ohio for an organization requesting information. It can be found at:
https://docs.google.com/document/d/1YjXOI4OCI3FuuBc42o7J4KRukyasWfAc6IIT...

After 25 years of teaching, I have been flagged by the state to be on an improvement plan because of my low value-added score from an OAA test two years ago in reading. I was rated highly effective in my SLO score from last year and my principal rated me accomplished in his evaluations of my teaching performance. I apparently have no recourse or way to fight this. Reading your article at least made me feel better about how invalid the student growth measure really is. This evaluation system is so unfair and makes no sense.

Add new comment

Target Image