September 19, 2009

Absurd definitions of achievement : Deborah Meier

September 17, 2009
We're Relying on an Absurd Definition of Achievement
Dear Diane,

Yes, the ways in which standardized bubble-in tests are open to abuse are rampant—and gaming of test scores has increasingly important repercussions. In the long run it leads to an increasingly toxic education in the name of “standards.” I urge you to read the two chapters in "In Schools We Trust" on my own experience and subsequent investigation of standardized testing for 7-year-olds. I discovered, in my effort to “prep” them, that while reading tests may in part test the ability to read—comprehend (vs. re-code into sounds)—they are even better at detecting one’s class and cultural sub-group. Of course, there were exceptions. There are ways to raise scores via prepping, but it can only diminish the “gap” if those at the top get less effective prepping. I published the recorded interviews I conducted with students—in 1972, and Jay Rosner has shown how it works for the SATs, and many have followed. Between 1980 and 1990, it seemed we had won the battle. By 2000, they had returned with a vengeance. Thus many friends and allies suggested I give up on this particular line of reasoning.

Meanwhile, we’ve witnessed a massive redistribution of wealth. Everyone has their culprit. Too often, "lazy" children, parents, or educators/teachers? In a nation that ranks near the bottom in services for the young, not to mention a bigger gap between rich and poor than we’ve seen since the 1920s, maybe this is myopic? “Forget it,” friends suggest. You’ll just be accused of making excuses for bad schools.

What you have documented, Diane, is eerily close to the kind of data abuse that helped create our current financial crisis. We go blindly ahead. There seem equally few lessons learned from either crisis. It’s apparently easier to hold children and teachers’ “feet to the fire” than the creators of the greatest economic crisis of our lives—especially Harvard grads. Note: The latter did fine on their SATs.

Many reputable studies have also demonstrated that, based on such tests, no evidence exists for most of the Race to the Top-NCLB-like reforms. Just rhetoric. Just more shifting of power away from the public sector. The CREDO study you mention suggests that only 17 percent of the charter schools do better on tests than their comparable noncharters, and more do worse. Try that with a drug test, and how long would it remain on the shelf? (Even though, in fact, low scores is not one of my complaints against charters.)

What’s the alternative, critics argue! Is a bad drug better than none at all, I respond?

In fact, we have alternatives—some pioneered in the good old USA and others in those much-vaunted international comparisons. For example, there is no competitor that relies as much as much as we do on testing of our sort. None. None. None. (A light bulb should go on.)

Alas, there are no simple magic bullets even among the reforms I like. Partly because we don’t all have the same agenda when it comes to outcomes—our priorities differ, what we’re willing to trade off or risk differs. Also, any reform package depends on its implementation and few recipes are foolproof. Trying to copy the KIPP model or the Deborah Meier “model” won’t produce the same thing.

And, then there’s the ornery fact that when today’s reformers refer to proof of “achievement” they mean something different than you and I do, Diane. Achievement equals standardized test scores in reading and math; others add test scores in other subjects, including aptitude/IQ tests. Everything else gets called “soft skills.” It’s as absurd as calling the written driving test the real achievement and the road test a measure of “soft” skills.

I’d argue that this applies to most intellectual knowledge and skills. We’re relying on an absurd definition of achievement—at best. It’s not lack of alternatives, but a lack of interest in having real standards that take into account that we are all not the same and that we actually don’t want to be all the same. Our interests, passions, talents, and even our priorities differ. High standards can be met in honest and serious ways if we are prepared to take the harder route—starting with each child. If we care enough about both means and ends consistent with democracy it will not be easy and will not be based on the need to rank order individuals, schools, or nations. There are plenty of good alternatives.

Any publicly funded school must serve, for better or worse, the public good—the complex demands of a modern democracy. But the public good is also met best when each individual's private good (his/her passions and interests) are also met. But how we sort out what best serves both will always be an art, not a hard or exact science.

There is no single “best” model—but there are bad ones. Neither the common good nor personal good can come out of schools that do not treat all members of their community with respect. (Respect, of course, is no easier to define than achievement.) Perhaps the reason I love "Tales of Priut Almus*" by Robert Belenky—which describes the time he spent in a Russian shelter for homeless youth—is the unmeasurable respectfulness with which they responded to the young people in their care. Belenky says “offering a short-term, flexible, familial, community-based home may be the most useful gift (offered) these young people.” But he actually describes far more than that. So, too, in any respectful school there is more than that—but nothing without that.

Deb
From: Education Week blog.

Posted by Deborah Meier at 7:24 AM | Permalink | Comments (19) | TrackBacks (0)

Labels: , , ,

August 12, 2009

The problem of testing

Testing Issues
Added to the shortcomings of the standards movement is the recent heavy emphasis on high-stakes testing to determine the achievement of the standards. Decades of work have highlighted the effectiveness of authentic assessments (portfolios, student exhibitions, scoring rubrics, etc.) as tools for informing teachers’ instructional practices and methods for communicating to students and parents the knowledge that has been gained. However, the current practice in testing is the standardized, norm-reference test, consisting almost wholly of multiple choice questions. It is problematic that policy makers would on one hand mandate the development of elaborate content standards only to couple such policies with low-level and narrow assessments (Nichols and Berliner, 2007, Dorn 2007)
The long and troubled history of testing and test development in the United States was severely damaged by racism, which the test producers have yet to overcome (Berlak, 2000; Valdéz & Figueroa, 1994). Standardized, usually multiple choice, tests are preferred because they can be mandated by political leaders, implemented, and deliver clear results. But, as in the case of positivism and reductionism (from which these tests come), they are measuring and evaluating only a small segment of the important learning goals of schools. Low level tests do not measure human relations, respect, civic courage, and critical thinking, for example. Standardized testing is a political act that often forces teachers to change their teaching strategies. Teachers need to examine the limits of the testing processes and use classroom based assessments to inform their teaching. In many states, including Texas, California, and Massachusetts, and in many school districts, including New York, Chicago, and Philadelphia, standards and a test driven curriculum have been used to reduce teacher professional choice and decision-making.
The most basic failure of the testing/accountability model was to refuse to recognize that public education is far more than production; it includes, at a minimum, facts, concepts, generalizations, skills, attitudes, critical thinking, and citizenship. Thus, a business-production model, based on low-level, multiple choice testing, only measures a small part of the important issues of schools in a democratic society (see Renzulli, 2002, Dorn 2007). Meanwhile, governors and legislators committed to the testing movement ignore other parts of the business production model, including providing as much support, including tools, training, and technical assistance for the workers as needed.
Testing systems have grown in part because they are very profitable for the companies that produce and score these low-quality tests—companies that lobby the legislatures to establish testing systems. State funding for testing grew in Texas from $19.5 million in 1995 to $ 68.6 million in 2001 and at similar rates in other states (Gluckman, Jan. 2002). Bloomberg News estimated in 2006 that the testing industry makes over $ 2.5 billion per year ( Gloven and Evans, 2006). Funding increases for testing and test preparation usually is matched by a reduction in funding for other classroom items such as textbooks, dictionaries, libraries, and teacher support. In spite of these large investments test-based accountability systems, without major improvements in the quality of testing and investments in teacher capacity-building, will not produce significant improvement in student achievement in high-risk neighborhoods (Kober, 2001; Popham, 2003, Nichols and Berliner, 2007).
Extensive evidence shows that the current testing emphasis has driven instruction away from important issues of developing democratic and multicultural content, away from critical thinking, and away from the development of citizenship and prodemocratic values (Neill, 2003; Renzulli, 2002). Available testing, particularly multiple choice testing, is not the only form of assessment. Other assessment devices include teacher observations, rubrics, student presentations, and portfolios ( Wood, Darling-Hammond, Neil, and Roschewski, 2007 ) These forms of assessment can be used to measure progress on goals of critical thinking, democracy, and important multicultural goals such as mutual respect.
Scores on most standardized skill tests actually teach us very little; they measure very imprecisely. Current objective tests measure whether the student can identify letters, words, and rhyming words, but they do not measure comprehension of a paragraph or the ability to write a creative essay. They measure skills and isolated facts rather than significant academic achievement. Tests are usually not actual measures of competencies, but measures of isolated skills that can be drilled without improving the student’s education. Rather than investing more money in the current low-quality testing systems, we could develop appropriate and useful assessments, including using computer technology, which would help the teacher. There are good uses for standardized testing. They should be short tests given frequently that assist the teacher in making decisions about individual students, teaching and review. But that is now what is happening with testing in k-12 today.
From: Choosing Democracy; a practical guide to multicultural education. 4th. ed. 2010. p. 381.
Duane Campbell

Labels: , , ,

June 28, 2007

Supreme court decision on school integration

Today’s Supreme Court decision on school integration plans.

Lets be honest: The Supreme court decision was a victory for conservatives. It reflects the Bush Supreme Court. Our opponents, such as the Linda Chavez Equal Opportunity Commission, Ward Connerly and others, oppose the sue of racial and ethnic measures to make school assignments. But, they do nothing to improve the quality of schooling in the poverty schools. They want to fight about racial categories but not work to improve schools. It is important to not ask the oppressor to design the remedies for past discrimination.

But, our moderate allies and elected officials share the responsibility for this defeat.
The school integration policies of cities have not worked, they have not produced high levels of school achievement for all. So, parents fight sending their children to integrated schools because many of these schools are dysfunctional. ( See Choosing Democracy, 2004)
School districts in Seattle and Louisville were defeated ( as will be Sacramento, Los Angeles and others) because they relied upon the lawyers approach and refused to listen to or to change the educators approach.
Since the 1990’s, we have had a school reform plans usually based upon writing standards and standardized testing, placing pressure on teachers to follow a specific curriculum and a focus on testing. The data on this process is in- it has not significantly improved most schools.
While there are exciting examples of improved achievement, in general, school achievement has remained stagnant. (NAEP) There is only limited evidence that this process actually improves schools , student achievement, or improves student opportunity. (Rothstein)
There has been limited improvement in most schools because the interventions used do not deal with basic causes of low achievement, unequal funding of schools, high teacher turn over, family disruption, un safe schools, crime, safety, unemployment have not changed—and therefore the local school is unlikely to change.
We need to improve the schools. The current dominant school leadership and legislative leadership has not improved the schools. The refuse to make the necessary changes to improve schools. The lawyers approach of integration- without school improvement, has failed.
In most cities, including Sacramento, Los Angeles, S.F., Oakland, etc, we have legal integration, but we also have extreme inequality of opportunity. The goal is not diversity, the goal is equal educational opportunity. These lawyers- these school administrators- have not worked for equal opportunity.
Now, in response to the Supreme Court decision, it is time to improve the schools. It is time to stop using lawyers as a substitute for school reform.
Our opponents, such as the Linda Chavez Equal Opportunity Commission, Ward Connerly and others, oppose the sue of racial and ethnic measures to make school assignments. But, they do nothing to improve the quality of schooling in the poverty schools. They want to fight about racial categories but not work to improve schools. It is important to not ask the oppressor to design the remedies for past discrimination.


Duane Campbell
author, Choosing Democracy, a practical guide to multicultural education. (2004)

Labels: , ,

April 19, 2007

Reading first and honest use of data

At least we should be able to agree to look at data honestly. The President and his staff can not make this claim.

The President and Secretary of Education constantly
maintain that Reading First has had a positive effect
on national tests, because of an increase on NAEP
trend test scores for fourth grade reading between
1999 and 2004, from 212 to 219. (Margaret Spellings
said this at the NCLB summit in April, 2006, and it
was repeated a year later, in April, 2007 on the White
House website, with many repetitions over the last
year.)

Trend tests, considered appropriate for comparison,
were not given between 1999 and 2004. Gerald Bracey
(2006a) has pointed out that Reading First did not go
into effect until 2002-2003, so it is not clear that
Reading First deserves the credit for the increase
(according to the recent GAO report, 25 states were
funded in 2002-2003 and 25 more in 2003-2004). In
fact, several analyses show that reading scores have
not improved since Reading First went into effect.
(Fuller et. al., 2006; Lee, 2006).

Bracey (2006b) also doubted that many children in
districts that did Reading First took the NAEP in
2004. The test is given to nine year olds, and Reading
First is aimed at grades three and lower.

Data from the recent GAO report confirms Bracey’s
suspicion. The report tells us that 1,200 districts in
the US were awarded Reading First grants. That’s only
seven percent of all districts in the US.

Let us give Reading First the benefit of the doubt and
ignore all the counterarguments presented above. Let
us give Reading First even more benefit of the doubt
and assume that even though only seven percent of
districts did Reading First, these districts were
gigantic, and that 21% of all children who took the
NAEP in 2004 had Reading First. If Reading First is
to take credit for the seven-point gain, those
children would have had to outscore their 1999
counterparts by a fantastic 33 points, a score that is
nearly exactly at the 75% percentile. If the Reading
First districts contributed only seven percent of
those who took the test, they would have had to score
312, outscoring their 1999 counterparts by 100 points
(The 90% percentile in 2005 was 263).

Again, several studies tell us that there was no gain
on national tests after Reading First went into
effect, and Bracey gives us reason to doubt that many
Reading First children were old enough to take the
test. But even ignoring these arguments, all estimates
of the number of children who took the test who were
in Reading First districts makes it highly unlikely
that Reading First deserves any credit for the
1999-2004 increase.
S. Krashen



Bracey, Gerald. 2006a. The 16th Bracey Report on the
Condition of Public Education. Phi Delta Kappan 18
(2): 151-166

Bracey, Gerald. 2006b. Letter to Congressperson George
Miller and Senator Edward Kennedy, September 25, 2006.

Fuller, Bruce, Gesicki, Kathryn, Kang, Erin, and
Wright, Joseph. 2006. Is the No Child Left Behind Act
Working? The Reliability of How States Track
Achievement. University of California, Berkeley:
Policy Analysis for California Education

GAO 2007. Reading First GAO 07-161 (States report
improvements in reading instruction, but additional
procedures would clarify Education’s role in ensuring
proper implementation by states.)

Lee, Jaekyung. 2006. Tracking achievement gaps and
assessing the impact of NCLB on the gaps: An in-depth
look into national and state reading and math outcome
trends.
Cambridge, MA: The Civil Rights Project at Harvard
University.

Labels: , ,

February 19, 2007

Imposed school reforms cost students $, do not improve schools

We know that the politically imposed school reforms have not worked, but the legislature does not.
Unfortunately the legislature usually will not look at the data. A new report from Education Testing Service again shows that school achievement in the nation and in California has –at best-been stagnant for the last twenty five years. (Kirsh, Braun, Yamamoto and Sum, 2007) The achievement gap between mostly middle class and white students and the growing poor and working class Latino and Black student populations, whether measured by test scores, drop out rates, or college attendance narrowed slightly between 1970 and 1988. There has been little improvement since 1990. ( NAEP, 2005. Perie and Moran)
California’s scores are lower than the national average and they are not significantly different from those in 1992. Contrary to claims by the California Dept. of Education, on the national NAEP test, the decade of test based school reform has not significantly improved scores in reading and math. (NAEP, 2005) There are good reasons why most schools have not improved. ( Rothstein, 2004, Anyon, 2005)
Working together at the school site can work. There are many examples of improvements of specific schools
In July 2006, The California legislature passed SB 1209 (Scott), entitled Teachers: teacher credentialing: out-of-state teachers: professional growth programs: teacher compensation. SB 1209 had 19 provisions, and claimed to be the implementation of existing law particularly SB 2042. Important issues in the bill include expansion of funding for charter schools, revision of teacher preparation laws for interns, provisions requiring ELL preparation and others. Provisions of the bill also mandated implementation of the teacher performance assessment system in all credential programs as of July 1,2008. These provisions required the application of the unproven process of teacher performance assessment WITHOUT BUDGET ALLOCATIONS. Interestingly, teacher preparation programs already used performance assessment for their fundamental decisions, however the assessment process is based upon clinical supervision over time rather than upon a single test as favored by testing advocates at the CCTC.
SB 1209 was presented to the legislature by Senator Scott as implementation of the several ideas advocated in reports by the Center for Teaching and Learning at UC Santa Cruz. Both major teachers unions agreed to SB 1209 in support of other provisions in the bill. Many of the provisions are indeed designed to improve teacher preparation. However the items in dispute – changes to Section 44259 (3) of the Education Code will create havoc in teacher preparation and is already reducing teacher preparation to teaching to the test.
There was no discussion of the testing mandates in committee hearings while the legislature considered SB 1209. Indeed it is doubtful if the legislature or their staff understand what performance assessment is, and certainly do not understand the major weaknesses of the current state of assessment.
The bill SB 1209 (Scott) from 2006 imposes an unfunded mandate which will cost new teachers hundreds of dollars. One possible consequence of focusing on this new performance assessment in a test situation is to significantly reduce the dollars spent on teaching or supervision in order to pay for assessment. A second possibility is to pass along the significant cost of testing to teaching candidates. Our brief estimate is that it would cost $400 - $500 per candidate. Is this the way to recruit new people to teaching?

A major function of bureaucracies, in this case the Commission on Teacher Credentialing and their staff is to keep the public out of decision making. Their role is to substitute their own views for information. Since the 1990’s, the CTC and the California School board have been dominated by ultra conservative, pro corporate elements.
CTC staff and members make certain that the persons who do the work, teachers, faculty in teacher preparation, are not listened to.
In those rare occasions when the public breaks through the bureaucratic front, both CTC and the State School Board arrange for hearings where only their own “experts” are allowed to speak. In summary, the role of these bureaucracies, and at times of legislators and consultants to is block or prevent democracy.
We are presently looking for a legislator willing to sponsor legislation to repeal this unfunded mandate.

The issue is similar to the legislature deciding upon a reading program or a math program. In this case, the legislature accepted the CCTCredential Commission’s narrow definitions teacher preparation and assessment. The imposition of a regressive system offensive to teacher preparation professionals was passed with no discussion in the legislature of the impact. This is not democracy. This is incompetent meddling by the poorly informed.

Duane Campbell

Labels: , ,