Research Roundup 16, 1 (Fall 1999)
Accountability
Principals are no strangers to accountability; whenever a problem occurs in a school, heads automatically turn toward the office. However, the recent emphasis on high-stakes, standards-driven accountability systems poses a new set of problems for school leaders.
The new accountability assumes that systematic assessment of school performance on the basis of clearly identified standards will lead to school improvement. People who know the expectations tend to live up to them, especially when results are linked to consequences. Unlike the days when annual test scores were simply archived, accountability emphasizes public disclosure and possible sanctions, including the closing of nonperforming schools.
Although principals are more than willing to take responsibility for what happens in their schools, the current accountability movement presents some special challenges:
- mobilizing human and fiscal resources to reach standards that are not just higher but more sophisticated
- avoiding unintended side effects, such as the tendency of assessment to drive nontested content out of the curriculum
- managing public perceptions when test scores are published with little explanatory context
- maintaining teacher morale in schools identified as low-achieving
- ensuring equity for students with special needs or students from disadvantaged backgrounds
The answers are far from clear, but the following readings provide a useful starting point for principals looking for a way through the accountability maze. Each views accountability through a slightly different lens, but collectively these readings argue that even in an era of state-mandated assessment, principals can pursue proactive strategies that will serve the needs of their students.
James A. Watts and colleagues describe the nature of comprehensive accountability systems.
Charles Abelmann and colleagues show that schools have internal accountability systems that influence the success of externally imposed standards.
Karen Levesque and colleagues provide school leaders with a guide to using school data to help chart the improvement process.
Mack McCary and colleagues describe the efforts of one district to develop a "culture of accountability" throughout the system.
Robin Lake and colleagues examine the strategies used by Washington State schools to respond to state assessments.
Watts, James A.; Gale F. Gaines; and Joseph D. Creech. Getting Results: A Fresh Look at School Accountability. Atlanta, Georgia: Southern Regional Education Board, 1998. 27 pages. Available from: Southern Regional Education Board, (404) 875-9211 x236. $5.00 plus shipping and handling. www.sreb.org
Advocates of accountability often see it as a simple matter of testing results. But as the authors point out, effective accountability is a system that links standards, testing, professional development, reporting, and consequences. Without careful alignment of the component parts, testing alone will have little effect.
The heart of the system is a set of clear content standards that point the way for students, teachers, and policy-makers. These standards should be developed with stakeholder input, and should be understandable, rigorous, reasonable, well-focused, and measurable. Tests must be directly aligned to these standards; be reliable, valid, and fair; have a clear purpose; be operationally feasible; and be useful for school improvement by showing each school how its students are performing and where its instructional strengths and weaknesses are.
While accountability can be mandated at the state level, improvement can only occur in schools, which must have the capacity to make the necessary improvements. Thus, effective accountability requires a well-coordinated professional development program that is keyed to the standards and assessment, focused on student achievement, responsive to school needs, accessible for teachers, and well funded.
The fourth component is the process of informing the public of a school’s progress. Effective reports focus on student learning, reveal trends, and communicate with a variety of audiences. A good report shows schools where they are, where they need to go, and what strengths and weaknesses they bring to the task.
Finally, accountability requires a bottom line, with rewards, sanctions, and assistance triggered by results. Highly controversial, rewards and sanctions require careful attention to fairness, consistency, and equity. For example, accountability systems should recognize improvement as well as attainment; otherwise schools in a deep academic hole may be too discouraged to make the effort.
Abelmann, Charles, and Richard Elmore; with Johanna Even, Susan Kenyon, and Joanne Marshall. When Accountability Knocks, Will Anyone Answer? No. RR-042. Philadelphia: Consortium for Policy Research in Education, 1999. 51 pages. Available from: CPRE, Graduate School of Education, University of Pennsylvania, Philadelphia, PA 19104. (215) 573-0700. $10.00 (includes shipping and handling). Prepaid check made payable to CPRE is required. http://www.upenn.edu/gse/cpre/
The current accountability movement assumes that without external pressure schools would evade accountability. Charles Abelmann and colleagues challenge this assumption, arguing that schools have internal standards of accountability that influence the way they respond to state-imposed standards.
The authors conducted case studies in a diverse sample of twenty public, private, and charter schools. They found that these schools had preexisting standards of accountability that took three forms. First, individual teachers showed a personal sense of responsibility, with strong beliefs about what level of work to expect from students, what kinds of assignments are appropriate, and what students should be learning. These beliefs were based on personal values and often differed from teacher to teacher within the same school.
Second, teachers in a school sometimes developed shared expectations by working together on schoolwide issues. Some schools, for instance, had achieved consensus on what constituted an acceptable reading level for first-graders, what level of corridor noise was acceptable, or how much homework should be assigned.
Finally, some schools had formal accountability mechanisms such as handing in weekly lesson plans, adhering to common disciplinary procedures, or following the established curriculum guides. The authors note that these formal accountability measures were more powerful when they were congruent with individual values and collective expectations. If responsibility, expectations, and accountability measures were not aligned, accountability systems were weak.
The authors note that the strength of the internal system affects the response to externally imposed measures. Schools with weak internal accountability will respond idiosyncratically and erratically to external accountability. Schools with a strong internal system will accept outside standards when consistent with existing values, resist them when not, or simply adopt whatever imposed standards are consistent and ignore the rest.
Because their sample of schools did not operate in a strong external accountability environment, the authors are cautious about generalizing. But they conclude that schools with weak systems view external accountability like the weather–something that will affect them, "but not something they could or should do anything about." Thus, accountability begins at home. The lesson for school leaders is that strengthening teachers’ collective expectations through vision-building will enhance the school’s ability to respond coherently and effectively to state-mandated accountability.
Levesque, Karen; Denise Bradby; Kristi Rossi; and Peter Teitelbaum. At Your Fingertips: Using Everyday Data to Improve Schools. Published jointly by MPR Associates, National Center for Research in Vocational Education, and AASA, 1998. 208 pages plus 64 worksheets. ISBN 0-9662883-0-0. Item #269-001. Available from: AASA Distribution Center, PO Box 411, Annapolis Junction, MD 20701-0411. AASA member price: $34.95, List $39.95. (888) 782-2272. http://www.aasa.org
Meeting the new standards requires sophisticated leadership to maintain a steady focus on improvement while still satisfying the relentless everyday demands of constituents. Karen Levesque and colleagues suggest that developing a set of "vital indicators" can serve as a reliable compass.
Schools typically generate reams of data for routine reports to state agencies. But instead of filing and forgetting this information, schools can use it to maintain control over their reform efforts rather than being totally driven by external tests. The authors outline a six-step process that begins with the identification of priority goals that may reflect national, state, or local concerns, but that should be meaningful, realistic, complementary (mutually consistent), measurable, prioritized, and agreed to by stakeholders.
Because system goals are often stated broadly, a second step is needed to sharpen expectations. The most important outcomes focus on student learning, but tracking inputs (the resources available) and school practices (such as curriculum and assessment procedures) can help identify reasons for the school’s success or failure.
The third step identifies existing data sources that may serve as useful indicators and creates new information through questionnaires, teacher logs, focus groups with stakeholders, and classroom observations. However, the raw data alone are of limited use. Leaders must sort through a variety of complex issues such as what numerical form will be most meaningful, or who, what, and when to measure.
The fourth step, examination of the data, requires thoughtful analysis and interpretation. For example, a daily attendance rate of 92 percent may initially sound good (seemingly equivalent to a grade of A), but it also represents more than three weeks of missed classes. Similarly, overall figures may disguise significant variations; perhaps attendance is much lower on Fridays than the rest of the week, or absenteeism may be higher for sixth-graders than for second-graders.
With the information in hand, schools can set performance targets by focusing on three key questions. "How are we currently performing?" and "What do you want to reach for?" will be answered by matching local results against state and national standards, exemplary schools, and stakeholder expectations. The third question, "How do we get there?", is answered by identifying strategies that are likely to move the program in the desired direction.
Finally, to make improvement a continuing process rather than a one-time event, the first five steps have to be institutionalized into a recurrent cycle of goal setting and data evaluation. Although this kind of effort is a formidable challenge, the authors provide numerous examples, worksheets, and suggested activities that will give schools a running start.
McCary, Mack; Joe Peel; and Wendy McColskey. Using Accountability as a Lever for Changing the Culture of Schools: Examining District Strategies. Greensboro, North Carolina: SouthEastern Regional Vision for Education, 1997. 69 pages. ED 408 697. Available from: SERVE, 345 South Magnolia Drive, Suite D-23, Tallahassee, FL 32301. Item No. RDUAL. $8.00 plus $2.50 shipping and handling. (919) 334-3211. http://www.serve.org
Pressure for results can lead schools to an almost obsessive focus on test scores. In this report, Mack McCary and colleagues advocate an alternative bottom-up approach that weaves accountability into the entire school culture.
The authors start from the premise that an excessive emphasis on testing can lead to low teacher morale, a narrowed curricular focus, a diminished sense of professionalism among teachers, and unethical placement practices. By contrast, accountability can be viewed in terms of Total Quality Management–everything a school does to ensure continual improvement and quality. Here the goal is steady progress rather than a single-minded focus on the bottom line.
The authors’ North Carolina district built a "culture of quality" with several years of intensive effort. The starting point was developing meaningful indicators that went beyond test scores and that reflected the school’s educational vision. Included in their forty-two new indicators were data such as entrance requirements for different grade levels, dropout rates, student participation in community-service projects, involvement of parent and community volunteers, the number of student books read at home, and faculty involvement in extracurricular activities.
District personnel took an active lead in identifying the new indicators and building commitment to making them a daily part of institutional life.
Self-assessment is a major piece of the effort. For example, a year of workshops and examination of research led to a self-assessment tool on key strategies for teaching K-6 communication skills. This document in turn contributed to an emerging common vision about effective instruction. Likewise, student self-evaluation was encouraged through the development of rubrics and similar tools.
The authors emphasize that achieving a culture of accountability does not happen overnight. Their efforts required active engagement with the entire school community and a capacity for listening and responding to staff concerns. They found no simple recipes, but relentless effort led to progress.
Lake, Robin J.; Paul T. Hill; Lauren O’ Toole; and Mary Beth Celio. Making Standards Work: Active Voices, Focused Learning. Seattle, Washington: Center on Reinventing Public Education, 1999. (206) 616-5769. Download whole report for free at http://www.gspa.washington.edu/crpe/crpehome.html
Accountability systems ostensibly offer incentives that will spur schools to improve instruction. Some schools do, others do not. What accounts for the difference?
Robin Lake and colleagues explored this question in their study of forty Washington State schools, thirty of which had significantly raised scores on the new fourth-grade assessment. They interviewed principals to find out how their schools had responded to the initial assessment.
The authors found that significant improvement came when schools took a proactive approach–assessing their strengths and weaknesses, focusing on key priorities, and seeking help when they needed it. In particular, they found that effective improvement efforts are focused and schoolwide, going beyond tinkering with new textbooks or adding small course modules; often, the improving schools had made major philosophical shifts. Teachers in improving schools functioned as teams, not as individuals. Improving schools were more likely to align professional development activities with improvement goals. Improving schools actively sought assistance, taking the attitude, "Don’t wait for help. Go out and find it."
Improving schools focused all available resources on key instructional goals. They were also more likely to involve parents in the process. It was also apparent that improving schools were responding as the accountability system predicted; stimulated by professional pride or community pressure, they felt compelled to do better on future assessments.
The authors note that their study was too limited to generate a prescription for improvement. There may have been schools that adopted the same strategies but were unsuccessful, and no schools in the sample brought all their students up to state expectations. Nonetheless, the study suggests that principals can respond positively to assessment results by identifying performance deficiencies, involving the whole school in a focused improvement strategy, aligning resources to support the goals, and seeking outside help when necessary.
| 
