| |
Research Roundup 14, 2 (Winter 1997-98)
Measuring Leadership
Larry Lashway
After yet another day of crisis management or battles with the
bureaucracy, many principals must wonder about the whole concept of
leadership. What they do to get through each day seems far
removed from the visions of dynamic, inspirational leadership they
once nurtured. In their darkest hours, they may wonder whether they
really are leaders.
Principals have two reasons to be troubled by these doubts. First,
they are responsible for identifying leadership in others, especially
when interviewing candidates for administrative or faculty positions.
Given today's quickly changing school environment, districts must
identify candidates who can act with initiative, raise questions, and
take intelligent risks.
Second, as leaders themselves, principals must be analytical and
reflective about their own capabilities. Are they leading their
schools effectively? What skills need improvement? How should they
direct their personal and professional growth?
Because of such questions, there is always a strong market for
instruments that promise valid and reliable measurement of leadership
qualities. Leadership tests are abundant, but do they deliver on
their claims? The fairest answer to that question is "Yes, but...."
Measurements of leadership can provide valuable information not
available from informal interviews, observations, or intuition.
However, users must select assessment procedures wisely, recognizing
that no single approach is perfect and that their choices must be
tailored to the needs of the school.
The literature on these issues is surprisingly limited, but the
works discussed in this issue provide a variety of helpful
perspectives.
Frederick C. Wendel and colleagues review some basic
concepts of measurement and apply them to leadership testing.
Richard L. Hughes and colleagues discuss some important
techniques for measuring leadership qualities and the effects of
leadership.
Kenneth Leithwood and colleagues examine the ways that
schools measure leadership and provide recommendations for
improvement.
Richard P. Santeusanio describes the 360-degree feedback
system used in his school district and provides guidelines for using
multirater feedback.
Frank Freeman and colleagues provide capsule descriptions
of some commonly used leadership tests, and the ERIC Clearinghouse
for Assessment and Evaluation offers an online database of leadership
tests.
Wendel, Frederick C.; Schmidt, Allan H.; and Loch, James.
Measurements of Personality and Leadership: Some
Relationships. Lincoln, Neb.,1992. 121 pages. ED350 694.
Available from: ERIC Document Reproduction Service, 7420
Fullerton Rd., Suite 110, Springfield, VA 22153-2852. 800-443-3742.
$20.40 plus postage.
Measuring leadership is an attractive concept that seems to
promise precise, scientifically valid "proof" that a person has (or
does not have) appropriate qualities. Unfortunately, this promise can
be misleading, since all measurement instruments have important
limitations. In this monograph, Frederick Wendel and colleagues
present a comprehensive but highly readable review of measurement
concepts.
The authors begin by cautioning that abstract concepts such as
personality and leadership are always more than the sum of precisely
measured parts. We can, for example, accurately measure
characteristics such as weight, height, and hair color, but the
concept of beauty still remains in the eye of the beholder,
A crucial point to understand is that every leadership test is
based on some definition of leadership, and current tests are based
on a wide range of definitions. Some focus on personal traits, such
as drive or interpersonal skill. Some zero in on specific behaviors
demonstrated by effective leaders. Others call attention to the way
that leaders think or use power, and still others assume that
leadership varies with situations.
The practical implication is that principals must have a clear
understanding of how leadership is defined in their institution. For
example, if a school has adopted site-based decision making, an
appropriate test will provide feedback on how a leader facilitates
the work of others.
The authors provide capsule descriptions of 16 commonly used
leadership tests, including information on their validity and
reliability. Even more helpful is a discussion of assessment centers,
in which participants perform simulations of actual leadership tasks,
thus producing "samples" rather than "signs" of leadership. (A
paper-and-pencil test may reveal an individual's knowledge of
leadership principles but provide no information on his or her
ability to carry out leadership tasks in a real-world setting.)
Wendel and colleagues cite studies showing that 90 percent of
those scoring at the highest level in the NASSP assessment centers
receive similar high ratings in later job performance. Although
assessment centers are considerably more expensive than
paper-and-pencil tests, they may be cost-effective, especially for
entry-level candidates with no track record.
The authors concede that measuring leadership is still an inexact
science requiring careful reflection and caution, yet the right test,
"one that fits the purpose of the examiner, if administered properly,
if scored with exactitude, and if interpreted correctly, will yield
good results." Conversely, those who rely on intuition or
unsystematic procedures are taking major risks.
Hughes, Richard L; Ginnett, Robert C.; and Curphy, Gordon J.
"Assessing Leadership and Measuring Its Effects." In
Leadership: Enhancing the Lessons of Experience. Homewood,
Ill.: Irwin, 1993. 543 pages. ED363 927. Available from: Richard D.
Irwin, Inc., 1333 Burridge Parkway, Burridge, IL 60521. 630-
789-4000. $42.95.
In this chapter of a leadership textbook, the authors summarize
the various ways that leadership qualities can be assessed and also
discuss ways of measuring leadership effects.
The critical-incidents technique asks subordinates, peers,
or superiors to describe an incident in which the leader did a
particularly good or bad job. This method typically generates
context-rich descriptions of the situation, the leader's behaviors,
and the impact of the leader's actions on others. However, this
technique requires considerable time, effort, and expense. In
addition, incidents may differ dramatically, making it difficult to
compare leaders by the same standard. The critical-incidents approach
may be better suited for individual professional development rather
than for selecting leaders.
The interview is a more familiar tool for assessing
leadership skill. Structured interviews, in which leaders are asked a
predetermined set of questions, are generally preferable to informal,
unstructured interviews, since a common set of questions makes
comparisons much easier.
Observation is another common technique that can be
structured (with a predetermined set of categories) or unstructured
(with observers simply recording anything that happens). Observations
can yield excellent information about how leaders actually behave,
but systematically observing and recording on-the-job behavior is
both difficult and time-consuming.
Two other types of assessment are the familiar
paper-and-pencil measures (such as surveys and questionnaires)
and the assessment center, which challenges examinees with a
series of interviews and realistic leadership tasks, using trained
assessors to rate their performance.
While these methods will generate information on the qualities and
skills that leaders possess, it is also possible to assess the
effects of leadership. Here, the key issue is identifying the
criteria by which performance is to be judged; that is, what will we
accept as evidence that the leader is successful? One common answer
is that successful leaders are those whose unit performs well. (An
obvious example, in regard to a school leader, would be test scores).
The problem with such criteria is that they may be affected by
factors other than the leader's skills. For example, test scores may
be influenced far more by student background than by the principal's
leadership skills. The authors recommend the use of multiple
criteria, with emphasis on those most responsive to leader actions.
Leithwood, Kenneth; Begley, Paul T.; and Cousins, J. Bradley.
"Performance Appraisal and Selection of School Leaders: Selection
Processes and Measurement Issues." In Developing Expert
Leadership for Future Schools. London: Falmer Press, 1994. 331
pages. ED394 172. Available from: Falmer Press, Taylor &
Francis, Inc., 1900 Frost Road, Suite 101, Bristol, PA 19007.
215785-5800. $27 paperback.
This selection reviews the ways that schools measure leadership
and provides guidelines for improving selection, appraisal, and
professional development.
Leithwood and his colleagues note that the literature on leader
selection is surprisingly thin, considering the consequences of
making a poor choice. Many selection processes are unsystematic,
relying more on subjective perceptions than hard evidence. The
interview appears to be the most commonly used tool, with
standardized instruments infrequently used. The predictive power of
such methods is questionable.
However, after examining 14 formal instruments designed to measure
school leadership, the authors identified a number of potential
problems. First, some of the instruments based their questions on
observation of typical rather than expert leaders. The
fact that many leaders behave in a certain way does not make that
behavior desirable.
Second, the quantitative dimensions of tests may obscure the many
opportunities for subjectivity and human judgment in their
development and administration. No test is sufficiently valid to be
the sole measure for important decisions.
Third, even when tests are reliable, testers may not be. Tests can
be used inappropriately, administered carelessly, or the results
simply ignored.
Leithwood and colleagues present several guidelines to improve the
use of leadership assessment in schools:
- The first priority is careful preparation. Schools should
specify their testing policies and procedures, stating clearly why
they are testing and what they hope to learn. While results may
sometimes be used to help make personnel decisions, the major goal
of measuring leadership should be professional growth and
improvement.
- Those responsible for selecting or evaluating leaders should
be well trained in assessment procedures.
- Schools can improve credibility of their assessment process by
using multiple data sources rather than any single technique.
Formal instruments may be most useful as an early screening tool.
- Leadership assessment should not be left to single
individuals. Panels involving people with different roles will
ensure fairness as well as the perception of fairness. Leaders
should be involved with their assessment, and should receive
feedback that allows them to develop plans for future growth.
The authors acknowledge that their recommendations are ambitious,
and may best be viewed as worthwhile directions for the future rather
than as reforms to be achieved overnight. But they argue that the
stakes are too high to ignore these issues.
Santeusanio, Richard P. "Using Multi-Raters in Superintendent
Evaluation." The School Administrator 54: 3 (March 1997):
12. Available from: American Association of School
Administrators Distribution Center, P.O. Box 411, Annapolis Jct., MD
20701-0411. 888-782-2272.
One of the newer approaches to leadership assessment is
"360-degree feedback," which relies on ratings from multiple sources.
Typically, a leader's performance and characteristics will be
assessed by subordinates, peers, and superiors.
Superintendent Richard Santeusanio succinctly describes the
five-step process used in his Danvers, Massachusetts, school district
for administrators and teachers. The first step is to identify the
competencies related to the job and format them into a survey. The
second step is to select the evaluation team. For Santeusanio's own
assessment, the team consisted of 21 people, including board members,
principals, curriculum specialists, teachers, and parents. Teams with
somewhat different makeup might be used to assess other
administrators.
The third and fourth steps are to conduct and score the survey.
The final&emdash;and most important&emdash;step is to develop an
action plan based on the strengths and weaknesses identified by the
survey. Santeusanio reports that he and others in the district have
found this collective feedback to be more valid and meaningful than
other methods. It has led not only to a clearer understanding of
performance standards, but has fostered change for professional
improvement.
He notes that 360-degree feedback solves some problems common with
single-source evaluations, including lack of fairness, accuracy,
credibility, and usefulness to the person being evaluated. His
district has found it to be both balanced and time-efficient.
Freeman, Frank H.; Knott, Katherine B.; and Schwartz, Mary K.
Leadership Education: A Source Book, 1996-1997. Sixth Edition,
Volume 2. Greensboro, N.C.: Center for Creative Leadership, 1996. 352
pages. Available from: Center for Creative Leadership, One
Leadership Place, P.O. Box 26300, Greensboro, NC 27438- 6300.
910288-7210. $35.00 plus $4.00 shipping and handling. [Also
available: Volume 1, 457 pages, $35.00; both volumes, $60.00.]
Finding an appropriate leadership instrument is a potentially
time-consuming task, traditionally requiring a hit-or-miss search
through many sources. In recent years, however, a number of groups
and publications have begun making this information more easily
accessible.
The Center for Creative Leadership's Source Book includes a
section with brief descriptions of 73 commonly used leadership tests,
including target audiences, administration procedures, and comments
on their validity and reliability. Vendor addresses are provided.
Another source is the test-locator website maintained by the ERIC
Clearinghouse for Assessment and Evaluation (http://ericae.net). The
database provides quick access to information on more than 10,000
instruments of various types. For each test there is a brief
descriptive abstract and the publisher's address. In addition, the
site can point the searcher to published reviews of some tests, and
it also has a section providing guidelines for choosing instruments.
Later this year, the ERIC Clearinghouse on Educational Management
will publish a book on measuring leadership that will be aimed at
meeting the specific needs of school leaders.
Larry Lashway is a research analyst and writer for the ERIC
Clearinghouse on Educational Management at the University of
Oregon.
|
