[The Montana Professor 21.1, Fall 2010 <http://mtprof.msun.edu>]
Amanda H. Goodall
Princeton and Oxford: Princeton University Press, 2009
183 pp., $29.95 hc
George M. Dennison
President Emeritus and Prof. of History
UM-Missoula
First, a confession, I lusted for the opportunity to review this book once the title caught my eye. Name an academic who will miss the chance to refute the recent drivel urging recruitment from business or the military to secure the kind of leadership sorely needed in higher education. Never mind that colleges and universities rival most human institutions for longevity, having survived and evolved through centuries of political, social, cultural, institutional, and technological change despite periodic predictions of their imminent demise—the most recent by management guru Peter Drucker. Never mind as well that business and the military have captured attention during the late 20th and early 21st centuries for egregious errors and failures, while colleges and universities have attracted increasing public acclaim for the wage and quality of life premiums to individuals and their "spillover effects" on the economy and society at large, using Amanda Goodall's terminology (2). In brief, colleges and universities have higher standing than business and the military (83), so does leadership matter?
The dust jacket boasts comments from a number of deep thinkers encouraging people to read and consider the arguments in the book responding to that query. Bill Bowen said he "started reading...with some skepticism, since I doubted that a convincing empirical case could be made for its key proposition—that research universities should be led by top scholars. I was wrong," he concluded. Warren Bennis praised the book for presenting an "invaluable and convincing case for...knowledge-based organizations...[to] increase the probability of success by choosing...their top leadership...[from among]...acknowledged thought leaders and experts in their fields." Unfortunately, while I have a strong affinity for Goodall's general argument, I cannot concur with these assessments.
The sub-title of the book sets an ambitious goal, described subsequently in various ways. "This book attempts to reconnect leadership with the business of research universities—that of scholarship" (11). Stated somewhat differently, "The aim of this book is to provide evidence about the kind of leader under which a research university will thrive" (139). "This is a book about the leadership of experts," according to the author, whose "conclusion is that better scholars make better leaders" (xiii). Essentially, Goodall intends to refute the venerable if apocryphal myth that "academics do not make good managers or leaders," held and voiced by "scholars, administrators, and those outside universities, including politicians, civil servants, and business people." Only a moment's reflection casts doubt on the assertion, whatever the source, since the existing evidence does not identify any one group of people as "natural managers," but rather suggests a roughly even if random "distribution [of leadership ability] across all professions" (xiii). Throughout the book, she maintains a distinction between management and leadership, noting that organizations can and do contract for management services but must have effective leadership for success. Goodall confines her attention to academic leaders—presidents, vice chancellors, rectors, etc., and subordinates, i.e., provosts, pro-vice chancellors, deans, chairs, heads, program leaders, and the like—leaving "nonacademic administrators" who "perform a critical role in universities" to others (xiv).
If these statements aptly summarize Goodall's thesis, the review ends here. However, she aims for much more, nothing less than "a contribution to the study of leadership, and to our understanding of research universities" (xiv). Despite the claimed intent, Goodall offers little about leadership per se, or even how to define it. Instead, she asserts "that the world needs research universities, and, therefore, it matters who leads them" (2). She has no interest in other types of higher education institutions, and accepts as a given that "research universities" derive their utility and importance from their "positive external effect on economic growth," as evidenced by government investment in them. The "spillover effects" of "research universities...occur when the creativity or knowledge...spreads outward, resulting in [more] growth." As the source of "more inventions and ideas," research universities generate a high rate of return on investment (2-3). For these claims she relies on existing scholarship, presenting no new evidence, and ignores completely the broader societal role of public research universities that educate 70% of all students and confer 58% of all doctoral degrees in the United States./1/
Goodall focuses on proving "a fairly smooth relationship between the leader's level of scholarship and a university's quality. The greater the first, the greater the second" (3). While she considers the reverse of that proposition, she never analyzes it seriously. In various formulations, she repeats her thesis throughout the book, seemingly on the assumption that repetition leads to acceptance. Quite correctly, she notes that no other study of university leadership has either mentioned or attempted to establish the alleged empirical relationship between the scholarly standing of the leader and university performance (4). In Goodall's view, with research excellence as the "core business" of a research university, it follows that those responsible should appoint leaders experienced in that "core business" (7-8, 11, 81, 134, 139).
As the intellectual model for her thesis, Goodall argues that effective leadership in learning or knowledge-based organizations requires the putative leader to possess and exhibit the four critical attributes of "being credible, bearing the standard, having expert knowledge, and signaling priorities" (7-8, 85). In research universities, the prototypical knowledge-based organization, and other professional firms, standing rises as the leader demonstrates prior possession of these attributes (124-35). While she recognizes the differences and does not analyze the other organizations in detail, she emphasizes the importance of "expert" status to effective leadership in organizations of this type (135).
To test the thesis, Goodall analyzes the leadership patterns characteristic of the 100 universities around the world in the Jiao Tong University Institute of Education (Shanghai) league table (26 ff., appendix 3). To measure scholarly standing, she counted the lifetime citations in 2004 of the 100 university leaders, normalizing them with a "P-score" she established by dividing the number of individual citations by the disciplinary average number of citations (24-5, appendix 2). Not all experts accept citation counts as the proxy for scholarly standing, but the approach has its supporters. Goodall notes the differences of opinion, but argues that the proxy works for her purposes. All but two of the leaders came from the academy, and all (including the two outliers) had research doctoral degrees and respectable P-scores (32-45). Using regression analyses, she found a statistically significant correlation or association between the P-score of the leader and the Jiao Tong ranking of the research university (20, 34-7). Once again, she states the relationship with the P-score or scholarly status of the leader as the determinant. However, after disaggregation, she found that the correlation held only for the 51 US research universities, not for the 49 non-US universities as a group (20, 38-44). On the basis of this finding, Goodall argues that US universities select distinctly different leader types, specifically the scholar-leader, through perhaps equally different selection processes, than non-US universities, but she does not document the latter point (42, 139). Nonetheless, she appears to lose sight of this differentiation in further refinements of her argument.
Goodall concedes straightaway that a correlation may suggest a causal relationship but does not prove it exists (55). Moreover, she recognizes that the Jiao Tong league table has at best ill-defined and questionable validity concerning institutional research excellence. For a solid ranking scheme, she constructed yet another database, this one consisting of 55 UK research universities, only 11 of which made the Jiao Tong top 100 (57-8, appendix 5). For the new database, she selected the 55 UK research universities that scored highest in the UK Research Assessment Exercise (RAE) administered roughly every five years (1992, 1996, and 2001) as the basis for allocating the research funding provided by the UK government (57-9, 158). The RAE generates a score for each participating institution based on 1) the number of research units—essentially departments—submitted for review, and 2) the number of units receiving excellent ratings. Institutions submitting a minority of their departments and thus faculty for review automatically receive lower scores and fewer dollars (64-6, appendix 5). Goodall ranks the universities (158) according to the three RAE scores (essentially counting the number of excellent departments). As an aside, she noted that the UK Teaching Quality Assessment scores "correlate highly with RAE scores," but she relies only on the RAE (65). These highly ranked universities receive the lion's share of the research funding.
Having established a ranking of the 55 research universities based on research excellence, Goodall calculates P-scores for each of the 157 vice chancellors who led those institutions during the nine-year period, about 3 per university, consisting of 9 non-academics, 8 "career" vice chancellors (appeared twice in the pool), and 140 academic scholar-leaders (60-3). Then she regresses the RAE scores of the institutions in 1996 and 2001 against the independent variable of the P-scores of the leaders in 1992 and 1996, allowing time lags of 4, 5, and 9 years—i.e., 1992-1996, 1996-2001, and 1992-2001 (69-77). She concludes (69-70) that, in every case, the "university performance in the RAE" in 1996 and 2001 correlates directly with the "lifetime citations" normalized in a P-score for the leader (using the mean of 1992 and 1996 P-scores for the 9-year analysis).
Because she understands that the "citation levels of the vice chancellors will not, of course, fully explain leadership success and institutional performance," she conducts additional regression analyses using three other independent variables—organizational revenue and the age and discipline of the leader—that had differing effects on the statistical significance of the leader P-scores (67, 70-7). She does not explain why she chooses these variables, indicating that she simply wants "to control for other factors that might influence a university's performance," and that the sample size limits the number of variables (57). On an earlier occasion, she concedes that "scholarship might just be a proxy for management ability or leadership skills," but she makes no effort to isolate scholarship from the other two through analysis (55). Age and discipline had no statistical significance at all. One wonders why anyone thinks they would. However, organizational revenue had a significant effect on the rankings in 1996 and again in 2001, but no significant effect on the incremental changes over time in the number of highly ranked departments in 1996 and 2001 compared to 1992 (70-4). She concludes that organizational revenue functioned in the analyses as a proxy for institutional size, thus having statistical significance for the number of top ranked departments in each RAE because larger universities have more departments and more money to pay for the quality revealed in each separate RAE. However, the incremental number of improved departments over the time periods apparently did not depend on size or amount of money, proving as she says that "cash is not everything" (74). The conclusion does not follow from the premises, as explained later.
Nonetheless, Goodall uses this finding, along with the observation that "Universities that improved most in the RAE [between 1992 and 2001]—that were led by better scholars—were not among the most prestigious universities," to argue that improvement over time depended on the influence of scholar-leaders, not wealth, stature, or institutional pedigree (86)./2/ As further corroboration, she conducts one more regression using the number of top departments an institution had in 1992 as an additional independent variable and the change in the number of top departments between 1992 and 2001 as the dependent variable. This analysis produced no change in the statistical significance of the P-scores of the leaders or of the other variables of money, age, and discipline (77). But why expect any change in the statistical significance of the P-scores by using as an independent variable the RAE score in 1992, the beginning point for the dependent variable, i.e., the change in top departments over the period? A regression analysis omitting the P-scores of the leaders but using the institutional proportion of top departments in 1992 as the independent variable with the incremental change in the institutional proportion of top departments between 1992 and 2001 as the dependent variable offers a more interesting test. However, Goodall did not pursue that approach. She did, however, include still another analysis regressing the P-scores of the leaders in 2001 against the RAE scores of 1992, finding once again no statistical significance (77).
She concludes that these results 1) rule out the common assumption that older, wealthier, and more prestigious universities perform better precisely because of their wealth, stature, and power; and 2) go "some way to disproving" the contention that "top universities simply select distinguished researchers [as leaders] as a matter of course, or because they can" (74-7). The leaders in 2001, institutionally as well as individually, "are...not merely a reflection of yesterday's performance." Those institutions making the greatest progress over the entire period did so because of the influence of exceptional scholar-leaders and "are not doing so merely because they had the furthest [sic] scope to move" (77).
As mentioned, I find these claims questionable, or least deserving of more analysis. In the first instance, the major gainers in the incremental increase of top departments all fall into the second tier of established UK research universities—i.e., not among the oldest, wealthiest, and most prestigious institutions. It defies common sense to believe that their lower starting points—measured by the total number of departments, the number of top departments, and the proportion of total departments ranked as top departments—provides no advantage over other universities starting from higher levels similarly measured. Goodall does not discuss the proportion of top departments per institution in either 1992 or 2001, except in passing. In the second instance, the claim that the leaders of 2001 did not merely reflect prior institutional performance or prominence at once appears irrelevant or over the top, or requires much more than a single regression analysis to verify. At the very best, these claims appear unsubstantiated by the evidence or analyses provided.
Having presented the quantitative evidence for the relationships between P-scores and institutional research performance, Goodall discusses some qualitative evidence drawn from interviews with a relatively small sample of 26 US and UK research university leaders and deans (143, appendix 1)./3/ To a person, the leaders actually interviewed agreed with Goodall's model for scholar-leaders of research universities—know and understand the culture and core business, live the standard in order to enforce it, make concern for research quality evident in institutional priorities and decisions, and signal that commitment internally and externally—as do most academics (chapter 5). Even the nonacademic leaders she interviewed concurred, although they did not emphasize "credibility and intellectual values" in their responses as the academics did (88). She concludes that the qualitative evidence supports the quantitative associations and correlations, thus verifying her argument that "universities should be led by...strong researchers" (104-5).
To buttress that conclusion, she looked briefly at practices in other organizations dependent on expert knowledge. For example, she and her colleagues found that professional basketball coaches who had achieved star status as players positively influence their teams' win-loss records and post-season successes compared to coaches who played but did not become stars or who did not play professionally (124-7). Goodall suggests that the same pattern of the significance of expert leaders appears in symphonies, museums, and practice firms of professionals such as architects, accountants, lawyers, and consultants (127-35). Relying on comments made by people she interviewed and a few secondary sources, she hypothesizes that further research will establish leadership patterns similar to those she identified in research universities (135, 144).
As a final source of relevant evidence concerning leaders for research universities, Goodall reports on 1) the appointment patterns for the 157 UK research university leaders in her database, 2) a single case study of the appointment process, and 3) some comments drawn once again from the interviews (chapter 6). Her analysis of the patterns evident after at least two successive leader appointments in each of 47 of the 55 UK universities she studies—not all had two successive appointments because of "career" vice chancellors—revealed a dominant pattern which she labels "an alternating-leader cycle;" 36 of the universities switched "from a stronger scholar to a weaker scholar and then back again" (106). Other studies and some of the interview comments suggested to Goodall that this pattern manifests a typically human tendency either to fix something deemed wrong or to choose something different (106-12).
The case study depicts a "random" process of identifying the desired attributes of the new leader by surveying a cross section of the stakeholders (with an uneven distribution of responses) rather than by reference to the institutional mission and strategic plan (113-20, 123). Because of this finding, she described the appointment process itself as dependent on "providence instead of planning" (119, 123). In the end, although the chair, committee members, and senior administrators agreed that the consensus supported the appointment of an eminent scholar, as actually happened, the very competitive runner-up, favored for appointment if the first choice declined, had a "traditional academic background" but lacked scholarly distinction (118). Goodall also notes that leaders typically develop and implement the institutional strategic plan rather than simply mirror a pre-existing one (14, 122, 139).
The qualitative evidence drawn from the interviews corroborated and reinforced the appointment pattern found among the UK research universities and in the case study (120-2). One respondent stated that neither "the chair...[nor] the committee...had a strategy, but they had a view about what was wrong" (120). Another, an eminent scholar, said he "was appointed because their first choice turned them down—he was a civil servant" (120). That scholar himself gave way six years later to "another civil servant" (120). Familiar patterns reappeared.
Goodall considers the random search process and actual appointment pattern as causes for concern, likely to result in serious problems for the appointing institutions if strong scholars set directions and then give way to weak scholars who abruptly change course (111-2). However, her analysis of the institutional performance of the 55 UK research universities based on the RAE scores does not reveal such radical institutional changes despite the "alternating-leader cycle" she found. In fact, most of the universities progressed quite well, since the number of high scores rose dramatically over the nine-year period, especially between 1996 and 2001, so much so that Goodall had to impose an arbitrary cut score in order to keep the number of top institutions manageable and meaningful for her study (53-4). That felt requirement strikes this reader as a strong indication of a flawed methodology or process, especially in light of the presence of the "alternating-leader cycle." Goodall argues that strong scholar-leaders improve institutional performance, and yet she does not explain why weak scholar-leaders thrown up by the "alternating-leader cycle" do not affect institutional performance.
Other weaknesses in the analysis and arguments reinforce the sense that Goodall, even if unconsciously, begins with a thesis and then structures the methodology and analyses to support it. She notes at the outset that "graduate students...[are commonly] introduced to academe via a disciplinary passage, thus coming out to see the world through a specific theoretical and methodological lens." She "adopted a different approach," choosing "the methodological tools" best suited to answer "an interesting research question" (xiv). In this reader's experience and view, such an approach works only when properly bounded by methodological logic and rigor, not always evident in her study.
Because Goodall recognizes that the Jiao Tong league table lacks credibility as a ranking of institutional research excellence, she changes databases to the 55 UK research universities, after noting that the correlations between P-scores and institutional performance she found in the Jiao Tong 100 universities (and the 51 US universities within the Jiao Tong 100) do not hold for the 49 non-US universities, with only a "modest relationship" between the P-scores and the Jiao Tong ranking of the 11 UK universities in the league table (40, 42, and discussion above). In addition, the P-scores of the leaders of the 11 UK universities fell well below the P-scores of the leaders of the other 38 non-US universities (42). How, then, can the study of UK universities prove or verify the major argument that the correlations found in the analysis of the Jiao Tong 100 actually have a causal effect in that scholar-leaders improve institutional performance? Goodall never explains, but indicates that she intends to ascertain whether appointing scholar-leaders makes a difference over time (45).
In sum, Goodall relies on one database to establish a correlation between the P-scores of the leaders and institutional ranking, but then turns to a very different one to test a correlation already deemed non-existent or "modest" and untrustworthy for the institutions that populate the second database. By way of some explanation, she notes that the "evidence I have presented does not enable judgments to be made about the weight assigned...to the research records of presidential candidates as distinct from other criteria such as managerial expertise or entrepreneurship" (43). Interestingly, she immediately drops the challenge of weighting these possibly quite influential variables—specifically, "managerial expertise" and "leadership"—and turns instead to the kind of analysis she employed for the Jiao Tong study to establish correlations between P-scores and institutional performance defined by how well the institutions perform in a tightly constrained and limited evaluative system designed for purposes of allocating government research funds. As she says, "I go beyond cross-section correlations and look at whether having better scholars at the helm actually improves performance" (45). One wonders whether this approach, using a different database and adding only the temporal perspective, will rectify the weaknesses she indentified from its earlier application. At the very least, it seems essential to establish and focus upon the significantly influential variables.
In this reader's judgment, Goodall failed to differentiate the effects of the independent variable she chose for her study from another independent variable likely to affect the dependent variables. In this regard, Goodall faithfully notes that all 400 leaders included in the study came to their positions with prior administrative experience (13, 138). Further, she insists that she does not view or use scholarly attainment or status as a proxy for "managerial" competence (12-3, 24, 56, 80, 85, 93, 137-8). On several occasions, she speculates that very possibly scholars simply make good leaders, but also realizes that not every scholar will become a good leader (xii, 56, 80, 85, 93, 137-8). Leaders, she argues, must have and exhibit "technical expertise about the core business" of the organizations they lead to succeed (13, 80). And, as mentioned earlier, she views research as the core business of a research university. Therefore, it follows that a successful scholar-leader must have a good understanding of research as a process and profession. Moreover, she specifically mentioned that "Scholarship might just be a proxy for management ability or leadership skills" (55). Quite clearly, she differentiates between leadership qualities and traits—sometimes referred to as "management ability"—and managerial competence.
Nonetheless, Goodall never seeks to test the significance of this attribute she found common to all 400 of the scholar-leaders she studies. Nor does she develop regression analyses to isolate scholarly status from managerial competence or leadership. Instead, she focuses her attention almost exclusively on the level of scholarly attainment of the leaders, even though finding that only 12 of the 100 leaders associated with the Jiao Tong top 100 universities made the list of the top "250 academic researchers" in the world based on number of citations (34). Equally important, she specifically notes that
When looking at the individuals who run these great universities, it is possible to find both a handful of heavily cited scholars and a handful of leaders with few or no research citations. It might be thought from this fact that there is no systematic link between research output and university leadership. Yet, as I will show, there is a pattern. A significant correlation exists between the research background of a leader and the position of their [sic] university in a world league table. (24)
However, rather than analyzing the attribute the leaders all have in common, Goodall uses great diligence and care to establish a P-score for all of the leaders, differentiating and ranking them into at least quintiles based on P-scores from strong to weak scholars (appendix 2, 147-52). Having done so, she demonstrates the correlations between the scholarly status of the leaders as established by the P-scores and institutional rankings based on performance. Unquestionably, the correlations exist; she does not manufacture them. But what do they signify?
Since all 400 leaders had prior administrative experience, it seems eminently logical to regress institutional ranking or performance against managerial competence. Of course, doing so requires establishing a proxy for managerial competence through a process different in detail but similar in function to Goodall's use of normalized citations to measure scholarly distinction. Doing so goes well beyond the bounds of a review of Goodall's work, but it certainly seems achievable by focusing on years and levels of successful experience and the extent and scope of prior training—as typically done more or less systematically by search committees seeking new leaders.
This suggested approach to a "study of leadership" in research universities has considerable promise in view of the attribute shared by all leaders, even if it offends academic sensibility. Moreover, following the proposed approach does not mean that all leaders will or must come from a specific discipline, rather that all leaders must have successful specific experience to improve their chances for further success. Just how much the specific type of prior experience and training will make a significant difference gives purpose to the study, and also allows for an identification of the statistical significance of scholarly standing versus "technical expertise" or managerial competence and leadership, using Goodall's terminology (13, 55, 80).
In this reader's view, much of Goodall's qualitative evidence points to this kind of approach as well. According to an administrator member of the search committee in the case study, "We were influenced by the quality of leaders and their experience. The main information came from academic members—strong research focus and credibility. Balance between academic and leadership" (118, emphasis supplied). A successful UK vice chancellor characterized the search process leading to his appointment as follows: "The selection committee were [sic] very keen to have certain things—someone with a public profile, engagement in public policy, knowing how to manage a complex organization. Beyond that it felt quite unclear" (120, emphasis supplied). Another said, "When it comes to selecting vice chancellors, I think lay members judge by character not by using objective evidence" (122). Still another explained his selection "because of a strategy he presented in interview" (18). Virtually all agreed that acceptance by the faculty made the critical difference. "Universities should be run by academics"; or "it is healthy for the sector that most university leaders have their roots in academe"; again, "faculty...would look askance if their presidents were not good academics with a research track record" (82-3).
These comments merely state and restate the obvious: a leader of academics must have what academics have to gain acceptance. They tell us little about the chances of any individual academic to become a successful leader, or about as much as Goodall's correlations between P-scores and institutional rankings. Equally pertinent for this review, observing that the overwhelming majority of the 400 leaders had academic backgrounds and respectable research records also states the obvious. That the leaders of research universities had increasingly distinguished prior research records corresponding to the ascending rank or performance of the institutions seems interesting as a correlation of individual status with institutional status, but does not suffice to explain institutional ranking or performance, as Goodall admits. Since most leaders selected by ranked institutions come from institutions similar to the ones they lead, and since progression through the academic ranks depends upon satisfying criteria relating fundamentally to research/creative and teaching performance, and since junior faculty rarely if ever receive serious consideration in searches for leaders, it follows that the leaders ultimately selected will have successful records of research and teaching; and that their institutional origins as faculty members will influence their levels of distinction. Goodall appears to concede these points in passing, and yet ignores their obvious significance (56-7 on criteria). She specifically notes that university leaders in the UK rarely if ever cross the boundary separating established from aspiring research universities (86).
However, to explain whether the research distinction or the level of "technical expertise" and leadership or some combination of both, using Goodall's terms again, influence institutional ranking or performance requires careful analysis to determine which, if either, has statistical significance, taking care to isolate one from the other insofar as possible. In her study, Goodall employed a methodology allowing these two variables to work their effects in tandem, with one masking the presence and effect of the other. In this reader's opinion, the successful prior administrative experience and training of the university leaders in all likelihood has a more demonstrable and consistent effect on institutional performance than prior research distinction per se of the leaders. To echo Goodall, "Being an expert, or a top researcher, is a necessary but not sufficient condition for being a good leader" (13).
In the final analysis, Goodall's arguments, analyses, and conclusions rest solidly on ideological and preferential grounds. Most academics share the preference for scholar-leaders, at least on an alternating cycle. By indulging her preference, Goodall misses an opportunity to identify precisely why some scholar-leaders succeed, some fail, and some never seek to become leaders; and to contribute significantly to the "study of leadership," as she hoped to do (xiv). Of course, that may ask too much of a diligent observer content to describe but not to explain. Perhaps the reasons she offers for the correlations she carefully identifies—i.e., the ease with which scholar-leaders satisfy the four descriptive criteria of her ideal model of the expert leader—have some explanatory power, but not as presented and without due consideration of prior successful administrative experience and training, the missing variable.
Notes
[The Montana Professor 21.1, Fall 2010 <http://mtprof.msun.edu>]