As I finalize the syllabus for my upcoming semester, I have decided to begin a habit that hopefully will address the shortcomings of existing indices of scholarly impact. Beginning this semester, I will contact the authors whose work I use in class and inform them of their impact on student learning. I will share my syllabus, class size, and other data that will help create something I call the “L-index.” The following is a short explanation of the L-index and a call to action for others to join me in expanding how we measure scholarly impact by including the impact of scholarship on student learning. I welcome feedback on this idea and ways to improve it.
Measuring scholarly impact
Scholarly merit is based on impact – impact on one’s field, students, institution, and the local, regional, national, or global community. Measuring impact has always been a challenge, but one with real impacts on hiring, promotion, salary, research funding, awards, and more.
In 2005, the physicist Jorge E. Hirsch proposed the h-index to assess the quantity and quality of a researcher’s scientific output (Hirsch 2005). Like the familiar journal impact factor (Garfield 2006), the h-index uses sophisticated statistical methods in an attempt to reduce bias in determining scholarly impact. However, such measures are inherently biased against scholarship in some fields, particularly the humanities, area studies, and social sciences, not only because of their focus on peer-review journals, but also because they fail to address the impact of scholarship on student learning.
To better understand this impact I propose the index L, in which L is for Learning. The L-index will allow scholars to demonstrate the impact of their scholarship beyond the statistics of journal impact factors and citations, by addressing how their scholarship is being used in teaching.
L-index, the non-index
In some ways, the L-index is a non-index. It does not involve an elegant formula, does not result in a single “objective” number, and cannot be derived from easily accessible data posted online. The L-index will result in a set of figures, but not in any standardized fashion. It will be up to the individual which figures to highlight. The follow are suggestions: Publication A is being assigned in W courses, for X students, in Y institutions, (in Z countries).
Because this data is not easily available online, the L-index requires scholars to share such details with others. The data in question can only be derived if scholars take the time to contact authors once a course syllabus is complete. Scholars will receive useful information about the impact of their scholarship on learning and hopefully, pay it forward by sharing similar information with others. This will result in meaningful data that can be used in important decisions about hiring and promotion, salary and bonuses, and awards.
Of course, qualitative data is also important in determining scholarly impact. This includes written assessments from peers both inside and outside one’s institution and student comments on course feedback forms, as well as comments from current and former students about significant learning memories, jobs landed, graduate schools entered, and more.
I am not suggesting we overlook this vital qualitative information, nor that we favor quantitative over qualitative data. I am advocating the inclusion of new quantitative data that recognizes scholarly impact otherwise overlooked. It seems futile to rely solely on quantitative measures developed by others, which are more suitable to other disciplines. If we must quantify our scholarly impact, we should introduce measures and collect data that take into account our unique scholarly outlets and contributions.
Whither the book chapter?
A colleague who recently prepared her tenure dossier inspired the L-index. While analyzing her publication and citation statistics, she sensed something was missing. According to the h-index and journal impact factor, a chapter in an edited volume published several years ago with few citations appeared to have little scholarly impact. However, after much digging through online course syllabi she found the chapter being taught in a number of institutions.
Subsequent conversations convinced me that scholarly impact tends to be calculated in a limited way, by not considering the impact of publications in coursework. In addition, I realized that existing methods for determining impact tend to ignore book chapters, a scholarly product more common in some disciplines and often more accessible for students than journal articles. It occurred to me that a chapter with only one or two citations might be read by hundreds or thousands of students in classes around world. However, there currently is no way to access this impact on student learning. The L-index offers one possible solution to this serious oversight.
I conclude by returning to Hirsch, who admitted that the idea of summarizing scholarly impact with a single number was “potentially distasteful” but necessary “in a world of limited resources.” I am not suggesting we throw out the h-index. Instead I suggest that there are other ways of thinking about and representing scholarly impact. In a world of increasing emphasis on quantitative measures it can only help to be armed with meaningful statistics that represent one’s impact on student learning.
Call to action
Developing an L-index requires action. As I pointed out above, currently no online service can glean this information. Until all course syllabi are posted online and routinely updated, this kind of “big data” will be impossible to create. An L-index requires “small data,” which only comes from one-to-one communication.
If you agree that the L-index is worth pursuing, please make it a habit to email the authors on your syllabus each semester. Attach your syllabus and share the course size and any other details that might give the author a sense of the impact of her/his book, book chapter, or article. Be willing to answer questions about how you use the work of others and its impact in the classroom. Let’s give each other the tools to explain to colleagues and administrators the impact of our scholarship beyond citation statistics, on learning.
References
Garfield, E. (4 January, 2006) “The History and Meaning of the Journal Impact Factor,” Journal of the American Medical Association (JAMA) 293 (1): 90-93.
Hirsch, J. E. (15 November 2005). “An index to quantify an individual’s scientific research output,” Proceedings of the National Academy of Sciences (PNAS) 102 (46): 16569–16572.