[Contents Page] [Previous section] [Next section] Back to my homepage
| Previous page [2a]: | 2.1 Use |
| 2.2 Citation frequency | |
| 2.3 Citation analysis | |
| 2.4 Local applications | |
| 2.5 Other evaluation methods | |
| This page [2b]: | 2.6 The age of citations |
| 2.7 Other limitations of citation counts | |
| 2.8 Irrelevant applications |
Let us consider the age of a citation as an approximate parameter.[37] It has long been established that tends towards a time-dependent decaying exponential function of that "age" parameter.[38] Of course, this exponential curve is a maximum, or envelope, which the actual distribution of citations does not reach immediately, because authors cannot cite sources unknown to them; but, as time increases, the "real" citation age-curve should approximate the envelope. That is, the "disadoption" of reference sources in any field has a "half-life," [39] and the curve is expected to be like:[40]
One should not cancel a serial subscription solely because of its low citation rate,[41] nor does a failure to be cited indicate a lack of virtue.[42] Indeed Seglen, analyzing science citations, agues that the progress of knowledge relies on the uncited, as well as cited, articles.[43] Moreover,
The rate of scientific progress can furthermore be stimulated by an increase in the overall volume of science, but not by sanitation [i.e., eliminating the least-cited papers or authors].[44]
His arguments have some validity for all fields of knowledge --- especially broad subjects, those with high interdisciplinarity, or where results are published in a wide range of resources. Furthermore, there may be intrinsic, statistical reasons why some works and authors are more cited than others:
There would thus seem to be a fundamental, irreducible variability in [science] article citedness, which is independent even of the publishing author. ... The distributions of author article citedness, author productivity, and athletic top performance are strikingly similar to the distribution of physical tallness, suggesting that the extreme-property distribution and the topnormal distribution may be closely related.[45]
This is because the non-producers, the zero frequency, will form the overwhelming proportion of the entire population.[46]
For an academic library, some journals and articles are well used but less cited, e.g., current awareness periodicals;[47] applied or clinical journals (when compared with research journals);[48] etc. Obviously citation counts disregard primarily educational materials; nor do they necessarily reflect "the casual, day-to-day use of library materials," [49] even by genuine researchers.
There are several criticisms of citation counting that are inapplicable (see §2.8 and §6.2 below). However, one timely warning is the possible invalidity of predicting future demands through the extrapolation of historical citation trends.[50] Of course this applies to all "use" studies; but the problem can be exacerbated by studying citations made over too long a period. Furthermore, a citation count measures the products of scholarly activity conducted during a recent (?), but ill-defined, period.
Reinsch and Reinsch also list several limitations of citation counts:
Citation analysis, like every research method, has limitations. Raw citation data do not distinguish between a favorable citing and an unfavorable citing, can be distorted by a prolific writer's self-citations, are subject to sampling error when examining small samples (e.g., citations of a single work), and may slight women or minorities. But, when used appropriately, citation analysis data can contribute to an assessment of the status of and relationship between articles, journals, and academic fields.[51]
Actually, excepting the negative citation of notorious or sensationalistic items,[52] the "failure" to distinguish between favourable & unfavourable citations is, itself, debatable. The process of scholarly deliberation requires a familiarity with both the pro and the con. Indeed, the favoured and the unfavoured are often cited side-by-side: "Negational references were often found to be used with a countervailing positive credit, currency or social consensus reference."[53] Besides, truly frank criticism is disproportionately rare --- e.g.,
The end result is that there are few negational references in the scientific literature. What criticism remains is disguised, reflected away from important people, or referenced perfunctorily.[54]
Self-citation, like most of the objections to citation analysis, only becomes a problem if the citation analysis is used to quantify or evaluate the productivity, impact, or quality of the cited authors.[55] But that is confusing the values of the tool with the intention of its wielder.
Similarly, citation analyses can reveal the apparent, relative influence of individual authors or institutions;[56] therefore they are sometimes [mis]used to evaluated the performance of researchers.[57] Again, this dubious practice does not invalidate the profitable advantages of citation studies. And although Reinsch and Reinsch's point that "Raw citation data ... may slight women or minorities"[58] partially oversimplifies Ferber's deductions,[59] her final conclusion is indeed that citations "should not be regarded as unbiased indicators of merit."[60]
Sometimes citation counts consider the inter-relationship between authors [co-citation analysis]. Some counts consider the interdisciplinarity of the cited sources,[61] or whether there is a network of specific topics within particular fields.[62] However both approaches are too peripheral to the current project's focus.
37. In order to reconcile the valid concern of Alexander Sandison, "Thinking about Citation Analysis," Journal of Documentation 45 (Mar. 1989), 61, for example. Namely:
Much citation analysis has been concerned with the ages of cited papers. The age that might influence citability is not the age when published but the age when the citing author was writing: this can be very different even for authors publishing in the same issue of a journal, but its date of publication may be the only date known. It is much safer to refer to the 'date of publication' of any cited paper than to its age.
38. George A. Barnett, Edward L. Fink, and Mary Beth Debus, "A Mathematical Model of Academic Citation Age." Communication Research 16, no. 4 (Aug. 1989), 513, quoting various sources. Also: L. Egghe and I. K. Ravichandra Rao, "Citation Age Data and the Obsolescence Function: Fits and Explanations," Information Processing & Management 28, no. 2 (1992), 201-2.
39. By knowing the half-life (or a similar rate constant), we may determine the ideal minimum age for items to be relegated to storage.
40. I introduce this diagram here because Fig.1 of Egghe & Ravichandra Rao "Citation Age Data," 202, is highly misleading. It more resembles y = (c.x)^(-1) , rather than y = (k.e)^(-x) , since their graph suggests that the curve does not intercept the y-axis.
41. Broadus, "Proposed Method," 33-4.
42. Sandison, "Thinking," 64.
43. Seglen, "Skewness," 634-5.
44. Seglen, "Skewness," 636.
45. Seglen, "Skewness," 632-5.
46. Quentin L. Burrell, "On the Growth of Bibliographies with Time: An Exercise in Bibliometric Prediction," Journal of Documentation 45 (Dec. 1989), 304.
47. Dombrowski, "Journal Evaluation," 176.
48. Dombrowski, "Journal Evaluation," 176. Some comments of Smith, "The Dangers of Citation Counting," 220 are applicable here. See also Peritz, "On the Objectives," 1992, p. 450:
In a recent analysis of clinical trials (Peritz, 1990), it was found that trials that were well-designed, i.e., randomized, carried out by a double-blind procedure, and with a relatively large number of subjects, tended to be less cited than those that lack these characteristics. The tentative interpretation offered was that a study suggesting new ideas and approaches, even if it is only preliminary, is more likely to be cited than a definitive treatment of the question.
49. Griscom, "Periodical Use," 38.
50. See Swigger and Wilkes, "Use of Citation Data," 44:
... citation data are historical. Past patterns of behavior are not guarantors of future behavior; faculty research interests may change, so past citations of a journal may not predict future use.
51. N. Lamar Reinsch, Jr. and Janet W. Reinsch, "Some Assessments of Business Communication Scholarship from Social Science Citations," Journal of Business & Technical Communication 10, no. 1 (January 1996), 29.
52. E.g., Cave et al., The Use of Performance Indicators, 183:
Jensen's controversial paper on racial superiority was one of the most highly cited social science papers. Many of the citations, however, were critical of, or outraged by, the paper primarily because of its political and ethical characteristics.
Similarly, Sandison, "Thinking," 64:
... erroneous and incompetent work attracts citations of contradiction ... --- such citations indicate the opposite of value.
53. Liu, "Progress," 394.
54. Michael H. MacRoberts and Barbara R. MacRoberts, "The Negational Reference: or the Art of Dissembling," Social Studies of Science 14 (1984), 93.
55. The problem is summarized by Cave et al., The Use of Performance Indicators, 183:
One particular potential source of measurement error is self-citation. ... A citation by a peer indicates recognition of quality. Can self-citation be regarded in this light? In general one might say that self-citations should not be included since they do not, in principle, follow the process of peer review. However, the exclusion of self-citations may introduce a potentially serious bias into the construction of an index for scientists who are researching new or innovative ideas, since there may be little or no work, other than their own, to be cited.
56. Bracken and Tucker. "Characteristics," 669; John M. Budd, and Charles A. Seavey, "Characteristics of Journal Authorship by Academic Librarians," College & Research Libraries 51 (Sep. 1990), tables 1 & 3; Budd, "Literature of Academic Libraries," table 6; etc.
57. Bensman, "Journal Collection Management," 20ff. Similarly Cave et al., The Use of Performance Indicators, gives the arguments for (pp. 178-82) and against (pp. 182-7) citation counting as a performance indicator. See also Adrian Smith, "The Dangers of Citation Counting," Library Association Record 90 (Apr. 15, 1988), 220.
58. Reinsch and Reinsch, "Some Assessments," 29.
59. E.g., see Marianne A. Ferber, "Citations: Are They an Objective Measure of Scholarly Merit," Signs: Journal of Women in Culture & Society 11, no. 2 (1986), 388-9, for her hypothesis and conclusion
... that researchers tend to cite a larger proportion of authors of their own sex than they do of the opposite sex. ...
Whatever the reason for the imbalance in citations, it unquestionably has substantial consequences in a field where men constitute a large majority. The issue here is not that men or women -or both- consciously discriminate against the opposite sex but rather that any affinity between authors of the same sex works to the disadvantage of those in the minority. Citations, then should not be regarded as unbiased indicators of merit.
60. See the above quote: ibid. 389.
61. Buchanan and Herubel, "Profiling PhD Dissertation Bibliographies," 6ff; Budd, "Literature of Academic Libraries," 295; etc.
62. E.g., H.G. Small and D. Crane, "Specialties and Disciplines in Science and Social Science: An Examination of Their Structure Using Citation Indexes," Scientometrics 1, no. 5-6 (1979), 445-461.