[Contents Page]     [Previous section]     [Next section]       Back to my homepage

6 --- Applying the model

Sections:

6.1   Suitability
6.2   Defining the citation
6.3   Answering or minimizing limitations from the literature review

6.1 Suitability

This project may use the Barnett model because

... the model presented here uses articles written in a given year [i.e., the 1996 theses] and examines the pattern of citation made by these articles [theses] to articles previously published. [90]

Actually, this current project is more appropriate to the model than Barnett's testing data was, because it is based on the works of only one given year, whereas Barnett's sources have many publication dates;[91] i.e., this project is comparatively independent of historical changes in citation practice.[92]   This project will be a citation study because that is expected to give a truer indication of the relative use of the various sources.[93]   Because the data will be recorded on a spreadsheet, we may also approximate a reference study (by substituting "1" if n > 1).[94]   Unlike a simple reference count, we have automatically eliminated the "over-stuffing" present in many of the theses - items listed in the bibliography which are never directly cited.

6.2 Defining the citation

Unfortunately, deciding on a citation study requires us to define what should be counted as a citation. E.g., how many citations are in the following mock-up?

Sandison (1989: 59, 61, 62-3) emphasizes the need to ensure statistics compare only like-to-like; i.e., "numbers of citations only become meaningfully comparable when expressed in relation to the same pool (Sandison 1989, 61)" --- especially when complicated by literature growth (p. 62) or the differing practices of various journals (p. 63). He continues:           the counts should be expressed in units which take           the citable pools into account (Sandison 1989, 61).

To be consistent, the following practices were observed.

• In any one sentence, multiple citations to the same source were counted as only one citation --- even if the references were to different topics or pages.
• unattributed quotations were not counted.
• undated references were not counted, unless it was clear that the item actually had no date.
• All quotations were counted as one citation, regardless of length.
• Copulative links (such as "They continue:" or "She, therefore, concludes:") between quotes were ignored, unless the two quotes were clearly on different topics and from different sections of the source.
• If an author-date citation did not match an item in the bibliography exactly, but almost did so,[95] it was assumed that the thesis writer had made a simple typographical error.
• If the error was not simple, or there were several near-misses, then the citation was counted separately and added to the tally sheet as a new item. Both simple [presumed] errors, and unrecognizable substitutions were appallingly frequent --- especially in the Education theses. [96]

6.3 Answering or minimizing limitations from the literature review

The vagueness of citation age[97]

Fortunately, this project will minimize that vagueness by nominally studying the products of only one year's academic research. It is possible that some students had received extensions on theses due, and perhaps researched, in 1995. Similarly the theses may incorporate recycled material from previous essays, projects, or research. Nonetheless, it is reasonable to assume that any older material has been renovated in its outdated parts.[98]   Furthermore, in comparison with doctoral or post-doctoral research, a normal master's thesis has a significantly smaller "time-window" of research which the author might incorporate into current projects. Combining both points, we assume that the 1996 theses generally represent valid samples of 1996-published research.

The failure to distinguish positive acclaim from negative criticism[99]

As indicated above (§2.7), this is arguably not a serious consideration. Furthermore, the aim of this particular project is not to quantify the didactic value of various resources, but rather how often they were "used" locally.

Self-citation[100]

As mentioned above (§2.8), self-citation is only a problem when the citation analysis is used for author evaluation. Besides, self-citation should be negligible or non-existent in a Master's thesis.

Sampling errors[101]

The concerns over sampling errors, is reasonable and relevant. For this project, we hope to reduce such errors by rejecting subjects with less than five theses, and by processing all theses extant in acceptable subjects. The sampling error is interconnected with "extrapolation of historical trends" assumption; we must not assume the thesis topics of 1996 will represent those of any later year --- but we do hope that reference materials of similar type and age will be employed, especially the secondary sources.

Evaluation of author influence or merit[102]

Such evaluations are irrelevant to a quantitative study of the what was actually used by the patrons --- as distinct from any judgment of the merits of what they used, or ought to have used. It is conceded that any popularly cited works not already held will be considered for future acquisition; and therefore a list of such items will reflect the citation biases of the successful M.A. candidates.

Footnotes for Chapter 6

90.  Barnett, Fink, and Debus, "Mathematical Model," 516.

91.  And obviously different ranges, at that, since the SSCI & AHCI included 4 years (1983-6) after the data from SCI. --- Barnett, Fink, and Debus, "Mathematical Model," 518.

92.  Cf. Barnett, Fink, and Debus, "Mathematical Model," 525. Similarly MacRoberts and MacRoberts, "Problems," 345:

The modern pattern of explicit citation ... continues to evolve, with the trend towards increasing numbers of references.

93.  As discussed in §2.3 above.

94.  Thus we may still spread items over several years (n= 1/3, etc). Of course if several items are in the same volume of a periodical those items will only have a total count of "1" - whereas, e.g., a reference analysis based on SSCI would count each separately. However cf. Cullars, "Characteristics ... British & American Literary Studies."

95.  E.g., "(Broadus 1989)" appears in the text but there is no such item in the bibliography, but there is a 1985 Broadus article.

96.  Two authors, in particular, were numerically illiterate. Although there are proportionally more unidentifiable citations for Psychology (5.22%) than Education (3.28%), the dates given for Education citations were noticeably more erratic.

97.  Sandison, "Thinking," 63; also ibid., 61 --- see n.37 above.

98.  Of course we may argue that the theses are more up-to-date than research published elsewhere which may be the result of several years' research. However such research may also renovate older sections, and we may hope the delay between an article's first submission and its actual publication is probably less than a year --- putting it on a par with the production schedule of our M.A. theses.

99.  E.g., Cave et al., The Use of Performance Indicators, 183; Sandison, "Thinking," 63; Reinsch and Reinsch, "Some Assessments," 29; etc. --- see also and n.57  above.

100.  E.g., Cave et al., The Use of Performance Indicators, 183; Sandison, "Thinking," 63; Reinsch and Reinsch, "Some Assessments," 29; etc.

101.  Reinsch, "Some Assessments," 29; etc. --- see §2.7 above.

102.  Reinsch, "Some Assessments," 29; etc. --- see §2.7 and §2.8 above.