EvaluationEvaluationThe first question we need to address is: what determines whether or not profiling is beneficial to violent crime investigations? Clinicians, law enforcement personnel, and academics each have their own perspectives on the issue, but by coming to agreement on what constitutes "success" or "failure" of the technique, perhaps research can be guided in ways that will bridge the various disciplines.
Clearly one measure of profiling's utility is its reliability. if each expert profiler came up with altogether different profiles of the same crime scene, we would have to question the assumptions and process of this technique. The FBI attempted to address this in a 1985 study of the interrater reliability amongst the Behavioral Sciene Unit (BSU) special agents trained in the technique.
Six BSU agents with varying levels of profiling experience were selected to participate. Sixty-four (64) murder scenes, of either a sexual or nonsexual nature, were presented to the sample. These cases also varied according to the structure of the crime scene (i.e. organized/disorganized). For each, one agent thoroughly familiar with the case also provided a short description of the crime scene. Questions were allowed after each presentation to clear up any misunderstandings resulting from the presentation. Subsequently, each agent was asked to classify both the type and structure of each crime scene. The presenting agent, who had more information available to him or her, also classified each scene to act as a baseline for comparison.
The profile is based on a totality of crime scene information including victimology, forensic pathology, environment, etc. This study only gave immediate information: no laboratory tests or periperal data were divulged. The FBI felt that if these were discussed by the presenter, the Agents might be influenced by him or her in reaching their opinions. This was rationalized by stating that if they could demonstrate good agreement between the presenter and other agents with only minimal information given, then it could be assumed that more details would only enhance these findings.
For classification of crime type, the agreement rate of each agent compared to any other Agent ranged from 77% to 100%. Eighty-one percent of the total classifications made agreed with the presenter. Agreement on crime scene structure was somewhat lower, with 74% of the total classifications matching the presenter. Agreement between any two agents ranged from 45 to 89%. The authors felt that the substantial variance between Agents in agreement with the presenter was due to the differing experience levels in classifying crime scenes. Thus, they also looked at the rates of the three Agents most involved in the process; with these Agents, the range was from 62% to 80%.
Given the study design, and extremely small sample size, this study does not answer conclusively the question of whether or not there is a strong interrater reliability amongst profilers. What it does demonstrate, however, is that utility might vary greatly according to experience, suggesting that profiling is not a skill anyone can practice without a great deal of training and practical experience.
Some preliminary studies have been conducted on the validity of profiling and its central assumptions, but few of the findings have ever been replicated in any systematic way. Recognizing that they are only preliminary findings, the initial results have been positive.
Those researchers that find fault in much of the theory discussed above can use these studies as either a base to prove replication is not possible, or to spark their own unique methodologies. It is the opinion of this author that given its grounding in sound, testable, psychological theory, profiling will withstand such empirical measures, even as it has to be modified in the process.
In 1981, the FBI conducted a cost-benefit study of profiling's utility. (cited in Pinizzotto, 1984). Based on a total of 192 profiled cases, 46% were solved. However, of these only 17% of the suspects were identified directly by the profile. According to the standard of apprehension this is not a significant rate. Before we dismiss profiling on the basis of these figures though, it was also foundd that investigations were properly focused by profiling in 77% of the cases. additionally, possible suspects were located in 20%, and successful prosecution was aided in 6% of the profiled crimes. Only in 17% of the solved cases was profiling considered to be of little or no assistance (as rated by detectives).
Unlike the difficulties faced in risk assessment research (Monahan, 1994), whereby sound empirical designs are not feasible because the researcher cannot control who is or is not released, profiling theory can be studied in several reliable ways. The first is to study the decision "process" of profiling and see whether or not profilers offer qualitatively different perspectives than regular detectives, and the degree to which they affect the actual outcome.
A study by Pinnizzotto and Finkel (1990) provides initial data on the utility of profiling in such a manner. Using a closed case method (known offender), the authors compared profilers, detectives, and college students on the quality of the profiles written about particular cases, a test of 15 questions about the identity of suspects (age, sex, employment, etc.), and a task to rank order suspects for each case in a line-up from most likely to least likely to have committed each of the crimes.
Although profilers in this study did not differ significantly from detectives in terms of how they processed the evidence, they were more accurate in answering specific questions about sex-related offenses. There was no difference on the regular homicide cases, but this could be due to the lack of psychopathology inherent in the presented crime scenes. Profilers were also better able to pick the sex related offenders out of a line-up than the other two groups.
Another method of empirical testing is to study correlations between offender and offense characteristics. Such an approach has been taken by several researchers, each providing preliminary support for the central premises of signature and organized/disorganized typology that profiling relies on (Ressler et al, 1988;Canter and Heritage, 1990; Gratzer and Bradford, 1995; Dietz, Hazelwood, and Warren, 1990).
The field of investigative psychology, pioneered at the University of Liverpool, offers promise fir future advances in this area. Canter and Heritage (1990) attempt to establish the empirical validity of the underlying assumptions inherent in profiling. They make the distinction that, unlike clinical research on offender classification, profiling must rely on the classification of offense behavior. They note that "it is the variety of offense behavior that happen in sexual attacks that indicate the different modes of relations offenders have with their victims" (Canter and Heritage, pg.188). Canter and Heritage claim that the empirical basis of profiling requires the identification of these mean variations in offender actions in relation to a given offense.
Using the statistical technique of Smallest Space Analysis, a form of multi-dimensional scaling in which the more highly correlated two variables are the closer they will occur on a "three-dimensional" map of the results, Canter and Heritage (1990) examined 66 rapes in which the victim and offender were unknown to each other. At the core of the map were basic sexual actions common to most sexual offenses; however, other actions did reflect the more distinct motivations of particular assaults. The researchers maintain that this can show that "the differences in an offender’s repertoire of offense behavior can therefore be clearly established" (Canter and Heritage, pg.204). This study provides strong support for the use of "signature" in profiling as discussed above, aiding in the connection of two offenses with the same offender.
Profiling theory and practice will be enhanced through the use of such rigorous statistical tools and analysis. Another possible application for empirical testing and development is use of the Chi-Squared Automated Interaction Detection (CHAID). CHAID analyzes categorical data to develop a model of predictors which are most likely to relate to the dependent variable (Magidson, 1993), and then graphically displays the relationship.
With the increasing availability of detailed databases containing victim, offender, and crime scene characteristics, there are many possibilities for validation research. This author is currently using a logistic regression model to determine whether or not the crime scene characteristics and victimology suggested by the FBI actually predict the profile characteristics they outline for the organized/disorganized typology (Grant, 1997). Future studies of this kind will provide profiling practitioners with a known or potential error rate, which can then be used to inform investigative recommendations and future editions of manuals such as the Crime Classification Manual. Such findings will also have practical implications with regards to the use of profilers as expert witnesses.
Emergence of the ProblemAssessment of the ProblemDecision-makingImplementationReferences