The following are summaries of some articles regarding what the MCAT really measures, how well it does this, and how medical schools use it.
When are students most at risk of encountering academic difficulty? A study of the 1992 matriculants to U.S. medical schools.
Huff KL; Fang D
Section for the Medical College Admission Test, Association of American Medical Colleges, Washington, D.C., USA. khuff@aamc.org
Acad Med 1999 Apr;74(4):454-60
The study indicated that (1) while the risk and timing of academic difficulty varied across the groups studied, a majority of the students who experienced academic difficulty eventually graduated from medical school and (2) students with non-science undergraduate majors did not have a greater risk of academic difficulty. The results confirm previous findings that increased risk of academic difficulty is associated with low MCAT scores, low science GPA, low undergraduate institutional selectivity, being a woman, being a member of a racial-ethnic underrepresented minority, or being older.
Correlation of students' characteristics with their learning styles as they begin medical school.
Aaron S; Skakun E
University of Alberta, Edmonton, Canada. stephen.aaron@ualberta.ca
Acad Med 1999 Mar;74(3):260-2
PURPOSE: To investigate the relationship between learning styles (surface, strategic, and deep learning) and admission data for an incoming class of medical students. METHOD: In 1997, the Approaches and Study Skills Inventory (ASSIST) was administered to the University of Alberta Medical School's incoming class as part of their orientation. Ninety percent of the class completed the questionnaire, the results of which were correlated with prerequisite grade-point average (GPA), MCAT scores, number of years of premedical experience, and scores on autobiography, interview, and letters of reference. RESULTS: Higher surface-learning scores correlated significantly with younger age at admission to medical school, as well as with higher GPA. There was a positive correlation between GPA and surface learning in the group of students with more than four years of premedical experience. CONCLUSIONS: The need to compete for grades in prerequisite courses may be a factor contributing to surface learning in new medical students.
Predicting medical students' academic performances by their cognitive abilities and personality characteristics.
Shen H; Comrey AL
Neuropsychiatric Institute, University of California, Los Angeles, USA.
Acad Med 1997 Sep;72(9):781-6
PURPOSE: To study the relationships among students' cognitive abilities, personality traits, and medical school performances at the University of Los Angeles, California, UCLA School of Medicine. METHOD: Ninety-seven "not-disadvantaged" students' pre-medical GPAs, MCAT scores, and personality traits as measured by the Comrey Personality Scales (administered at their application to medical school in 1985) were used to predict their medical school performances according to several criteria. "Disadvantaged" students were excluded from the study because their poor performances on all criteria would confound the relationships of personality, cognitive ability, and performance. RESULTS: The MCAT score was a strong predictor of medical school performances, particularly those criteria measured by medical school GPAs and the National Board of Medical Examiners examination scores, but its predictive power dropped sharply when clinical performance and personal suitability were part of the performance evaluation. Specific personality traits not only strengthened the predictive power of cognitive and personality variables jointly, they became the primary predictors of clinical performance and personal suitability.
Evaluating GPAs and MCAT scores as predictors of NBME I and clerkship performances based on students' data from one undergraduate institution
Silver B; Hodgson CS
University of California, School of Medicine, Los Angeles 90095-1722, USA.
Acad Med 1997 May;72(5):394-6
Although undergraduate GPAs and MCAT scores are good indicators of NBME I performance, they are not useful in predicting clinical performance.
Prediction of students' USMLE step 2 performances based on premedical credentials related to verbal skills.
Roth KS; Riley WT; Brandt RB; Seibel HR
Department of Pediatrics, Virginia Commonwealth University Medical College at Virginia School of Medicine, Richmond 23298, USA.
Acad Med 1996 Feb;71(2):176-80
PURPOSE. To examine the relationship between the objective premedical credentials and performances on Step 2 on the United States Medical Licensing Examination (USMLE) of 480 students in three classes at the Virginia Commonwealth University Medical College of Virginia School of Medicine. The purpose of the study was to seek those selection criteria that might best predict performance on an examination designed to assess problem-solving skills, the essence of clinical medicine. METHOD. Premedical data from two classes (1993, 1994) were analyzed, and a regression equation was used to calculate theoretical USMLE Step 2 scores for the students in the class of 1995, who had not yet taken this examination. The premedical variables were scores on the verbal and math section on the Scholastic Aptitude Test (SAT), scores on the six sections of the pre-1991 Medical College Admission Test (MCAT), grade-point average (GPA) in science courses required of premedical students, and undergraduate major. Once the class of 1995 had taken the USMLE Step 2, the equation was cross validated, and the theoretical and actual scores of the class of 1995 were correlated. RESULTS. The correlation between theoretical and actual scores was r = .443. The single variables most highly predictive of USMLE Step 2 performance were scores on the verbal section of the SAT (r = .317) and the Skills Analysis: Reading section of the MCAT (r = .331) suggesting that high verbal aptitude serves one well, even when coping with complex scientific concepts.
Influence of medical school applicants' demographic and cognitive characteristics on interviewers' ratings of noncognitive traits.
Shaw DL; Martz DM; Lancaster CJ; Sade RM
Department of Psychiatry and Behavioral Sciences, Medical University of South Carolina, College of Medicine (MUSCCM), Charleston 29425-0742, USA.
Acad Med 1995 Jun;70(6):532-6
BACKGROUND. Interviews are commonly used to measure noncognitive traits of medical school applicants. The present study investigated the influence of knowledge of applicants' cognitive abilities on interviewers' ratings of noncognitive traits. METHOD. Academic and demographic predictors of interview ratings of applicants' noncognitive traits were examined at the Medical University of South Carolina College of Medicine during two years: 1992, when applicants' Medical College Admission Test (MCAT) total scores and undergraduate grade- point averages (GPAs) were available to interviewers; and 1993, when MCAT and GPA data were not available. In 1992, 226 applicants met study criteria (i.e. they received ratings from three interviewers in addition to having MCAT and GPA data on file); in 1993, 245 applicants met the criteria. RESULTS. GPA was the best predictor for both years but accounted for double the amount of variance in interview ratings in 1992 (15.7%) compared with 1993 (7.4%). The reliability coefficients for the interviewers were .496 for 1992 and .473 for 1993. CONCLUSION. If the goal of the medical school admission interview is to assess noncognitive traits independently from academic skills, the authors recommend that MCAT and GPA data not be available to interviewers during interviews. The authors found that interview scores were only moderately reliable across different interviewers.
The decline and rise of the medical school applicant pool.
Kassebaum DG; Szenas PL
Division of Educational Research and Assessment, Association of American Medical Colleges, Washington, D.C., USA.
Acad Med 1995 Apr;70(4):334-40
The authors characterize the demographic changes that transpired with the decline and rise of the medical school applicant pool over the past decade, and describe the variations in academic antecedents, attrition, and graduation rates of students matriculated during that time. Data over the ten-year cycle, derived from the AAMC's Student and Applicant Information Management System (SAIMS), were examined in the context of published education and employment statistics. The contraction and expansion of the applicant pool were related to changes in the number and pattern of undergraduate majors and to changes in the employment conditions for college-educated youth. Furthermore, a significant part of the variations in size of the applicant pool is an artifact of changes in the number of repeat applications. Matriculants' pre-medical grades and MCAT scores dropped slightly during the period of applicant decline, and rebounded as admission committees were able to exercise greater selection when the pool expanded. The attrition of medical students rose and fell during this time, but the changes were small and of little discrete influence on graduation rates during the period. The downturn and rebound in applications over the past decade appear to be more related to cycles in the employment market for college graduates than to applicants' perceptions of unfavorable/favorable conditions in medical education and practice.
Assessing the validity of the updated Medical College Admission Test.
Mitchell K; Haynes R; Koenig J
Center for Research on Evaluation, Standards, and Student Testing, RAND Corporation, Washington, D.C.
Acad Med 1994 May;69(5):394-401
This report represents early information about the validity of the updated Medical College Admission Test (MCAT) battery introduced in 1991. Data are given on both the use of the new examination in the selection of medical students and its estimated predictive value for freshman students' performance. Admission officials from 114 institutions responded to a survey on medical school admission practices and on the use of MCAT data; also elicited were assessments of the examination in relation to its design and implementation objectives. Regression-based evaluations of the predictive validity of the MCAT, undergraduate grades, and of undergraduate selectivity data for first-year grades at 12 institutions are described. Survey responses suggest that MCAT data are viewed in relation to the varied information needs of admission decision making and are supplemented by other academic and nonacademic information in selection. The new battery generally is judged favorably by admission officials. Predictive validity results reflect the usefulness of MCAT scores, undergraduate grades, and selectivity data for identifying individuals apt to succeed in medical school. The authors conclude that preliminary information about the results of the new examination is encouraging.
Students' psychosocial characteristics as predictors of academic performance in medical school.
Hojat M; Robeson M; Damjanov I; Veloski JJ; Glaser K; Gonnella JS
Center for Research in Medical Education and Health Care, Jefferson Medical College, Philadelphia, PA 19107-5083.
Acad Med 1993 Aug;68(8):635-7
PURPOSE: To investigate the incremental effects of selected psychosocial measures--beyond the effects of conventional admission measures--in predicting students' academic performances in medical school. METHODS. In 1989-90, 210 second-year students at Jefferson Medical College were each asked to complete 11 psychosocial questionnaires that were then used as predictors of performance measures in medical school. The students' scores on three subtests of the Medical College Admission Test (MCAT) were also used as predictors. Three composite measures of performance were used as the criterion measures: basic science examination grades, clinical examination grades, and ratings of clinical competence. A multiple regression algorithm (general linear model) was used for statistical analysis. RESULTS. The response rate was 83% (175 students). When the psychosocial measures were added to the statistical models in which the common variances of the MCAT scores were already determined, significant increments in the common variances were observed for two of the three performance measures: basic science grades and clinical examination grades. Whereas only 4% of the common variance in the ratings of clinical competence could be accounted for by the MCAT scores, 14% could be accounted for by the psychosocial measures. CONCLUSION. The "noncognitive," or psychosocial, measures increased the magnitude of the relationships between the predictive and criterion measures of the students' academic performances, beyond the magnitude attained when only the conventional admission measures were used. Therefore, psychosocial measures should be considered as significant and unique predictors of performance in medical school.
Performances on the NBME I, II, and III by medical students in the problem-based learning and conventional tracks at the University of New Mexico
Mennin SP; Friedman M; Skipper B; Kalishman S; Snyder J
University of New Mexico School of Medicine, Albuquerque 87131-5211.
Acad Med 1993 Aug;68(8):616-24
BACKGROUND. Problem-based learning curricula are growing in popularity, and questions have been raised about the appropriateness of standardized examinations, such as the National Board of Medical Examiners (NBME) Parts I, II, and III examinations, for assessing students in these new curricula. CONCLUSION. In the short run, the more teacher-centered and structured conventional curriculum better prepared the students for the NBME I, while in the long run, the more student-centered problem-based curriculum better prepared the students for the NBME III.
A twelve-year profile of students' SAT scores, GPAs, and MCAT scores from a small university's premedical program.
Montague JR; Frei JK
Barry University School of Natural and Health Sciences, Miami Shores, Florida.
Acad Med 1993 Apr;68(4):306-8
Students' SAT scores and GPAs proved to be statistically reliable predictors of MCAT scores.
