Blog #5: Dissertation– Predictive Validity of MC Test for CC Placement

Verbout, Mary F. Predictive Validity of a Multiple-choice Test for Placement in a Community College. Diss. Indiana University of Pennsylvania, 2013. Ann Arbor, 2013. 3592228. Web. 18 Mar. 2016.

Verbout studied scores from Compass to prove that the cut-off scores used for placement did not correlate with success in the courses. The program in the study was optimal because while students were required to take the test, they were not required to use the results when selecting their courses.

 

According to the test creators, “The test is efficient; an algorithm recalculates the students’ overall score with each answer, and as soon as the internal calculation achieves certainty, the test is over, and the score is calculated (ACT Inc., 2006)” (3). It should be noted that “The eight domains addressed by the writing diagnostics are: punctuation, spelling, capitalization, usage, verb formation/agreement, relationships of clauses, shifts in construction, organization” (75-76), so the test does not coincide with the overall goals of the courses.

 

Research questions: does the placement based on Compass scores predict success in FYC1 and 2; is there a significant difference between the mean scores for White,  Hispanic, and Native American students; is there a significant difference between success rates for White, Hispanic and Native American students in FYC 1 and 2 (67).

 

SPSS and Excel were the tools used to process data. The tests were ANOVA and Chi-Square. Of the students who scored in the 13-37 range on Compass, 23% of the students who enrolled in BW1 passed FYC, 39% of BW2 passed, and 76% of the students who chose to enroll in FYC1 against recommendation passed it!

 

The group of test scores for direct BW2 enrollment ended up with pass rates below the students who chose to skip BW2 and enroll directly into FYC1 (46% to 81%) (82).

The results about race were: the test scores placed Hispanic and Native American students into BW at a higher rate than White students. The pass rates for students in FYC were not significantly different. What was of interest is that the Hispanic and Native American students were more likely to enroll in the course that was recommended for them based on test scores than the White students, who were more likely to choose to take FYC against test score recommendation (84).  “With one exception (Hispanic students scoring 13 -37, first  course BW2 students in each score range and ethnic category completed FYC1 and FYC2 at higher rates when they began in a more advanced course” (85).

The researcher’s conclusions were to discontinue use of Compass and consider creation of studio-model BW courses similar to CCBS’s Accelerated Learning Program (ALP).
The results and methodology sections were not as full as the excellent literature review that combines educational and composition theorists. Verbout does not stipulate the year range for this study and how long students were followed nor include whether the students in BW1 and BW2 also had additional remedial courses to take that could slow their entrance into FYC. More information would be needed in order for the study to be replicable, but the statistics were clear, direct, and surprising.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s