Concurrent Placement Validity Studies
ACES Concurrent Placement Validity Studies
- How can I benefit from a concurrent placement validity study?
- How do I design a placement validity study using College-Level Examination Program® (CLEP®) scores?
- What does an ACES™ placement validity report look like?
How can I benefit from a concurrent placement validity study?
A concurrent placement validity study will help you translate a student's score on CLEP® exams, and up to three additional institutional tests or measures that were administered in the same time frame, into a probability that the student has the same level of ability as students successfully completing a particular course or program.
Making appropriate course placements for students in your institution benefits both the institution and the student. Appropriate course placement ensures that valuable course time, student time, and other resources are not wasted because students are not prepared to benefit from a course, or because their level of preparation is such that they would benefit from a higher-level course.
ACES™ reports can be shared with policymakers and provide the type of documentation required to support placement decisions and policies at your institution.
How do I design a concurrent placement validity study using CLEP examination scores?
There are several decisions you will need to make when designing a concurrent placement study. The information below will help you.
- Choosing a criterion
- Selecting predictor variables
- Whom to include in the study
- Student records used in the analysis
- How to collect data
- How to submit data
- Format for data submission
Choosing a criterion
The criterion used for your placement validity study will determine whether or not students were correctly placed. Determining whether or not a test is valid for placement purposes depends on choosing the correct criterion against which to evaluate the test. To choose your criterion for a concurrent placement validity study:
- Decide out of which course you will place your students (rather than the course into which you will place students).
- Decide the best measure of whether or not students were correctly placed in the course—usually determined at the end of the course. This measure is the criterion.
ACES uses end-of-course grades as the criterion in placement validity studies. All the problems of GPA reliability as a criterion apply as well to the criterion of course grade. Use of only one grade from one instructor per student increases the problems. The course grade, however, is often the only available or feasible criterion.
A more reliable criterion, if available, would be a well-designed common departmental examination, graded centrally without indication of a student's instructor or section. This increases the chance that the study results will be useful, regardless of the instructor to which future students are assigned.
Selecting predictor variables
For concurrent placement validity studies, you are predicting success on the criterion (end-of-course grade) using the predictor variables. The predictor information is gathered at or near the completion of the particular course.
A variety of tests and other measures can be used as predictors for placement purposes. In a concurrent validity study, the College Board's College-Level Examination Program (CLEP) tests can be given to students completing the relevant course to determine how students known to have the necessary knowledge will perform. Local tests used for placement can be included to see how they perform relative to a new test being considered, as long as they are administered within the same time frame. ACES can be used to help validate any of these measures or to determine their best combination.
The specific predictors you choose should first pass content and face validity examinations. In addition, you will want to think through the availability and cost of the predictors you might choose. It may be that a test with several open-ended questions matches your course material, is deemed appropriate by your colleagues, and does a wonderful job of predicting success in your course. However, the cost of such a test may be more than your department or the students are willing to pay, or the test may take so long to score that decisions cannot be made in time for class assignments.
Whom to include in the study
General guidelines
To be included in the sample for a validity study, the students you use must have scores on the predictor variables and on the criterion. To predict the probability of success of future students, data derived from the sample of students is used to generate the validity study results. Predictions are more likely to be accurate if the sample of students used for the study is similar to the group of students whose success you want to predict, both in ways that are and are not measured by the predictors. Characteristics of the population, such as gender, ethnic/racial makeup, and age, should be represented in the sample.
For the study to be accurate, the predictor and criterion need to be collected from all students at the same point in the course sequence.
Specific guidelines
Students with Pass/Fail grades
In order to effectively evaluate how well a predictor is working, ACES requires a range of at least four points on a scale for the criterion. Students with Pass/Fail grades should not be included in the study.
Students with a Fail, Incomplete, or Withdraw grade
Generally, we have found that at most institutions failing grades, such as F or E, are calculated into the students' grade point average and are equivalent to zero grade/quality points; and Incomplete or Withdraw grades, such as I or W, are not incorporated into the GPA calculation. In these cases, students with failing grades should be included in your placement validity study, while students with Incomplete or Withdraw grades should not.
These are only given as general guidelines. If you have a large number of students who failed the course you wish to study for any reason other than doing poorly on the tests and/or class work (such as, dropping the class or leaving the school without officially withdrawing), or if your Incomplete or Withdraw grades are calculated into the GPA, please contact the ACES staff at aces@info.collegeboard.org or by calling (609) 921-9000 to discuss your specific situation.
Number of students in the study
The minimum number of students required for a study depends on the number of predictors used. If 1 to 3 predictors are used, a minimum of 30 students are required; for 4 predictors, a minimum of 40 students are required; and for 5 predictors, a minimum of 50 students are required.
Student records used in the analysis
College-Level Examination Program® (CLEP®) scores
For a concurrent study using CLEP data, ACES begins by matching your students to the national CLEP database. If a match cannot be found, the student will not be included in your concurrent placement study.
How to collect data
There are substantial differences between how you collect data for a predictive placement validity study and a concurrent placement validity study. For a predictive placement validity study, information from the tests or measures that are used as predictive variables is collected prior to the student entering the course of interest. For a concurrent placement validity study, data from the tests or measures that are used as predictive variables are collected at or near the end of the course; that is, the predictive measures are typically collected concurrently with the criterion measures. The CLEP exam scores and other test scores or measures that will be used in your concurrent placement validity study should, therefore, be administered at or near the end of the course. It is important that the students taking the CLEP exam for use as the predictive variable consider the test seriously.
How to submit data
For a concurrent placement validity study the college may submit their files of student data as one merged file or two separate files. If two files are sent, both must be in a horizontal format. Each file must contain the key student identifying fields of student name, social security number, date of birth, and gender. All of the key identifying data fields must be in the same format on both files, including the layout of the social security number, format of the birth date, the gender designation, and the upper/lowercase format for the name. The ACES data submission form provides a space for you to identify how the student names are formatted in your files. The first file must contain the test scores, if they are being provided by your institution; whereas, the second file will consist of the course data.
If two files are submitted, ACES begins by matching the data in the two files. If students are found in one file but not the other, these students will not be included in your study.
Format for data submission
Data may be submitted as a fixed-length ASCII file, a comma-delimited ASCII file, a tab-delimited ASCII file, a Microsoft ACCESS® database, a Microsoft EXCEL® workbook, a SAS® transport file, or an SPSS® portable file.
What does an ACES™ placement validity report look like?
View a sample placement validity report. (.pdf/586K)
Requires Adobe Reader (latest version recommended).
Return to Understanding Placement Validity Studies