Existing Placement Program
Evaluating an Existing Placement Program
- How can I benefit from an evaluation of an existing placement program?
- How do I design an evaluation of an existing placement program?
- What does an ACES™ placement validity report look like?
How can I benefit from an evaluation of an existing placement program?
Even the most carefully designed, thoroughly evaluated placement programs can become dated or have unintended consequences. Shifts in the characteristics of the students in the program, normal shifts in test content, and the natural evolution of course content can all affect the validity of a placement program. While a new predictive or concurrent placement study should be completed on a regular basis, simple checks of an existing placement program can ensure that validity is being maintained.
Note: Although Advanced Placement Program® (AP®) Exams and College Level Examination Program® (CLEP®) exams may be used to place students into an advanced course, these two tests are given to different students. AP Exams are intended for students completing a specific course sequence. CLEP exams are intended for students who have earned their knowledge through self-study or life experience. Therefore, it is not advisable to include scores from both of these programs in the same study.
How do I design an evaluation of an existing placement program?
There are several decisions you will need to make when designing an evaluation of an existing placement program using AP Exams. The information below will help you design a placement study that uses AP exam grades as predictors.
- Choosing a criterion
- Whom to include in the study
- Student records used in the analysis
- How to collect the data
- How to submit the data
- Format for data submission
Choosing a criterion
ACES™ uses end-of-course grade as the criterion in placement validity studies. When carrying out a validation of an existing placement test, such as AP Examinations, it is assumed that the test was used to exempt some of the students from a prerequisite course. The study helps you to determine whether the placement exam cutoff grades you use are adequately predicting success in the second-level course. The analysis requires information on the students' grades in the second-level course they have taken. It is helpful, but not necessary, to include information about grades earned in the prerequisite course by students who were not placed directly into the higher-level course.
Whom to include in the study
General guidelines
Two groups are contrasted in the validation of an existing placement program: those who have taken the course of interest after completing a prerequisite and those who are placed into the course using your predictor variables (AP Examination grades). In order to be included in the sample, information on the criterion and predictors must be included. Students who placed into the course will need to have grades on the AP Examination you wish to validate, while students who took the prerequisite will need to have the prerequisite course grades.
It is important that the sample be similar to the population for which you will be continuing to use the test in ways that are and are not measured by the placement test. Characteristics of the population, such as gender, ethnic/racial makeup, and age, should be represented in the sample.
Specific guidelines
Students with Pass/Fail grades
In order to effectively evaluate how well a predictor is working, ACES requires a range of at least four points on a scale for the criterion. Students with Pass/Fail grades should not be included in the study.
Students with a Fail, Incomplete, or Withdraw grade
Generally, we have found that at most institutions failing grades, such as F or E, are calculated into the students' grade point average and are equivalent to zero grade/quality points; and Incomplete or Withdraw grades, such as I or W, are not incorporated into the GPA calculation. In these cases, students with failing grades should be included in your placement validity study, while students with Incomplete or Withdraw grades should not.
These are only given as general guidelines. If you have a large number of students who failed the course you wish to study for any reason other than doing poorly on the tests and/or class work (such as, dropping the class or leaving the school without officially withdrawing), or if your Incomplete or Withdraw grades are calculated into the GPA, please contact the ACES staff at aces@info.collegeboard.org or by calling (609) 921-9000 to discuss your specific situation.
Number of students in the study
The more students you include in your analysis, the more valid the results. There must be at least 30 students with both a valid grade in the course being analyzed and an AP Exam score. If a prerequisite course is included in the analysis, there must be at least an additional 30 students with valid grades in the course and in the prerequisite course.
Students who are missing AP exam grades
It is not necessary for your data to include the AP exam grades. ACES will match your students to our national database using name, social security number, date of birth, and gender. If any of this information is missing, ACES will try to match the student with our database based on the available information, but we cannot guarantee that we will be able to find the student's information. ACES may not be able to match AP exam grades for students who do not have a social security number and were, therefore, assigned a college ID number.
Student records used in the analysis
Advanced Placement Program (AP®) exam grades
For an ACES placement study using AP exam grades, ACES begins by matching your students to the national AP database. If a match cannot be found, the student will not be included in the ACES study.
How to collect data
The ACES process for validating an existing placement test is useful when your chief concern is predicting student success in a particular course on the basis of scores on tests they took before they entered college. An example of this would be grades obtained from an AP Exam. The grades of students who have taken a course are useful for predicting whether other students will succeed if they in turn take the same course.
One important piece of information you will want to look at is the characteristics of students who enter the course through the course prerequisite and those who place into the course using the particular placement exam, such as an AP Exam. You may also want to know the average performance of selected subgroups of students in the course.
For an AP study, there must be at least 30 students with both a valid grade in the course being analyzed and an AP Exam score. If a prerequisite course is included in the analysis, there must be at least an additional 30 students with valid grades in the course and in the prerequisite course. Please note that small sample sizes lead to unstable results. The larger the sample of students, the more likely the results will generalize from one year to the next.
How to submit data
For a placement study using AP Examination grades, the college should submit their files of student data as a single horizontal or vertical file containing the key student identifying fields of student name, social security number, date of birth, and gender, as well as, the course data for the courses they choose to study and the prerequisite course grades, if any.
Format for data submission
Data may be submitted as a fixed-length ASCII file, a comma-delimited ASCII file, a tab-delimited ASCII file, a Microsoft ACCESS® database, a Microsoft EXCEL® workbook, a SAS® transport file, or an SPSS® portable file.
What does an ACES™ placement validity report look like?
View a sample placement validity report (.pdf/658K). Requires Adobe Reader (latest version recommended).
Return to Understanding Placement Validity Studies