The Psychometric Department is looking for two doctoral summer interns for 2020 and each intern will work with two mentors on a specific project in the area of psychometric.
About the College Board
We are a mission focused organization that believes that investing in education is investing in the future. From our earliest days, we have devoted ourselves to educational opportunity and achievement. We have promoted innovation, equity and excellence for generations of students. We are advocates for children and parents; we empower teachers and educators; and we are a strong presence in thousands of schools and communities across the country through programs and services - the SAT, Advanced Placement (AP®) and Pre-AP are just few. Our work falls broadly into four categories: College Readiness, College Connection & Success, Student Opportunities and Advocacy.
Named by Fast Company as one of the most innovative education companies, the College Board is a mission-focused organization. This job requires a strong focus on improving educational opportunities and outcomes, particularly for disadvantaged students, in the context of a competitive business environment.
The internship spans 8 weeks, starting on June 1st and ending on July 24th, with an expected weekly full-time workload (40 hours per week). This eight-week internship is designed to provide interns with opportunity to work closely with psychometricians and gain hands-on working experience with College Board data and projects. Interns are expected to perform a literature review, conduct analysis, write a research report, and present the research to College Board staff at the conclusion of the project.
To be eligible:
- Interns must be full time doctoral students at an accredited 4‐year university
- A strong preference will be made toward advanced students in the process of completing their dissertations
- Graduate students in psychometrics, measurement, quantitative & mathematical psychology, educational psychology, industrial‐organizational psychology, statistics or related fields are invited to submit applications
- Experience with statistical software (SAS, SPSS, and/or R) is required and working knowledge of Classical Test Theory, and Item Response Theory are desired
- Interns are expected to perform a literature review, conduct analysis, write a research report, and present the research to College Board staff at the conclusion of the project
- Students must be eligible to be legally employed in the United States (international F1 visa students please read details below)
Housing stipend can be offered when deemed necessary
Please indicate in your Cover Letter which project you would like to be considered
Interns will work with two College Board mentors on a specific project in the area of psychometrics. Possible topics include:
Project 1: Investigation of Methods to Evaluate Item Fit Plot
In an operational testing environment, it is imperative that psychometric work be completed in a timely manner. One task that can be time consuming is the evaluation of item characteristic curve plots for fit. The proposed study is to evaluate variants of a method to evaluate item fit plots, with the goal of alleviating the need for human review. The study will utilize operational data and requires some advanced programming in R and familiarity with FlexMIRT. Familiarity with aberrancy detection methods is a plus.
The goal of this project is to investigate an automated method to determine if the empirical data in an item plot fits the item characteristic curve (ICC). If it shows sufficient effectiveness, then the process developed could be used to reduce the amount of time and resources spent reviewing IRT item plots for pretest items.
The basic procedure involves the following steps:
- The calibration data is split into two datasets. The procedure for splitting the data is described below.
- Two sets of item parameters are calibrated
- The ICCs of the items are compared to their corresponding ICCs
- Items with large differences between their ICCs are then removed
- The procedure is then repeated until no items are flagged for removal
- The excluded items are tagged as having poor item fit
The crux of this study is to identify a procedure that splits the data in a manner that identifies items with poor item fit. A few of the options being considered are:
- Institution size
- Lz statistic
- Admin Region
The final list of options will be identified in collaboration with the intern. We plan to use SAT data from a pretest administration for the analysis. The results will be compared to the list of pretest items flagged under the current review process.
Project 2: Impact of Population Variation on Equating Error and Scale Stability
Testing programs usually offer more than one administration throughout a year, which may lead to test-taker population variation across those administrations. In the meanwhile, equating results based on different populations may vary and it is important to evaluate the impact of population variation on equating to guide and inform operational equating practice.
This research study will evaluate whether equating invariance holds for tests administered to test-takers with various characteristics. Data will be simulated in the IRT framework and evaluated by both classical and IRT equating methods.
The primary research questions are (1) to evaluate whether equating based on test-takers from different administrations yield similar results, and (2) to identify robust equating methods that provide stable equating solutions. This study will also investigate the impact of population variation on equating based on different kinds of test forms (e.g., with essay vs. without essay, easy vs. hard, and high reliability vs. low reliability, etc.).
- Item Response theory
- Strong programming skills in C++, SAS, or R
International students who are studying at an accredited university under F1 visas are eligible to apply for the summer internship under Curricular Practical Training (CPT) stipulations. Please note that only two internship positions can be offered. International students should not apply for CPT unless accepted as a summer intern.
Upon acceptance to the summer internship, we urge students to contact their respective international advisers at their host university as soon as possible to apply for a practical training certificate, which permits F1 visa holders to receive compensation from the College Board for the work they will be completing over the summer. The process to clear a student for CPT may take six weeks or longer. Therefore, we urge students to initiate the process as soon as possible. Additionally, all international students must have a social security number in order to receive compensation.
Our mission is to clear a path for all students to own their future. The College Board is committed to diversity in the workplace and is an Equal Opportunity Employer. The College Board participates in E-Verify, a service of DHS and SSA, where required. Please understand that only qualified applicants will be contacted.