Evaluation of the Human Resources Products Services (HRPS) Staff Development Program Courses

 

    

PerformTech was hired by OPM, Human Resources Products and Services (HRPS), to plan and implement an extensive evaluation of 18 classroom-based courses that make up the HRPS Staff Development Program (SDP). The purpose of the SDP is to close the gaps in 13 identified core competencies for all HRPS staff. The program was launched in November 2004, following a comprehensive training needs assessment conducted by PerformTech. 

 

PerformTech designed an evaluation plan for conducting level 1 and level 3 evaluations of each of the courses over a 10-month period.  The plan provided for:

• Student-completed level 1 course evaluations
• A web-based pre-course survey distributed to students and their supervisors one week before  a scheduled course
• A web-based post-survey distributed to students and their supervisors four months after the course was completed
• One-on-one telephone interviews conducted with a random sample of students and their supervisors to gather more in-depth information

The pre-course surveys and interviews were designed to identify the reasons students and their supervisors selected particular courses and what competencies were expected to be addressed in each course.  Supervisors were also asked to assess their employee’s performance level for the chosen competency.  Four months later, employees were asked to indicate how well the course met their objectives and what impact it had on their performance.  Supervisors were then asked to rate the current performance level of their employees in the target competency and to provide examples of changes in observed performance attributable to the training. 

PerformTech created a web-based evaluation database to aggregate all of the data from the level 1 evaluation forms (1,300 completed forms) and the web-based survey and telephone interviews.  This database allowed PerformTech to compare student and supervisor responses across courses and different offerings of the same course.  We were also able to track changes in performance for individual employees and compare improvement levels across courses and competencies 

PerformTech summarized the evaluation data collected for each of the 18 courses.  These reports were combined with information gathered during a follow-up needs assessment to recommend revisions to individual courses and to the SDP as a whole.