University of Nebraska Kearney

Reading Graduate Program Assessment Report 2006

Submitted Spring, 2006

Methodology

There were six graduate candidates that applied for graduation during the 2005-2006 year. One completed her work in November 2005, three completed their work in March 2006 and two completed their work in July 2006. They were required to develop a case method for the evaluation of the second program objective. The objective (Program Objective #2) being evaluated was:

• Students will use strategies to analyze, synthesize, and evaluate seminal research and original inquiry within the field of literacy.

The case method topic focused on how the candidate utilized current research to address a classroom instructional problem in the area of reading instruction. The candidate was required to address the following items within the case method.

1. Exploration of the “Reading First” initiative.
2. Explain and assess why the five major areas are covered.
3. Identify elements that should be added to an intermediate grades curriculum.
4. Give rationale for elements identified.
5. Evaluate the pros and cons of each of the options.
6. Support position with evidence from masters program classes.
7. Support position with research citations, etc.

The case methods were scored using a rubric that rated the following traits.

• Explanation and analysis of initiative
• Analysis of issues
• Essential elements and rationale
• Relevant research
• Position support
• Professional sources

Results

The results are as follows:

Conclusions

1. Five of the students were primary teachers and the other one was secondary.  All six of them handled that case appropriately.  They have been inundated with material about "reading first" at the primary level so to conceptualize the material and apply it to a higher level is a stretch.
2. This indicates that our students are using the class preparation as well as their professional settings to grow in knowledge.  

Use of Results

1. We will continue to use this case in our classes at this time as it involves all of our students as they strive toward the reading endorsement.
2. We will also prepare for surveys of past students and administrators now that we have a workable number to analyze and disseminate.

REVISION OF ORIGINAL PLAN

The Original Assessment Plan

Schedule

The assessment of candidates will be conducted as the comprehensive exam. The comprehensive exam is administered independent of individual courses during one of the last semesters of the candidate’s completion of the program of study. The comprehensive exam is typically scheduled at the first of November (fall), April (spring), and July (summer).

Four program objectives were identified and assessment tasks and rubrics were developed. The assessment cycle for these objectives will be as follows:
     Objective 1: November
     Objective 2: April
     Objective 3: July
     Objective 4: November
There are usually one to three candidates completing the comprehensive exam each semester. Therefore, this schedule will provide for consistent collection of assessment data. In the event that no one is scheduled for an exam in a particular semester, the schedule will be followed for the next semester.

Process

The assessment task will be given to the candidate for completion during the scheduled week for the comprehensive exam. The candidate will complete the task independently and submit it to the faculty member supervising the assessment task for that semester.

A team with a minimum number of three faculty members will be used to assess the assessment task completed and submitted by the candidate. Each faculty member will use the assessment rubric and conduct a “blind” scoring of the assessment task. The team will then compare scores as a reliability check and resolve major differences in scoring to determine if the candidate passed the assessment.

Each spring semester, the members of the literacy faculty will review the results of the assessment of the program objectives and recommend program changes.

REEVALUATE THE ASSESSMENT

After 3 years of data collection and reporting, a revision of the assessment was completed.  Assessment data for Objective 1 and 4 will be collected when students are enrolled in TE 816 b (or TE 816c, d, or e).  The original rubric will be used to assess this material, a reporting guide will be constructed and data will begin with Spring 2007. 

Objective 3 data will be collected from the Critical Friend Survey and the Alumni Survey to be conducted in the Spring of 2007.

This revision will permit the collection of data on all objectives in a more effective manner than the original schedule permitted.

Revised Reading Program Rubric