University of Nebraska Kearney

Curriculum and Instruction Graduate Program Assessment Report 2006

Submitted Fall, 2006

Department of Teacher Education - 2006 Assessment Report
Master of Arts in Education: Curriculum and Instruction

Assessment Mission Statement

Master degree students graduating from the Department of Teacher Education's graduate programs will demonstrate an understanding of the manner in which perspectives and beliefs influence instructional behaviors, extend their knowledge of the central concepts of selected disciplines, and obtain and use knowledge and skills which will support increased instructional effectiveness with diverse learners.

DEPARTMENT of Teacher Education Graduate Program Objectives (*)

1. Students will demonstrate their leadership and administrative skills, which include promoting and implementing professional and state standards; (COE Desired Outcomes #2, #6, #7,  #9, #10)
    
2. Students will effectively design instruction and materials for all learners; (COE Desired Outcomes #2, #3, #4, #6, #8)
    
3. Students will demonstrate how to use a variety of productivity, research, and software tools in various instructional settings; (COE Desired Outcomes #2, #3, #4)
    
4. a.  Students will be able to apply, analyze, synthesize and evaluate critical and contemporary issues; and (COE Desired Outcomes #1, #2, #5, #9)
   
   b.  Students will be knowledgeable consumers and generators of research (COE Desired Outcomes #2, #5)

*   The Department of Teacher Education Graduate Program Objectives are linked to the ten Desired Outcomes for Graduates of the UNK College of Education. 

Program-Specific Objectives—MAED:  Curriculum and Instruction (**)

1. Students will demonstrate their leadership and administrative skills, which include the ability to promote and implement professional and state standards relating to curriculum and multicultural education. (Department Objective 1)
    
2. Students will effectively select/design/develop instruction and materials for all learners. (Department Objective 2)
    
3. Students will know how to evaluate, select, and implement a variety of productivity, research, and software tools in various instructional settings. (Department Objective 3, 4b)
    
4. Students will be able to apply, analyze, synthesize and evaluate critical and contemporary issues, particularly philosophical perspectives as they play out in the program structures and processes of schooling. (Department Objective 4a)

**   The Program-Specific Objectives for the MAED: Curriculum and Instruction, then, are linked to the Teacher Education Graduate Program Objectives.

Assessment Instruments

Direct Measure

Department Faculty Assessment of Comprehensive Examination Performance

Indirect Measures

Alumni/Graduate Survey
Critical Friends/Principal Survey 

Report Overview

This 2006 MAED: Curriculum and Instruction NCA Report is different than expected; the NCA assessment plan on file is focused on studying Curriculum and Instruction graduates who are a part of a cohort.  The historical rationale for this cohort-based assessment plan is simple; over the past 7 years nearly all of our CI graduates have been members of a cohort.  However, because none of our cohorts finished in 2005-2006 and also because we were very pleased in 2005-2006 to graduate ten non-cohort students from the Curriculum and Instruction Program, the Department decided to collect indirect data from non-cohort graduates of the CI program and, where applicable (not all non-cohort graduates in 05-06) are teaching), to also collect direct data from critical friends (school principals).  The same instrument used in the previous year with two cohort-based groups of graduates was used.  This provides the Department with an interesting lens for viewing the experiences, learning, and satisfaction level of non-cohort versus cohort-based graduates.  Unfortunately, however, direct data was not collected or reported because the format for the comprehensive examination used with non-cohort-based students is appreciably different from what is used in the cohorts. Thus, data comparison was not possible.

Direct Measure - Department Faculty Assessment of Comprehensive Examination Performance

As per our assessment plan, because none of our Curriculum and Instruction cohorts finished their work in 2005-2006, we do NOT have any direct measure data to report.

Indirect Measures

In September 2006, the Alumni/Graduate survey was disseminated to the ten (10) 2005-2006 non-cohort graduates of the Curriculum and Instruction Masters Degree Program. Four of the ten graduates responded to the survey (Return Rate = 40 %). Table One presents results for the Alumni/Graduate Survey.

In September 2006, the Critical Friends/Principal Survey was mailed to the school principals that currently work in the school buildings where the aforementioned ten (10) non-cohort graduates of the CI Program are teaching. Nine surveys were mailed (one graduate is currently not teaching) and four were returned (Return Rate = 45 %). Table Two presents results for the Critical Friends/Principal Survey.

TABLE ONE
Alumni/Graduate Survey –Fall 2006 

TABLE TWO
Critical Friends/Principal's Survey – Fall 2006 

Data Analysis

Recommendations:

The high percentage of agree/strongly agree ratings submitted by graduates and also by the critical friends of 2006 graduates was affirming; non-cohort graduates of the CI Masters Degree Program, as per self-report and supervisor report, are learning the desired content. On the basis of the data, the Teacher Education Department identified two improvement targets for the coming year(s). Both recommendations relate most closely to Program-Specific Objective #4 (Students will be able to apply, analyze, synthesize and evaluate critical and contemporary issues, particularly philosophical perspectives as they play out in the program structures and processes of schooling). 

1. The Department will consider / adopt strategies designed to bolster the “democracy” component of Program-Specific Objective #4.
   1. This recommendation is consistent with a previously declared Department desire to more effectively infuse the agenda of the National Network for Educational Renewal throughout all graduate programs (A part of the Department response to our Spring 2005 Academic Program Review)
   2. The roles and purposes for schools/schooling in United States society is a theme that is central to the NNER Agenda and also prominently showcased in our programs.
   3. We will more seriously explore the possibility of requiring TE 819P: Teacher Development Training in Service Learning. TE 819P would be provide an excellent democracy extension for the grounding that is provided in TE 803: Philosophy of Education
    
2. The Department will also consider curriculum renewals that might more effectively promote / foster student understanding of the diversity/multicultural education component of Program-Specific Objective #4.
   1. One first step will be to carefully review the content of the TE 804 course.
    

Assessment of the Assessment Process

The quality of the Assessment Plan and Process was onsidered. Key questions included:

• Are the desired outcomes for the graduates of our department relevant and defensible?
• Does the current means of assessing actually assess the department’s desired outcomes for graduates and provide information that allows for the continuous improvement of our program?   
• Is the scope and focus of our assessment process reasonable?   
• Do we need to discontinue or add any assessment activities?  

Four recommendations relative to the assessment process were identified:

Recommendation #1—Standardization of our Direct Assessment Plan.   The significant but also very recent increase in the number of non-cohort graduates of the CI Masters Degree Program clearly indicates a need for standardizing the direct measures assessment that is used.  Accomplishing this task is one goal for 2006-2007. Taking this step will make our assessment report more complete and it will also facilitate easy comparison of cohort versus non-cohort graduates.

Recommendation #2—Data Collection Plan for Alumni/Graduate Survey and for Critical Friends Survey.   The Department will consider alternative strategies for administering the Exit Alumni/Graduate Survey and the Critical Friends Survey. The overall rates of return (40% and 45%), even though they mirror last year’s return rates, ae still unacceptable.Whereas in 2005 we used OPINIO software (which encountered serious glitches), the data collection plan in 2006 included direct mailing of the survey. Still, the response rate remained unchanged. One option might be to reconsider when we distribute the surveys; perhaps sending them about the same time as graduation or perhaps as a part of the final semester when comprehensive exams are completed would result in a higher rate of return.

Recommendation #3—Reconsideration of Program-Specific Objectives—MAED:  Curriculum and Instruction.   The assessment plan for the MAED: CI declares four program-specific objectives.  The themes addressed by those four objectives are as follows: (1) leadership and administrative skills—especially including promoting and implementing professional and state standards relating to curriculum and multicultural education; (2) instructional skills; (3) use of productivity, research, and software tools; and (4) analysis skills – particularly relating to contemporary issues and philosophical perspectives.  It is time to reconsider the utility and future use of these objectives.

Recommendation #4—Review Linkage of NCA and NCATE Assessment Plans and Processes.  The assessment plan and process will be revisited as part of a NCATE unit review and potential realignment of assessments process that will be undertaken, in 2006-2007, for graduate programs.