University of Nebraska Kearney

MSEd (Principal/Building Level) Graduate Program Assessment Report 2006

Submitted Fall, 2006

Educational Administration
Assessment Report for Building Level Administrators
September 2006

Assessment Mission Statement

Graduates will be competent building level school administrators, who are responsive, reflective, and collaborative decision makers and who have the requisite knowledge, skills, and technological expertise to organize learning environments that embrace the principles of democracy, diversity, and equity.

Assessment Goal:

Graduates will become competent school administrators.

A.  Principal/Building Level Objectives: 

Knowledge

Learning Objective 1:
Candidates who complete the program are educational leaders who have the knowledge and ability to promote the success of all students by facilitating the development, articulation, implementation, and stewardship of a vision of learning that is shared and supported by the school district.

Learning Objective 2:
Candidates who complete the program are educational leaders who have the knowledge and ability to promote the success of all students by promoting a positive school culture, providing an effective instructional program, applying best practices to student learning, and designing comprehensive professional growth plans for staff.

Learning Objective 3:
Candidates who complete the program are educational leaders who have the knowledge and ability to promote the success of all students by understanding, responding to, and influencing the larger political, social, economic, legal, and cultural index.

Learning Objective 4:
Candidates who complete the program are educational leaders who have the knowledge and ability to promote the success of all students by managing the organization, operations, and resources in a way that promotes a safe, efficient, and effective learning environment

Skills:
Learning Objective 5:
Candidates who complete the program are educational leaders who communicate effectively both orally and in writing.

Learning Objective 6:
Practicum.  Candidates who complete the program are educational leaders who have the knowledge to synthesize and apply the knowledge and skills identified in state and local standards through substantial, sustained, standards-based work in real settings, planned and guided cooperatively by the institution and school district personnel for graduate credit.

Values:
Learning Objective 7:
Candidates who complete the program are educational leaders who have the knowledge and ability to promote the success of all students by acting with integrity, fairly, and in an ethical manner.

Learning Objective 8:
Candidates who complete the program are educational leaders who have the knowledge and ability to promote the success of all students by collaborating with families and other community members, responding to diverse community interests and needs, and mobilizing community resources.

Learning Objective 9:
Candidates who complete the program are educational leaders who seek to increase their education by formal and informal means.

2006 Assessment Report

Both direct and indirect measures will be used for assessment, including Comprehensive Assessment of Standards, Practicum Internship Assessment, Parallel Surveys for Alumni and Employers, Advisory Committee Surveys, and a database on graduate placements.
  
Direct:
   D1   Practicum Internship Assessment
   D2   Comprehensive Assessment of Standards

Indirect:
   Ind. 1 Alumni Parallel Survey
   Ind. 2 Employers Parallel Survey
   Ind. 3 Advisory Committee Survey
   Ind. 4 Placement (Database)

Direct Assessment 1: Practicum Internship Assessment (Objectives 1,2,3,4,5,7,8,9)

Assessment

All candidates participated in Practicum, a course designed to provide practical learning experiences relevant to elementary and secondary school principals.  Experiences included discussion of needs and problems, utilization of resource personnel, development of a portfolio, school visitations and an internship of no less than 60 clock hours.

Visitations were scheduled in Omaha Publics Schools and Millard Public Schools. Candidates “shadowed” administrators over a two-day period and submitted a narrative summary of the visitations that included information received from the schools visited.  As part of their summaries, the 29 students reflected on the challenges of diversity in large school districts compared to out-state Nebraska.  Diversity was a major topic of discussion at the final class session.

Candidates were required to develop a portfolio, an accumulation of evidence about the candidate’s intern experience. The following was included: Internship log, budget procedures, staff development plan, teacher evaluation, parent involvement, book reports, Board of Education meeting reviews, journal reviews, resume/application, book report, class presentations, and school visitations.

Both the site mentor and the faculty member evaluated the internship experience according to specific criteria in the Practicum Internship Assessment.  All students successfully completed their internship experiences.

Recommendations

To continue with the 2004 recommendation that students have the option to submit their assignments either in a bound notebook or in an electronic portfolio.

Direct Assessment 2: Comprehensive Assessment of Standards (Objectives 1,2,3,4,5)

Assessment

All master level candidates in Educational Administration were given a written comprehensive assessment of standards based on the principal and or building level learning objectives in the form of case scenarios.  All faculty members were involved in the development of the assessment that required the candidate to apply knowledge and skills to specific problems of practice.  On a rotating basis, all faculty members are involved with the assessment of the comprehensive assessment of the standards to determine the candidate’s mastery of the standards and his/her ability to apply that knowledge to a practical situation. Thirty six students completed the written comprehensive assessment, four in fall 2005, 20 in spring 2006 and 12 in summer 2006.  Results on the Comprehensive Assessment of Standards were collected and analyzed to evaluate overall program effectiveness.

 

 

 

Recommendation:

The findings indicate that our department goals and objectives are being met by our department offerings.

Indirect Assessment 1 and 2: Surveys (Objectives 1,2,3,4,8,9,10)

Assessment

To provide insights into the levels of preparedness graduates experience when becoming administrators and the strengths and weaknesses of the EDAD program in contributing to their readiness, parallel surveys were sent to employers and alumni.   Every three years parallel surveys are sent to a random sampling of UNK EDAD Program graduates who are currently practicing school administrators in Nebraska and to their Board Presidents. Results of the survey were compiled and examined to determine if there were implications for changes in EDAD programs.  Information was shared with both inter-and external audience, and for making recommendations and, if needed, program changes.

On April 1, 2005 the University of Nebraska at Kearny’s Educational Administration Department (UNK EDAD) sent out its triennial graduate survey. Completed questionnaires were received from 58 of 95 Master degree graduates for a return rate of 61%. Completed surveys were returned from 27 of 76 Specialist degree graduates for a return rate of 36%.

Two additional components were added to the 2005 survey.  Superintendents who supervised recent graduates of the UNK EDAD Masters program were asked to rate those administrators using the same areas as the self-rating questionnaire sent to the principals. Board Presidents who were on boards of education that employed recent graduates of UNK EDAD Specialist program were asked to rate those superintendents using the same areas as the self-rating questionnaire sent to the superintendents. Completed questionnaires were received from 27 of 76 superintendents for a return rate of 36%. Completed surveys were returned from 20 of 76 board presidents for a return rate of 26%.

All 2005 survey information was provided in the 2005 Assessment Report.  The next survey will be conducted in 2008.

Indirect Assessment 3: Advisory Committee Evaluation (Objectives 8, 9, 10)

Assessment

To provide statewide insights into the perceived quality of our UNK administrators, alumni and members of the Nebraska Department of Education (NDE) were invited to serve on the EDAD Advisory Committee. An agenda was mailed prior to the meeting and time was allocated for concerns or comments. At the annual meeting, committee members were requested to react to a set of questions that focused on their perceptions of UNK graduates. Responses were compiled, analyzed, and evaluated. The members of the committee rated our graduates high in all areas.

Eight committee members completed assessments. The following questions were posed: (1) Graduates have the knowledge and ability to promote the success of all students.
(2) The graduates are sought as candidates for administrative positions. (3) State educators are both aware and have respect for the EDAD graduate. Based on agree or disagree, all committee members responded, agree, on all questions. On questions four, Summative Evaluation: How would you rank the EDAD graduates at the University of Nebraska at Kearney: exemplary--3; excellent—4; and one who couldn’t decide, marked both Exemplary and Excellence

Each question contained a place for Strengths and Concerns and Suggestions. Questions one and five received most comments. To better make program improvement, we focused on the Concerns and Suggestions.

Question 1:

5. Continue to determine the best ways to balance the need for face-to-face contacts with online courses.
6. Perhaps, use local mentoring, especially for those taking courses online.
7. I like the idea of “passing the torch” by having practicing administrators identify candidates.
8. In the dark ages when I went through the program, I learned as much from students (and their experience in their schools) as I did from instructors. My concern is that efforts need to be made to generate that [student interaction] and it is difficult when not face-to-face.

Question 5:

10. Broaden perspective to national and international without losing local/state understanding.
11. Continue broadening delivery systems as technology becomes available.
12. More face-to-face opportunities interspersed with online instruction.
13. Get the students together earlier in the course works, especially if the course is delivered via internet.
14. Continue to solicit and listen to current practitioners.
15. Either a mentor concept or an emphasis on assignments that regularly require students to access direct input from experienced administrators.
16. Local mentor recruitment for all EdAd students
17. Be visible and involved in off-campus conversations about leadership in Nebraska Schools ( High school reform efforts, NDE regional and state meetings)
18. Put together an aggressive “on site” recruitment programs.

Recommendation:

The findings indicate that our department goals and objectives are being met by our department offerings; however, based on the comments, the department needs to continue to seek innovative technology to create a face-to-face environment in online classes and to continue to maintain the partnership between public school administrators and educational administration students.

Indirect Assessment 4: Placement (Objective 8)

Assessment

In addition to program-specific assessments, another measurement was included to assess program outcome effectiveness.  Job placements of candidates were recorded in a database to document program effectiveness and success for all students. 

Map depicting EDAD Graduate Employment by County, 1999- 2006.

Recommendation

The findings indicate that our department goals and objectives are being met by our department offerings.
 

Assessment of the Assessment Process

The faculty members have considered the following:

1. Is the data collected relevant to the department objectives?

   The department objectives are aligned with the College of Education (COE) Ten Desired Outcomes; COE Dispositions; COE Conceptual Framework; Nebraska Department of Education (NDE) Rule #24; and the National Policy Board for Educational Administration (NPBEA) Standards.  The assessments are aligned with the objectives; therefore the faculty felt that the data collected was relevant to the department objectives. 

2. Is the focus of the assessment based on “best practices”?

   In the Instruction to Implement Standards for Advanced Programs In Educational Leadership, it is noted: “Tomorrow’s educational leaders must be able to work with diverse groups and to integrate ideas to solve a continuous flow of problems.” The NPBEA standards provide a guide to ensure that our graduates can accomplish tasks associated with each standard and to participate in an extensive internship. 

3. Is the assessment data being used for department decision making?

   At the 2006 spring Advisory Council meeting, a concern was expressed about maintaining a face-to-face environment in an online setting.  Suggested was to maintain a close relationship with area practitioners.
   
   This fall, several department members added Audacity and the web cam to better interact with students.  With the use of Audacity, the students are able to not only read class notes and comments, but are able to hear them also.  With the use of the web cam, another dimension is added.  Students are able to see their instructors.  Department faculty members are continually moving out of their comfort zone to use current technology to create an environment of community. 

4. Are the various strategies for assessing student learning outcomes successful?

   The faculty members feel that there is an appropriate mix of direct and indirect assessment strategies. 

5. Is there a need to discontinue or add any assessment activities?

   Starting after the 2002 Triennial Graduate Survey, there had been a gradual move to offer the Educational Specialist Program online.  Currently, the Educational Specialist Program in online. 
   
   Overall, department members believe that our assessment practices are relevant, provide excellent feedback for the evaluation of our program and present direction for program changes.