University of Nebraska Kearney

Special Education: Gifted Graduate Program Assessment Report 2006

Submitted Fall, 2006

Distance Special Education: Gifted Assessment Report
Academic Year 2005-2007

Mission

Master degree students graduating from the Department of Teacher Education's graduate programs will demonstrate an understanding of the manner in which perspectives and beliefs influence instructional behaviors, extend their knowledge of the central concepts of selected disciplines, and obtain and use knowledge and skills which will support increased instructional effectiveness with diverse learners.

Graduate Program Objectives

1. Students will demonstrate their leadership and administrative skills, which include promoting and implementing professional and state standards;
2. Students will effectively design instruction and materials for all learners;
3. Students will demonstrate how to use a variety of productivity, research, and software tools in various instructional settings;
4. a. Students will be able to apply, analyze, synthesize and evaluate critical and contemporary issues; and,
   b. Students will be knowledgeable consumers and generators of research.

Special Education: Gifted Specialization Objectives

1. Students will demonstrate the ability to design cognitively complex learning experiences for gifted learners based on their cognitive and affective characteristics.
2. Students will demonstrate the ability to design guidance/counseling strategies to facilitate the social and emotional development of gifted learners based on their cognitive and affective characteristics.
3. Students will demonstrate the ability to select and apply a combination of standardized and informal methods of assessment to identify gifted learners.
4. Students will demonstrate the ability to develop programs for gifted learners based on philosophical, theoretical, and empirical evidence and understand the management of such programs.
5. Students will demonstrate the ability to develop and disseminate materials to increase educator, parent, and community understanding of gifted learners and their cognitive and affective need for appropriate programming.
6. Students will demonstrate the ability to work effectively with K-12 gifted learners, their parents, educational colleagues, and other stakeholders in the community.
7. Students will demonstrate the ability to analyze, synthesize, and evaluate seminal research and original inquiry within the field of gifted education.

Data Report

The majority of students obtaining gifted specialization are focusing on the endorsement and only a small percentage obtain the masters degree. As of late spring 2006, there has been an increase in masters-level students because both the endorsement and master’s degree are being advertised on eLearners. Nevertheless, it is essential to assess both of these groups of students in order to obtain an accurate picture of the effectiveness of the 18 hour endorsement which constitutes one half of the Masters degree. Direct measures assess student products that demonstrate mastery of major course objectives. Two courses include graduate students who are obtaining professional development in the area of gifted education (TESE 822P and TESE 823P). It is not always possible to know who these students are, and because their ability to master course objectives and demonstrate through key products that they too have gained an understanding of the cognitive and affective needs of gifted learners is equally important, their data are included in this report where appropriate.

During the 2005 – 2006 academic year, five program objectives were evaluated by a minimum of 10 to a maximum of 17 graduate students, depending on which direct measure was being employed since these assessments are tied to the demonstration of concepts and skills through class projects. Since only two masters degree students and two endorsement-only students completed their programs this academic year, they were not assessed using indirect measures because anonymity wasn’t possible. These students and their immediate supervisors will be evaluated with the group that complete their programs in the 2006-2007academic year.
 

Direct Measures

Affective Plan Rubric
Case Study Rubric
Identification Checklist
Needs Assessment Rating Scale
Program Design Checklist
Program Evaluation Rating Scale
Program Management Criteria Rating Scale
Public Relations Rating Scale
Research Paper Rubric
Staff Development Rating Scale

Indirect Measures

None were employed during this academic year.

Assessment Interpretation and Recommendations

Direct Measures (Special Education: Gifted Specialization Objectives #2)

Methodology: 

Each student completing the Affective Plan during the fall of 2005 was evaluated by the Affective Plan Rubric in December, 2005. Program Objective #2 was being evaluated:

2. Students will demonstrate the ability to design guidance/counseling strategies to facilitate the social and emotional development of gifted learners based on their cognitive and affective characteristics. (Departmental Objectives 2, 4a)

Students were required to develop an affective plan to help gifted/talented learners build a positive self-concept, cope with stress, avoid underachievement, and/or learn acceptance of their unique characteristics. This was a class syllabus requirement for all graduate students in the endorsement class TESE 823P Guiding the Social and Emotional Development of the Gifted/Talented Learner.

Results: 

Conclusions: 

1. These are very positive results; all 14 graduates met the Acceptable level on the Affective Plan and 13 of those students produced Commendable level work.

Use of Results: 

1. The project that addresses this skill must, of necessity, be very open-ended allowing a range of responses. I have modified the directions several times to make them sufficiently specific and clear. Within those parameters, there is a lot of latitude. As a result, some projects are far more elaborate than others. All projects that meet the criteria described in the rubric receive high evaluations, regardless of complexity. My expectation is that most of the students should be able to produce an acceptable affective plan, even if this is their first class on gifted education, provided they follow the directions. These results tell me that I am teaching the skill well and my directions are clear.

Methodology: 

Each student completing the Case Study during the fall of 2005 was evaluated by the Case Study Rubric in December, 2005. Program Objective #2 was being evaluated:

2. Students will demonstrate the ability to design guidance/counseling strategies to facilitate the social and emotional development of gifted learners based on their cognitive and affective characteristics. (Departmental Objectives 4a, 4b)

Students were required to develop a Case study of an individual student to relieve academic underachievement or investigate academic and career options. This was a class syllabus requirement for all graduate students in the endorsement class TESE 823P Guiding the Social and Emotional Development of the Gifted/Talented Learner.

Results: 

Conclusions: 

1. Nine students achieved at the Acceptable level and five of them produced Commendable level work. I am very pleased so many achieved at the highest level. This is a very difficult assignment requiring multiple level skills in areas that may be very new for some students. That said, only 9 of 12 (75%) students completing the project achieved at the acceptable level or better.
2. Two students did not complete their Case Study during the 2005-2006 academic year and 3 students did not achieve at an acceptable level. This means five of the 14 students (35.7%) did not perform acceptably. Even though this assignment is complex, these results are not acceptable.

Use of Results: 

1. These results clearly indicate that I need to improve the directions and support for this skill. The directions in the syllabus are explicit; however, some of the requirements for this project are not included in the case study format as it is used with permission by the developer. All three of the students who rated Unacceptable omitted some or all of the materials that were not included in this model, despite the components being listed in the syllabus detailed directions and in the directions posted on Blackboard. I will revise the case study format packet to include these 4 components. Two of the students’ would very likely have ranked at the Commendable level if this practice had been in effect as their case studies were only 3 points below the Acceptable level between them.
2. I don’t like incomplete assignments, but they are difficult to avoid without being insensitive to students’ needs. They frequently have unexpected challenges in their lives that make it difficult if not impossible to complete the class. Nevertheless, developing supports to promote completion of all projects in this and other courses is a necessary revision to course/program policies.

Methodology: 

Each student completing the Identification Procedures assignment during the spring of 2006 was evaluated by the Identification Checklist in May, 2006. Program Objective #3 was being evaluated:

3. Students will demonstrate the ability to select and apply a combination of standardized and informal methods of assessment to identify gifted learners. (Departmental Objectives 4a)

Students were required to develop identification procedures for their target population based on the results of their needs assessment. This project was required to include a multi-level set of procedures for identifying children who need differentiation or special programs resulting from specific skill strengths. Procedures needed to be included for finding students who might not be recognized in the regular classroom. The identification plan was expected to be multidimensional and inclusive at each level. Requirements are the same as in TESE 824. This assignment was a class syllabus requirement for all graduate students in the endorsement class TESE 826 Program Design for Gifted/Talented Education.

Results: 

Conclusions: 

1. This was a complex task that required a radical paradigm shift from the students. Eleven students completed the assignment at or above the Acceptable level; ten at the Commendable level.
2. Two students’ work was unacceptable, one because it was incomplete (she quit participating for several weeks and the other apparently had not fully understood the assignment.
3. Seven of the 10 at the Commendable level had completed the assignment in TESE 824 as had the student with the Acceptable level, so this was a chance for them to improve their understanding. For three of the students producing Commendable work, this was a new assignment. On the whole, I was please with these results.

Use of Results: 

1. This was a group assignment. For most of the students, that helped them build their understanding. I think the main problem for the one student who really worked at the assignment and still earned just under the acceptable level was that he needed the more extensive background on identification provided by TESE 824.
2. This is a class sequence problem. Despite students needing the TESE 824 Identification, Assessment, and Evaluation class prior to TESE 826 Program Design, I have had to allow students to take TESE 826 first because it allows for a more flexible course rotation. The students who take TESE 824 last should then demonstrate at least the Acceptable level work and I should be able to confirm that when the class is taught the summer of 2007. I was aware when I changed the rotation that particular skill would be affected. Despite this arrangement not being the most educationally sound, I will continue it and monitor the results since I am unable to offer more than one class a semester and the flexibility has provided a needed increase in enrollment.
3. The results from the summer 2005 TESE 824 class were one Acceptable, three Commendable, and two Incomplete. The two incompletes have since been cleared; one was Commendable and the other Unacceptable. This was their first exposure to the assignment. The student whose work was unacceptable appeared to benefit from working in a group for the current project, but I am not sure she could produce Acceptable work on her own. For that reason, I do not think I will use the group approach again. This will create a problem that I will have to solve when several students work in the same school district. I think what I will do is allow the students to complete the TESE 826 Needs Assessment together, and then develop the remaining materials separately. 

Methodology: 

Each student completing the Needs Assessment in the spring of 2006 was evaluated by Needs Assessment Rating Scale. It was used to evaluate each student in May, 2006. Program Objective #4 was being evaluated:

4. Students will demonstrate the ability to develop programs for gifted learners based on philosophical, theoretical, and empirical evidence and understand the management of such programs. (Departmental Objectives 1, 4a)

Students were required to prepare and conduct a needs assessment for their school or district that included two or more populations in the school, two or more methods of gathering data, how to compile both quantitative and qualitative results into a meaningful report, analyze the results, prioritize the major needs, indicate which needs were possible and which would have to be deferred along with the reasons for deferring them. This was a class syllabus requirement for all graduate students in the endorsement class TESE 826 Program Design for Gifted/Talented Education

Results: 

Conclusions: 

1. Some of the students worked in groups while two students preferred to work alone. All thirteen students achieved at the Acceptable level or better, and 12 achieved the Commendable level. These results demonstrate that the students understand how to conduct, analyze, and prioritize the results of a small scale needs assessment.

Use of Results: 

1. This task was not easy as most teachers have no background conducting such an assessment. Nevertheless, these teachers rose to the challenge and submitted excellent work. I do not plan to make any changes at this time. I think it is more productive to focus on the use of the data, which is assessed in the following sections of this report.

Methodology: 

Each student completing the Program Design in the spring of 2006 was evaluated by Program Design Checklist. It was used to evaluate each student in May, 2006. Program Objective #4 was being evaluated:

4. Students will demonstrate the ability to develop programs for gifted learners based on philosophical, theoretical, and empirical evidence and understand the management of such programs. (Departmental Objectives 1, 4a)

Students were required to design a program of services for gifted/talented learners including philosophy and rationale for the program, definition of giftedness, goals, curriculum model(s), methods of differentiating the curriculum, and program delivery format(s). This was a class syllabus requirement for all graduate students in the endorsement class TESE 826 Program Design for Gifted/Talented Education.

Results: 

Conclusions:

1. Twelve of the 13 students worked at the Acceptable level with 9 of those students at the Commendable level. These results demonstrate that all but one of the students understand how to design a coherent program of services for gifted/talented learners that included the basic components.

Use of Results: 

1. The program design task is highly complex since each component needs to be based on the previous ones and be appropriate for gifted learners within their districts. This skill requires an in depth understanding of all the major areas of gifted education and must demonstrate a clear connection to the results of their needs assessment. I do not plan to make changes in the assignment or its directions; however, I will not allow group assignments. As noted above, they can work together on the needs assessment as that is difficult to do in some districts, but the program design will need to be individual so that I can be sure that each student knows the skill.
2. The one student who did not submit an acceptable product has a severe learning disability which affects each of his written products. Orally he was able to explain the connections; however, he was not able to demonstrate his knowledge on the written work even with two opportunities to rewrite.

Methodology: 

Each student completing the Program Evaluation Project in the spring of 2006 was evaluated by Program Evaluation Rating Scale. It was used to evaluate each student in May, 2006. Program Objective #4 was being evaluated:

4. Students will demonstrate the ability to develop programs for gifted learners based on philosophical, theoretical, and empirical evidence and understand the management of such programs. (Departmental Objectives 1, 4a)

Students were required to develop the beginning components for evaluating their program using the Borland model and at least one other method. This was a class syllabus requirement for all graduate students in the endorsement class TESE 826 Program Design for Gifted/Talented Education.

Results: 

Conclusions: 

1. These results were very poor even for a difficult still. Only four students completed the project at the Acceptable level (33.3%), three of whom reached the Commendable level.
2. The work of eight students was Unacceptable and another did not complete the work in time for this evaluation. Three of the Unacceptable results as well as the Incomplete were from students who had not yet taken TESE 824 in which the Borland model was also taught. However, one of the Commendable results was by a student who had not yet taken TESE 824. Five of the Unacceptable results were by students who had also completed the assignment in TESE 824,

Use of Results: 

1. While this assignment is one of the most difficult because of the level of analysis and evaluation required to complete it, the results are still disturbing. The biggest problem was that students did not follow the directions carefully even though they were clearly written and explained. There were many difficult projects required as part of this class and it is possible that the students who performed poorly did not allot enough time to this last assignment. When I next teach this skill, I will spend more time having the students practice using the Borland model in class. That is the most difficult part of the assignment, yet it is crucial because it is a direct measure of student learning.
2. One cause of Unacceptable results for several of the students was not including the second, indirect method of evaluation. That part was easy; it just was omitted. I don’t know how I can be more explicit about the directions.
3. This assignment has been one of the last for this class. It can be moved earlier in the semester as it is not depended on the Needs Assessment data. Therefore, students can work on this while they are waiting for all the data to be collected. That way the skill can be taught multiple times if necessary. Also, it the indirect measures are discussed first and they discuss these measures on Blackboard before we move on to the Borland model, each part may receive enough attention. I will make changes in the syllabus and course structure to incorporate these changes.

Methodology: 

Each student completing the Management Outline in the spring of 2006 was evaluated by Program Management Criteria Rating Scale. It was used to evaluate each student in May, 2006. Program Objective #4 was being evaluated:

4. Students will demonstrate the ability to develop programs for gifted learners based on philosophical, theoretical, and empirical evidence and understand the management of such programs. (Departmental Objectives 1, 4a)

Students were required to craft an outline of a plan for managing their gifted program, including roles of key personnel, means of communicating with constituents, budget, facilities, and method(s) for evaluating strengths and weaknesses. This was a class syllabus requirement for all graduate students in the endorsement class TESE 826 Program Design for Gifted/Talented Education.

Results: 

Conclusions: 

1. Ten of the thirteen students submitted Commendable level work. This was not a difficult assignment as long as they followed the directions.
2. One student forgot to include this portion of the class assignment and two students were not involved in the group decision making for this project, a good reason not to use groups since I have no way of knowing if they can complete the skill satisfactorily.

Use of Results: 

1. Those students who worked in groups were supposed to work together on all parts, but one group did not on this and several other assignments. The only change I intend to make is to require individual work.
2. I already have a thorough listing of all the components that need to be included for the full portfolio of skills that need to be demonstrated in this class. One student didn’t submit this project with everything else. In order to avoid this kind of a problem and a few others too, I will have students submit portions of the requirements throughout the semester. There are many advantages to having a all the components in a single portfolio, but there are disadvantages too. They can build their portfolios as they submit their separate components to me.

Methodology: 

Each student completing the (Tiered) Staff Development Plan in the spring of 2006 was evaluated by Staff Development Rating Scale. It was used to evaluate each student in May, 2006. Program Objective #5 was being evaluated:

5. Students will demonstrate the ability to develop and disseminate materials to increase educator, parent, and community understanding of gifted learners and their cognitive and affective need for appropriate programming. (Departmental Objectives 1, 4a)

Students were required to develop a (tiered) staff development plan for their school that included a description of target group’s experience level, a bulleted outline of a series of four staff developments, an explanation of how they would differentiate each staff development for different entry levels of experience with gifted learners, a description of the proposed outcomes, an explanation of how the plan met district needs as identified in the needs assessment, and a brief description of next year’s topic and the reason(s) for the choice. This was a class syllabus requirement for all graduate students in the endorsement class TESE 826 Program Design for Gifted/Talented Education.

Results: 

Conclusions: 

1. Ten students’ work was Commendable and I am please that they can use their accumulated knowledge to plan staff development for their colleagues.
2. Two students as a group submitted a product that showed they had not read the directions carefully or had not taken them seriously. Their revision was not satisfactory either. One student did not participate with his group on this assignment so I do not know whether he understands the skill or not.

Use of Results: 

1. The directions were clear and the concepts were not new. I will have students submit individual products after discussing them among themselves on Blackboard. Peer direction seems to be an effective for method for building understanding in my Educational Research classes, so I will make use of it here too.

Methodology: 

Each student completing the Program Design in the spring of 2006 was evaluated by Public Relations Rating Scale. It was used to evaluate each student in May, 2006. Program Objective #5 was being evaluated:

5. Students will demonstrate the ability to develop and disseminate materials to increase educator, parent, and community understanding of gifted learners and their cognitive and affective need for appropriate programming. (Departmental Objectives 1, 4a)

Students were required to develop a public relations framework for gifted educational services in their school or district. They were to describe two things that could actually be carried out and the target populations that met the following criteria: Each activity must improve the understanding of parents, educators, and/or the general public in the community about the needs of gifted and talented learners. One activity was to be for use during the current school year and the other to be used at the beginning of next school year. In addition, they were to describe how they would involve others in advocating for gifted learners and their programs and identify key people to serve as members of an advocacy group for gifted education and their responsibilities. This was a class syllabus requirement for all graduate students in the endorsement class TESE 826 Program Design for Gifted/Talented Education.

Results: 

Conclusions: 

1. Eleven students worked at the Commendable level. This is not a difficult assignment, so I expect at least the acceptable level.
2. Two students from one group did not participate in this assignment so I do not have an indication of their competence with this skill.

Use of Results: 

1. As stated above, these results demonstrate the problems with group projects. They can work well, but didn’t with this group in particular. The assignment will be individual in the future. There is no reason why all students could not achieve at the Commendable level as demonstrated from past classes.
2. This is another skill that can be completed early in the semester, and might be a good one to use at the beginning as a bridge from previous classes. Discussions from all the previous classes have shown that the students want the opportunity to advocate for gifted learners. That is the purpose of this assignment, to increase the knowledge of various publics about gifted learners and their educational needs.

Methodology: 

Each student completing the Expert Research Paper during the fall of 2005 was evaluated by the Research Paper Rubric in December, 2005. Program Objective #7 was being evaluated:

7. Students will demonstrate the ability to analyze, synthesize, and evaluate seminal research and original inquiry within the field of gifted education.

Students were required to identify a topic of interest related to the content of the class, conduct a search of the professional literature, and write an “expert” paper using APA format. This was a class syllabus requirement for all graduate students in the endorsement class TESE 823P Guiding the Social and Emotional Development of the Gifted/Talented Learner.

Results: 

Conclusions: 

1. Eleven of 14 students, or 78.6%, were at or above the Acceptable level, and nine (64.3%) achieved the Commendable level. These students’ results suggest that between my teaching and the support for distance access library research provided by the UNK library, particularly Sheryl Heidenreich, students are producing at an appropriate level.
2. The three students at the Unacceptable level are a concern. One student has a language disability and struggles with all written products. The other student whose work was judged Unacceptable has continually struggled with her fear of writing papers of any kind. I worked with her over several semesters to help her have enough confidence to submit anything. She needs more assistance than I can provide. Both her written products and oral presentations demonstrate limited skill in self expression. By her own admission, this has been a life long problem. The Incomplete grade as from a student who dropped out without officially dropping the class. She did not respond to my emails or calls.
3. These results are comparable, although not quite as strong, as those reported for the 2004-2005 academic year based on essentially the same assignment with similar students in TESE 822P The Psychology and Education of Gifted and Talented Learners. Of the 22 students in that class, 19 (86.4%) achieved at least the Acceptable level and 13 (59.1%) were Commendable. In this class there were also three incompletes.

Use of Results: 

1. I have already provided more supportive guidelines and library links in the second teaching of TESE 822P since I began officially evaluating student skills. Those changes will be incorporated into TESE 823P.
2. I stated in my “Use of Results” for TESE 822P that I would “work with Sheryl to provide additional supports for the research activities of off-campus students.” I asked her and Steve McGahan, Continuing Education’s lead Instructional Design Specialist, to develop high quality videos demonstrating some of the key database search skills. They produced some excellent videos that I have included them in my Library Resources folder since they were developed in late fall 2005. These videos have helped my Educational Research students who also need to conduct library research. I will make the videos available to the TESE 823P students.
3. To assist students who are new to professional writing such as those in TESE 823P (and TESE 822P), I have begun posting the basics of APA format for references and citations in text within the weekly Assignment directions a few weeks into the period when they are gathering their research and noting their sources. I began doing that in my Educational Research classes because so many students didn’t bother to make use of the excellent on-line resources that I provided. Because their work has improved as a result of this extra guidance, I will incorporate the support in TESE 823P.
4. My biggest concern after general research and writing skills is preventing incompletes. Adult students have babies and experience serious health or other problems within their personal lives that interfere with regular class work on occasions, and I hesitate to punish them for these challenges beyond their control. I do not want to add unnecessary barriers to their increasing their education. I still want to prevent incompletes as much as possible without being inflexible. To this end, I will build in additional structure to Blackboard by using the model I have incorporated into my Educational Research classes for two semesters as well as the tighter timelines and small group discussion groups that report to the full class for further processing of difficult content. I have clear evidence that these changes work with students with essentially the same level of experience with graduate classes; therefore, they should also help the students in TESE 823P.

Assessment of the Assessment Process

Lessons gained from this second official assessment of the effectiveness of the Special Education: Gifted Specialization graduate program and endorsement only program include the following: 

• The desired outcomes for graduates in this program, both masters level and endorsement level, are at an acceptable level assessed by direct measures. 
• Direct measures data are collected at the course level in a manner that is valid because class assignments that demonstrate the assessed skill are used to collect the data. The reliability of the data is reasonable since the same individual assesses all students’ work. This method of data collection is not a burden on the instructor or program evaluator, thus making the method easy to maintain. Equally important, data collection gathered from the evaluation of major class products facilitates identifying areas of strength and those requiring improvement within course instruction itself.  
• One direct measure was not officially assessed in this report although that was the original intention. Last spring, three students completed the practicum, whose products and observation assess Departmental Objective 6: Students will demonstrate the ability to work effectively with K-12 gifted learners, their parents, educational colleagues, and other stakeholders in the community. Midway through the semester I was informed that the new CEC/NAGC Standards that I have helped develop would be presented at the April 2006 University Networking Conference In Washington, DC. Because I would immediately revise both the practicum and TESE 822P for fall 2006, it did not seem practical to collect data with one set of instruments that would not be comparable with the next semester’s data since the revisions are extensive. The new assessment instruments, Teaching Observation Rubric and Reflection Paper Rubric, are included with this report and are guiding the instruction of the students in the fall 2006 practicum class. Although the Standards will not be officially presented to the members of NAGC until the national conference beginning November 1, 2006, I saw no useful reason to wait implementing them. 
• Indirect measures data were not collected this assessment year because only two students completed either the endorsement and two students completed the Masters program. Their data will be collected and combined with the data in the 2006-2007 report to provide anonymity for both groups.  
• One critical issue with the indirect measures needs to be addressed before the 2006-2007 gathering of data. The 12 credit hours of core classes were not assessed in last year’s report. This was an oversight that was recognized at the time of submission of the 2005-2006 report. It will be corrected by the next report. The Teacher Education Department Graduate Committee of which I am chair is developing a common assessment of these core classes that will be employed in all of the graduate programs. Data will then be analyzed from all the programs to determine if there are any differences that might improve the whole. Students in the Gifted Education endorsement program will not be affected.