University of Nebraska Kearney

Communication Disorders Assessment Report 2006

Submitted Fall, 2006

Department of Communication Disorders
Assessment Report
August 2006

Assessment Mission Statement

Students who successfully complete programs in the Department of Communication Disorders will be responsive, reflective, and collaborative decision makers and who have the requisite knowledge, skills, and technological expertise to practice ethically and successfully.

Upon completion of the graduate program, the student in speech/language pathology will be able to:

1. Analyze ideas and make critical evaluations.
2. Understand and apply research literature to selected disciplines.
3. Demonstrate professional communication skills.
4. Understand and apply fundamental information in anatomy, physiology, neurology, and the psychology of communication.
5. Assess speech, language, and hearing performance and appropriately interpret and apply information.
6. Plan, implement, evaluate and modify educational or clinical interventions across a wide range of patients.
7. Develop effective professional relationships with patients, caregivers and other professionals.
8. Appreciate, understand and apply multicultural information in assessment and intervention.
9. Recognize the need for and participate in professional activities that promote lifelong learning.
10. Earn appropriate credential, licenses and/or certifications.

Data Report

During the academic year of 2005/06 a total of 26 graduate student majors, 12 program graduates, 3 employers of program graduates and 15 undergraduates completed assessment activities. Both direct and indirect measures were used for assessment, including comprehensive examinations, evaluation of clinical skills, and parallel surveys for alumni and employers.

Direct MeasuresClinical Competency Evaluation
Written Comprehensive Examination
Clinic, Field Placement, and Student Teaching Evaluation
Praxis Examination
Indirect MeasuresAlumni Survey Employer Survey

Assessment Interpretation and Recommendations

Direct Assessment 1: Clinical Competency Evaluation (Objectives 1,3,5,6,7,8)

Assessment

During a graduate student’s fourth semester, she/he is required to complete a clinical competency evaluation, which requires five one-hour sessions during which time the student will evaluate a client’s communication abilities and complete enough trial therapy to write comprehensive goals and objectives. Students must obtain a minimum of 176 points (out of a total of 220) in order to pass the evaluation. Two faculty members evaluate each student.

Twelve students participated in the Clinical Competency Evaluation. All students passed. The overall ratings ranged from 189 – 218 with the average at 199.6. The assessment allowed for feedback to the student and faculty members as to whether the learning objectives were being met and if there was a need for program changes. Faculty members were in agreement that the competency evaluation measured the designated learning objectives.

Recommendations

Data from this and previous administrations of the Clinical Competency Evaluation indicate that the current program is meeting the department goals and objectives included under this assessment.

Direct Assessment 2: Written Comprehensive Exam (Objectives 1,2,4,5,6,7,9)

Assessment

The comprehensive exam is a four-part exam (8 hours) covering normal processes of communication, clinical issues in children, clinical issues in adults, and professional issues. Each question is graded by two faculty members using the criteria of high pass, pass, low pass, and fail. Candidates must achieve an 80% overall in order to pass.

Thirteen students wrote comprehensive exams during the 2005/2006 academic year. All students passed with the overall ratings ranging from 85% to 97% and an average score of 91%. Category breakdowns are as follows: High Pass = 4 students; Pass = 8 students; Low Pass = 1 student; Fail = 0 students.

Recommendations

Data from this and previous administrations of this assessment indicate that the department is providing the appropriate knowledge base and clinical experiences for our students. Based on the 2004/2005 academic year, the faculty decided that information concerning verification of students for SLP services in the pubic schools should be added to several courses at the graduate level. Each of the courses (4) involve assessment and treatment of communication disorders that would be present in a public school caseload. Additionally, the information was first introduced to undergraduates in the undergraduate course, CDIS 488: Senior Seminar. As a result of adding this material to coursework, all students were able to successfully complete related questions on the comprehensive exam.

Direct Assessment 3: Clinic, Field Placement, and Student Teaching Evaluations (Objectives 1,3,4,5,6,7,8)

Assessment

Each semester clinic and field supervisors complete an evaluation of students under their supervision.

Clinic Evaluations: Each student (both undergraduates and graduates) enrolled in clinical practicum is evaluated based on performance in evaluation and treatment of clients with communication disorders. The grading scale recognizes that students new to clinical practice need more guidance and do not perform as well as students about to leave the program. Therefore, as a student progresses through practicum assignments, they are expected to score higher and higher on the scale in order to maintain an acceptable grade.

1st Semester Undergraduates – 15 undergraduates were assessed. A score of 3.05 or better is required in order to pass. The scores ranged from 3.25 to 5.95 with an average of 4.55.

2nd Semester Undergraduates – 10 undergraduates were assessed. A score of 3.80 or better is required in order to pass. The scores ranged from 3.85 to 6.10 with an average of 4.45.

1st Semester Graduate – 15 graduate students were assessed. A score of 5.0 or better is required in order to pass. The scores ranged from 5.45 to 7.35 with an average of 6.45.

2nd Semester Graduate – 14 graduate students were assessed. A score of 5.35 or better is required in order to pass. The scores ranged from 5.35 to 7.45 with an average of 6.10.

3rd Semester Graduate – 14 graduate students were assessed. A score of 5.75 or better is required to pass. The scores ranged from 6.30 to 8.55 with an average of 7.55.

4th Semester Graduate – 12 graduate students were assessed. A score of 6.50 or better is required in order to pass. The scores ranged from 7.10 to 9.00 with an average of 8.45.

 

Field Placement and Student Teaching Evaluations: During her/his last semester (fifth) of graduate study, each student must complete two internship placements, one in a clinical or hospital setting and the other in a public school setting. The site supervisors completed a survey evaluating the performance of the students who interned in their place of employment. Students were assessed on a scale from 2-Minimally competent to 5 – Extremely competent with a 1 for Not applicable. Students were rated in fifteen areas with a possible high score of 75 and a score of 60 is required in order to pass.

Twelve students were assessed and each student was assessed twice, once in each setting, yielding 24 completed evaluations. Scores ranged from 75 to 90 with an average score of 84.35.
 

Recommendations

These performance assessments allow for feedback to the students and the faculty members as to whether the learning objectives were being met and if there was a need for program changes. Department members are in agreement that this assessment adequately measures knowledge base and clinical skills of our students.

At the bottom of each form is a section for comments from supervisors. Analysis of these comments indicates that no major deficiencies were noted with our student interns. During the 2004/2005 assessments process, recommendations from the supervisors included that student interns receive more training in swallowing evaluation and tracheotomy care prior to the internship placement. The faculty modified course requirements to reflect these recommendations. During the summer of 2005, the program utilized priority monies to purchase medical simulators that allow our students to practice invasive techniques such as trach suctioning and care, nasal and oral endoscopy, and otoscopy. Practice on the simulators was integrated into two undergraduate courses, two graduate courses, and all clinical practicum at the graduate level. As a result of these modifications, none of the supervisors mentioned a deficiency in this area. Many of the supervisors are the same supervisors from last year.

Direct Assessment 4: Praxis Examination (Objectives 1,2,4,5,8,9,10)

Assessment

All master degree candidates must take the national examination required for national certification in speech-language pathology. These scores provide specific information related to the effectiveness of the program.

Fifteen students were assessed during the 2005/2006 academic year. A minimum score of 600 is required in order to pass the exam. Scores ranged from 590 to 760 with an average score of 650. Thirteen of the fourteen students passed the exam giving our program an 92.8% first time pass rate. This is well above the national first time pass rate of 78.9%. The student who failed has not retaken the exam.

Recommendations

The scores from 2005/2006 academic year indicate that our program is adequately preparing students for the national certification exam. Unfortunately, only scores are reported to us. It would be helpful if specific test items that were missed were reported.

Indirect Assessments 1 and 2: Alumni Survey and Employer Survey (Objectives 1,2,3,4,5,6,7,8,9,10)

Assessment

To provide insights into the levels of preparedness graduates experience and the strengths and weaknesses of the CDIS program in contributing to their readiness, parallel surveys were sent to alumni and their employers one year after date of graduation. The department has been conducting these surveys for over ten years and has used the information gathered to make curricular clinical changes in the program.

The alumni survey was sent to 13 program graduates one year after their date of graduation and 9 surveys were returned. Responses from graduates indicated that they would have preferred more information on the following: preparation of reports in a school setting; collaborative intervention, and autism. The students also commented on a need for more emphasis on specific treatment in all areas

Recommendations

Based on the results of the 2004/2005 survey, the faculty made some minor curriculum changes. A pediatric dysphagia course was added for the Fall 2005 semester. Additionally, the department contracted with a specialist in autism who offered an on-campus workshop for our students. These opportunities provided students with information they indicated was needed.

The employer survey was sent to 10 employers and 3 were returned. Employers generally rated program graduates as excellent. Employers indicated that students are generally knowledgeable in most areas, but have limited experience in IEP mangagement.
 

Recommendations

Program faculty addressed the need for incorporating information regarding case load management and IEP preparation and management following the 2004/05 assessment process. This information has been added to several courses which address assessment and treatment of specific communication disorders. As a result, supervisors and employers did not rate this as an area needing improvement.

Assessment of the Assessment Process

During the spring semester of 2006 the faculty of the department considered the following points:

• Are the desired outcomes for the graduates of our department relevant and defensible? The faculty of the department believes the answer to the above question is yes. 
• Does the current means of assessing actually assess the department’s desired outcomes for graduates and provide information that allows for the continuous improvement of our program? Given that the process allows for assessment to occur at both the course level and at the end of the program (internships) the faculty believe that data collected will allow for improvement of the program. 
• Is the scope and focus of our assessment process reasonable? The faculty believes that while the process is time consuming it is reliable. We have been required to do this assessment for a number of years in order to maintain accreditation. Analysis of our assessments over the past seven years has prompted us to review and update our curriculum and clinical experiences.
   
• Do we need to discontinue or add any assessment activities? We have been examining our curriculum and clinical experiences in light of the 2005 ASHA standards. The new standards require us to document student outcomes in very specific ways. At this point, we believe that our assessment practices are strong and provide excellent feedback to our program. We feel that our assessment activities clearly meet the new standards.