Submitted Fall, 2005
I. Direct Measures
The Department of Modern Languages was able to follow its Undergraduate Assessment Plan (particularly in the case of Spanish) throughout the 2003-2004 Academic Year. Objective A was assessed for FREN/GERM/SPAN 200-201 courses using the BrighamYoung University Exam. (It also conducted at the end of Spring 2005 an oral proficiency interview of SPAN 201 students as a pilot program for General Studies Assessment, which will be discussed at the end of the undergraduate portion of this document.) Objective B was assessed in SPAN 305 (Spanish Civilization) in Fall 2004, and Objective C was evaluated in SPAN 304 (Advanced Conversation and Composition) in Spring 2005. Although FREN 304 (Conversation, Composition & Culture) was offered in Spring 2005, because of the absence of prior experience in the assessment of Objective C for French, the instructor did not conduct such an assessment. For the first time we were able to receive the Student Teacher Evaluation Forms from the College of Education so we could assess Objective D.
Objective A
In order to assess the basic communication skills at the intermediate level, students in FREN/GERM/SPAN 200 are expected to take the Brigham Young University (computer adaptive) Exam at the beginning of each semester when the courses are offered, and students in FREN/GERM/SPAN 201 need to take the same (but variable) exam at the end of this course. (Please note: these two classes satisfy Option A for the B.A. requirement and are the initial courses in most of the ML majors and minors. Also currently one of them can count for 3 hours of General Studies credit.) According to the ML Assessment Plan (see V. Rubrics and Interpretation of Data #1) “individual students are expected to improve their score by at least 10% and the average improvement should be 15%.”
In Fall 2004 and Spring 2005 six French students took both exams. In FREN 200 the scores ranged from 255 to 398 and the average score was 340.3, and in FREN 201 the range was from 376 to 475 and the average was 424.5. The average improvement was 24.7%, which is well above the expected 15%. Four of the six students exceeded the 10% individual improvement goal (the highest was by 86.3%) but two did not, making the success rate 67%. In 2004-2005 seven German students took both exams. For GERM 200 the range was from 256 to 477 and the average score was 369, while for GERM 201 the range was from 332 to 554 and the average score was 459.4. The average improvement of these students was 24.5 %, which is also higher than the expected 15%. We are pleased to report that all ten (100%) of the students improved their scores by at least 10% (the highest was by 45.2%).
Since SPAN 200 and 201 are offered every semester (unlike French and German), we were able to assess two groups: the ten students that took the two-course series in Spring 2004 and Fall 2004 and the twenty students that began in Fall 2004 and ended in Spring 2005. In the first case, the range for SPAN 200 was from 164 to 455 and the average score was 333.7, while the range for SPAN 201 was from 262 to 557 and the average score was 392. The average improvement was 17.5%, which is not high but is above the expected 15%. Of the ten students, six exceeded the 10% individual improvement goal (with the highest being 54.7%) but four did not. For the second group (Fall 2004 and Spring 2005), the range for SPAN 200 was from 133 to 383 and the average score was 294. In SPAN 201 the range was from 256 to 466 and the average score was 356.1. The average improvement was 21.1% (with the highest being 107.6%), which was better than for the previous group. Likewise for the 10% individual improvement goal the students did better because thirteen of the twenty students were successful, while seven were not.
In summary, we can observe that all three languages met the 15% average improvement goal (French 24.7%, German 24.5% and Spanish 17.5% and 21.1%). As we compare these percentages with the previous year (French 16.6%, German 20.5% and Spanish 31.8%), we can conclude that French improved the most, German was the most consistent but also improved, while Spanish actually declined. To explain the latter case I would point out that Spanish was considerably higher last year than the others, and that for some reason (perhaps due to carelessness) two students in the first group and four students in the second received worse scores in 201 than in 200. Looking at the individual 10% improvement expectation, we can note that only German was successful for 100% of its students. For French the percentage was 67 and the two Spanish groups were 60 and 65 respectively. As was suggested last year, if all students were required to receive at least a C- in these courses to satisfy their B.A., G.S, major and minor requirements, few, if any, would fall below the expected individual improvement level. Such a change, however, would be difficult because of the governing bodies involved. The Department of Modern Languages could only rule upon the majors and minors.
Objective B
In order to demonstrate the understanding and appreciation that they have acquired concerning Hispanic cultures, the students in SPAN 305 Spanish Civilization or SPAN 360 Latin American Civilization (one of which is required for all Spanish majors and minors) write a reflective essay on this general area. According to the Cultural/ Literary Rubric (see Assessment Plan V # 2; also note that the general levels are the following: l: Novice, 2: Intermediate, 3: Advanced, 4: Superior, 5: Native), “the students are expected to perform at the Intermediate-High (2.5) or Advanced (3) level .”
In Fall 2004 Dr. Hart had her twenty-eight SPAN 305 students write an essay to show their knowledge of the contributions of Spanish culture. Both she and Dr. Craig read the essays, made their assessments according the cultural rubric and came to an agreement about each essay. The scores ranged from 1.8 (Intermediate-Low) to 4.5 (Superior-High), and the students averaged 2.56 on the rubric scale. Twenty students reached the expected level of Intermediate-High (2.5) but eight did not. Thus the success rate was 71.4%. This is not as high as last year, which was 90.5%, but seems acceptable. Following the experience now of two years of applying this type of assessment, it may be advisable to revisit the rubric to modify certain aspects of its definition (e.g. those that refer to literature and more than one culture) and to define more than just two levels (Intermediate-High and Advanced) because some of the students fall above or below these. It suggested that if the prerequisite for SPAN 305/360 were raised from SPAN 201 to SPAN 204 (Culture, Conversation & Writing) the students would have a better cultural base for their civilization course and assessment and would also have improved writing skills for their essay and other exams. The department faculty agreed to this proposal, which should be implemented.
Objective C
To demonstrate their speaking and writing proficiencies in Spanish, students in SPAN 304 Advanced Conversation and Composition (which is required for all Spanish majors and minors) take both a speaking and a writing proficiency exam (see Assessment Plan V. # 3, please note again that the general levels are the following: 1: Novice, 2: Intermediate, 3: Advanced, 4: Superior, 5: Native). For speaking, “the students are expected to perform at the Intermediate High (2.5), Advanced (3) or Superior (4) level,” while for writing, “the students are expected to perform at the Advanced-Low (3-), Advanced (3), Advanced-High (3.5) or Superior (4) level” (Note: the ML Assessment Committee agreed to accept last year’s recommendation that Advanced-Low be considered sufficient).
As has been our normal procedure since 1999, the professors of the two sections of SPAN 304, Dr. Eduardo González and Dr. Herbert Craig, gave in Spring 2005 an Oral Proficiency Interview to each other’s students in order to insure reliability. Thirty-three students were interviewed. The scores ranged from 1.8 (Intermediate-Low) to 5 (Native), and the average was 2.93. Twenty-five reached the expected minimum of 2.5 (and 18 attained the next level of Advanced-Low or 2.8) but seven did not. The average of 2.93 is in the exact range of that of 2001 (2.91) and 2002 (2.96) but is lower than that of 2003 (3.33) and 2004 (3.29). The current success rate of 78.8% is better than that of 2002 (75%), but lower than that of the other years (2001: 86%, 2003: 82.8% and 2004: 93%). Although the Intermediate-High (2.5) expectation might appear to still be reasonable according to the success rate, it may be a good idea to raise this minimum level to Advanced-Low (2.8) (although for this year, this would imply only a success rate of 54.5%), as was suggested as a possibility last year. Since the Nebraska Council on Teacher Education is now considering as a proposed change to Rule 24 for Foreign Language Endorsements that the expected oral proficiency level be Advanced-Low instead of Intermediate-High, we should soon do the same for all of your 304 students.
With this goal in mind, it was suggested that the prerequisite of SPAN 304 be raised from SPAN 201 to SPAN 204 (Culture, Conversation & Writing) because this would provide the students with more conversation, as well as composition, practice before they even started 304. Since the department faculty believed in the advantageous results of this measure, it was decided to implement it.
This year only Dr. Craig gave a writing proficiency exam to the thirteen students in his section of SPAN 304, and he alone assessed their level. The scores ranged from 2.7 (Intermediate-High/Advanced) to 3.5 (Advanced-High) and their average was 2.85. Nine of the students attained the new expectation level of Advanced-Low (2.8) but four did not. The current average of 2.85 is better than that of 2001 (2.71) and 2003 (2.4), but not as good as that of 2004 (3.11) (note that this assessment was not performed in 2002). Using the new expectation level of Advanced-Low (2.8) as our point of comparison, we can see that the 2005 success rate of 69.2% is better than it would have been for that of 2001 (44.4%) and 2003 (36.4%) but not quite as good as 2004 (71.4%). We can observe that the new level (which is precisely the one stated in the proposed Rule 24) is appropriate for our students. Just the same the aforementioned decision to raise the prerequisite for SPAN 304 from 201 to 204 should also have a beneficial effect with regard to writing, as well as to speaking.
Objective D
According to the Modern Languages Assessment Plan, the standard form filled out by the supervising teacher for student teaching (Learning & the Learner, Instruction and Professional Practice) was to be used to determine the competency of student teachers. Successful student teachers were “expected to receive an average rating of 3 and should have no rating of 1.” However, since the recently instituted form uses a four point instead of a five point scale (as we were led to believe when we created our Plan) we have needed to interpret the data provided and in the near future we will have to modify the Plan itself. Also because we have not assessed this area before, we will include data for Fall 2003, as well as for Spring and Fall 2004, which constitute the most recent data available (there were no student teachers for Modern Languages in Spring 2005).
The new Student Teacher Evaluation Form is divided into three general areas: I. Lesson Planning, II. Instructional Delivery, and III. Teacher Dispositions. The four levels are termed Beginning, Progressing, Proficient and Advanced, to which we have assigned the corresponding numerical values from 1 to 4.
In Fall 2003 the one student teacher, who was working on a German 7-12 Subject Endorsement, performed very well. He received the following average ratings for the three areas: 3.25, 3.33 and 3.46, and none of his subcategory ratings was 1 or even 2. In Spring 2004 the one German student teacher did even better with average ratings of 3.75, 4.00 and 3.85, but the one in French and Spanish did worse with average ratings of 2.69, 2.72 and 3.15. She did not receive any ratings of 1 but had ratings of 2 in 13 subcategories. There were two Spanish student teachers in Fall 2004. One received good average ratings of 3.13, 3.0 and 3.23, but the other was initially quite low (2.56, 2.75, and 2.54), but apparently the university supervisor gave a higher score in a few subcategories making the final average ratings 2.68, 2.81 and 2.62. This student received one subcategory rating of 1 (for activities outside of the classroom, but her distance from the school was considered a legitimate excuse) and 16 ratings of 2. To summarize we can say that the five student teachers averaged together 3.16, 3.17, and 3.26, all of which ratings are more than proficient (and certainly above 3.0 on the 5 point scale of the original plan). Even the final weak student teacher was higher than 2.4 (which is 60% of the maximum 4, just as 3.0 is 60% of the original 5). Along with the written comments which were nearly all favorable, this data suggests that the FREN/GERM/SPAN 7-12 Subject endorsement students have been prepared to be successful foreign language teachers.
II. Indirect Measures
The survey forms for students completing an undergraduate program in Modern Languages at UNK were sent to graduating major and minor students immediately after the Fall and Spring Commencements. In the former case there were three majors and four minors, while in the latter case there was a single major and fourteen minors. Unfortunately, the forms were completed and returned by only one major and one minor for the Fall Graduation and one major and one minor for the Spring Graduation. (Perhaps a gift or incentive should be offered to the students so that they will return their forms more frequently.) Since the number is so small and the answers for the major are essentially the same as the others, they will all be considered together. In assessing their own abilities, all four graduates believed that both their speaking and writing proficiencies in the appropriate language were advanced or slightly higher (one made a mark for writing between advanced and superior). Two considered their cultural fluency advanced and another superior (the fourth student did not respond to this item). Concerning their knowledge of the literature(s) associated with the language, one believed his level to intermediate, while the other three advanced. The one endorsement student considered her knowledge of FL teaching methods to be superior. These ratings along with some written comments suggest that these students were quite satisfied with their programs in Modern Languages.