Submitted Fall, 2004
Direct Measurements
The Department of Modern Languages has followed carefully its Undergraduate Assessment Plan throughout the 2003-2004 Academic Year. Accordingly, Objective A was assessed for French, German and Spanish using the BYU Exam, Objective B was assessed for Spanish in SPAN 360 (Latin American Civilization) in Fall 2003, and Objective C was assessed for Spanish in SPAN 304 (Advanced Conversation and Composition) in Spring 2004. Since neither French nor German offered the corresponding courses, assessments for Objectives B and C could not be conducted.
In order to assess the basic communication skills at the intermediate level, students in FREN/GERM/SPAN 200 are expected to take the computerized Brigham Young University Exam at the beginning of each semester and students in FREN/GERM/SPAN 201 need to take the same (but variable) exam at the end of this course. (Please note: these two classes satisfy the B.A. requirement and are the initial courses in most of the ML majors and minors. Also currently one of them can count in the General Studies program.) According to the ML Assessment Plan (see V. Rubrics and Interpretation of Data #1) "individual students are expected to improve their score by at least 10% and the average improvement should be 15%."
In Fall 2003 and Spring 2004 five French students took both exams. (We should note at the outset that this sample is quite small, and a low score by one or two students can greatly affect the results, but these are the only figures that we have.) In FREN 200 the scores ranged from 67 to 480, and the average score for these students was 207.4. In FREN 201 the range was from 117 to 445 and the average was 241.8. The average improvement was 16.6% which reached the assessment goal of 15%. Three of the five students exceeded the 10% individual improvement level (and some by much more), but two did not. In 2003-2004 four German students took both exams. For GERM 200 the range was from 333 to 477 and the average score was 379, while for GERM 201 the range was from 388 to 508, and the average score was 456.8. This means that the average improvement was 20.5 %. Three of the students reached the 10% individual goal but one did not. In Fall 2003 and Spring 2004 twenty-two Spanish students took both exams. For SPAN 200 the range was from 135 to 412, and the average score was 265.3. For SPAN 201 the range was from 229 to 517, and the average score was 349.7. Thus for Spanish the average improvement was 31.8%. Of the 22 students, 16 exceeded the 10% improvement goal but 6 did not. In short, the average improvement goal of 15% was met, especially for German and Spanish, but there were some individual students that did not meet the expectation for them (40% for French, 25% for German and 27% for Spanish). Our intermediate language courses are doing a good job of helping the students improve their level of basic communication skills, but a few students need to strive harder. Perhaps if all students were required to receive at least a C- in these courses to satisfy their B.A., G.S., major or minor requirements, few, if any, would fall below the expected individual improvement level.
In order to demonstrate the understanding and appreciation that they have acquired concerning Hispanic cultures, the students in SPAN 305 Spanish Civilization or SPAN 360 Latin American Civilization (one of which is required for Spanish Majors and Minors) write a reflective essay on this general area. According to the Cultural/Literary Rubric (see Assessment Plan V # 2, also note that the general levels are the following: l: Novice, 2: Intermediate, 3: Advanced, 4: Superior, 5: Native), "the students are expected to perform at the Intermediate-High (2.5) or Advanced (3) level ."
In Fall 2003 twenty-one SPAN 360 students wrote the required general essay on Hispanic cultures. The scores ranged from 1.5 (Novice-high) to 3.5 (Advanced-high), and the students averaged 2.7 on the rubric scale. Nineteen reached the expected level of 2.5 but two did not. Thus the success rate was 90.5%. This was the first time that we have performed this type of cultural assessment, but the results appear to be satisfactory.
To demonstrate their speaking and writing proficiencies in Spanish, students in SPAN 304 Advanced Conversation and Composition (which is required for all Spanish Majors and Minors) take both a speaking and a writing proficiency exam (see Assessment Plan V. # 3, also note that the general levels are the following: 1: Novice, 2: Intermediate, 3: Advanced, 4: Superior, 5: Native). For speaking, "the students are expected to perform at the Intermediate High (2.5), Advanced (3) or Superior (4) level," while for writing, "the students are expected to perform at the Advanced (3), Advanced-High (3.5) or Superior (4) level".
As has been our normal procedure since 1999, the professors of the two section of SPAN 304, Dr. Lon Pearson and Dr. Herbert Craig, gave in Spring 2004 an Oral Proficiency Interview to each other's students in order to insure reliability. Twenty-nine students were interviewed. The scores ranged from 1.8 (Intermediate-low) to 4.9 (Native-low/Native) and the average was 3.29. Twenty-seven reached the expected minimum of 2.5 (and 22 reached the next level, Advanced Low, of 2.8) but two did not. The average of 3.29 is nearly as high as that for 2003 (3.33) and better than the preceding two years (2001: 2.91, 2002: 2.96). Furthermore the success rate of 93% is higher than any of the previous years (2001: 86%, 2002: 75%, 2003: 82.8%). As in the Assessment Report of Fall 2003, we can conclude that the Intermediate-high goal of 2.5 is still reasonable, but if the improvement continues, we may wish to raise this goal to Advanced-low (2.8). Also we can conclude that the oral practice that the students receive in our advanced classes (especially SPAN 304) is quite good.
This year both Dr. Pearson and Dr. Craig gave a writing proficiency exam to their SPAN 304 classes. Each rated and graded his own students, but each professor checked the exams and ratings of the other. In a few cases the second instructor suggested slight alterations, and some scores were modified accordingly. The scores for the twenty-eight students tested ranged from 1.8 (Intermediate-low) to 4.25 (Superior/Superor-high) and the average was 3.11. Eighteen students reached the 3.0 (Advanced) goal while 10 did not. The success rate was only 64.3%; thus, as in the previous Assessment Report, the Committee Chair suggested that we consider lowering the goal to be more like that of Speaking Proficiency: 2.5 (Intermediate-high). This would produce a 92.9% success rate, while a goal of Advanced-low (2.8) would render a 78.6% success rate.
For the sake of comparison, we should note the averages and success rates (with a 3.0 goal) for 2001 and 2003 (the only years for which we have data). The former averaged 2.71 and had a 70% success rate, while the latter averaged 2.4 and had a 54.5% success rate. Dr. Pearson thought that the better performance of his students was due to their monitoring and correcting of each other's journals. This procedure, (which we suggested in the December 2003 Response to the ML Assessment Report Evaluation) should definitely be attempted in both sections of SPAN 304 next spring. The Committee Chair proposed that the goal for Writing Proficiency be lowered from Advanced (3.0) to Advanced-low (2.8), and the Assessment Committee agreed, so this change will be incorporated into the Assessment Plan.
Indirect Measurements
The new form of the Senior Surveys, which allows the majors and minors from the Department of Modern Languages at the time of their graduation to assess their own Speaking, Writing and Cultural Fluency for their chosen language(s), as well as their knowledge of the appropriate literature(s), language teaching methods and/or translation-interpreting skills, have now been distributed for three semesters. These are sent with a cover letter immediately after each graduation ceremony and are returned voluntarily by the former students.
In Spring 2003 Senior Surveys were sent to eight graduating majors and eleven minors. Four of the former and two of the latter completed and returned their surveys. The four majors distributed equally their self-ratings for Speaking Proficiency, that is, one marked intermediate, one marked advanced, one marked superior and one marked native-like. In contrast, all four believed that their Writing Proficiency was superior. Concerning Cultural Proficiency, one was rated as advanced, two were superior and one was native-like. In the area of literary knowledge one was intermediate, two were advanced and one was superior. For teaching methods one was advanced and two were superior, for T-I two were advanced and one was superior.
Thus, although one of the majors did not rate her Speaking Proficiency level as even advanced (which might be expected for a language graduate), the others saw themselves within the normal range. Furthermore they are all apparently confident about their Writing and Cultural Proficiency. With one exception for literature, they saw themselves as having an advanced or superior knowledge of the three areas surveyed. In contrast, the two minors that returned the survey saw their Speaking, Writing and Cultural Proficiency, as well as the knowledge areas, as being merely intermediate in all cases.
After the Fall 2003 graduation, Senior Surveys were sent to five majors and three minors, but only one minor returned her survey. She considered her three Proficiency areas to be advanced and her three knowledge areas to be intermediate.
At the end of Spring 2004, Senior Surveys were sent to eleven majors and two minors. As of September 24, only three majors had returned their forms. Two of them considered their Speaking Proficiency to be superior and one native-like. One considered her Writing Proficiency to be advanced and two superior. All three saw their Cultural Proficiency to be superior. One believed her knowledge of literature to be only intermediate, but the other two found it to be superior. Only two had studied teaching methods and thought their knowledge of this area to be superior. For translation-interpreting one was advanced and two were superior. In short, these three Spring 2004 graduates are apparently very satisfied with their proficiencies and areas of knowledge because their most frequent self-rating was superior.