University of Nebraska Kearney

Philosophy Assessment Report 2006

Submitted Fall, 2006

Philosophy Program Assessment Report 2006

I. Objectives

The Philosophy Program Assessment Plan is designed to measure the following objectives for students:

1. The ability to comprehend a variety of philosophical texts and ideas.
    
2. The ability to see the implications of, and draw logical conclusions from, philosophical ideas, especially as these ideas relate to their own lives.
    
3. The ability to demonstrate critical thinking through clear written communication.
    
4. The ability to apply basic ethical principles to moral issues arising within one’s private and professional life.
    
5. The ability to understand the modes of inquiry used in various disciplines and to view these from a critical interdisciplinary perspective.
    
6. The ability to appreciate the intellectual achievements of geographically and temporally diverse thinkers and cultures.
    
7. The ability to recognize and analyze intellectual problems and to propose possible solutions to them.
    
8. A commitment to the value of a critical analysis and comprehension of philosophical texts and ideas as they bear on self-examination.
    
9. A basic knowledge of the major areas of philosophy and of the methods of philosophical reasoning and analysis.
    
10. Adequate preparation for admission to graduate or professional school.

II. Measures

The Philosophy Program Assessment Plan specifies two means of direct measurement:

1. Writing Samples (Measures Objectives 1,2,3,4,5,7, and 8)

For each student in the major or minor program, the following steps will be taken, as time and opportunity allows, to assess the effectiveness of the program through their written work:

1. In order to establish a baseline, a writing sample—such as an essay exam, a position paper, or a formalized portion of a student’s journal—will be collected from each major and minor in one of the required introductory courses (Introduction to Philosophy or Introduction to Ethics). The Assessment Committee (which will consist of all the members of the philosophy program) will read and evaluate these writing samples each semester, using the Writing Sample Assessment Rubric (see attachment A). These samples will remain anonymous, using only a number for identification, until after a consensus has been reached by the committee on the rating each sample ought to receive. These samples, along with their ratings, will be filed for comparison with writing samples from later courses.
    
2. Upon completion of the program, another writing sample—such as an essay exam, a position paper, or a formalized portion of a student’s journal—will be collected from each major and minor in one of the program’s capstone courses. (For majors, the capstone course is PHIL 490; for minors, any 400 level Philosophy course will serve as a capstone course.) The Assessment Committee will read and evaluate these samples, using the same rubric as was used for the introductory level samples, and again maintaining anonymity until an evaluative consensus has been reached.
    
3. The introductory level samples and the capstone level samples will then be compared to assess the effectiveness of the program as a whole as evidenced by the improvement (or the lack of it) in the writing of these students, viewed as a group.
    
4. The Philosophy faculty will keep a portfolio for our majors and minors containing samples of their written work as they progress through the Philosophy curriculum. Before graduation, each portfolio will be assessed in terms of the objectives to determine the effectiveness of the program. A steady improvement in meeting the objectives by a majority of these students will be considered a positive assessment of the effectiveness of the program; a failure to see such improvement will be considered a negative assessment of the program. In the case of a negative assessment, all the member of the faculty will meet to discuss and implement whatever changes are needed to improve program effectiveness.

A consensus is defined as an average rating by at least two members of the committee that is the same or within one rating level of each other. Samples are read and rated independently, but if a consensus does not emerge after the independent reading, members of the committee will re-read the sample and discuss its merits (or demerits) until a consensus is reached.

2. Capstone Course Survey (Measures Objectives 4,5,6,7,8,9, and 10)

For majors, the capstone course is PHIL 490; for minors, any 400 level Philosophy course will serve as a capstone course. Students in these courses will be asked to complete an assessment survey (see attachment B) near the end of each semester. The Assessment Committee will utilize these surveys to assess the effectiveness of the teaching and the curriculum of the Philosophy program.

The assessment plan also includes indirect measures:

We encourage our major and minor students to submit papers for presentation at conferences and for the O.K. Bouwsma and Donovan Welch Family scholarships, both of which are judged (on the basis of superior reasoning and writing) and awarded by the Philosophy faculty. The awarding of these scholarships to some of our majors and minors will be an indication of the quality of student writing. Since most of these papers will originate in writing they are required to do for our courses, they will indirectly indicate the effectiveness of the philosophy program.

III. Summary and Explanation of Results

The information gathered for this report will be given by measure.

1. Writing Samples (Measures Objectives 1,2,3,4,5,7, and 8)

In the Fall of 2004 the Assessment Committee (i.e., all the members of the Philosophy Program) began to collect data from all of the students minoring in Philosophy for the writing sample measure. That year we evaluated essays from ten students who had declared Philosophy as their minor and who were enrolled in either of our introductory courses—Intro. to Philosophy or Intro. to Ethics. Last year we collected data from ten more students majoring or minoring in philosophy: short papers and essay exams were blindly evaluated using the writing sample rubric (see below). This year we collected data from eight more students. Since this is now the third year we have collected assessment data under our current Assessment Plan, most of our data still is most useful in establishing a writing sample baseline to put in the portfolio of each of our majors and minors. Next year’s data will mark a point at which we can use the data more constructively in evaluating the long-term effectiveness of our programs.

As in the past two years, each student paper was identified by a number, followed by the semester and year the paper was evaluated. The rating scale was A, B, C, or D, with A being the highest and D being the lowest.

The average rating for each student paper was as follows:

As was the case in the last two years, scores were generally higher in the area of Comprehension of texts/ideas (measuring objectives 1 and 7) than in the areas of Content (measuring objectives 2, 3, and 8), Form (measuring objectives 3 and 5), and Application (measuring objectives 2, 4, and 8). These results are not unexpected, as the latter three areas are pedagogically posterior to, and dependent upon, the first area. These students’ portfolios now have a baseline for further assessment using the writing sample measure. We will evaluate writing samples of these same students as they progress through the program.

Three students majoring in Philosophy and one student minoring in Philosophy whose initial assessments were done two years ago were in capstone courses this past year. Longer papers from these students written for their respective capstone courses were blindly evaluated using the writing sample rubric (see below). Each student paper was identified by a number, followed by the semester and year the paper was evaluated. The rating scale was A, B, C, or D, with A being the highest and D being the lowest. Once the blind evaluation was completed, each student was identified, and their final rating was compared to their initial rating from 2003.

The average rating for each student paper was as follows:

As the data shows, there was not a lot of improvement in the first student’s writing because there wasn’t much room for it: their initial rating was already very high. The fourth student showed significant improvement, with the second and third students showing slight improvement. Still, there was definite improvement in all of the four cases at the level of specific indicators. (See the rubric for these indicators.) Because of the relatively scarce data from students in capstone courses up to this point, it is still a bit premature to draw any conclusions from this data with regard to program assessment. There has been improvement in the area of student writing, comprehension and critical thought, based on the data up to this point, so we will continue with the current program and the current assessment plan until we have a more comprehensive set of data.

2. Capstone Course Survey (Measures Objectives 4,5,6,7,8,9, and 10)

One student with a major in Philosophy graduated in the Fall of 2004 and two others who majored in Philosophy graduated in the Spring of 2005. These students were asked to complete the Capstone Course Survey. We received responses from all three of these students.

The average scores from Section 1 of the survey are as follows:

1. To what extent have your Philosophy classes increased your:

Comments from Section 2 of the survey clarified these scores. The relatively lower scores on questions (d) and (g) were due primarily to three factors:

1. The personal choices of the students in selecting classes. Many students acknowledged that, though the program offered courses in a wide variety of philosophical areas, they chose to concentrate in one or two areas of interest.
2. The infrequency of offerings. Many students noted that upper division courses can only be offered infrequently, due to the low number of instructors (3) in the program.
3. The differential in the quality of both the studied writings and the teaching in the courses that are generally categorized as diverse. Several students noted that there was a great diversity among the works studied even in those courses that are not usually classified as diverse.

The areas of strength noted in the surveys were:

• The high quality of instruction; specifically, the love of learning and care for the students shown by the instructors, and the rigorousness of the courses.
• The method of instruction; specifically, the emphasis on classroom discussion and on the use of primary texts.
• The value of the study of philosophical ideas for present and future life.

The areas of weakness most commonly noted in the survey were:

• Not having a major in philosophy until very recently (last year).
• The infrequency of certain course offerings at the upper division level, due to a small staff.

Suggestions for improvement were:

• To offer more honors sections in philosophy.
• To increase the number of instructors.

3. Indirect measures

Two students majoring in philosophy submitted papers in the Spring of 2005 to conferences (one to an undergraduate conference, the other to a conference for both students and professors); both were accepted. Twelve students either majoring or minoring in philosophy submitted papers for the O.K. Bouwsma and the Donovan Welch Family scholarship competitions in late March of 2006. Two of those students were awarded $2200 scholarships; three were awarded $1200 scholarships; two were awarded $600 scholarships; four were awarded $500 scholarships; and two more were awarded $400 (one-course) scholarships for the 2006-2007 school year.

IV. Recommendations

At this point the results of our assessment plan indicate that our program is fairly successful in achieving the objectives we have set. The writing sample measure is still in the first stages of use, so it has not shown us anything conclusive. However, the capstone survey indicates that our objectives have been, and are being successfully achieved by our current program. Our recommendation is to follow our current course of action in terms of curriculum and instruction, and to continue to gather data under the current assessment plan until we have compelling evidence to make other changes.

V. Assessment of the Assessment Plan

Because of the newness of our current assessment plan, as well as the apparent success of the program that the assessment plan has shown, we see no need to make any adjustments in the assessment plan at this juncture.