University of Nebraska Kearney

Psychology Assessment Report 2006

Submitted Fall, 2006

Goal 1. Students will develop a comprehensive knowledge base (basic knowledge of psychology.

Objective 1.1: For each ACAT content area (except experimental and statistics), the departmental assessment committee will compare the scores of students who have had the relevant coursework to those who have not over a two-year period. Students with the relevant coursework will show a higher mean score in that content area than those without the relevant coursework.

     Objective 1.1 Using and Reporting Results. A comparison of students with and without the relevant coursework in each of the ACAT content areas (except experimental and statistics) will be posted and reviewed every two years by the assessment committee. If the students with the relevant coursework for a given content area fail to show a higher mean score than those without the relevant coursework (assuming that there are at least 10 students in each group), the department chair will discuss the possible reasons for the apparent lack of increase in knowledge with the relevant instructor(s). The instructor(s), in conjunction with the department chair and departmental assessment committee, will make changes that are likely to increase the effect of taking the course(s). The changes undertaken will be posted on the next available assessment report.

     Objective 1.1 Results: Below is a comparison of mean scores for students with and without the corresponding coursework for each ACAT content area from Fall 2004 – Spring 2005; N = 28 and Spring 2006; N = 23 (No ACAT was given to graduating seniors Fall 2005). The mean and standard deviation from the standardization sample are 500 and 100, respectively. The number of students is in parentheses.

     Objective 1.1 Use of Results: During past assessment reports, only one class was used for each of the content areas in counting whether or not the course was taken. As some sections, have information that could have come from many of the advanced courses in the area, a change was made by adding a column "With Similar Course" to include these additional classes. The table below summarizes the classes used for each section. The areas of Human Learning/Conditioning, Personality and Sensation and Perception had no similar courses to include.

In addition, for the section on Personality, the ACAT uses questions from an experimental personality perspective and the personality class offered at UNK is a Theories of Personality perspective. The assessment committee will discuss what can be done to provide a more accurate picture of content learning for this area.

The assessment committee will discuss whether or not to continue giving the ACAT for the next few years.

Objective 1.2: Students, as a group for the academic year, will score above the 50th percentile in each of the eight content areas noted above.

     Objective 1.2 Using and Reporting Results: The group percentile ranking of students who took the ACAT during the past two academic years will be posted with the next assessment report and reviewed by the assessment committee. If students fall below the 50th percentile in a given area for two consecutive years, the same procedures as described above for comparison 1.1 will be implemented.

     Objective 1.2 Results: Below are the mean scores for our graduating seniors across years for each content area. The overall mean for the standardization sample is 500 and the standard deviation is 100 for each area.

     Objective 1.2 Use of Results: The scores in general remain above the national mean, particularly the overall scores. The only exception is Sensation and Perception but it should be noted that only 4 graduating seniors taking the ACAT have taken the Sensation and Perception course over the past two years. The committee will determine whether or not to continue using the ACAT in the next few years.

Objective 1.3: Field experience and practicum students, as a group for the academic year, will score above 3 on the five-point scale above in each area assessed (using end of field experience data where beginning and end points are assessed).

     Objective 1.3 Using and Reporting Results: The results of the Field Experience and practicum evaluations will be posted with the next assessment report and reviewed by the assessment committee. If students, as a group, score 3 or below on any area assessed, the department chair will meet with the Field Experience or Practicum instructor and representative from the field placement to discuss ways of improving student learning at the placement. Final decisions on how to improve learning at the placement will be posted on the next assessment report.

     Objective 1.3 Results: The data represent the mean scores for external evaluations of Field Experience students, Spring 2006(N = 7). 3 = acceptable; 4 = above expectations; 5 = far above expectations.

Field Experience Ratings of Student Learning 

      Objective 1.3 Use of Results: The scores have decreased slightly since the last report but it is worth noting that there were 7 students this year compared to only 2 last year that participated in the clinical field experience. We will maintain the curriculum offerings that support the field experience (Psychopathology, Intro to Clinical). We will examine ways to expand this type of student experience.

Goal 2. Students will be familiar with the methodology for furthering ideas, theory, and knowledge within the discipline.

Objective 2.1: Students will score above the 50th percentile on the experimental and statistics areas of the ACAT.

     Objective 2.1 Using and Reporting Results: The percentile raking of students who took the experimental and statistical portions of the ACAT during the year will be posted with the next assessment report and reviewed by the assessment committee. If students fall below the 50th percentile in a given area for two consecutive years, the same procedures as described above for goal #1 will be implemented.

     Objective 2.1 Results: Below are the mean scores for our graduating seniors for each Experimental Psychology and Statistics class for the past two years. The overall mean for the standardization sample is 500 and the standard deviation is 100 for each area.

     Objective 2.1 Use of Results: In 1998, our students were scoring around the mean (Experimental: 490, Stat: 507) and there has been significant improvement since that time, though during Spring 2005, the scores had dipped slightly. Though the scores are up again this past Spring, we will continue to monitor the scores to see if they fall again or whether the Spring 2005 set of scores is unusual.

Objective 2.2: At least 20% of exiting seniors will produce a paper or poster that is presented at a regional, national, or international psychology conference.

     Objective 2.2 Using and Reporting Results: The percentage of our exiting seniors who present a paper or poster at a regional, national, or international psychology conference within each academic year, will be reviewed by the assessment committee. If less than 20% of our exiting seniors within this period present a paper or poster at this level (the national average is 19.6%), the entire department will meet to discuss how to increase student research activity and dissemination. Final decisions on how to increase productivity will be posted with the next assessment report.

     Objective 2.2 Results: Of 30 exiting senior Psychology majors in 2005-6, 17 or 56% produced research that was presented at a regional-level conference or above.

     Objective 2.2 Use of Results: We will continue to encourage students to present their work at conferences and explore more venues for their presentations.

Objective 2.3: UNK Psychology will rank in the top 20% of institutions represented at the annual Great Plains Student Psychology Conference for the number of awards for papers or posters at each conference.

     Objective 2.3 Using and Reporting Results: The number of awards for papers or posters by UNK Psychology students will be compared to the number of awards by the other schools represented at the Great Plains Psychology Student Conference. The assessment committee will review these results. If UNK Psychology does not place in the top 20% for the number of awards at two consecutive conferences, the department will construct a plan to better prepare students for these events and post that plan on the next assessment report.

     Objective 2.3 Results: Of the 30 graduate and undergraduate schools represented at Great Plains in 2006, 16 schools received at least one award and 13 received multiple awards. UNK received 7 awards out of 12 presentations (58%). Only 4 schools had 10 or more presentations (including UNK). Three schools had a higher percentage of awards than UNK, though only two of those schools (Creighton and Washburn) had more than 10 presentations. Creighton (13 presentations), Morningside (9 presentations) and Washburn (12 presentations) had 62%, 67% and 67% respectively.

     Objective 2.3 Use of Results: The department routinely encourages students to rehearse their presentation before live audiences that include several faculty and other students. After the rehearsed presentation, the presenters are given a thorough oral critique with suggestions for improvement and refinement. We believe this largely accounts for our students’ success, and we will make sure to continue these rehearsals. In addition, we routinely encourage students who are not presenting papers to attend conferences, where we believe they develop realistic expectations and often become interested in pursuing their own research ideas. This practice will also be continued.

Goal 3. Students will communicate their knowledge clearly through both spoken and written media.

Objective 3.1: There will be a statistically significant increase in scores from the PSY 270 papers to the upper-level papers.

     Objective 3.1 Using and Reporting Results: If there is not a statistically significant increase in scores, the entire department will meet to review procedures for teaching students to write within the discipline. Final decisions on how to improve student writing will be posted with the next assessment report.

     Objective 3.1 Results: Below are the results for student writing skills assessed during the Experimental Psychology course (PSY 270) and later during an advanced course (PSY 300-499) from Fall 2005 – Spring 2006. This component of our assessment plan was only recently reinstated, so there is limited data. The data only include 22 student scores. Separate results are displayed for writing style, APA Format, and Level of content. Each area was assessed on a 1 –5 point.

     Objective 3.1 Use of Results: It appears that in the past our methods of writing and revising APA-style papers in Experimental Psychology has been effective in teaching students the writing standards of the discipline. We will continue this practice of scoring papers.

Objective 3.2: At least 10% of our exiting seniors will present an oral paper at a regional, national, or international psychology conference.

     Objective 3.2 Using and Reporting Results: The percentage of our exiting seniors who present an oral paper at a regional, national, or international psychology conference within each academic year, , will be reviewed by the assessment committee. If less than 10% of our juniors and seniors present an oral paper, the entire department will meet to discuss how to increase student research activity and dissemination. Final decisions on how to increase productivity will be posted with the next assessment report.

     Objective 3.2 Results: Of 30 exiting seniors, 14 or 47% authored a presentation that was given orally at a regional or national conference during Fall 2005 or Spring 2006. (Based on exit interview).

     Objective 3.2 Use of Results: We encourage our students to make at least one of their conference presentations oral rather than poster, and we will continue this practice. Because this objective seems to be met with much room to spare, we will raise the objective to 25% of our exiting seniors presenting oral papers at conferences.

Objective 3.3: UNK Psychology Majors will rank in the top 20% for the number of awards for oral papers at each conference.

     Objective 3.3 Using and Reporting Results: The number of awards for oral papers by UNK Psychology students will be compared to the number of awards by the other schools represented at each conference. The assessment committee will review the results. If UNK Psychology does not place in the top 20% for the number of awards at two consecutive conferences, the department will construct a plan to better prepare students for these events and post that plan on the next assessment report.

     Objective 3.3 Results: Of the 30 schools represented at Great Plains in 2006, UNK was in the top 3 in terms of number of awards for oral presentations. UNK received 4 awards for oral presentations out of 7 oral presentations made (57%) and only two other schools received 5 or more awards for oral presentations. Washburn received the highest number of oral presentations, 8, out of 11 oral presentations made (73%). UNL received 5 awards out of 9 oral presentations made (56%). UNK students continue to compete well with other students at regional conferences.

     Objective 3.3 Use of Results: The department routinely encourages students to rehearse their presentation before live audiences that include several faculty and other students. After the rehearsed presentation, the presenters are given a thorough oral critique with suggestions for improvement and refinement. We believe this largely accounts for our students’ success, and we will make sure to continue these rehearsals. In addition, we routinely encourage students who are not presenting papers to nevertheless attend conferences, where we believe they develop realistic expectations and often become interested in pursuing their own research ideas. This practice will also be continued.

Objective 3.4: UNK Psychology Majors will rank in the top 20% for the number of awards for poster presentations at each conference.

     Objective 3.4 Using and Reporting Results: The number of awards for poster presentations by UNK Psychology students will be compared to the number of awards by the other schools represented at each conference. The assessment committee will review the results. If UNK Psychology does not place in the top 20% for the number of awards at two consecutive conferences, the department will construct a plan to better prepare students for these events and post that plan on the next assessment report.

     Objective 3.4 Results: Of the 30 schools represented at Great Plains in 2006, 12 schools received awards for posters, and 6 received multiple awards. UNK received 3 awards for poster presentations out of 5 poster presentations made (60%). Upper Iowa University also had 3 awards for posters out of 18 presented (17%). Creighton University and Morningside had the highest number of awards for posters, 5 each, with Creighton's percentage at 56% (out of 9 posters) and Morningside had the highest percentage awards, 63% (out of 8 posters).

     Objective 3.4 Use of Results: The department routinely encourages students to prepare their posters ahead of time and have them reviewed by faculty members. Students are given suggestions for improvement and refinement. We believe this largely accounts for our students’ success, and we will make sure to continue these practices.

Goal 4. Students will apply their knowledge with:

1. tolerance of conflicting viewpoints 
2. critical thinking approaches 
3. appropriate professional and scientific ethics and values. 

Objective 4.1: Graduating seniors will score higher than freshman and sophomores on each of the items of both a social desirability scale and a locally developed value of science survey.

     Objective 4.1 Using and Reporting Results: If graduating seniors do not score higher on one or more of the items for both scales, the department will meet to discuss how to better develop the given skills or values in our students.

     Objective 4.1 Results: The Marlowe-Crown Scale is a 15-item scale used to measure social desirability. Students in PSY 250 and exiting seniors were compared on each of the items. There were no significant differences on any of the items on the scale. There was general trend showing that seniors tend to score in a more positive direction than PSY 250 students.
The Values of Science Survey is an 8-item scale used to assess students’ investment in reasoning, facts, and science, for forming beliefs, opinions and decision-making. Students in PSY 250 and exiting seniors were compared on each of the items. There were no significant differences on any of the items on the scale. There was general trend showing that seniors tend to score in a more positive direction than PSY 250 students. There was one item that was particularly interesting, though not significantly different. The statement was “people can share sensitive information with me and be confident that I will not share it with someone else” was higher for the senior group (M = 6.25) than the PSY 250 group (M = 5.67). This item suggests that as students advance in their academic careers, they internalize the importance of keeping sensitive information private.

     Objective 4.1 Use of Results: The data reported includes only one class from PSY 250. We will continue to collect the data from each group (PSY 250 and exiting seniors) and determine if the lack of change in attitudes remains fairly consistent. It is important to note that the means for the PSY 250 students are not really very low suggesting that students come into the program with some tolerance of conflicting viewpoints, critical thinking approaches and appropriate professional and scientific ethics and values as we are measuring them. We will continue to collect more data and determine if we may need to develop or find a more sensitive measure.

Objective 4.2: Field experience and practicum students, as a group for the academic year, will score above 3 on the five-point scale above in each area assessed.

     Objective 4.2 Using and Reporting Results: The results of the Field Experience and Practicum evaluations will be posted with the next assessment report and reviewed by the assessment committee. If students, as a group, score 3 or below on any area assessed, the department chair will meet with the Field Experience instructor and representative from the field placement to discuss ways of improving student learning at the placement. Final decisions will be posted on the next assessment report.

     Objective 4.2 Results: Below are the mean scores for external evaluations of Field Experience students, Spring 2006(N = 7). 3 = acceptable; 4 = above expectations; 5 = far above expectations.

Field Experience Ratings of Ethics and Values 

     Objective 4.2 Use of Results: Again, the scores in this area have dipped slightly, but must be looked at with caution given the small number of students enrolled in the Field Experience the past year. This year there were 7 students compared to only 2 students last year. We will continue to monitor this area during the next year.

Goal: 5. Students will be prepared for future academic and/or professional endeavors.

Objective 5.1: Our alumni will exceed the standards in each of the relevant areas as determined by the national statistics compiled by ETS.

     Objective 5.1 Using and Reporting Results: If our alumni group fails to exceed the performance of the ETS normative group for one or more of the survey items above, the departmental assessment committee will consult with several alumni to determine how their career needs could have been better served by the department. Final decisions of programmatic changes will be made by the department chair in conjunction with the assessment committee and be posted with the next assessment report.

     Objective 5.1 Results: No new data have been collected from alumni since 2002.

     Objective 5.1 Use of Results: The last assessment report indicated that the particular assessment instrument used in the past would be discontinued. The department continues to explore other means of assessing this objective. The current plan is to send a questionnaire to alumni through the department newsletter and query alumni about different aspects of their experiences as they relate to their profession and/or continued education. Results of this survey will be included in the next assessment report. Another plan to assess this objective will include gathering information from exiting seniors about their immediate plans, whether they have applied to and been accepted to graduate school or whether they have applied for and been accepted into a position in the psychology field.

Objective 5.2: Exiting seniors will be successful in their application to graduate schools or be hired in the job market of their chosen field.

     Objective 5.2 Using and Reporting Results: If our exiting seniors fail to be accepted into graduate school at a rate of more than 50% or fail to find employment in their chosen field at a rate of more than 50%, the departmental assessment committee will consult with several alumni to determine how their career needs could have been better served by the department. Final decisions of programmatic changes will be made by the department chair in conjunction with the assessment committee and be posted with the next assessment report.

     Objective 5.2 Results: Twenty-one of 27 students reporting on the exit interview indicated that they applied to graduate school. An attempt to contact the graduating students during the Fall of 2006 will be made to determine whether they were accepted and began attending graduate school or if they were able to find employment in their chosen field.

     Objective 5.2 Use of Results: The current plan is to develop a questionnaire to give to exiting seniors asking them about their plans and applications to graduate school or for employment. They will be asked if they have made a decision about where they will be attending graduate school or what job they will be taking if provided more than one option. We will then follow up the next year and ask how well prepared they felt they were for graduate school or for their employment compared to their cohorts. Results of this objective will be included in the 2007 assessment report.

Goal 6: Students in General Studies psychology classes will meet the objectives of the overall general studies program as well as the social and behavioral science objectives.

Objective 6.1: Students who have taken a psychology social science general studies course will score higher on test items that measure the following social and behavioral science general studies objectives than students who have not taken a general studies course:

     6.1a. demonstrate an understanding of human experiences of individuals and groups and be able to relate them to the present;
     6.1b. demonstrate the ability to understand the application of the empirical research methods used in the social sciences to understand individual behavior as well as the interrelationships among people, including the ability to critically evaluate the appropriateness of scientific methods and the accuracy of findings in psychology;
     6.1c. demonstrate the ability to comprehend how social scientific concepts originate, are validated and refined in psychology; and
     6.1d. demonstrate the ability to evaluate the strengths and limitations of the behavioral sciences and the explanations they offer for contemporary life and human behavior.

     Objective 6.1 Using and Reporting Results: A comparison of students who are taking their first psychology general studies course (having taken no previous social science courses) with students that have taken previous social science courses will be posted and reviewed every two years by the assessment committee. If the students who are taking their first general studies course in psychology do not score as high in each of the objective areas for the social and behavioral sciences as students that have taken previous general studies courses, the department chair will discuss the possible reasons for the lack of meeting the social and behavioral science objectives. The instructors, in conjunction with the department chair and departmental assessment committee, will make changes that are likely to increase the students’ ability to meet the objectives.

     Objective 6.1 Results: Pretest and posttest scores. There was a significant difference between pre-test and post-test scores, F(1, 271) = 16.92, p =.001, where students scores increased from the pretest to the posttest. There was a significant difference between the scores of males and females on the post-test only, F(9, 271) = 6.33, p < .05. Females scored higher on the posttest than males did. There were no significant differences between the scores of males and females at the pretest, suggesting that females learned more regarding the objectives measured.

Objectives by time 1 and time 2There was a significant increase in scores from the pretest to the posttest for objective 1, F(1, 274) = 7.61, p < .05, objective 3 F(1, 274) = 24.70, p < .01; and objective 4.

High school psychology class
Students were asked whether or not they had a psychology class in high school. Their response to that question was then assessed with the pretest scores for each objective. Objectives 2 was nearing significance and objective 5 was the only significant result, F(1, 274) = 7.37, p < .01 The results indicate that students that took a high school psychology course, scored lower as compared to students that did not have a high school psychology course. This perhaps makes sense as the general studies objectives and social and behavioral objectives are not likely to be variables focused on in the high school courses.

Instructor by Objectives on Time 1 and Time 2
There were no significant results regarding the scores between time 1 and time 2 overall or for each objective based on which professor the student had. Objective 4 was close F(1, 268) = 2.06, p = .058.

     Objective 6.1 Use of Results: The results suggest that students are developing many of the skills measured by the social and behavioral science objectives 6.1a, 6.1c and 6.1d. The lack of improvement for 6.1b may simply be a matter that students are gaining the information regarding the application of empirical research methods used in the social sciences through other classes and that their general psychology course may not be the first introduction to such information.

Objective 6.2: Students who have taken a psychology social science general studies course will score higher on test items that measure the overall general studies objectives than students who have not taken a general studies course:

     6.2a. demonstrate the ability to locate and gather information;
     6.2b. demonstrate the capability for critical thinking, reasoning, and analyzing;
     6.2c. demonstrate effective communication skills including the ability to read, speak and write effectively, using the materials, ideas, and discourse modes of specific academic areas’
     6.2d. demonstrate an understanding of the experiences and values of groups and cultures which have been historically under-represented, and
     6.2e. demonstrate a motivation for life-long learning.

     Objective 6.2 Using and Reporting Results: A comparison of students who are taking their first psychology general studies course (having taken no previous social science courses) with students that have taken previous social science courses will be posted and reviewed every two years by the assessment committee. If the students who are taking their first general studies course in psychology do not score as high in each of the objective areas for the overall general studies program as students that have taken previous general studies courses, the department chair will discuss the possible reasons for the lack of meeting the overall general studies objectives. The instructors, in conjunction with the department chair and departmental assessment committee, will make changes that are likely to increase the students’ ability to meet the objectives.

     Objective 6.2 Results: Pretest and posttest scores. There was a significant difference between pre-test and post-test scores, F(1, 271) = 16.92, p =.001, where students scores increased from the pretest to the posttest. There was a significant difference between the scores of males and females on the post-test only, F(9, 271) = 6.33, p < .05. Females scored higher on the posttest than males did. There were no significant differences between the scores of males and females at the pretest, suggesting that females learned more regarding the objectives measured.

Objectives by time 1 and time 2
There was a significant increase in scores from the pretest to the posttest for objective 6, F(1, 274) = 9.12, p < .01; objective 7 F(1, 274) = 16.56, p < .01, and objective 8, F(1, 274) = 11.55, p < .01.

Motivation for life-long learning questions
Regarding the addition of the ninth objective that was used to measure motivation for life-long learning, two questions were asked. How likely are you to read a psychology book over the summer and how likely are you to use psychological concepts to analyze characters you see in a movie? The response choices were from very unlikely to very likely. Both questions showed a significant increase in rating the likelihood to engage in the activities identified, F(1, 274) = 9.56, p < .01 and F(1, 274) = 13.92, p < .01, respectively.

Perspectives
Students were asked to identify what other classes in the social and behavioral sciences area that they have taken. Based on their responses, we analyzed the scores from time 1 only with those students who had not previously taken a class in the social and behavioral science area.
In the social perspective, objective 7 was nearing significance, F(1, 274) = 3.03, p = .08 but nothing else was at all significant. In the political perspective, objectives 6, F(1, 274) = 8.57, p < .01; objective 8 F(1, 274) = 7.89, p < .01; and the overall score F(1, 274) = 7.94, p < .01 was significant. In the geography perspective, only objective 8 was significant F(1, 274) = 4.64, p < .05. In the behavioral perspective, none of the objectives were significant.

     Objective 6.2 Use of Results: The results suggest that students are developing many of the skills measured by the social and behavioral science objectives 6.2b, 6.2c, 6.2d and 6.2e. The lack of improvement for 6.2a may again be a matter that students are gaining the information regarding the ability to locate information through other classes and that their general psychology course may not be the first introduction to such information.

How we plan to use the dataWe will use inter-rater reliability with the general psychology instructors for the questions being used to measure each of the objectives. In addition, the list of objectives provided in the syllabi of those instructors for each of the general psychology sections will be discussed and an attempt will be made to find some consensus in the objectives for the PSY 203 course.

Additional Assessment Activities carried out in 2005-2006 (These activities will be added to the department's assessment plan for future reference and review):

1. ACAT/grade study. We compared the ACAT scores of students over the past 3-4 years to their grades in the classes that correspond to each ACAT score. We found that the students' overall ACAT scores are significantly correlated with their overall GPA in Psychology (r = .425, p < .002). However, only 3 ACAT subtest scores were significantly correlated with their students' grades in corresponding courses: Learning and Conditioning (r = .494, p < .019. Statistics (r = .329, p < .021, and Human Learning and Cognition (r = .358, p < .037. The remaining scores were not predictive of students' grades in the corresponding content areas. In trying to determine why there was such a discrepancy, we had each of the instructors for those specific courses for which there was not a correlation between ACAT scores and grades look at the test items in the ACAT to identify what might have caused the discrepancy. It was found that there was not a significant amount of overlap between what is focused on in the specific classes and the ACAT test items for those classes. For example, our course in Physiological Psychology provides students with a much more complete understanding of psychopharmacology than the ACAT requires. We will continue using the ACAT test, at least for a while to be able to document how our students compare to other psychology majors across the country.

2. Bloom’s taxonomy. Dr. Mandernach has developed a research strategy to analyze the cognitive complexity targeted by the student evaluation processes used in various psychology courses at the University of Nebraska at Kearney as measured through a comparison of learning objectives in relation to the depth of understanding required by course assessments.

Results. A pilot study was conducted that examined whether upper division courses utilize exam items which indeed measure higher order learning as defined by Bloom’s taxonomy. Professors from three departments voluntarily submitted current exams, which were analyzed according to Bloom’s taxonomy. The results of the analysis indicated a trend for overall, lower level courses to utilize exams which included an overwhelming number of times that targeted knowledge-based skills. While upper division courses had a comparatively decreased reliance on knowledge abilities (Evaluation skills accounted for a larger percentage of items as compared to lower level courses). This provided partial support for the hypothesis that upper-division classes are using exams that include a greater number of higher order items as compared to lower-division classes, although the difference was not statistically significant. To address the methodological concern that exams are only one component of coursework and that most classes integrate complex learning tools and evaluations which may include discussions, projects, or papers, a follow-up study examined the relevance of course assessments in relation to course learning objectives, course level, and depth of understanding. Three target courses (two lower division, one upper division) provided course learning outcomes and all assignments and exams. The assignments and exams were analyzed according to Bloom’s taxonomy. For the first lower division course, analysis of learning objectives indicated that 100% of learning objectives target lower outcomes with 3 of 4 focusing on knowledge and 1 of 4 targeting application. For the second lower division course, analysis of learning objectives indicated that 83% of learning objectives target lower outcomes (4 of 6 focusing on knowledge and 1 of 6 targeting application) and 17% target upper-level learning outcomes (1 of 6 focusing on synthesis). For the upper division course, analysis of learning objectives indicated that 60% of learning objectives target lower outcomes (1 of 5 focusing on knowledge and 1 of 5 on comprehension and 1 of 5 targeting application) and 40% target upper-level learning outcomes (1 of 5 focusing on analysis, 1 of 5 targets evaluation). Contrary to the assumption that upper division courses would focus more on higher-order learning objectives, this research suggests a fairly even split between lower-level and higher-level learning outcomes. While this may seem counter-intuitive in an upper division course, the changing content of each course may mandate a focus on course-specific, basic knowledge prior to the integration of more advance cognitive goals. Future research will exam the requirements in advanced lab courses, which may be the place where psychology students are required to use higher order learning skills.

3. Advising effectiveness. We developed a questionnaire to assess the effectiveness of advising in the psychology department. If the results indicate that students believe that that department advising need improvement, the department chair will work with the faculty to enhance their advising program.

Results - The questionnaire that was developed for use throughout the campus to assess the effectiveness of advising was modified so that we would get data for each faculty advisor within our department. The Psychology Department distributed this advising questionnaire last Spring. We analyzed the results to determine where our strengths and weaknesses are with regard to advising

The survey was distributed to majors at the end of advising sessions in the Spring semester 2006. Fifty-five surveys were returned. Based on the responses, 66% of students indicated that they had attended 2 advising sessions during the academic year (the number required). Slightly more than 23% of students indicated that they had had more than the 2 required sessions in the past year.

A series of 15 questions describing a good advisor were analyzed. The items included such characteristics as: being a good listener, understands general studies requirements, easy to contact, connects me with campus resources. The questions were accompanied by 5 point Likert scales ranging from strongly disagree = 1 to strongly agree = 5. No significant differences were found between faculty advisors in our department as reflected in students’ ratings for what makes a good advisor. The maximum score on this composite index is 75 points. The scores for psychology advisors ranged from 60.5 to 72.5 out of 75, indicating a high degree of student satisfaction with their advisor.

No significant differences were found between advisors in our department on the question asking about satisfaction in general with the academic advising received. The mean scores on the general satisfaction question (1-5) ranged from 4.25 to 5.00 indicating a high degree of satisfaction. No significant differences were found between advisors in our department regarding students’ beliefs about receiving accurate information about courses, programs, and requirements. . The mean scores on the accurate information question (1-5) ranged from 4.17 to 5.00 indicating a high degree of satisfaction No significant differences were found between advisors in our department on students’ perceived access to their advisor. The mean scores on the access question (1-5) ranged from 4.17 to 5.00 indicating a high degree of satisfaction. Also, no significant differences were found between advisors in our department on students’ perception of having sufficient time during advising sessions. The mean scores on the sufficient time question (1-5) ranged from 4.42 to 5.00 indicating a high degree of satisfaction The scores on all of these questions indicated a high degree of satisfaction with student advising by psychology majors.
No significant gender differences were found for students’ ratings of satisfaction with their advisor or their beliefs regarding receiving accurate information. No significant differences were found using age as an independent variable though there was a linear trend suggested that older students rated their advisors better on those items describing what makes a good advisor.

Significant differences were found regarding what students consider important in the advising session F(1, 52) = 3605.18, p < 001. The means of those items are presented below with the items identified as most important (high scores) first.

When asked if students seek advising from someone other than their assigned advisors, 65% of students reported that they also seek advising from peers, 34% from other faculty, 7% from the department chair and 21% from faculty outside their major.

No changes will be made at this time regarding the advising process in the department.

4. On-line course offerings. We plan to determine if grade distributions for on-line course offerings are similar to traditional course offerings. If we find a discrepancy in grade distributions, the department chair will work with the faculty to determine what areas need to be addressed and how.

Results: Grades from three professors who teach the same courses both in a traditional format and an online format were analyzed. A significant difference was found between traditional and online classes with regard to grades, F(1, 521) = 4.489, p < .05. Online classes had higher grades (M = 3.05) than traditional classes (M = 2.81).

A significant interaction between professor and class type was found on the mean grade received, F(2, 521) = 3.66, p < .05. The means for traditional vs. online courses by the 3 different professors are found in Table 1.

Table 1

Mean grades of traditional and online classes by professor 

While the grades for teacher A and teacher C do not differ between traditional and online classes, teacher B shows higher grades in the online classes than in the traditional classes. The department chair will meet with teacher B to discuss why this happened.

5. Rubrics. Psychology majors are required to take PSY 270 Experimental Psychology, preferably at the end of their sophomore year. It is a 4-hour, writing intensive course in which the students conduct an empirical study and write a paper in APA style that describes that study (Introduction, Method, Results, and Discussion). The students undertake multiple revisions of that paper. For assessment purposes, we have for several years examined their level of skill on their first draft and their final draft of the required paper. PSY 270 is a prerequisite for any advanced lab course of which psychology majors must take two. In general, the model for an advanced lab course requires students to engage in empirical research and to prepare a paper in APA style. Again, for assessment purposes, we examine the first draft of their advanced lab paper to determine the extent to which what they learned in PSY 270 is retained. This information has been regularly reported in our Assessment Reports.

Our plan to implement the new WI requirements is to make each of our advanced lab courses a writing intensive course. The lab courses are all 1 credit hour courses so that every psychology major would complete 6 hours of WI course work within the major. We have developed a more sophisticated rubric for grading their writing skills that we will use in PSY 270 as well as the advanced lab courses.

Additional Assessment Activities to be carried out in 2006-2007:

1. Faculty vs. student evaluations. For over ten years now, we have required faculty to evaluate each of their courses using many of the same items used by the students in their formal course evaluation. We plan to compare the course evaluations completed by students with evaluations that faculty have completed. We will try to determine if the results from the student evaluations correlate to the results of the faculty evaluations of their classes and if not, what then might account for the areas of discrepancy.

2. General studies assessment of PSY 230 Human Development. Similar to the research conducted last year with students in PSY 203, an evaluation of how well students meet general studies objectives will be conducted. For control purposes, we will focus on those students who are taking their first psychology general studies course in PSY 230 (having taken no previous social science courses) with students that have taken previous social science courses will be reviewed. We will try to determine if the students who are taking PSY 230 as their first general studies course in psychology score higher at the end of the course in each of the objective areas for the overall general studies program and for the social and behavioral studies objectives than they did at the beginning of the course. We will also compare the data to previous data from PSY 203 to see if one or the other course does a better job in meeting General Studies objectives.

3. Traditional vs. condensed online course. We will identify classes, which have been taught in the traditional 16-week face-to-face format, and classes that have been taught in a more condensed time frame (3 or 4 weeks) including courses taught using the online course format. We will send those students an overall final exam several months after course completion to determine if there is a difference between the students in each type of class in their retention of the information.

4. Pod-casting assessment. We will identify classes (6 this semester) that include pod-casting as part of the resources that students have available to them. In one case, students will have had pod-casting available after the first exam so we will compare how those students performed from the first test to the second test. For other classes, we will compare student performance in podcast vs. non-podcast equivalent courses throughout the course. We will also assess the course satisfaction and student performance on the final exam between similar classes that do and do not use pod-casting.

5. New course assessment: PSY 300 PSY 300 is a redeveloped course that is an introduction to the Psychology Major. We plan to use a Pre and post-test appraisal of how well students reach course objectives. The course objectives are identified in the survey (Appendix D).