University of Nebraska Kearney

General Studies Program Assessment Report 2005-2006

2005-2006 General Studies Program
Assessment Report
Prepared by Daren Snider

Overview of Assessment Procedure

Assessment of the General Studies (GS) program in 2005-06 included direct and indirect measures of student learning. Direct measures were embedded in the GS courses and reported by the departments. Indirect measures consisted of a survey of graduating students and a student focus group designed to measure satisfaction with learning outcomes. In addition, data were reviewed and analyzed from previous student and faculty surveys and from the NSSE.

These assessment activities led to a campus-wide review of the GS program in the form of Roundtable discussions facilitated by the Faculty Senate with participation from the GS Council. During 2005-06, the Roundtables and the GS Council reviewed data on student learning, as well as faculty and student opinions about the structure and effectiveness of the GS program. This extensive ongoing renewal initiative is a major result of assessment efforts at UNK.

I. Direct Measures

Departmental Assessment

Each department offering a GS course is required to assess learning outcomes of those courses annually. The majority of GS courses were assessed during the fall semester of 2005 and most of the remaining courses during spring 2006.

Reports were submitted from all 23 departments but only 3 departments met the October 1st deadline. The lateness may be because the report is a new and unfamiliar requirement. As reports were submitted, they were checked for completeness against a 10-point rubric by the Assessment / GS graduate assistant, then by the director. Eight department reports included all of the required components. A majority of reports addressed most of the components, though one department seemed to seriously misunderstand assessment methodology and procedures and is being assisted in revising their report.

Recommendations / Actions:

• The GS director will work closely with the Assessment Office in publicizing deadlines and will stay in close contact with departments early in the assessment cycle.
• Departments will provide the name of a contact person on the report for follow up.
• The GS director has provided clear guidelines and sample reports (.pdf) on the GS website to standardize the format of reports and make the process easier on departments.
• The GS Council will create clear guidelines and sample assessment plans on the GS website to help departments formulate their plan that is due May 1st.
• During next assessment cycle, departments will be required to assess learning outcomes not only of the four general objectives of GS, but also for the category and perspective objectives.
• Perspective objectives will be published in the UNK catalog and put on the GS website so that departments may reference them when making assessment plans and reports.

Nationally-normed Standardized Test

Several nationally-normed instruments were evaluated as potential direct measures of GS objectives. (Identification of potential tests began in early 2005 and more specific analysis was conducted in 2005-06.) Tests were analyzed using criteria designed by the GS Council. The tests included: College BASE, Collegiate Learning Assessment, Collegiate Assessment of Academic Proficiency, Information and Communications Technology Assessment, and the Measure of Academic Proficiency and Progress. Part of the review process included an analysis of results of a focus group who took the College BASE the spring before.

Recommendations / Actions:

• After analysis of the focus group results, the College BASE was not adopted because it did not satisfy evaluative criteria 1, 2, and 5.
• The GS Council investigated with the director of assessment and the senior vice chancellor the possibility of administering the Collegiate Learning Assessment to a pilot group during spring 2007. The discussion is ongoing.
• The GS Council and Assessment Office discussed possibly creating a summative learning instrument to assess freshmen and seniors, and piloting this during spring 2007.
• The GS Council will continue reviewing nationally-normed instruments to identify an appropriate one.

II. Indirect Measures

A Graduating Student Survey was conducted to gauge student satisfaction with their learning experience in GS courses. Further, data from prior surveys about the GS program were analyzed and used in the current GS renewal process. These included data from online surveys for students and faculty, from a student focus group, and from the NSSE.

Graduating Student Survey

In spring 2006 the Registrar’s Office administered a survey to graduating seniors to measure satisfaction with their educational experience. The GS Council developed survey items for inclusion in the survey to measure student satisfaction in meeting GS program objectives.

The sample included only those graduates (34%, 180 out of 527) who chose return the questionnaire. The data indicate a general satisfaction with their academic experiences at UNK. Significantly more students (44%) said they were academically challenged by their GS courses than those who did not (18%). Students from the College of Natural and Social Sciences (mean = 3.04) rated their ability to integrate GS course material into other classes was significantly lower than for students from the College of Fine Arts and Humanities (mean = 3.81). Students from the Colleges of Business and Technology (mean = 3.54) and Fine Arts and Humanities (mean = 3.65) rated the critical thinking and problem solving skills gained from GS courses significantly higher than students from Natural and Social Sciences (mean = 2.93).

Means for each of the seven questions in the survey were above the midpoint of 3 on a 5-point scale. Particularly high were students perceptions of how well their GS courses helped them develop skills in communication (mean=3.59), critical thinking (mean=3.44), and locating information (mean=3.41).

Recommendations / Actions:

• The GS Council discussed ways that GS courses can increase students’ understanding of cultures other than their own.
• The GS Council will revise the questions for future administration of this survey to reflect other assessable GS outcomes.

Student Perception of the GS Program

The Student Assessment Committee (SAC) created this survey to measure student perceptions of the GS Program. It is a modified form of a questionnaire given to faculty. During 2005-06 the results from the survey were shared by the SAC with the GS Council. The two groups collaborated to modify the survey to reflect changing informational needs. The renewal work of the GS Roundtable discussions have used the data in their deliberations, which showed that 61.33% of student participants agreed or strongly agreed that changes should be made in the GS program.

Faculty Questionnaire

The faculty questionnaire addressed GS curriculum, structure, and governance. In 2005-06, data from this survey were analyzed and compared to results from the student survey. The analysis found that faculty respondents felt the program has strong support (mean=3.43) from students, faculty, graduates, trustees, and employers. However, there were some significant differences between faculty perceptions and student perceptions of the GS program.

Faculty (mean = 3.13) rated the clarity of purpose of the GS program significantly higher than did students (mean = 2.83). Faculty (mean = 2.87) rated faculty understanding of the rationale of the GS curriculum significantly lower than did students (mean = 3.34). However, faculty (mean = 3.28) rated their enthusiasm for teaching GS courses significantly higher than did students (mean = 2.66). Further, students (mean = 2.73) rated the disciplinary links between their major and the GS program significantly lower than did faculty (mean = 3.18).

Recommendations / Actions:

• Faculty will be informed about the differences in their perceptions and the students’ perceptions about the GS program.
• A “Comments” section was added to the student survey for future administrations of it.
• Some questions on the student survey were revised to allow for a “Not Applicable” option.
• The next version of the student survey will focus more on students’ learning experiences.
• The GS may create a survey for freshmen (Have you heard about GS?, etc.)
• The GS Council will continue to collaborate with the SAC to give the student survey annually.
• The GS Council will survey the faculty about whether they think their GS courses meet GS objectives.
• In future administrations of the faculty questionnaire, participants will report how long they have taught at UNK.
• Faculty questions will be reworded to be personalized to the participant, rather than asking what the participant believes other faculty think.
• A "Don't Know / Not Applicable" response option will be added to the faculty questionnaire.
• The GS Council will seek input from the SAC before administering the next version of the faculty questionnaire.

Student Focus Group

The GS Council collaborated with the SAC for the purpose of conducting a student focus group, which the SAC conducted prior to the assessment cycle. Discussion items addressed their experiences with and desired improvements to the GS program. In 2005-06 the data were interpreted. The results will be disseminated, the process reviewed, and a timeline established for future focus groups.

The participants (n=6) met in a 36-minute discussion group moderated by two members of the SAC. The focus group consisted of six males and one female. Their majors were: chemistry (3), business administration (1), business marketing (1), and telecommunications (1). During the discussion, no indication was given as to their class status, transfer status, academic record, or demographics. The students were given a gift certificate to a bookstore as an incentive to participate. The discussion centered on four questions:

1. What do you feel is the purpose of the GS program?
   Students responded that the courses make one well rounded and give opportunities to learn about different cultures. They also pointed out that GS courses give opportunities to take courses that otherwise would not have been taken.
    
2. How difficult were your GS courses compared to other courses you have taken at UNK?
   Participants seemed to agree that while many GS courses seem too easy, others are challenging. They expressed the desire, however, to avoid repeating the curriculum of high school.
    
3. What are some of the strengths of the GS program and GS courses?
   This question elicited some complaints about specific departments. However, several participants noted that courses in English composition and literature were very helpful in improving their communication and critical thinking skills. Math courses were also thought to be beneficial, especially for major courses that require the use of math, such as advanced economics.
    
4. What changes would you like to see in the current GS program?
   The participants seemed to believe that attendance in GS courses should be mandatory and that instructors should take roll to enforce it.

Recommendations / Actions:

• Results of the student focus group will be reported to participants of the Roundtable on renewal of the GS program.
• The GS Council initiated discussions about setting guidelines on academic rigor for GS courses. The discussions are ongoing in the GS Council and were presented to participants of the GS Roundtables. The Roundtables are currently considering setting guidelines to address academic standards.
• The GS Council will revise the focus group discussion questions and is planning possible future student focus groups on various aspects of the GS program.

National Survey of Student Engagement (NSSE)

The NSSE was administered prior to the assessment cycle to establish baseline data and will be administered again in spring 2007. In 2005-06, the GS Council and Dr. Deborah Bridges, a member of the GS Council, prepared an extensive data analysis (Appendix G) of previous administrations of the NSSE. They also outlined direct linkages (Appendix H) between NSSE questions and the four Objectives of GS. The GS Council met several times to consider how the data should influence the renewal work of the GS Roundtable.

Recommendations / Actions:

• NSSE results and analysis were put on the GS website and disseminated to the members of the GS Roundtable for their consideration during the process of renewing the GS program.
• The GS Council discussed the analysis of NSSE data in deliberations about creating standards for academic rigor.
• The GS Council will continue to analyze NSSE data also for use in the work of campus committees on student success and retention in GS courses.
• Future administrations of NSSE will show trends that will be used in creating assessable objectives for the emerging new GS program.

III. Evaluation of the Assessment Process

The GS assessment process in 2005-06 was somewhat hindered because it is a new requirement, and because there was administrative turnover in the Director of GS and in the Office of Assessment. It was difficult to obtain timely GS assessment reports from departments, some of which confused them with the major/minor reports that have been required for some years. Also, because some departments lack experience in assessment, the quality of assessment plans and reports was somewhat uneven.

Recommendations / Actions

• The GS Council will work to streamline and clarify the GS assessment process.
• Departments should be notified before fall semester if there are inconsistencies or other problems in their GS assessment plans.