University of Nebraska Kearney

General Studies Program Comprehensive Assessment Report Academic Year 2001-2002: Appendix A

Perhaps, the most intriguing part of the internal evaluation of the General Studies (GS) Program at the University of Nebraska at Kearney (U.N.K.) was the faculty survey. A mailed questionnaire was sent to all faculty members in the fall of 2000. When all was said and done, 269 members of the faculty had responded from a mailing originally sent to almost 330. Once bad addresses had been culled, we had 280 good addresses and a response rate of 96%. Figure One (below) shows that most faculty members gave the GS program a good grade but many, almost two in five, graded the program as only average or below average. This report explores the sources of differences among the faculty, including their evaluation of the status of reported strengths of the General Studies Program from the previous program review.

Figure One

Description of Respondents

To begin, the data that describes faculty and their teaching of GS classes is instructive. Table One (below) is a correlation matrix and it offers a glimpse of faculty activities in the GS program. The table offers a few clear observations. For one, few of the correlations between faculty responses to our questions are significant. One strong correlation (.321) was the result of faculty who reported they taught more students in their GS courses and used lectures more often. Another significant correlation occurred among respondents who reported they taught larger classes and taught more GS classes in an average semester (.214). There was also a significant correlation between employment (whether faculty were tenured, tenure-track, lecturers, visiting professors or adjuncts) and the number of years faculty members had taught GS classes (.392). Since tenured faculty were coded as the first category, tenure-track the next category, and so on, the correlation indicates that as time passes tenured faculty are less likely to teach GS courses than other faculty (such as lecturers

Table One

** significant @ /001 or less

What can we glean from these significant relationships? Some of it makes good sense. Larger classes probably demand the use of lectures more often. Teaching more classes may also lend to the use of lecture since lectures do not need to be dramatically modified to suit the needs of different classes. Other teaching techniques, such as group work, may need to be adjusted quite dramatically to fit class size and the room setting. Many instructors have tried to hold group meetings in lecture halls, where the seats are fixed, only to find that the technique met with limited success—it is difficult for students to interact in a group when they have to twist their torsos and crane their necks just to look at other group members! Unfortunately, the strong correlation means that many instructors in GS classes are employing lectures much of the time.

Figure Two indicates that over half of the General Studies Instructors reported using lecture techniques most of the time in their GS classes. Only sixteen percent reported that they used lecture techniques a quarter of the time and almost another quarter reported they used lectures half of the time. Certainly, lecturing has a long history in university classrooms and it is a fine method for reporting knowledge. It may not be the best method for teaching and the lecture may also be suspect as the best method for creating learning situations that effectively work to meet the goals of the GS program.

Program Mechanics

One area of concern we wished to understand related to program mechanics. We wanted to know if students were having difficulties with the GS program because of rules or administrative actions. A series of statements were made on the questionnaire and faculty were asked to respond by marking boxes that indicated their level of agreement with those statements. Response patterns were based on a modified Likert Scale that ranged from “strongly agree” to “strongly disagree.” Table Two, below, shows that faculty responded in a consistent manner to statements about GS program mechanics.

Table Two
Correlation Matrix of Program Mechanics 

Correlation is significant at the 0.01 level

While each correlation reported above was statistically significant, there was some variation in the strength of the correlations. For instance, respondents marked boxes in a less than uniform manner when reacting to statements about advising practices and problems students have finding classes, whether the students reside in or outside of the Kearney area (.161 and .171). On the other end of the spectrum, faculty responses to the statement about completing the program in 45 hours were strongly correlated with responses to all other statement about program mechanics. In other words, faculty who thought it was routine to complete the GS program in 45 hours were also more likely to think that students did not need to adjust their program significantly to complete it and the reverse was true for faculty who believed students did have a difficult time completing the program in 45 hours.

For the most part, faculty members seemed to think there are few problems with mechanics. A quick look at Table Two shows the frequencies for faculty responses to the statement about finishing the program in the required number of hours. In this table, the numbers make it apparent most respondents (71.2%) “agreed” or “strongly agreed” that students had few problems finishing the program in the 45 hours it was designed to take. This same kind of response pattern was found in the frequencies of all the variables that assessed elements of GS program mechanics.

Table Three
Students have Few Problems Finishing in 45 Hours 

Program Strengths

A response pattern similar to the one that emerged with the questions about program mechanics was found with responses to questions about program strengths. During the last program review, several strengths were identified. To try to determine the status of program strengths, as identified by the previous review, we wrote statements about the GS program’s strengths and asked faculty to indicate their level of agreement. We once again used the modified Likert Scale response pattern. Table Three provides a correlation matrix of each of the variables for program strength. In general, respondents were consistent in their responses. In fact, the matrix suggests that most faculty were answering all of the questions in either a positive or negative manner—few mixed their answers from negative to positive when it came to analyzing program strengths

Table Four
Program Strength Correlation Matrix 

Correlation is significant at the 0.01 level

Generally speaking, faculty members who like the program seem to agree that all of the strengths listed in the previous program review are still present.

Table Four offers another nice snapshot of the level of agreement found among UNK faculty. In this table, the frequencies for responses to a statement about the GS program meeting the needs of the university are reported. Again, a majority of the faculty agreed (at some level) that the philosophy and objectives of the GS program met the needs of the university but a much smaller percentage (58.3%) registered either an “agree” or “strongly agree” response. A quick review of some of the comments respondents offered about the program’s strengths indicates that some faculty are not as happy with the program’s objectives. As one respondent put it, “We need people who aren’t afraid of change, who know what the small private schools are doing, to take the reins and create a new 'enlightened' GS framework.”

Table Five
Philosophy and Objectives Meet Needs of University 

In fact, a little deeper analysis of faculty member responses to the question about whether the philosophy and objectives of the GS program met university needs helps explain how faculty graded the overall program. As Table Five (below) shows, a linear regression of the overall grade measure and the regression shows that the philosophy and objectives question explained quite a bit of the variance in the overall program grades (32.6%). While it may not be statistically appropriate to use regression analysis with ordinal variables, this regression shows there may be more than a simple association between the variables; there may even be a strong causal relationship. It is likely (according to the F statistic) that the very same faculty who graded the overall program low also believed that the objectives/philosophy of the GS program need revising.

Table Six
Regression Coefficients 

Dependent Variable: Overall Assessment of GS Program
Adjusted R-square = .326
F = 116.036, significant @ .000

In sum, most faculty respondents believed the program strengths identified in the previous review were still strengths. A sizeable minority of faculty respondents had questions about how well the philosophy and objectives of the GS program fit with university needs. Many of those who questioned the philosophy and objectives of the program also gave it an overall grade that was average or below average.

Improvements?

The final portion of the questionnaire asked faculty to assess certain areas where improvements might be made. This section was designed with a format similar to the previous sections of the questionnaire. After numerous comments from faculty editors of the questionnaire, statements in this section were made into positive statements. Each statement was written as though it were a strength and faculty were asked to mark whether they “strongly agree,” “agree,” “somewhat agree,” “somewhat disagree,” “disagree,” “strongly disagree,” or “do not know.” Table Six shows that the correlations between the responses to all of the questions were, once again, strong and significant. We found that only one correlation did not reach the .01 level of significance, that was the correlation between responses to the question about having adequate support for teaching GS classes and the question about whether students had enough opportunities to learn about other cultures through GS classes. In the case of this one association, the measure was still significant at the .05 level and therefore probably a result of some causal relationship. We can say that faculty members were also amazingly consistent in their responses to this set of questions, being either consistently positive or negative.

Table Seven
Program Improvement Correlation Matrix 

           Correlation is significant at the 0.01 level

While there was some variation in response patterns, it was not large. Again, take a look at the frequency of responses for one variable (see Table Seven) and note how most faculty (over 63%) either “strongly agreed” or “agreed” or “somewhat agreed” to the statement about having ample opportunities to evaluate GS classes. This response pattern held throughout the questions asked on areas where the program could be improved. It must be added that roughly 37% of the respondents did disagree to the statement and this suggests that faculty involvement with the program might be improved.

Table Eight
Evaluation Opportunities for Courses have been Sufficient 

What is most important is that the responses from faculty members, from positive to negative, were also associated with their overall evaluation of the program. In fact, when the variable for evaluation opportunities (used above) was included in a regression model (Table Eight) designed to explain faculty respondents’ overall assessment of the program, the model’s fit improved. In fact, when questions about philosophy and opportunities to change GS are used to explain the overall assessment 51% of the variance is explained. Also, a quick examination of the standardized coefficients show that both the variable for improving opportunities to evaluate the program and the variable for assessing whether the objectives fit the university have nearly the same influence on explaining the overall assessment grade given by faculty responding to our survey.

Table Nine
Regression Coefficients 

Dependent Variable: Overall Assessment of GS Program
Adjusted R-square = .510
F = 98.808, significant @ .000

< /BLOCKQUOTE >

One should be careful making causal arguments in the case of responses to attitudinal questions, but it is plain to see that those who graded the GS program high also thought that its philosophy fit with the university’s needs and that there were ample opportunities to make changes to it. In terms of understanding why a sizeable minority of faculty gave the program a low grade (C or lower), we do not have much we can learn about who gave those grades; however, we do know that they, too, were consistent. Those who gave lower grades also tended to disagree to statements that the program offered ample opportunities to make changes and that its philosophy and objectives met university needs.

 

Summary:

Overall, most faculty liked the program and saw few problems with it. These positive evaluations seemed to be true whether people taught GS courses or not. The positive nature of evaluations also could not be explained by any kind of demographic factor. On the other hand, the consistency of the evaluations also meant that faculty who did not evaluate the GS program positively also disagreed to almost all of our statements about the program’s mechanics, strengths and areas for improvement. One could easily argue there is no need to rush to any negative judgments about the GS program based on faculty input but that there are enough faculty members who have serious doubts about the program to warrant serious discussions of its merits. Furthermore, there is some evidence that improving faculty involvement with the program’s evaluation might assuage negative thinking that the program is not as strong as it might be.