University of Nebraska Kearney

UNK
Random UNK images
Assessment
related links

Economics Assessment Report 2004 

Submitted Fall, 2004 

Economics students were assessed using two direct and one indirect assessment measures.

The direct measures were scores on the Major Field Exam and scores on the assessment rubric for student research papers. The nationally-normed Major Field Exam scores were generated by students taking the exam in BMGT495 - Administrative Strategy and Policy during Final Week of Fall 03, Spring 04 and Summer 04. The results, scored and provided by ETS, were then compared to previous results. To assess the student research papers, two sets of three faculty reviewers each read ten randomly selected student papers from upper division Economics courses. Each faculty panel read a different set of papers, so a total of twenty student papers were reviewed using the Department's assessment rubric.

The indirect measure is the College of Business and Technology Skills Competencies Survey administered by the Director of the Center for Rural Research and Development.

Direct Measures

Major Field Exam 

The following tables compare the performance of the Economics Comprehensive Business and Agri-Business students to all College of Business & Technology students. Due to small numbers of Econ/AgBusiness students, they are grouped together in the exam report. The number in parenthesis following the raw score is the national percentile.

Fall 2003
Students B & T Econ/Agri−Business Difference
Mean (%ile) 151.5 (45) 153.1 (55) 1.6
Std. Dev. 11.6 10.5  
Std. Error 1.31 3.51  
n 80 10  
Mean % correct      
Accounting (%ile) 43.6 (43) 44.6 (51) 1.0
Economics (%ile) 40.3 (33) 48.3 (77) 8.0
Management (%ile) 61.2 (62) 60.8 (59) -0.4
Quantitative Business Analysis (%ile) 54.4 (37) 59.1 (61) 4.7
Finance (%ile) 35.6 (46) 38.3 (62) 2.7
Marketing (%ile) 49.0 (60) 46.9 (45) -2.1
Legal and Social Environment (%ile) 50.0 (50) 44.3 (17) -5.7
International Issues (%ile) 41.4 (32) 42.2 (36) 0.8
Spring 2004
Students B & T Econ/Agri−Business Difference
Mean (%ile) 155.9 (65) 152.0 (40) -3.9
Std. Dev. 12.8 16.2  
Std. Error 1.47 7.25  
n 76 5  
Mean % correct      
Accounting (%ile) 47.8 (64) 43.0 (30) -4.8
Economics (%ile) 44.7 (54) 52.0 (85) 7.3
Management (%ile) 63.2 (71) 54.8 (24) -8.4
Quantitative Business Analysis (%ile) 58.3 (48) 57.0 (45) -1.3
Finance (%ile) 43.1 (81) 46.2 (86) 3.1
Marketing (%ile) 51.5 (68) 44.4 (27) -7.1
Legal and Social Environment (%ile) 53.9 (65) 43.4 (12) -10.5
International Issues (%ile) 46.2 (56) 54.0 (85) 7.8

A total of 15 Business Economic Comprehensive or Agri-Business students graduating in 2003-04 took the Major Field Exam. In general, they performed close to other business students. Beginning in Fall 2003, Econ students and Agri-Business students were combined on the test results to justify an identified column in the results. Previously, they had been included in the "Other" category. Henceforth, then, the results for both Business Econ Comprehensive and Agri-Business will be combined in annual assessment reports. The results of student performance on the Major Field exam are summarized in the following table:

Mean Score
  Economics Students B&T Difference
Semester Score No. of students    
Spring 2002 156 (1) 153.6 2.4
Fall 2002 157 (1) 156.4 0.6
Spring 2003 154 (4) ***  
Econ/Agri−Business Students
Fall 2003 153.1 (10) 151.5 1.6
Spring 2004 152 (5) 155.9 -3.9

Given the small sample size, little can be said about the performance of Economics students relative to other business students. As one can see from the table, in Spring 2002, Fall 2002, and Fall 2003 the Econ/ AgBusiness students did slightly better than the mean for all students taking the exam. Because there was a major violation of testing protocol in Spring 2003, a UNK mean was not calculated. It is not known if some of the Economics student scores were impacted. If, in the future, the Spring 2003 score appears to be an anomaly it will be discarded.

Student Research Papers 

The Economics Department uses an assessment rubric to evaluate student papers regarding the student's learning outcomes. A randomly selected sample of student papers from upper division Economics courses was evaluated by two three-faculty member panels. Papers were evaluated regarding the ability of the student to demonstrate the following:

  1. An understanding of the core knowledge of the discipline
  2. The ability to find relevant economic data and use it in appropriate ways.
  3. The ability to read and comprehend economic literature.
  4. The ability to write in a manner appropriate to the discipline.

The results are given in the linked table. The 2002-2003 Academic Year was the first year that this assessment activity was undertaken, so the 2003-2004 results are compared with the previous year.

Indirect Measures

The Economics Department sends out a questionnaire to alumni every five years. As part of that survey, alumni are asked to rank undergraduate economics courses on a scale of 1 to 5, with 1 indicating a course now ranked by the alum as most useful and 5 least useful. The latest Alumni survey was sent out in Fall 2000. While there are no new numbers, the results of the Fall 2000 are included in this report. When the next survey is done in the Fall 2005, the Fall 2000 responses will serve as a baseline to begin tracking results over time.

Economics Alumni Questionnaire Results (1 = most useful on scale of 1-5) 

25 respondents - Fall 2000

Rank 100 270 271 314 320 321 322 335 350 380 395 398 420 430 465 499
1   3 4 1 2 1 6 1 11     12 1   1 3
2 2 7 5 2 4 4     2     2   2    
3 1 4 5 1 2 3     1   1   3 1    
4 2 2 2 5     1   3   1 2        
5   3   2 1   2           1 1    

Results of the College of Business and Technology Skills Competencies Survey administered by the Director of the Center for Rural Research and Development, assess the business division's curriculum & program.

Recommendations for Continuous Improvement

The following three paragraphs contain the recommendations made by the faculty as part of the 2002- 2003 assessment.

  1. On the basis of the Major Field exam scores, students in the "Other" category which contained the Economics students tended to score relatively low in Quantitative Business Analysis and Legal and Social Environment. All professors teaching upper division economics courses will be informed of this, and asked to make changes in course material and assignments to improve student performance in these areas.
  2. On the basis of faculty review of student papers, all professors teaching upper division economics courses will be asked to make changes in course material and assignments to improve student performance regarding the use of economic theory, appropriate methodology, data use, and writing specific to the major.
  3. Faculty will be informed of the results of the Fall 2000 alumni survey, and asked to make appropriate changes in course content and assignments in courses identified as not useful. 
 

Results of 2002 - 2003 Recommendations 

  1. Professors in upper division courses included additional quantitative assignments, which seemed to improve student performance on the Major Field Exam. In Fall 2003, Econ/AgBusiness students did better than the College overall on this criteria. In Spring 2004, Econ/AgBusiness students were slightly below the College score.
  2. The number of professors requiring research papers in upper division courses increased.
  3. The Department made a change in operating procedures to improve the overall quality of upper division courses. The Department now tries to schedule each professor so he/she has only one upper division course in each semester. The Department will monitor student performance to see if this change brings improvement.
  4. In 2003, the Department added a capstone course as a program requirement for economics students. This will take effect for students entering UNK in 2004. The capstone course will be an important component of the assessment effort in the future. 
 

Recommendations for Continuous Improvement from the 2003 -2004 Assessment 

  1. Faculty discussed the pattern of low scores on the Legal and Social Environment component of the Major Field Exam. Because we have never seen any of the test questions, there is some concern regarding the exact nature of the appropriate response. Where possible, all professors will try to increase coverage of regulatory issues. Also, the Department will add Industrial Organization to the course rotation. IO has not been offered for several years. Regulatory issues are central to the course content in Industrial Organization. The Agri-Business program was changed in 2002 to require students to take Business Ethics. As students under the new requirements begin to take the Major Field exam, scores should improve in the Legal and Social Environment section.
  2. As a general rule, the Department will limit class size to 45 students for Principles courses and 30 students in upper division courses.
  3. More faculty are now requiring student writing beginning with the Fall 2004 semester.
  4. The Department will explore acquiring the Test of Understanding in College Economics as a direct measure for the General Studies course offerings. 
 

Ongoing Assessment of the Assessment Plan

From 2002 -2003
From this first iteration, the Department needs to make slight changes in its Assessment activities. First, Economics students need have more details reported from the Major Field Exam. Right now, Economics students are grouped into the "Other" category, so the only information available to the Department is the overall score of each individual student. Second, all upper-division courses with research paper assignments should keep a copy so the "sample" of student papers is not coming from only two courses. In the future, once the capstone course is in place, this issue will disappear.

From 2003 -2004
By grouping Economics students with Agri-Business Students, we were able to get a dedicated column in the Major Field Exam report matrix. This is useful. The sample of student papers is now coming from 4 courses rather than 2. This number will increase in future semesters as more professors require research papers. We need to develop an appropriate assessment plan for the General Studies offerings in the Department. The Department will review the Test of Understanding in College Economics as a possible direct measure.