Chemistry Reports

Report - 2003

Chemistry Assessment Report 2003: Goal 1

Submitted Fall, 2003

Department Learning Goal #1 -- Upon completing a course required for a Chemistry degree, students will have developed the ability to critically evaluate and solve problems through the application of appropriate chemistry principles.

Measurement Tools and Participants

All of the above learning objectives were measured using standardized national exams (produced by the American Chemical Society (ACS). Participants were all students in the following courses, each of which has an associated subject-specific ACS Exam.

Learning Objective Subject Area Course(s) Associated ACS Standardized Exam
General Chemistry
  • Fall CHEM 160 - 1st−Semester General Chemistry
    Spring CHEM 161 - 2nd−Semester General Chemistry
 
  • ACS 1st−Term General Chemistry Exam
  • ACS General Chemistry Exam (2−semester content)
 
Organic Chemistry
  • CHEM 361 - 2nd-Semester Organic Chemistry
 
  • ACS Organic Chemistry Exam (2-semester content)
 
Analytical Chemistry
  • CHEM 301 - Techniques of Chemical Analysis
  • CHEM 475 - Instrumental Analysis
 
  • ACS Analytical Chem. Exam
  • ACS Instrumental Analysis Exam
 
Physical Chemistry
  • CHEM 480 - 1st-Semester Physical Chemistry
  • CHEM 481 - 2nd-Semester Physical Chemistry
 
  • ACS Thermodynamics Exam
  • ACS Dynamics and Quantum Chemistry Exams
 

Benchmark criterion for successful achievement:

Student exam scores will average at or above the 50th percentile (the national average)

Results and Discussion

   Learning Objective   
Subject Area
Courses ACS Exam Average(percentile) Meets Criterion?
General Chemistry    CHEM 160
   CHEM 161
   49   
58
   no   
yes
Organic Chemistry    CHEM 361    71       yes   
Analytical Chemistry    CHEM 301
   CHEM 475
Not available (as of 8/28)
64

yes
Physical Chemistry    CHEM 480
   CHEM 481 (dynamics)
   CHEM 481 (quantum)
   37   
81
not given*
   no   
yes

*this exam has been discontinued in CHEM 481 

All courses except CHEM 160 and 480 met the benchmark criterion. Future standardized exam results from these courses will be monitored. Two more successive years of failing to meet the benchmark criterion will trigger a more detailed review.


Chemistry Assessment Report 2003: Goal 2

Submitted Fall, 2003

 

Department Learning Goal #2 -- Chemistry graduates will be able to coherently organize and present chemical information through written and oral presentation.

Measurement Tools and Participants

The learning objectives were measured using an evaluation of the final oral presentations given by all students enrolled in CHEM 469 (Chemistry Seminar). Because all chemistry majors are required to complete this course, this measure evaluates all chemistry graduates. A copy of the evaluation form is attached as Appendix I.

Benchmark criterion for successful achievement:

  • Minimum Proficiency for All Chemistry Graduates - The median evaluation question score for each student will be no lower than 3 points on the 5-point scale.
  • Desired Proficiency for Best-Performing Students -- The median evaluation question score for each student will be no lower than 4 points on the 5-point scale.

Results and Discussion

A summary of evaluation scores is given below:

Average Rating Student #1 Student #2 Student #3 Student #4 Student #5 Class Avg.
Presentation Organization 4.5 5.0 4.0 5.0 4.0 4.5
Effectiveness of Communication 3.5 5.0 3.5 5.0 4.0 4.2
Effectiveness of Illustrations 4.0 3.5 3.5 5.0 3.0 3.8
Understanding & Applications of Chemical Principles 3.5 4.0 3.0 5.0 4.0 3.9
Appropriate Level for Audience 3.5 4.5 4.0 5.0 4.0 4.2
Effective Analysis & Synthesis of Themes from Multiple Sources 3.0 4.0 3.0 5.0 3.0 3.6
Depth of Analysis 2.0 3.5 3.0 5.0 3.0 3.3
Student Median Score 3.5 4.0 3.5 5.0 4.0  

The minimum proficiency criterion was met; however, only one of the two best-performing students met the desired proficiency criterion. No formal review of the CHEM 469 structure and format is required at this time, but future median score/course grade data correlations will be monitored.


Chemistry Assessment Report 2003: Goals 3-4

Submitted Fall, 2003

Department Learning Goal #3 -- Chemistry graduates will have the necessary skills and knowledge to acquire entry-level positions in the field (including industry and high school teaching) or for admission to their graduate or professional schools of choice.

and

Department Learning Goal #4 -- Chemistry graduates will value how their education enables them to make more informed assessments of and take actions on chemistry-related health and environmental issues.

Measurement Tools and Participants

All of the above learning objectives were measured using surveys of graduating seniors The survey was developed by the UNK Department of Chemistry

Benchmark criterion for successful achievement:

  • At least 75% of chemistry graduates who choose to pursue a chemistry-related profession will acquire employment with the company of their choice or gain admission to the graduate or professional school of their choice
  • All scores for questions pertaining to the value of chemistry course content and course activities/resources in the graduates' learning of chemistry will be at least 3 (on the 5-point scale).
  • All scores for questions pertaining to the recommendation of attending UNK and being a chemistry major - "If you were starting your college career over, would you attend UNK?" and if so, ". . . would you be a chemistry major?" - will be at least 3 (on the 5-point scale).
  • Average value-question scores from surveys received from 4-year alumni will be at or above the average from the surveys they had completed just before graduation.

Results and Discussion

Survey Results 

While 6 out of 7 survey respondents (83%) gained employment or admission in their chosen field, only 4 (57%) gained employment or admission in the school or company of their choice - fails to meet benchmark criterion. 

Out of the three graduates who did not gain employment or admission in their preferred company or school, two hold industry positions in the area of analytical chemistry, and one is in graduate school in biochemistry. These three graduates rated the value of courses most closely related to their chosen profession at an average of 4.33 out of 5 points. This indicates that the graduates did not perceive that inadequate chemistry course preparation prevented hiring or admission by the company or school or their choice.

On questions pertaining to the value of chemistry course content, course activities/resources, and recommendation of attending UNK and being a chemistry major, most ratings were very positive, but the following scores below 3 (out of the 5-point scale) were noted:

Rating for Preparation for Profession
General Chemistry (1 response below 3 pts.)
Biochemistry (2 responses below 3 pts.)

Rating for Enhancing Understanding of Chemistry as a Discipline
General Chemistry (1 response below 3 pts.)
Biochemistry (1 response below 3 pts.)

Rating for Enhancing Informed Decision-Making on Health/Environmental Issues
General Chemistry (2 responses below 3 pts.)

Rating for Value of Department Resource or Activity
ACS Student Affiliate - or Chemistry Club (2 responses below 3 pts.)

"Would You Attend UNK Again?" - 1 response below 3 pts. 

"Would You Major in Chemistry Again?" - 1 response below 3 pts. 

General Chemistry. The 2002-03 graduates took CHEM 161 in the Spring of 1998, 1999 or 2000. Standardized ACS final exams scores from these semesters averaged in the 31st, 42nd and 31st national percentiles, respectively. While a link between teaching effectiveness and the low student value responses may be proposed, it is important to point out that three of the four sub-benchmark responses came from a student who received a C+ grade in the course and graduated with a 2.76 GPA. It is likely that the sub-benchmark responses represent an individual satisfaction instead of a systemic problem. This coupled with the most recent CHEM 161 ACS exam average in the 49th percentile indicates that extensive modifications in the General Chemistry curriculum are not warranted at this time.

Biochemistry. The three sub-benchmark responses came from two graduates - one pursuing a career in analytical chemistry (industry) and the other in materials chemistry (graduate school). It is likely that the sub-benchmark scores come from individual dissatisfaction or relevance to individual career choices. The need for extensive modifications in the Biochemistry curriculum is not indicated.

Recommendation for attending UNK and majoring in Chemistry. One of the seven graduates gave sub-benchmark responses to these two questions, but he followed in the comments sections with several complements about the department and faculty. His personal situation would have him attending college out of state if he were starting over.


Chemistry Assessment Report 2003: Assessment of Assessment

Submitted Fall, 2003

Existing Goals, Objectives, Measurement Tools, and Criteria

Based on the 2002-03 assessment results, the existing learning goals, objectives, and measurement tools successfully serve the department's needs. One benchmark criterion for the graduate survey assessment will be modified for the 2003-04 assessment. Currently, the benchmark regarding job/school placement is as follows:

  • At least 75% of chemistry graduates who choose to pursue a chemistry-related profession will acquire employment with the company of their choice or gain admission to the graduate or professional school of their choice

It turns out that 100% of this year's survey respondents gained employment or admission in the job/school field of their choice - the survey indicated 83%, but the 7th and last respondent student secured employment a few weeks after the survey - if not with their preferred company/school. Because so many individual factors not related to curriculum delivery (such as appropriateness of student expectations, job market conditions, specific company/school standards) can significantly contribute to probability of a student gaining employment/admission with/to the specific company or school of their choice, the department will modify the above criterion as follows:

  • At least 75% of chemistry graduates who choose to pursue a chemistry-related profession will acquire employment with a company or gain admission to a graduate or professional school in the field of their choice 

Additional Measurement for Department Learning Goal #1

Because introductory chemistry classes (CHEM 145 - Introductory Chemistry and CHEM 150 - Intro to Organic & Biochemistry) do not use standardized exams, academic achievement in these courses were not adequately assessed.

Local final exams will be used for this assessment measurement. CHEM 145 and 150 instructors will divide their final exam questions into sections corresponding to material covered in the course. The instructors will report the number of students taking the exam and average score on each section. While individual exam questions may vary slightly from year-to-year, the types of exam sections and relative weighting will remain consistent.

Benchmark criterion for successful achievement - The average score on each exam section for each course will be no lower than 70%.

If this benchmark is not met in three successive years, the format and delivery of the course will be reviewed and modified as appropriate.

Additional Objective and Measurement for Department Learning Goal #2

The current assessment plan does not evaluate student writing skills, a critical component in an undergraduate chemistry education. To emphasize the importance of this, an additional learning objective will be incorporated into department learning goal #2:

Learning Objective -- Chemistry graduates will demonstrate the ability to relay proper understanding and application of chemistry principles through written reports. 

Portfolios will be used for this assessment measurement. Files will be established for all declared chemistry majors. From each applicable chemistry course, two samples of writing will be placed in each student's file. Prior to graduation, each student's materials will be evaluated for improvement in organization, clarity, effectiveness of illustrations (when applicable), and understanding and application of chemistry principles.

Benchmark criterion for successful achievement - Each year, faculty evaluation of portfolios for graduating seniors will reveal at least a 90% rate of satisfactory improvement.

If this benchmark is not met in three successive years, the writing components of the chemistry curriculum will be reviewed and modified as appropriate.

Additional Objective and Measurement for Department Learning Goal #4

This goal currently focuses on the value that graduates place on their chemistry course areas. However, value perceptions of General Studies students are particularly important, given the broader scope of student backgrounds and expectations. An additional learning objective will be incorporated into department learning goal #4:

Learning Objective -- Chemistry students in General Studies courses will value the learning of their chemistry course content for the understanding of chemistry as a scientific discipline and enabling them to make more informed decisions on public issues of a scientific nature. 

Surveys will be used for this assessment measurement. Questions will be derived from the Chemistry Department graduate survey and the UNK General Studies objectives. This short survey will be administered in every Chemistry General Studies course (CHEM 145, 150, 160, 161) along with the teaching evaluation.

Students will be asked to respond to the following statements: "This course enhanced my ability to . . ."

  • Use critical thinking, reasoning, and analyzing in solving problems.
  • Apply scientific methodology in a laboratory setting.
  • Comprehend how scientific concepts originate and are validated and refined.
  • Use specialized vocabulary to understand matter and energy.
  • Make more informed decisions about public issues pertaining to science.

Responses will be rated 1 through 5 according to the following scale:

  • 1 = strongly disagree
  • 2 = disagree
  • 3 = neither agree nor disagree
  • 4 = agree
  • 5 = strongly agree

Benchmark criterion for successful achievement - Each year for each General Studies course, the average survey response for each question will be no lower than 3.5.

If this benchmark is not met in three successive years, the General Studies course format and delivery will be reviewed and modified as appropriate.

Additional Department Learning Goal (#5), Objective, and Measurement

The current assessment plan does not evaluate student laboratory skills, a critical component of a chemists function and an undergraduate chemistry education. Because research projects most thoroughly integrate principles and skills learned in individual chemistry courses, an evaluation of student research will be used to assess student laboratory skills. An new department learning goal and learning objective will be added to the assessment plan:

Department Learning Goal #5 - Chemistry students will have developed the ability to use proper scientific methodology and laboratory technique in the investigation of chemical principles through experiment. 

Learning Objective -- Students will demonstrate the ability to critically evaluate and solve problems through the application of appropriate laboratory principles. 

Two measurement tools will be used for assessment pertaining to this learning objective:

National exams will be used for this assessment measurement. Laboratory questions are a part of the ACS standardized exams given in General Chemistry courses (CHEM 160, 161).

Benchmark criterion for successful achievement - Each year, the average laboratory question section score will correspond to the national exam average or better (56% correct on the latest CHEM160 exam and 50% correct on the latest CHEM 161 exam)

If this benchmark is not met in three successive years, the General Chemistry laboratory format and delivery will be reviewed and modified as appropriate.

Projects will be used to directly evaluate General Chemistry student (CHEM 160, 161) laboratory skills. As proper technique directly relates to the quality of experiment results, the results from one quantitative analysis experiment in each course will be assessed.

Benchmark criterion for successful achievement - The best student results in each experiment will lie within the relative error associated with the primary measurement error source. The average student results will deviate from the accepted results by no more than three times the relative error associated with the primary measurement error source. The desired proficiency from the entire student group assessed is that 90% of the experiment results will lie within three standard deviations of the average student results.

If this benchmark is not met in three successive years, the General Chemistry laboratory format and delivery will be reviewed and modified as appropriate.