Chemistry Reports

Report - 2004

Chemistry Assessment Report 2004: Goal 1

Submitted Fall, 2004

Department Learning Goal #1 -- Upon completing a course required for a Chemistry degree, students will have developed the ability to critically evaluate and solve problems through the application of appropriate chemistry principles

Measurement Tools and Participants

The learning objectives associated with Department Learning Goal #1 were measured in two ways:

Standardized national exams (Direct Measure category 1. a. i.) produced by the American Chemical Society (ACS). Participants were all students in the following courses, each of which has an associated subject-specific ACS Exam.

Learning Objective Subject Area Course(s) Associated ACS Standardized Exam
General Chemistry
  • Fall CHEM 160 - 1st−Semester General Chemistry
    Spring CHEM 161 - 2nd−Semester General Chemistry
 
  • ACS 1st−Term General Chemistry Exam
  • ACS General Chemistry Exam (2−semester content)
 
Organic Chemistry
  • CHEM 361 - 2nd-Semester Organic Chemistry
 
  • ACS Organic Chemistry Exam (2-semester content)
 
Analytical Chemistry
  • CHEM 301 - Techniques of Chemical Analysis
  • CHEM 475 - Instrumental Analysis
 
  • ACS Analytical Chem. Exam
  • ACS Instrumental Analysis Exam
 
Physical Chemistry
  • CHEM 480 - 1st-Semester Physical Chemistry
  • CHEM 481 - 2nd-Semester Physical Chemistry
 
  • ACS Thermodynamics Exam
  • ACS Dynamics and Quantum Chemistry Exams
 

Direct measure 1. a. ii. - local final exams - were used for assessing academic achievement in CHEM 145 (Introductory Chemistry) and 150 (Introduction to Organic & Biochemistry). Participants were all students in these courses. Instructors divided their final exam questions into sections corresponding to material covered in the course. While individual exam questions may vary slightly from year-to-year, the types of exam sections and relative weighting will remain consistent.

Benchmark criterion for successful achievement:

  • Standardized national exams -- student exam scores will average at or above the 50th percentile (the national average)
  • CHEM 145, 150 Exams -- The average score on each exam section for each course will be no lower than 70%. If this benchmark is not met in three successive years, the format and delivery of the course will be reviewed and modified as appropriate.

Results and Discussion

   Learning Objective   
Subject Area
Courses ACS Exam Average(percentile) Meets Criterion?
General Chemistry    CHEM 160
   CHEM 161
   51.6   
54.6
   yes   
yes
Organic Chemistry    CHEM 361    77       yes   
Analytical Chemistry    CHEM 301
   CHEM 475
   61   
course not offered
yes
n/a
Physical Chemistry    CHEM 480
   CHEM 481 (dynamics)
   CHEM 481 (quantum)
   55   
course not offered
   yes   
n/a

All courses met the benchmark criterion. This included CHEM 160 and 480, courses that failed to meet the benchmark criterion last year. Future standardized exam results from all of the above courses will be monitored. Three successive years of failing to meet the benchmark criterion will trigger a more detailed review.

CHEM 145 Exam Section Exam Section Average Score (%) CHEM 150 Exam Section Exam Section Average Score (%)
Atomic Theory 71 Molecular Structure 73
Elements, Atoms, Ions 75 Functional Groups 73
Stoichiometry 72 Molecular Properties 73
Bonding 63* Chemical Reactions 73
Intermolecular Forces 68* Biochemistry 73
States of Matter 70    
*did not meet benchmark criterion

Scores from two content areas in CHEM 145 failed to meet the benchmark criterion. These results correlated with anecdotal laboratory evidence that students were weakest in the bonding and intermolecular forces material. Future exam results will be monitored. Two more successive years of failing to meet the benchmark criterion will trigger a more detailed review.

                                                         

Chemistry Assessment Report 2004: Goal 2

Submitted Fall, 2004

Department Learning Goal #2 -- Chemistry graduates will be able to coherently organize and present chemical information through written and oral presentation

Measurement Tools and Participants

The learning objectives were measured using an evaluation of the final oral presentations - direct measure category 1. b. x. -- given by all students enrolled in CHEM 469 (Chemistry Seminar). Because all chemistry majors are required to complete this course, this measure evaluates all chemistry graduates. A copy of the evaluation form is attached as Appendix I.

Benchmark criterion for successful achievement:

  • Minimum Proficiency for All Chemistry Graduates - The median evaluation question score for each student will be no lower than 3 points on the 5-point scale.
  • Desired Proficiency for Best-Performing Students -- The median evaluation question score for each student will be no lower than 4 points on the 5-point scale.

Results and Discussion

A summary of evaluation scores is given below:

  Student #1 Student #2 Student #3 Student #4 Student #5 Student #6 Student #7 Class Avg.
Presentation Organization 5.00 4.57 4.79 4.38 4.71 4.46 4.71 4.66
Effectiveness of Communication 4.14 4.07 4.67 3.88 4.00 3.96 4.07 4.11
Effectiveness of Illustrations 4.56 4.07 4.33 4.50 4.21 4.54 4.21 4.35
Understanding & Applications of Chemical Principles 4.75 4.14 4.79 4.25 3.79 3.79 4.07 4.23
Appropriate Level for Audience 4.50 4.57 4.50 4.50 4.21 4.46 4.64 4.48
Effective Analysis & Synthesis of Themes from Multiple Sources 4.56 3.93 4.75 4.13 4.43 4.08 4.00 4.27
Depth of Analysis 4.44 4.07 4.50 3.63 3.71 3.80 3.86 4.00
Median 4.57 4.25 4.56 4.06 4.21 4.17 4.29
Course Grade A+ B A B+ B B B-

(NOTE: The CHEM 469 course grade is based on a written term paper and assignments in additional to presentations)

The minimum and desired proficiency criteria were met. Additionally, there was an improvement from 2002-03 in correlation between presentation results and final course grade. The least-squares correlation coefficient, R, for the Fall 2003 course was 0.74, up from 0.61 in the Fall 2002 course.

Note on Portfolios

A second direct measure, 1. b. xii. - portfolios - is in preparation. Files are being established for all declared chemistry majors. From each applicable chemistry course, two samples of writing are placed in each student's file. Prior to graduation, each student's materials will be evaluated for improvement in organization, clarity, effectiveness of illustrations (when applicable), and understanding and application of chemistry principles. This evaluation will not commence until the 2006-07 academic year, when 4-year portfolios will first be available for graduating seniors. 


Chemistry Assessment Report 2004: Goals 3-4

Submitted Fall, 2004

Department Learning Goal #3 -- Chemistry graduates will have the necessary skills and knowledge to acquire entry-level positions in the field (including industry and high school teaching) or for admission to their graduate or professional schools of choice.

and

Department Learning Goal #4 -- Chemistry graduates will value how their education enables them to make more informed assessments of and take actions on chemistry-related health and environmental issues.

Measurement Tools and Participants

All of the learning objectives for Department Learning Goal #3 and most for Department Learning Goal #4 were measured using surveys of graduating seniors - indirect measure category 2. iii. The survey was developed by the UNK Department of Chemistry.

Benchmark criterion for successful achievement:

  • At least 75% of chemistry graduates who choose to pursue a chemistry-related profession will acquire employment with the company of their choice or gain admission to the graduate or professional school of their choice
  • All scores for questions pertaining to the value of chemistry course content and course activities/resources in the graduates' learning of chemistry will be at least 3 (on the 5-point scale).
  • All scores for questions pertaining to the recommendation of attending UNK and being a chemistry major - "If you were starting your college career over, would you attend UNK?" and if so, ". . . would you be a chemistry major?" - will be at least 3 (on the 5-point scale).
  • Average value-question scores from surveys received from 4-year alumni will be at or above the average from the surveys they had completed just before graduation.

Results and Discussion

Survey Results 

Out of five survey respondents, four (80%) gained employment or admission in their chosen field. This meets the benchmark criterion.

The one graduate who did not gain employment or admission in their preferred company or school is currently enrolled at UNK as a non-degree seeking graduate student. This student rated the value of courses most closely related to the chosen profession at an average of 4.0 out of 5 points. This indicates that the graduate did not perceive that inadequate chemistry course preparation prevented hiring or admission by the company or school or their choice.

On questions pertaining to the value of chemistry course content, course activities/resources, and recommendation of attending UNK and being a chemistry major, most ratings were very positive, but the following scores below 3 (out of the 5-point scale) were noted:

Rating for Preparation for ProfessionPhysical Chemistry (1 response below 3 pts.)

Rating for Enhancing Informed Decision-Making on Health/Environmental IssuesGeneral Chemistry (1 response below 3 pts.)

Rating for Value of Department Resource or ActivityACS Student Affiliate - or Chemistry Club (2 responses below 3 pts.)

Physical Chemistry. The one sub-benchmark response regarding the value of this course for preparation for one's profession came from a graduate pursuing a health-related career with the most closely-related courses lying on the biological side of chemistry - physical chemistry is considered to be on the "opposite" side of the chemistry content spectrum. With only one sub-benchmark score, the need for extensive modifications in the Physical Chemistry curriculum is not indicated.

General Chemistry From this single data set, it is difficult to discern the reason for the one sub-benchmark score pertaining to enhancing informed decision-making on health/environmental issues, but there seems to be some consistency with the General Studies survey described below.

Chemistry Club Survey scores in the category appear to simply reflect students' levels of activity in the department's ACS Student Affiliate. Before an extensive review of the Student Affiliate's role in the department is initiated, it would be useful to categorically separate this factor out in future surveys.

Additional Objective and Measurement for Department Learning Goal #4

In addition to the value that graduates place on their chemistry course areas, value perceptions of General Studies students are particularly important, given the broader scope of student backgrounds and expectations. An additional learning objective for department learning goal #4 is below:

Chemistry students in General Studies courses will value the learning of their chemistry course content for the understanding of chemistry as a scientific discipline and enabling them to make more informed decisions on public issues of a scientific nature.

Indirect measure 2. iii. - surveys - was used for this assessment measurement. Questions are derived from the Chemistry Department graduate survey and the UNK General Studies objectives. This short survey was administered in every Chemistry General Studies course (CHEM 145, 150, 160, 161) along with the teaching evaluation.

Students were asked to respond to the following statements: "This course enhanced my ability to . . ."

  • Use critical thinking, reasoning, and analyzing in solving problems.
  • Apply scientific methodology in a laboratory setting.
  • Comprehend how scientific concepts originate and are validated and refined.
  • Use specialized vocabulary to understand matter and energy.
  • Make more informed decisions about public issues pertaining to science.

Responses were rated 1 through 5 according to the following scale:

  • 1 = strongly disagree
  • 2 = disagree
  • 3 = neither agree nor disagree
  • 4 = agree
  • 5 = strongly agree

Benchmark criterion for successful achievement - Each year for each General Studies course, the average survey response for each question will be no lower than 3.5. If this benchmark is not met in three successive years, the General Studies course format and delivery will be reviewed and modified as appropriate.

Results and Discussion

A summary table of survey results from 2003-04 CHEM 145, 150, 160, and 161 students is below:

  Average Score (5-point scale)
General Studies Objective CHEM 145 CHEM 150 CHEM 160 CHEM161
Use critical thinking, reasoning, and analyzing in solving problems. 4.40 3.88 4.13 4.50
Apply scientific methodology in a laboratory setting. 4.51 3.82 4.18 4.22
Comprehend how scientific concepts originate and are validated and refined. 4.33   3.87 4.11
Use specialized vocabulary to understand matter and energy. 4.34   4.06 4.44
Make more informed decisions about public issues pertaining to science. 4.12   3.71 4.00

The benchmark criterion was met; however, the lowest score - 3.71 for CHEM 160 in the area of "making more informed decisions about public issues pertaining to science" - was noted by the department and there is a connection between this result and the General Chemistry sub-benchmark score for this same value category in the graduate surveys described above. Individual instructors will make a conscious effort increase connections between CHEM 160 material and public science issues where appropriate.


Chemistry Assessment Report 2004: Goal 5

Submitted Fall, 2004

Department Learning Goal #5 -- Chemistry students will have developed the ability to use proper scientific methodology and laboratory technique in the investigation of chemical principles through experiment.

Measurement Tools and Participants

Two measurement tools were used for assessment pertaining to this learning goal. All General Chemistry (CHEM 160 & 161) students were assessed

Direct measure category 1. a. i. - national exams - were used to assess laboratory content understanding. Laboratory questions are a part of the ACS standardized exams given in General Chemistry courses (CHEM 160, 161).

Direct measure category 1. b. xiii. - projects - were used to directly evaluate General Chemistry student (CHEM 160, 161) laboratory skills. As proper technique directly relates to the quality of experiment results, the results from one quantitative analysis experiment in each course will be assessed.

Benchmark criteria for successful achievement:

Laboratory Exam Questions. Each year, the average laboratory question section score will correspond to the national exam average or better (56% correct on the latest CHEM160 exam and 50% correct on the latest CHEM 161 exam)

If this benchmark is not met in three successive years, the General Chemistry laboratory format and delivery will be reviewed and modified as appropriate.

Laboratory Projects. The best student results in each experiment will lie within the relative error associated with the primary measurement error source. The average student results will deviate from the accepted results by no more than three times the relative error associated with the primary measurement error source. The desired proficiency from the entire student group assessed is that 90% of the experiment results will lie within three standard deviations of the average student results.

If this benchmark is not met in three successive years, the General Chemistry laboratory format and delivery will be reviewed and modified as appropriate.

Results and Discussion

The table below summarizes results from both assessment tools:

Laboratory Exam Questions % Correct Benchmark Criterion (%) Met Criterion?
CHEM 160 58.9 56 yes
CHEM 161 54.6 50 yes
       
Laboratory Projects % Students within 3 Standard Deviations Benchmark Criterion (%) Met Criterion?
CHEM 160 - Standardization of a NaOH Solution 92 90 yes
CHEM 161 - Determination of Solubility Product Constant of Calcium Hydroxide 93 90 yes

All benchmark criteria were met. Result trends over a four-year cycle will be monitored.


Chemistry Assessment Report 2004: Assessment of Assessment

Submitted Fall, 2004

Existing Goals, Objectives, Measurement Tools, and Criteria

Based on the 2003-04 assessment results, the existing learning goals, objectives, and measurement tools successfully serve the department's needs. The only modification that will be implemented for the 2004-05 assessment period is the addition of the following question to the graduate survey:

What was your level of activity in the department Chemistry Club? (circle one)

Very Active Somewhat Active Not Active

This will allow the department to determine if student values regarding the chemistry club simply reflect their level of participation or are a true value assessment of the role of the club in the department.

General Studies

The General Studies objectives over which CHEM 145, 150, 160, and 161 student are surveyed could be said to establish minimum levels of exposure to and proficiencies in Chemistry content and methodology for all UNK students. True fulfillment of these objectives would be most accurately assessed using data from non-science majors. Unfortunately, this audience makes up a very small portion of chemistry General Studies classes.

All UNK students must choose science courses from two out of four science departments -at least one of the two courses must include a laboratory component. As the table below shows, very few students choose a chemistry course as a General Studies elective.

2002-03 Major Distribution for Students Enrolled in Natural Science General Studies Courses

        Gen. Studies Courses             Chemistry         Biology         Physics         Geography    
% degree students* 79.6% 48.9% 33.6% 7.9%
% "elective" students** 11.2% 39.0% 55.8% 74.2%
% undeclared majors 9.3% 12.1% 10.7% 18.0%

*those taking a department's general studies course because it is required for their academic program

**those taking a course to meet a requirement for general studies, minor, or 2nd major

With such a small percentage of non-science majors (who come from the General Studies "elective" students group) in chemistry General Studies courses, data collected by the Chemistry Department for General Studies assessment is of very limited use. If administrative steps were taken to equalize non-science major enrollment across the four natural science departments, much more meaningful General Studies assessment could be conducted.

Undergraduate Research

Collaborative student/faculty research has become an increasingly essential component in the undergraduate chemistry curriculum delivery. There are a number of positive outcomes that are often discussed in anecdotal and qualitative fashions. Development of meaningful assessment tools has been an issue of national debate due to issues such as complicated control factors, small student numbers (per discipline), and the great breadth and paradigm variety among academic disciplines. The department is interested in exploring the development of undergraduate research assessment in cooperation with the Coordinator of Assessment.