Chemistry Reports

Report - 2005

Chemistry Assessment Report 2005: Goal 1

Submitted Fall, 2005

Department Learning Goal #1 -- Upon completing a course required for a Chemistry degree, students will have developed the ability to critically evaluate and solve problems through the application of appropriate chemistry principles.

Measurement Tools and Participants

The learning objectives associated with Department Learning Goal #1 were measured in two ways:

Standardized national exams (Direct Measure category 1. a. i.) produced by the American Chemical Society (ACS).  Participants were all students in the following courses, each of which has an associated subject-specific ACS Exam.

Learning Objective Subject Area Course(s) Associated ACS Standardized Exam
General Chemistry • Fall CHEM 160 - 1st-Semester General Chemistry
• Spring CHEM 161 - 2nd-Semester General Chemistry
• ACS 1st-Term General Chemistry Exam
• ACS General Chemistry Exam (2-semester content)
Organic Chemistry • CHEM 361 - 2nd-Semester Organic Chemistry • ACS Organic Chemistry Exam (2-semester content)
Analytical Chemistry • CHEM 301 - Techniques of Chemical Analysis
• CHEM 475 - Instrumental Analysis
• ACS Analytical Chem. Exam
• ACS Instrumental Analysis Exam
Physical Chemistry • CHEM 480 - 1st-Semester Physical Chemistry
• CHEM 481 - 2nd-Semester Physical Chemistry
• ACS Thermodynamics Exam
• ACS Dynamics Exam

Direct measure 1. a. ii. – local final exams – were used for assessing academic achievement in CHEM 145 (Introductory Chemistry) and 150 (Introduction to Organic & Biochemistry).  Participants were all students in these courses.  Instructors divided their final exam questions into sections corresponding to material covered in the course. While individual exam questions may vary slightly from year-to-year, the types of exam sections and relative weighting will remain consistent.

Benchmark criteria for successful achievement:

  • Standardized national exams -- student exam scores will average at or above the 50th percentile (the national average)
  • CHEM 145, 150 Exams -- The average score on each exam section for each course will be no lower than 70%.  If this benchmark is not met in three successive years, the format and delivery of the course will be reviewed and modified as appropriate.  
Learning Objective Subject Area Courses ACS Exam Average
(percentile)
Meets Criterion?
General Chemistry CHEM 160
CHEM 161
42
58
no
yes
Organic Chemistry CHEM 361 55 yes
Analytical Chemistry CHEM 301
CHEM 475
75
99
yes
yes
Physical Chemistry CHEM 480
CHEM 481
58
96
yes
yes

Only CHEM 160 failed to meet the benchmark criterion.  Exceptional results are noted for the Instrumental Analysis (CHEM 475) and Physical Chemistry (CHEM 481) Dynamics exams.  Future standardized exam results from all of the above courses will be monitored.  Three successive years of failing to meet the benchmark criterion will trigger a more detailed review.

CHEM 145 Exam Section Exam Section Average Score (%) CHEM 150 Exam Section Exam Section Average Score (%)
Atomic Theory 74 Molecular Structure 79
Elements, Atoms, Ions 85 Functional Groups 79
Stoichiometry 70 Molecular Properties 79
Bonding 72 Chemical Reactions 79
Intermolecular Forces 58* Biochemistry 79
States of Matter 60*    

*did not meet benchmark criterion

Scores from two content areas in CHEM 145 failed to meet the benchmark criterion.  This is the second year in a row for the intermolecular forces contact, while it is the first year for the states of matter content.  Future exam results will be monitored.  For any content area, three successive years of failing to meet the benchmark criterion will trigger a more detailed review.


Chemistry Assessment Report 2005: Goal 2

Submitted Fall, 2005

Department Learning Goal #2 -- Chemistry graduates will be able to coherently organize and present chemical information through written and oral presentation. 

Measurement Tools and Participants

The learning objectives were measured using an evaluation of the final oral presentations – direct measure category 1. b. x. -- given by all students enrolled in CHEM 469 (Chemistry Seminar).  Because all chemistry majors are required to complete this course, this measure evaluates all chemistry graduates.  A copy of the evaluation form is attached as Appendix I.

Benchmark criteria for successful achievement

  • Minimum Proficiency for All Chemistry Graduates – The median evaluation question score for each student will be no lower than 3 points on the 5-point scale.
  • Desired Proficiency for Best-Performing Students -- The median evaluation question score for each student will be no lower than 4 points on the 5-point scale.

Results and Discussion

A summary of evaluation scores is given below:

  Student #1 Student #2 Student #3 Student #4 Class Avg.
Presentation Organization 4.67 4.33 5.00 4.00 4.50
Effectiveness of Communication 5.00 3.67 4.00 3.67 4.09
Effectiveness of Illustrations 5.00 4.33 4.17 3.67 4.29
Understanding & Applications of Chemical Principles 5.00 5.00 4.50 4.33 4.71
Appropriate Level for Audience 5.00 3.67 4.00 3.67 4.09
Effective Analysis & Synthesis of Themes from Multiple Sources 5.00 5.00 4.67 4.33 4.75
Depth of Analysis 5.00 5.00 4.67 4.33 4.75
Median 5.00 4.33 4.50 4.00
Course Grade A A A A

(NOTE:  The CHEM 469 course grade is based on a written term paper and assignments in additional to presentations)

The minimum and desired proficiency criteria were met.  This assessment supports the “A” grades given to all four students.

Note on Portfolios 

A second direct measure, 1. b. xii. – portfolios – is in preparation.  Files are being established for all declared chemistry majors.  From each applicable chemistry course, two samples of writing are placed in each student’s file.  Prior to graduation, each student’s materials will be evaluated for improvement in organization, clarity, effectiveness of illustrations (when applicable), and understanding and application of chemistry principles.  This evaluation will not commence until the 2006-07 academic year, when 4-year portfolios will first be available for graduating seniors. 


Chemistry Assessment Report 2005: Goals 3-4

Submitted Fall, 2005

Department Learning Goal #3 -- Chemistry graduates will have the necessary skills and knowledge to acquire entry-level positions in the field (including industry and high school teaching) or for admission to their graduate or professional schools of choice.

and

Department Learning Goal #4 -- Chemistry graduates will value how their education enables them to make more informed assessments of and take actions on chemistry-related health and environmental issues.

Measurement Tools and Participants

All of the learning objectives for Department Learning Goal #3 and most for Department Learning Goal #4 were measured using surveys of graduating seniors – indirect measure category 2. iii.  The survey was developed by the UNK Department of Chemistry and can be found in Appendix II.

Benchmark criteria for successful achievement

  • At least 75% of chemistry graduates who choose to pursue a chemistry-related profession will acquire employment with the company of their choice or gain admission to the graduate or professional school of their choice
  • All scores for questions pertaining to the value of chemistry course content and course activities/resources in the graduates’ learning of chemistry will be at least 3 (on the 5-point scale).
  • All scores for questions pertaining to the recommendation of attending UNK and being a chemistry major – “If you were starting your college career over, would you attend UNK?” and if so, “. . . would you be a chemistry major?” – will be at least 3 (on the 5-point scale).
  • Average value-question scores from surveys received from 4-year alumni will be at or above the average from the surveys they had completed just before graduation.

Results and Discussion

A survey summary report is attached as Appendix III.  Results pertaining to benchmark criteria are discussed below.

Out of five survey respondents, four (80%) gained employment or admission in their chosen field.  This meets the benchmark criterion.

The one graduate who did not gain employment or admission in their preferred company or school is currently enrolled at UNK as a non-degree seeking graduate student.  This student rated the value of courses most closely related to the chosen profession at an average of 4.0 out of 5 points.  This indicates that the graduate did not perceive that inadequate chemistry course preparation prevented hiring or admission by the company or school or their choice.

On questions pertaining to the value of chemistry course content, course activities/resources, and recommendation of attending UNK and being a chemistry major, most ratings were very positive, but the following scores below 3 (out of the 5-point scale) were noted:

Rating for Preparation for Profession
Physical Chemistry (3 responses below 3 pts.)

Rating for Understanding of Chemistry
Biochemistry (1 response below 3 pts.)
Physical Chemistry (2 responses below 3 pts.)

Rating for Enhancing Informed Decision-Making on Health/Environmental Issues
Biochemistry (1 response below 3 pts.)
Physical Chemistry (3 responses below 3 pts.)

Rating for Value of Department Resource or Activity
Library Journal Holdings (1 response below 3 pts.)
Library Book Holdings (3 responses below 3 pts.)
ACS Student Affiliate – or Chemistry Club (2 responses below 3 pts.)

In each category, an occasional single low response can often be attributed to a student perceiving that his or her career aspiration poorly match one area of chemistry.  The multiple low responses for physical chemistry are sudden and came unexpectedly.  At this point, the data set is small to establish a reliable trend; however, the department will make greater efforts to point out the relevance and importance of physical chemistry.  This should be reinforced by the curricular change that is being adopted in the physical chemistry courses this year – rearrangement of topic order (now quantum chemistry, thermodynamics, and kinetics, respectively).  Associated with the one low response for library journal holdings was a favorable comment about electronic journals – the student interpreted “library journal holdings” as print-only volumes.  An aim to correct this perception will be made in next year’s survey.  Lastly, as reported last year, response to the value of the ACS Student Affiliate seems to correlate directly with the student’s level of activity in the club.  More is discussed in the Assessment of the Assessment section.

Additional Objective and Measurement for Department Learning Goal #4

In addition to the value that graduates place on their chemistry course areas, value perceptions of General Studies students are particularly important, given the broader scope of student backgrounds and expectations.  An additional learning objective for department learning goal #4 is below:

Chemistry students in General Studies courses will value the learning of their chemistry course content for the understanding of chemistry as a scientific discipline and enabling them to make more informed decisions on public issues of a scientific nature. 

Indirect measure 2. iii. – surveys – was used for this assessment measurement.  Questions are derived from the Chemistry Department graduate survey (Appendix II) and the UNK General Studies objectives.  This short survey was administered in every Chemistry General Studies course (CHEM 145, 150, 160, 161) along with the teaching evaluation.

Students were asked to respond to the following statements:  “This course enhanced my ability to . . .”

  • Use critical thinking, reasoning, and analyzing in solving problems.
  • Apply scientific methodology in a laboratory setting.
  • Comprehend how scientific concepts originate and are validated and refined.
  • Use specialized vocabulary to understand matter and energy.
  • Make more informed decisions about public issues pertaining to science.

Responses were rated 1 through 5 according to the following scale:

  • 1 = strongly disagree
  • 2 = disagree
  • 3 = neither agree nor disagree
  • 4 = agree
  • 5 = strongly agree

Benchmark criterion for successful achievement – Each year for each General Studies course, the average survey response for each question will be no lower than 3.5.  If this benchmark is not met in three successive years, the General Studies course format and delivery will be reviewed and modified as appropriate.

Results and Discussion

Due to miscommunications and inconsistencies in data collection, no data is reported this year.  These surveys will recommence in 2005-06.


Chemistry Assessment Report 2005: Goal 5

Submitted Fall, 2005

Department Learning Goal #5 -- Chemistry students will have developed the ability to use proper scientific methodology and laboratory technique in the investigation of chemical principles through experiment.

Measurement Tools and Participants 

Two measurement tools were used for assessment pertaining to this learning goal.  All General Chemistry (CHEM 160 & 161) students were assessed

Direct measure category 1. a. i. – national exams – were used to assess laboratory content understanding.  Laboratory questions are a part of the ACS standardized exams given in General Chemistry courses (CHEM 160, 161).

Direct measure category 1. b. xiii. – projects – were used to directly evaluate General Chemistry student (CHEM 160, 161) laboratory skills.  As proper technique directly relates to the quality of experiment results, the results from one quantitative analysis experiment in each course will be assessed.

Benchmark criteria for successful achievement 

Laboratory Exam Questions.  Each year, the average laboratory question section score will correspond to the national exam average or better (56% correct on the latest CHEM160 exam and 50% correct on the latest CHEM 161 exam)

If this benchmark is not met in three successive years, the General Chemistry laboratory format and delivery will be reviewed and modified as appropriate.

Laboratory Projects.  The best student results in each experiment will lie within the relative error associated with the primary measurement error source.  The average student results will deviate from the accepted results by no more than three times the relative error associated with the primary measurement error source.  The desired proficiency from the entire student group assessed is that 90% of the experiment results will lie within three standard deviations of the average student results.

If this benchmark is not met in three successive years, the General Chemistry laboratory format and delivery will be reviewed and modified as appropriate.

Results and Discussion 

The table below summarizes results from both assessment tools:

Laboratory Exam Questions % Correct Benchmark Criterion (%) Met Criterion?
CHEM 160 31 37.5 no
CHEM 161 68.4 50 yes
 
Laboratory Projects % Students within 3 Standard Deviations Benchmark Criterion (%) Met Criterion?
CHEM 160 - Standardization of a NaOH Solution 91 90 yes
CHEM 161 - Determination of Solubility Product Constant of Calcium Hydroxide 94 90 yes

The CHEM 160 standardized laboratory exam benchmark criterion was not met.  If the benchmark is not met for two more successive years, a detailed review will be triggered.  In all four assessment areas, result trends over a four-year cycle will be monitored.


Chemistry Assessment Report 2005: Assessment of Assessment

Submitted Fall, 2005

Existing Goals, Objectives, Measurement Tools, and Criteria

Based on the 2004-05 assessment results, the existing learning goals, objectives, and measurement tools successfully serve the department’s needs. The following are issues of concern that have arisen.

Addition of question to Graduate Survey.  As described in the 2003-04 report, it was intended to have the following question added to the graduate survey. Mistakenly, it was not; however, the question will be added to the 2005-06 surveys:

What was your level of activity in the department Chemistry Club? (circle one)

Very Active                 Somewhat Active                   Not Active

This will allow the department to determine if student values regarding the chemistry club simply reflect their level of participation or are a true value assessment of the role of the club in the department.

Laboratory Practical Exams. The department has determined that the existing laboratory project assessments reflects precision in student lab work at the potential expense of accuracy analysis. Starting with the 2005-2006 academic year, the department will establish laboratory final exams that include practical, lab activity-based exercises in CHEM 145, 160L, and 161L. Such lab final exams will more thoroughly and accurately assess student learning in these General Studies lab courses. Exam questions/activities and results will be described in next year’s report.

Potential Problems with 4-Year Portfolios. It has become apparent that many students do not declare chemistry majors until their sophomore or junior year. This limits the number of writing samples – and the timeframe of those samples – that can be collected for the portfolio assessment. In 2006-07, the first year that assessment data in this area will be collected, the impact of this potential problem will be noted.

General Studies

The General Studies objectives over which CHEM 145, 150, 160, and 161 student are surveyed could be said to establish minimum levels of exposure to and proficiencies in Chemistry content and methodology for all UNK students. True fulfillment of these objectives would be most accurately assessed using data from non-science majors. Unfortunately, this audience makes up a very small portion of chemistry General Studies classes.

All UNK students must choose science courses from two out of four science departments –at least one of the two courses must include a laboratory component. As the table below shows, very few students choose a chemistry course as a General Studies elective.

2002-03 Major Distribution for Students Enrolled in Natural Science General Studies Courses 

Gen. Studies Courses Chemistry Biology Physics Geography
% degree students* 79.6% 48.9% 33.6% 7.9%
% “elective” students** 11.2% 39.0% 55.8% 74.2%
% undeclared majors 9.3% 12.1% 10.7% 18.0%