PROGRAM
Platte Valley Assessment Conference
Best Practices in Assessment
April 29, 2005
8:00 to 4:00
Nebraskan Student Union
Great Room A&B
Cedar Room
Oak Room
University of Nebraska at Kearney
Kearney, Nebraska
|
10 Guidelines for Assessing As If Learning Matters Most
If learning really matters most, then our assessment practices should help students develop the skills, dispositions, and knowledge needed to:
Engage actively — intellectually and emotionally — in their academic work.
Set and maintain realistically high, personally meaningful expectations and goals.
Provide, receive, and make use of regular, timely, specific feedback.
Become explicitly aware of their values, beliefs, preconceptions, and prior learning, and be willing to unlearn when necessary.
Work in ways that recognize (and stretch) their present learning styles or preferences and levels of development.
Seek and find connections to and real-world applications of what they’re learning.
Understand and value the criteria, standards, and methods by which they are assessed and evaluated.
Work regularly and productively with academic staff.
Work regularly and productively with other students.
Invest as much engaged time and high-quality effort as possible in academic work.
Angelo, T. (1999). Doing assessment as if learning matters most. AAHE Bulletin, May 1999.
Friday, April 29, 2005
8:00 – 9:00
Great Room A&B
Registration and Continental Breakfast
9:00 – 10:00
Cedar Room
Dr. William Pallett
Dr. William Pallett has served as director of The IDEA Center (Individual Development and Educational Assessment Center) for the past eight years. Since joining the Center, he has been involved in all phases of the student rating system’s development and research.
Using Student Ratings for Multiple Purposes
The presentation will describe the IDEA Student Ratings of Instruction system and its use. With its unique focus on specific student learning goals, the data from the student ratings can be used to:
Support institutional or disciplinary accreditation and program review
Make informed decisions related to curricular/course revision
Samples of the instruments and reports will be provided.
10:00 – 11:00
Paper Sessions - Cedar Room
9:00 – 9:25
Vital or Vile: Faculty Perceptions of Student Evaluations at UNK
Jean Mandernach, Jessica Lieske and Demaris Dynak, Psychology UNK
Contradictory research indicates that student evaluations can be a reliable and moderately valid indicator of an instructor’s level of teaching effectiveness, but evaluations may be skewed according to grading leniency, instructor popularity, gender, course difficulty and other factors unrelated to instructional efficacy. A survey of UNK faculty revealed that most UNK instructors support the continued use of student evaluations. Models for enhancing the value of applicability of student evaluations will be discussed.
9:25 – 9:50
Meeting Social and Behavioral Science Objectives: The Case of General Psychology
Richard Miller and Theresa Wadkins, Psychology UNK
This study examined the extent of student’s knowledge, skills and values required by the UNK General Studies program for the social and behavioral sciences. A comprehensive examination addressing both general and specific objectives of the general studies program was administered electronically to two groups of students: (1) students who had never been enrolled and were not currently enrolled in a general studies course; (2) students whose first and only course in social and behavioral science was PSY 203.
Paper Sessions - Oak Room
9:00 – 9:25
The Challenge of Accepting Quality Students for a Masters in Counseling
Marlene Kuskie and David Hof, Counseling and School Psychology UNK
The challenge within any graduate program is to accept students that have the greatest probability of being successful in the program and profession. The CSP department has gone through an extensive revision of their application and intake process. Components and elements of the assessment process in accepting students will be shared with an overview of data that is being collected and analyzed.
9:25 – 9:50
Quality Assessment and the Accreditation of a College Academic Program
Teara Archwamety and Max McFarland, Counseling and School Psychology UNK
The histories of both quality assurance (manufacturing/business) and accreditation (educational Institutions) will be outlined. An ERIC database search indicating the merging of the two concepts in the last 20 years will be described. An account of how a school psychology program in the U.S. went through accreditation will be shared.
11:00 – 12:00
Panel Discussion - Oak Room
11:00 – 11:55
Teacher Work Samples: Teacher Candidates, College Faculty, and Cooperating Teachers Learning Together
David Rogers, Mary Beave, Barb Feezell, and Kass Rempp, Hastings College
One way to measure the performance of teacher candidates is with a Teacher Work Sample. A Teacher Work Sample is a collection of a teachers’ work which includes unit lesson plans that are aligned with learning targets or standards, pre and post assessment data, a reflection on student learning, and a description of the special characteristics of students. Two teacher candidates and their college faculty will discuss their experiences with work sampling.
Paper Sessions - Cedar Room
11:00 – 11:25
Assessment Plan for the Introductory Courses Offered by the Physics Department at UNK
Jose Mena-Werth, Physics UNK
Introductory courses in physics must meet the needs of more students than just our majors. Departments such as Aviation, Biology, chemistry, Health Sciences, Mathematics, Physical Education and Teacher Education require introductory courses in Physics and Physical Sciences. We currently are implementing our assessment plan to ensure that we meet the needs of students taking our algebra-based and calculus-based physics courses. The presentation will cover the Physics Assessment Plan that is in full operation this year.
11:25 – 11:55
Investment Portfolios: Direct Writing Assessment for Programs
Kathryn N. Benzel and Jeremy Schnieder, English/Writing Center UNK
The process used to create and implement a direct writing component assessment for the First Year program will be presented with an emphasis on the connection between the program objectives and the assessment “tool”. Furthermore, the results, the decisions, and the lessons learned may suggest a model for similar future projects.
Luncheon and Keynote Address
12:00 – 1:00
Great Room A&B
Dr. Arn Froese
As Assessment Director for Sterling College, Dr. Froese has assisted the College in developing a “clear and simple” assessment program. Froese presented Sterling’s integrated assessment model at the NCA convention in 2004 and has served on NCA’s “Think Tank” to develop new initiatives for supporting good practices in assessment.
“Righteous” Assessment: Coordinating Means and Ends
Dr. Froese suggests that righteousness exists when people use right means to achieve good ends. Assessment’s good ends involve improving student learning. Collecting data about student outcomes by itself is not an adequate means to achieve assessment’s ends. In his address, Froese will describe requisite “right means,” and illustrate how means contribute to good ends.
1:00 – 2:00
Poster Session - Great Room A&B
A. Examining Exams: Learning Outcomes as a Function of Course Level
Theresa Houlihan and Jean Mandernach, Psychology UNK
It is assumed that classes intended for Junior and Senior-level students incorporate progressively more advanced learning outcomes. As such, upper-division courses should go beyond simply providing more information; advanced courses should encourage a deeper understanding and application of a student’s knowledge. The current study analyzes the cognitive complexity targeted by various courses at UNK as measured through the depth of understanding required by typical course assessments. An examination of the type and depth of learning that occurs in a given course is an essential component of an effective program review.
B. Blackboard Versus Paper: Trends in Assessment at UNK
Jean Mandernach, Psychology UNK
Instructors have less time and money available for their teaching; thus it is essential to implement methods that maximize the educational effectiveness and efficiency of these limited resources. Online course management systems provide an effective, efficient means of supplementing a traditional face-to-face class and enhancing learning through the use of online assessment. The University of Nebraska at Kearney adopted the Blackboard course management system for campus-wide implementation in 2000; Blackboard usage rates have continued to rise since its introduction. As the popularity of Blackboard grows, so does its effectiveness as a means of supplemental online assessment for the traditional classroom.
C. Online Equals Alone: Perceptions of Group Work in Online Courses
Jean Mandernach, Psychology UNK
In early models of online instruction, assessments in online courses relied primarily on threaded discussions or papers. Advances in technology and a growing trend to diversity assessments have lead to the emergence of online group projects. While group projects have a well-established history in traditional educational environments, the value of group work in the online classroom is less clear. Online faculty report that group projects are a valuable tool as group work provides a realistic assessment of requisite professional skills. In contrast, online students indicate that group projects are counterproductive to their choice to pursue an online education.
D. Students’ Perceptions of the Current General Studies Program
Jennifer Bienhoff, Megan Boss, Chris Denton, Alisha Nabity, Tom Schlund and Tanis Saldivar, UNK Student Assessment Committee
As part of the commitment to student participation in assessment, the Student Assessment Committee adapted a General Studies Survey to allow student input on the current General Studies program. 594 undergraduate students responded to the online survey and indicated their agreement or disagreement with a series of statements about the current program. Results are provided for several of the key questions on the survey.
E. The Student Assessment Committee: Including Students in the Assessment Process
Jennifer Bienhoff, Megan Boss, Chris Denton, Alisha Nabity, Tom Schlund and Tanis Saldivar, UNK Student Assessment Committee
The key to developing a student culture of assessment is to first get student representatives involved in the process to give voice to student concerns and interests in assessment. The Student Assessment Committee was formed to represent the concerns of students; to disseminate information; and to educate fellow students about current assessment initiatives through
surveys, conversations, presentations, and focus groups.
1:00 – 2:00
Panel Discussion - Oak Room
1:00 – 1:55
Assessment Issues in Online Learning
Roxanne Sullivan, Bellevue University--Steve McGahan, Continuing Education UNK--Jeanne Butler, Office of Assessment UNK
Various issues of assessment in online courses/programs will be discussed. These issues include: (1) Assessment of students enrolled in online courses that serve as both general education and program requirements; (2) assessment techniques that meet both institutional and program goals/ objectives; (3) course evaluation instruments and delivery systems that meet the specific needs of online learners; (4) multi-faceted approaches to assessing all aspects of online courses and programs.
Paper Sessions - Cedar Room
1:00 – 1:25
Towards an Integrated Model of Assessment: Merging Accreditation and Institutional Practices
Maha Younes, Social Work UNK
This session provides rationale, guidelines, and one approach for integrating assessment practices mandated by accrediting bodies of professional programs with those required for higher education institutions. This integration requires careful consideration of the needs of each body, negotiation of areas of interest, and problem-solving potential differences.
1:25 – 1:50
Assessing Implementation of Assessment with the HLC/NCA’s “Assessment Culture Matrix”
Jessica Jonson, University of Nebraska Lincoln
How can an institution assess how well it is implementing assessment so that progress can be shared internally and externally? This session discusses an approach using the HLC/NCA’s “Assessment Culture Matrix” that identifies indicators for monitoring institutional progress and uses that information in accreditation reporting.
2:00 – 3:00
Panel Discussion - Oak Room
2:00 – 2:55
A Snapshot of the Assessment Process in UNK’s College of Business and Technology: Continuous Improvement in the Making
David Palmer, Sandy Lebsack, Tim Burkink, Marsha Yeagley, Greg Broekemier, College of Business and Technology UNK
The assessment plan for UNK’s College of Business and Technology will be presented along with the college-wide processes/issues of the senior self-assessment, alumni survey, and the employer focus group. The process/ issues of the Major Field Test for all business students will be discussed.
Paper Sessions - Cedar Room
2:00 – 2:25
Assessing Assessment—Focus Groups and Students’ Views of a Proposed Assessment Test
Michelle Fleig-Palmer and Ada Leung, UNK
In evaluating assessment tests, focus groups are one method used to solicit students’ opinions. This session will discuss one example of a pilot study conducted at the University of Nebraska Kearney. Focus group techniques were utilized to uncover students’ perceptions of a test measuring knowledge of general studies. Recent research on focus group methodology, as well as qualitative research software for analyzing students’ comments will be presented.
2:25 – 2:50
Implementation of an Assessment Plan for an Athletic Training Education Program: Transitioning to a Paperless Format
Scott Unruh, Athletic Training Education Program UNK
This discussion will present a master assessment plan for the Athletic Training Education Program at UNK. Summative and Formative measures for assessment of student learning and program effectiveness will be presented. The presentation will also outline progress toward implementation of technology to convert from a paper portfolio system of recording student progress.
3:00 – 4:00
Panel Discussion - Oak Room
3:00 – 3:55
Issues in General Studies Assessment
Michelle Lang, Art; Jeremy Dillon, Geography; Jeanne Butler, Office of Assessment UNK
This presentation is a summary of several issues of assessment in the UNK General Studies program. The issues to be covered include: (1) Process and content in designing a common assessment tool for different sections of Art Appreciation; (2) quantitative assessment of student achievement in course content and General Studies outcomes in a diverse discipline; (3) using multiple assessment methods to assess student learning in a General Studies Program.
Paper Sessions - Cedar Room
3:00 – 3:25
The Online vs. Traditional Classroom: What Do Students Learn and Retain?
Richard Miller, Theresa Wadkins, and Robert Rycek, Psychology UNK
This study compared student outcomes as measured by standard examinations completed by students enrolled in online psychology courses to students enrolled in the same courses taught in a traditional classroom. The extent of initial student learning was examined by comparing test performance on tests administered throughout the semester by instructors who taught the same course and used the same exams for both their online and traditional class. Retention was measured using a comprehensive examination of students six months after they had completed either the online or traditional course in Human Development.
3:25 – 3:50
Techniques for Grading Student Writing
Sandy Cook-Fong, Social Work UNK and Dan Fong, Kearney High School
Assessing student learning is a challenge for new faculty members and often continues to be a frustration for senior faculty members. This session will provide a model framework for assessing writing, as well as give suggestions for feedback to students that will empower them to improve their writing.