Platte Valley Assessment Conference
Using Assessment Data to Bring About Change
April 20, 2007
8:00 to 4:00
Nebraskan Student Union
Office of Assessment
University of Nebraska at Kearney
Using Assessment Data to Bring About Change
The best use of assessment results is to share them with faculty members and use the information as a tool for facilitating discussion about the continuous development and improvement of the curriculum or degree program. Assessment information should be used to justify things that are working well in the program or curriculum and to identify areas for development or improvement. Assessment information is of little value unless it is shared with appropriate audiences and used in meaningful ways (Palomba & Banta 2000).
Assessment data at UNK have been used to:
- Facilitate discussions at faculty meetings and retreats.
- Make changes in course content.
- Add/delete courses and modify course sequences.
- Alter degree requirements.
- Justify curriculum changes and show program improvements resulting from those changes.
- Make improvements in advising processes.
- Develop academic services for students.
- Obtain funds for laboratory equipment.
- Change emphases for new or vacant faculty positions.
- Enhance program-based websites to provide students and other stakeholders with academic, program, and alumni information.
- Refine the assessment methods or implement new assessment methods.
8:00 – 9:00
Great Room A&B
Registration and Continental Breakfast
9:00 – 10:00
Great Room A & B
Jean M Yerian
Virginia Commonweath University - WEAVEonline
Becoming more Intentional, more Accountable:
Assessment Management for the 21st Century
Beginning in 2001, Virginia Commonwealth University created the WEAVEonline Assessment Management System to capture and guide assessment practice in academic and administrative areas on three campuses. The institution very intentionally went beyond compliance for accreditation to put in place an ongoing system to support external accountability as well as internal quality enhancement. WEAVEonline is now a web-based central repository for a variety of information: it is used for assessment, accreditation, and program reviews. While viewing WEAVEonline as an example of leading-edge assessment management – from the entry of standard assessment cycle elements to advanced features of action tracking over time, annual reporting (no more paper!), and curriculum mapping – we will discuss VCU’s process of building assessment buy-in and practice.
Jean has a rich background in academic and student affairs areas, at VCU and elsewhere. She is one of the developers of WEAVEonline at VCU and, in retirement, continues work on this assessment management system, now in use across the country. Her education includes an A.B. in English from Duke University and a Higher Diploma in Anglo-Irish Literature of the 19th and 20th Centuries from Trinity College, Dublin, Ireland. She earned an M.A. in Student Personnel Work in Higher Education from The Ohio State University. She has served on the Council for the Advancement of Standards in Higher Education (CAS) and led the Virginia Assessment Group.
10:00 – 11:00
10:00 – 10:25 - Ponderosa C
Surveys that Monkeys Can Do (and analyze)
P. Artz, Bellevue University
This session demonstrates an inexpensive, easy to use, web-based survey service, presents sample results that are useful in program assessment, and presents preliminary analysis of university results which compare online versus face-to-face programs as well as accelerated versus course-by-course programs. Presently, Bellevue University uses a web-based interface called Survey Monkey. Our current survey is based on the seven characteristics for good practice in undergraduate education as presented by the research of Chickering and Gamson. The survey results are useful to individual program directors in their assessment reports as indirect measures of student learning. Further analysis at the university level is possible via the export feature of Survey Monkey.
10:00-10:25 - Ponderosa D
Advice on Advising: What Students Want
B. Hyatt, Graduate Assistant for General Studies, UNK
Academic advising is an integral part of the education process at Universities. Therefore, this presentation will focus on what students’ desire from academic advising. The speaker will describe results from a student survey conducted in the spring of 2006 by the Student Assessment Committee. This session will include possibilities for active participation from those who attend. Participants will be encouraged to evaluate their own academic advising process.
10:30-10:55 - Ponderosa D
What Assessment Looks Like when You're a "SPA"
R. Sobansky, T. McFarland, Counseling and School Psychology, UNK
As a Specialty Program Area (SPA) as recognized by NCATE, assessment components/data are tightly prescribed as six required and two optional assessments. This presentation will outline these required assessments and provide an overview of one program's preparation for an electronic accreditation folio review process.
11:00 – 12:00
Panel Discussion - Ponderosa D
11:00 – 11:55
Using Technology to Assist Assessment, Data Collection and Data Dissemination
G. Powell, S. Haberlan, D. Brandt, and J. Petersen, College of Education, UNK
The College of Education’s NCATE accredited Teacher Education unit must meet NCATE Standard 2 which specifies that all education candidates shall be assessed and the data used to facilitate the decision-making process. A significant number of students are assessed each semester. However, problems have been encountered with effectively collecting and disseminating the data. Technology can provide the solutions. This session focuses on the use of technology to solve problems with collecting assessment data on student teachers, storing the data, and disseminating it. Panelists will describe the problems and the processes used to make data collection, storage, and dissemination more effective with technology solutions. Implications for UNK will be discussed.
Luncheon and Keynote Address
12:00 – 1:30
Great Room A&B
Keynote Speaker: Maureen A McCarthy
Attributes of High Quality Programs: A Developmental Framework for Program Assessment
Although the Association of American Colleges (1992) offers a general framework for program review, I will offer more criteria. Eight broad dimensions— curriculum, assessment issues, student learning outcomes, program resources, student development, faculty characteristics, program climate, and administrative support— will be presented as a model for program evaluation. Evidence of support for these dimensions is derived from the undergraduate education in the liberal arts literature and from psychology specific pedagogy. This model also provides a mechanism for evaluation along a continuum, ranging from under-developed programs to departments that are functioning at a distinguished level. I will offer an overview of these dimensions, the continuum and specific evaluative criteria that can be used for productive program reviews.
Dr. McCarthy earned her M.S. degree in Counseling and Ph.D. in Research and Evaluation from Oklahoma State University. She also completed coursework in the area of Counseling Psychology at Tennessee State University. Her professional research interests include Teaching of Psychology, Measurement of Moral Reasoning, Perceptual Sensitivity. She is active as a member of the Society for Teaching of Psychology and the American Psychological Association
11:30 – 1:30
Poster Session - Great Room A&B
Presenters from several institutions will present findings of research on various topics in assessment today. See page seven for list of presenters and topics.
Great Room A&B
|Zimmerman/ Joeckel/ Heinrich
|Portfolio Assessment of College Goals
|Assessing Cognitive Complexity: Course Analysis by Outcomes and Level
|Miller/ Wadkins/ Wozniak/ Demoret
|Factors that Promote Course Engagement
|Bruckman/ Garban/ Becky
|Students’ Views on Exam Item Format
|Blackboard at UNK: Faculty Perspectives
|Myer/ Bethel/ Jones
|Faculty Perceptions of the UNK Writing Center
Babutzke/ Malcolm/ Johnson/ Putnam
|Students’ Perceptions of UNK’s 22 Million Dollar Housing Project
|Thompson/ Lamb/ Rhoads
|The Value of Course Exams from the Student Perspective
|Developing a “Culture of Assessment” Assessment
|Aligning General Studies Objectives with Introductory Courses
|Development of a Student Evaluation for Honors Courses
|Students’ Perceptions of the Women’s Study Program at UNK
|Student Assessment Committee
Office of Assessment
|Follow up to the General Studies Survey
1:35-2:00 - Ponderosa C
Knowledge Retention: The Effects of Instructional Approach
R. Miller, T. Wadkins, R. Rycek, Psychology, UNK
Two studies compared student's retention of course material several months after course completion for students in content equivalent courses taught in either a full semester or an accelerated three week format, and in either a traditional lecture/discussion style or as an online course. Comprehensive final exams were administered to students several months after the completion of the courses. No significant differences were found between students' scores as a function of course length or format, suggesting that neither length of the instructional approach nor the delivery format are key factors in student learning.
1:35-2:00 - Ponderosa D
The Educational Benchmarking (EBI) Undergraduate Business Exit Assessment
T. Burkink, Assoc. Dean College of Business and Technology, UNK
In 2006, UNK’s College of Business and Technology (business division) switched from using a locally developed senior exit survey to the Education Benchmarking (EBI) Undergraduate Business Exit Assessment. This measure allows for comparison of student outcomes to a selected group of six peer institutions as well as to all institutions in our Carnegie class. There are 66 items and 16 factors in the instrument. Examples of factors include Quality of Faculty and Instruction, Advisor, and Learning Outcomes - Effective Communication and Team Work.
2:05-2:30 - Ponderosa C
Assessment of iPod Use and Podcasting in Psychology Classes
W. Wozniak, J. Benz, Psychology, UNK
Instructors recorded their lectures and made them available to students via Blackboard or via a podcast in one introductory and two advanced Psychology Classes. Student outcome data were collected and compared to similar courses in which recorded lectures were not made available. Data concerning the technical and personal reasons for not using the recordings were also investigated.
2:05-2:30 - Ponderosa D
Acceditation and the Academic Program Review (APR)
T. Archwamety, M. McFarland, Counseling and School Psychology, UNK
The histories of both Accreditation and Academic Program Review (AOPR) will be outlined. An ERIC database search compared the historical trends of the two concepts in publication. The focus will then turn to APR. The presentation will outline the different purposes and different types of APR, the APR process as well as the benefits and possible negative outcomes of APR.
2:35-3:00 - Ponderosa C
Using Multi-Media Projects to Assess Communication Skills in Psychology
R. Sullivan, Bellevue University
Developing good communication skills, both in the classroom and in non-academic settings, is one of the University assessment goals at Bellevue University. To address this goal within the psychology major, students in our capstone course are required to produce multi-media projects designed to communicate current psychological research findings that have relevance to public concerns (e.g., bullying in schools). The projects are assessed on several dimensions by the students, as well as by members of the community during a public presentation. Results of these assessments, samples of the rubrics used, and video clips of the projects will be presented.
2:35-3:00 - Ponderosa D
A Case Study: Assessment of Student Learning through Industrial Interniships
R. Meznarich, S. Lightner, Industrial Technology, UNK
This session will focus on the assessment of student internships as a tool for enhancing student learning. The speakers will present qualitative and quantitative case study data collected from internship experiences with a major electrical distributor and document how the experience has improved student outcome learning and program improvement.
3:05-3:30 - Ponderosa C
Assessing Learning Outcomes from the Perspective of Employers, and Beyond
A. Leung, T. Burkink, D. Bridges, and M. Yeagley, Business and Technology, UNK
This paper assesses UNK graduates from the perspective of the
employer. In order to have a comprehensive understanding about how UNK prepares its graduates for their careers, multiple methods were utilized. Using a survey, the importance of competencies, and the performance of employees who graduated from UNK were compared to other institutions. Moreover, the participants were also guided to give qualitative interpretations of the competencies to understand what organizational skills, attitudes, and behaviors exemplify the competencies.
Panel Session - Ponderosa C
Student Course Evaluations: Strategies to Enhance the Value of Student Feedback
J. Mandernach, T. Wadkins, K. Fritson, and S. McGahan, Psychology UNK
Despite considerable research challenging the validity and usefulness of student course evaluations, they are a well-established component of faculty and department assessments at most institutions of higher education. As such, it is essential to implement strategies that ensure the data obtained through course evaluations can be used to simultaneously evaluate instruction and enhance student learning. This symposium will examine student perceptions on the role and function of course evaluations, unique considerations for evaluating online courses, benefits of mid-term teaching evaluations, and response rates for student evaluations conducted online.