Assessment Reporting: Deadlines and Upcoming Changes
 Congratulations! For the first time all reports were submitted (or accounted for) by October 1st! Thanks to all who contributed to this effort, especially those who are directly responsible for preparing annual reports. All departments/programs should receive report feedback by November 1st. The deadline to submit revised reports (when necessary) is December 1st.
We are in the process of purchasing a new assessment reporting product, WeaveOnline. We anticipate that this product will simplify the reporting process. This year separate reports were required for Department/Program and General Studies assessment. Next year, we hope to have all assessment reports submitted through WeaveOnline, including WI/CD, General Studies and departmental/program. Our office will take the reports that were turned in this year and enter them in to WeaveOnline. Then next year, departments will only need to update fields that change from year to year. To learn more about WeaveOnline or see a demo please visit: www.weaveonline.com
back to top
_________________________________________
.png)
UNK’s 2007 NSSE and FSSE Results are available
Since 2000, the National Survey of Student Engagement (NSSE) has been collecting data from students at four-year colleges and universities around the country to assess the extent to which students engage in a variety of educationally effective activities. NSSE is cosponsored by The Carnegie Foundation for the Advancement of Teaching and the Pew Forum on Undergraduate Learning. The University of Nebraska at Kearney has participated in the survey in 2002, 2003, 2004 and 2007.
The NSSE is designed to obtain, on an annual basis, information from scores of colleges and universities nationwide about student participation in programs and activities that institutions provide for their learning and personal development. The results will provide an estimate of how undergraduates spend their time and what they gain from attending college. Survey items on the NSSE represent empirically confirmed "good practices" in undergraduate education. That is, they reflect behaviors by students and institutions that are associated with desired outcomes of college.
In 2007, UNK also participated in the Faculty Survey of Student Engagement (FSSE). Paired with NSSE results, FSSE offers a comprehensive look at student engagement on the UNK campus. We selected the Course-Based FSSE option which asks faculty members to respond to questions about student engagement based on a course taught during the current academic year. Fifty-eight percent of faculty at UNK responded to FSSE.
A sampling of results from the NSSE:
What students are saying about their UNK experience.
Are students satisfied with their overall educational experience at UNK? According to NSSE data, 85% of FY students report a favorable image of this institution; 82% of seniors would choose this school again if they could start their college career over.
Are faculty members accessible and supportive?
40% of FY students say their faculty are available, helpful and sympathetic.
How many students work on research projects with faculty?
By their senior year, 19% of students have done research with a faculty member.
How often are topics from class discussed outside of the classroom? 41% of FY students frequently discuss readings or ideas from coursework outside of class.
Do students work together on projects – inside and outside of class? 44% of FY students frequently work with other students on projects in class, 35% work with peers on assignments outside of class.
How often do students interact with peers with different social, political, or religious views? 42% of FY students say they frequently have serious conversations with students who are different from themselves in terms of their religious, political, or personal beliefs.
How often do students interact with peers from different racial or ethnic backgrounds? 33% of FY students frequently have serious conversations with those of a different race.
How many students study in other countries? By their senior year, 7% of students have studied abroad.
What percentage of students participate in community service? By the time they are seniors, 66% of students have participated in community service or volunteer work.
To see more NSSE and FSSE results please visit: http://www.unk.edu/academicaffairs/assessment/University-Wide/index.php?id=4466.
To view the complete 2007 NSSE (and earlier versions) : http://nsse.iub.edu/html/survey_instruments_2007.cfm
To view the complete 2007 FSSE: http://fsse.iub.edu/pdf/FSSE%202007%20(CB)%20FINAL.pdf
back to top
_________________________________________
CAAP Administered for General Studies Program Assessment
.jpg)
In October, UNK participated in the ACT Collegiate Assessment of Academic Proficiency. CAAP is the standardized, nationally normed assessment program from ACT that enables postsecondary institutions to assess, evaluate, and enhance the outcomes of their general education programs.” ACT offered the UNK the opportunity to participate in the CAAP this year without cost.
Two of the four General Studies objectives were assessed using this measure, writing and critical thinking. Almost 400 freshmen and 400 seniors took portions of the CAAP during October. For more information on the CAAP and sample test items visit: http://www.act.org/caap/ Results from the CAAP will be reported in the Winter Issue of the Assessment Newsletter
Thank you to all faculty who allowed us to use their classes to administer CAAP measures of writing and critical thinking.
back to top
_________________________________________
2008 NCA Assessment Focused Visit
.jpg) On April 28-29, 2008, three Higher Learning Commission (HLC) evaluators will visit UNK for a focused visit. The purpose of this visit is to evaluate progress in assessment of student learning outcomes since their 2004 visit. On the assessment website homepage (http://www.unk.edu/assessment) there is now a link to a page called NCA 2008 that has been created to house all of the key assessment information for the visit. During the 2004 visit, the areas identified for review in the focused visit were:
- Faculty commitment to assessment
- Stable infrastructure to support assessment
- Assessment of Distance Learning
- Assessment of General Studies
- Assessment of the Writing Intensive program (WI)
- Assessment of the Cultural Diversity program (CD)
- Recognition of exemplary assessment practices
Evidence of progress in each of these areas is presented on the website. The site is dynamic with new information being added as it becomes available. The website will be the primary tool for evaluators to review what has happened in the last four years at UNK in the seven areas of assessment included in the NCA focused visit.
Since a focused visit by NCA is the final step before an institution loses its accreditation, it is critical that all departments and programs meet deadlines for assessment reporting and that all faculty and administrators are knowledgeable about the assessment process at UNK. NCA is very concerned that departments and programs analyze their data to identify strengths and areas for improvement, use the data to make necessary changes and improvements in their curriculum and programs, and that every faculty member on campus is involved in discussions in his/her department about assessment results and anticipated changes. These areas will be the focus of the evaluation team while they are on campus.
back to top
_________________________________________
Assessment Climate Survey
Thank you to everyone who participated in the Faculty Assessment Climate Survey! We had 123 faculty members respond to the survey representing each of the four colleges. The Assessment Climate Survey was designed to measure faculty involvement in and attitudes toward assessment of student learning at UNK. The Office of Assessment is using this survey data to gain a better understanding of progress at UNK in developing an effective and sustainable assessment program. The results of the survey are available at the assessment website and will be shared with the NCA focused visit team in April of 2008.
We were pleased to learn that almost 90% of faculty reported being involved in assessment activities. Many reported that their departments discuss assessment results frequently. Most reported having positive attitudes toward assessment of student learning.
.jpg "climate graph 1")
.jpg "climate gragh 2")
To see the Assessment Climate survey and read more about the Results please visit: http://www.unk.edu/academicaffairs/assessment/about/index.php?id=17614
back to top
_________________________________________
Cultural Diversity Survey
At UNK, the General Studies program diversity objective* and the Cultural Diversity course requirements** are two of the ways that diversity is integrated into the curriculum of all students on campus. Understanding and appreciating diversity is important because it allows students to see the world from a perspective different from their own. Through these programs it is hoped that students will develop an appreciation for the differences, similarities, and contributions at both national and global levels of persons belonging to various groups (e.g. cultural, racial, gender, sexual orientation, disability, and religious) and enhance their abilities to work with/interact with diverse individuals.
In order to gauge the impact of these programs (and the UNK experience) on student knowledge, attitudes, and behaviors related to cultural diversity, a survey was developed by the assessment office staff. The Cultural Diversity Survey was sent to all undergraduate students at UNK in Mid-September. As we begin to look at the data collected, we hope to find that upper division students 1) possess more knowledge about diversity, 2) have more positive attitudes about individuals who differ from themselves, rejecting any hierarchical conceptions of diversity (i.e. that one group is superior to another), and 3) seek information about issues of diversity and interaction with persons of different backgrounds.
Selected survey results will be posted on the Assessment website by mid-November www.unk.edu/assessment
* General Studies Diversity Objective: Students will develop and demonstrate an understanding of the experiences and values of groups and cultures which have been historically under-represented
** The UNK Cultural Diversity (CD) Program will cultivate knowledge, awareness, respect, understanding, and appreciation of the experiences and values of underrepresented groups and cultures to prepare students to live and work in a diverse world. Students completing the CD requirements will:
- be able to identify the major contributions of diverse groups to educational, social, and cultural institutions.
- seek information about diversity issues and be able to distinguish among facts, cultural assumptions, interpretations and opinions related to diversity.
- have more positive attitudes about individuals who differ from themselves, rejecting any hierarchical conceptions of diversity (i.e. that one group is superior to another)
- choose to interact with members of groups different from themselves.
back to top
_________________________________________
2007-2008 Assessment Committee
Jeanne Butler, Greg Anderson, Paul Bishop, Karin Covalt, Ron Crocker, Kim Elliott, Jessy Hansen, Mary Kommers, George Lawson, Tami Moore, Sarah von Schrader and Scott Unruh, Gloria Vavricka
2007-2008 Student Assessment Committee
Jessy Hansen, April Becker, Tim Davis, Bethany Hyatt
Not pictured: Russell Crawford and Heath Dooley
back to top
_________________________________________
Upcoming Assessment Conferences
The Office of Assessment can provide funding to faculty who are interested in attending any of the following assessment related conferences.
- IUPUI Assessment Institute
November 4-6, 2007
Indianapolis, IN
http://www.planning.iupui.edu/conferences/national/nationalconf.html
- Evaluation 2007: Evaluation and Learning
November 7-10, 2007
Baltimore, MD
http://www.eval.org/search07/search.asp
- AACU Annual Meeting
Intentional Learning, Unscripted Challenges
Knowledge and Imagination for an Interdependent World
January 23-26, 2008
Washington, D.C.
http://www.aacu.org/meetings/annualmeeting/index.cfm
- Integrative Designs for General Education and Assessment
February 21-23, 2008
http://www.aacu.org/meetings/index.cfm
- Higher Learning Commission
Finding Common Ground: Accreditation, Assessment and Accountability
April 11-15, 2008
Chicago, IL
http://www.ncahlc.org/annualmeeting/
back to top
______________________________
An Interview with Dr. Maureen McCarthy
“The Impact of Assessment on Students”
Jamie Babutzke
Erin Johnson
Anastacia Malcolm
The following is a transcript of an interview with Dr. Maureen McCarthy conducted by three UNK psychology students. Dr. McCarthy was on the UNK Campus as the keynote speaker of the Platte Valley Assessment Conference last April.
.jpg "mcarthy")
Dr. McCarthy is Professor of Psychology at Kennesaw State University in Georgia. Dr. McCarthy earned her MS degree in Counseling and PhD in Research and Evaluation from Oklahoma State University. She also completed coursework in the area of Counseling Psychology at Tennessee State University. Dr. McCarthy held positions in Counseling Centers at both Oklahoma State University and Tulsa Community College. She held faculty positions at Austin Peay State University prior to serving as the Executive Associate Director for Undergraduate and Precollege Programs at the American Psychological Association. Dr. McCarthy joined the faculty at KSU in Spring 2005. Her professional research interests include Teaching of Psychology, Measurement of Moral Reasoning, Perceptual Sensitivity. She is active as a member of the Society for Teaching of Psychology and the American Psychological Association.
Malcolm: The first question we have is how do you see students being affected by assessments? It seems like a very large looming shadow over the university system, how do you see students being affected?
McCarthy: There are a couple ofdifferent ways students can be affected. I would go back to what is happening in K-12 right now and the way students are affected in the grade schools and high schools. Assessment is being done to them. Students are being tested repeatedly. The literature has suggested that it is a stressful situation for the kids, for the parents, and I am not sure that it provides direct benefits back to the students.
If we consider how assessment is used in colleges and universities we have an opportunity to implement a system that is different. Provided that the system is not legislated, which is where they are going right now in Washington DC. If it is legislated, more tests will be administered.
Use of the ACAT, or any outcomes measure is a little bit stressful. If we consider, the UNK assessment committee, you’ve got input to help make assessment positive. What are the positive outcomes of assessment? You can use those assessment data as evidence of how you performed, how students at UNK performed, and the value of a UNK degree when searching for a job or graduate school. I see all those things as advantages. Assessments also provide information to individual faculty or the university to improve things for you, which I think is the ultimate goal of assessment.
Malcolm: I would agree. My parents are high school teachers so I am very familiar with assessment at the 8-12 level and in their opinion assessment is such a negative thing, because they are doing so much testing. In an university system where you do course evaluations are you seeing results from them?
McCarthy: I think that course evaluations do differ by campus and in a place like UNK and at Kennesaw State University; we do pay attention to what students say. But I am not sure we are always asking the right questions. Sometimes, I actually conduct additional assessments. For example, I offer 8 points extra credit and I ask “How do you feel about your time in this project?” and “What can I do to make things better when you are writing a research proposal?”
Johnson: What is the ideal cycle that should happen? Ideally, what do you think should happen with assessment?
McCarthy: The ideal cycle includes a couple of different assessments. There are assessments that happen during the semester called formative assessments. Two things should happen with formative assessment. It should provide feedback to a student, give them a sense of how they’re doing and give them a sense of pride in what they have learned. The assessment should also be used as feedback to the faculty member. If everybody in the class fails an item I don’t think it’s the students’ problem, I think it’s the faculty member’s problem and so it serves two purposes. It provides feedback to a faculty member and it provides feedback to the student throughout the semester. I use the information and I examine what I’m doing well or what I need to change. I also do informal assessments on a daily basis.
A summative assessment, for example, is the ACAT. It is an endpoint and I think the assessment should be examined by the entire faculty. In that sense, you can’t expect everyone to get a 100% so you would expect a normal distribution. I think that with the ACAT you can break it down and figure out what students are learning (for example stats or bio) and this helps to make changes.
There isn’t an ideal cycle, but a two-fold process should be ongoing. Sometimes the ACAT is administered and very little analysis takes place. Results of any outcomes measure should be used to inform teaching.
Johnson: What are the barriers that you are running into in today's imperfect world?
McCarthy: Ten years ago we didn’t have “no child left behind” so I think people are very scared because this law has been misused. Data from assessments can also be misused. Sometimes we close schools when a school doesn’t perform well, yet we may not account for the fact that students may not be doing well because they don’t have basic resources. At the college level we must balance an obligation to teach basic content with academic freedom, while at the same time providing evidence of learning.
Babutzke: Within those barriers do you feel that people have a misconception that assessment is evaluation?
McCarthy: I think that it is evaluation of student learning, not necessarily faculty effectiveness.
Babutzke: Can you explain a little more on that?
McCarthy: Although assessments can be misused, they can also be helpful. For example, if you successfully complete a course, and you are not able to communicate what you learned in the course, then our assessment may be flawed. Alternatively, successful performance on a test may be an indication of a specific fund of knowledge.
It is often difficult to parse whether we are measuring faculty performance, student knowledge, pedagogical innovation, or some other influencing factor.
Ten years ago, we had less of an emphasis on assessment at the college level. Today, we are experiencing increased pressure from accreditation bodies and external constituencies to produce evidence of student learning. If we want our students to walk out of here, saying this is what I can do, then we need to hold faculty partly responsible. The problem is; you can’t hold one entity fully responsible for the performance of someone else.
Babutzke: Students are interested in the investment of their education and whether assessment ultimately leads back to the students, what can the students do?
McCarthy: When I look at what is on my syllabus, I have goals for what students should know when they complete my course. Students also have a right to expect that and they can ask faculty throughout the semester questions about those learning outcomes. Asking for feedback is a way to strengthen student experience. As students, you have a right to expect that you are getting reasonable ongoing feedback. What else can students do? Another thing that you can do is to keep a record of work. For example, doing a poster at a conference is a reflection of what you learned in class. A poster is product.
Babuztke: Due to the “No Child Left Behind Act” by President Bush, assessment is a hot topic geared more towards K-12. How will it affect college students today as compared to high school students?
McCarthy: Something will soon be in place for all college students. Accrediting bodies are coming together to try to stop legal definitions from being quite so specific at the college level. Accrediting bodies are simply trying to maintain a balance between the need for accountability and flexibility to teach.
Babutzke: When do you think that these changes may be implemented?
McCarthy: It will be a few years before the changes are clear and implemented. Large institutions take time to change. There is a governing process that is in place at the university level and a governing process at the federal level. It will be a slow, incremental change for implementation of these regulations.
Malcolm: As our wrap up, from a students perspective, we're assessed, we know we are assessed, we take the evaluations, but we don’t feel that we see the results from those assessments and evaluations that we do, how do we make it meaningful to us? For instance when we take the ACAT, we take that test before we graduate; any changes that would be implemented will be seen after we graduate. We don’t see those changes, so how do we keep it meaningful? How do we keep from going in and answering the same to every question? How do we make assessment meaningful to ourselves?
McCarthy: You have the answer. You are making it better for the person after you. It is not uncommon to require a test near graduation. You complete this assessment and you can advocate and offer feedback to the university.
Babutzke: What about having two course evaluations during a semester. One in the middle that says here is what the students want and one at the end to evaluate if those needs were met?
McCarthy: Often, faculty take it upon themselves to do a mid-semester assessment. Some do this informally and others do it formally. For example, a faculty member might put together a team of students who act as an executive committee. Any time a person in class has a concern, they shuttle their concern to the executive committee. Several times a semester the committee meets with the faculty member and provides feedback. Use of a committee reduces the likelihood that a student will feel singled-out. It also provides feedback to the instructor that they can use to make changes throughout the semester.
Another way to conduct a mid-semester assessment is through an analysis of student performance. For example, I examine test performance. If everyone does poorly on an item, I consider it important to examine my contribution to the process. I don’t grade students on that item and I obtain feedback about the problem with the item. If the problem is instructional, this discussion allows me to make a mid-semester correction.
back to top
_________________________________________
|