University of Nebraska Kearney

Family Studies Assessment Report 2005 

Submitted Fall, 2005

Introduction 

This report reflects assessment activities for the Family Studies Program in the Department of Family Studies and Interior Design.

1. Methodology

   Learning Objectives: Knowledge

   1. Students will develop a broad knowledge and understanding of their disciplinary studies and how they may be applied in the working world.
      
      Assessment Tool – Survey: Family Studies Substance Areas of Knowledge Survey 

   Learning Objectives: Skills

   1. Students will demonstrate proficiency in the use of computer technologies essential for disciplinary proficiency and workplace competence. This is assessed every other year.
       
      Assessment Tool – Term paper and oral presentation in FSID 402 – Critical Reading in Marriage and Family Relationships. 
   2. Students will demonstrate the ability to write in their emerging field. This is assessed every other year.
       
      Assessment Tool – Term paper and oral presentation in FSID 402 – Critical Reading in Marriage and Family Relationships. 
   3. Students will be able to express themselves orally through creation and delivery of public presentations related to their disciplinary studies. This is assessed every other year.
       
      Assessment Tool – Term paper and oral presentation in FSID 402 – Critical Reading in Marriage and Family Relationships. 

   Learning Objectives: Values

   1. Students will demonstrate professional protocols of the workplace in regard to appropriate dress, deportment, sensitivity to diversity issues, as well as ethical and moral issues that may confront them.
       
      Assessment Tool – Internship and Field Experience Assessment rubric - Internships and Field Experiences will be assessed commencing 2005. 
   2. Students will demonstrate professional attitudes such as good work place habits, motivation, anticipating work demands, and ethical and social sensitivity, deemed essential for professional competence.
       
      Assessment Tool – Internship and Field Experience Assessment rubric - Internships and Field Experiences will be assessed commencing 2005. 
    
2. Results
    
   Learning Objectives: Knowledge
   1. Students will develop a broad knowledge and understanding of their disciplinary studies and how they may be applied in the working world.
       
      Assessment Tool – Term paper and oral presentation in FSID 402 – Critical Reading in Marriage and Family Relationships.
       
      The instrument that was developed last year was deemed to be less than appropriate, was not used, and will not be used in the future.  The Term paper and oral presentation in FSID 402 – Critical Reading in marriage and Family Relationships was used for assessment. A formal rubric (Appendix I) was used in the critical reading and evaluation of student papers submitted Spring, 2005. Four faculty members participated in this evaluation. One-third of the 24 papers were randomly pulled. Each faculty member read four. Each paper was read by 2 faculty members. If a difference of more than 20 pts occurred, the paper was read by a third faculty member. The scores were averaged for each rubric item and as a total for each paper evaluated.
       
   Learning Objectives: Skills
   1. Students will demonstrate proficiency in the use of computer technologies essential for disciplinary proficiency and workplace competence. This is assessed every other year.
       
      Note: The original assessment was to be conducted every two years beginning Spring 06. Family Studies faculty decided to initiate assessment activities Spring 05.
       
      Assessment Tool – Term paper and oral presentation in FSID 402 – Critical Reading in Marriage and Family Relationships.
       
      Research Papers. A formal rubric (Appendix I) was used in the critical reading and evaluation of student papers submitted Spring, 2005. Four faculty members participated in this evaluation. One-third of the twenty-four papers were randomly pulled. Each faculty member read four. Each paper was read by two faculty members. If a difference of more than twenty points occurred, the paper was read by a third faculty member. The scores were averaged for each rubric item and as a total for each paper evaluated.
       
      Research Presentations. A formal rubric (Appendix II) was used in the critical viewing and evaluation of student presentations Spring, 2005. Four faculty members participated in this evaluation. One-third of the presentations were randomly pulled. Each sample was viewed and evaluated by two faculty members. If a difference of more than twenty points occurred, the presentation was viewed by a third faculty member. The scores were averaged for each rubric item and as a total for each presentation.
   2. Students will demonstrate the ability to write in their emerging field. This is assessed every other year.
       
      Note: The original assessment was to be conducted every two years beginning Spring 06. Family Studies faculty decided to initiate assessment activities Spring 05.
       
      Assessment Tool – Term paper and oral presentation in FSID 402 – Critical Reading in Marriage and Family Relationships.
       
      Research Papers. A formal rubric (Appendix I) was used in the critical reading and evaluation of student papers submitted Spring, 2005. Four faculty members participated in this evaluation. One-third of the twenty-four papers were randomly pulled. Each faculty member read four. Each paper was read by two faculty members. If a difference of more than twenty points occurred, the paper was read by a third faculty member. The scores were averaged for each rubric item and as a total for each paper evaluated.
       
      Research Presentations. A formal rubric (Appendix II) was used in the critical viewing and evaluation of student presentations Spring, 2005. Four faculty members participated in this evaluation. One-third of the presentations were randomly pulled. Each sample was viewed and evaluated by two faculty members. If a difference of more than twenty points occurred, the presentation was viewed by a third faculty member. The scores were averaged for each rubric item and as a total for each presentation.
   3. Students will be able to express themselves orally through creation and delivery of public presentations related to their disciplinary studies. This is assessed every other year.
       
      Note: The original assessment was to be conducted every two years beginning Spring 06. Family Studies faculty decided to initiate assessment activities Spring 05.
       
      Assessment Tool – Term paper and oral presentation in FSID 402 – Critical Reading in Marriage and Family Relationships.
       
      Research Papers. A formal rubric (Appendix I) was used in the critical reading and evaluation of student papers submitted Spring, 2005. Four faculty members participated in this evaluation. One-third of the twenty-four papers were randomly pulled. Each faculty member read four. Each paper was read by two faculty members. If a difference of more than twenty points occurred, the paper was read by a third faculty member. The scores were averaged for each rubric item and as a total for each paper evaluated.
       
      Research Presentations. A formal rubric (Appendix II) was used in the critical viewing and evaluation of student presentations Spring, 2005. Four faculty members participated in this evaluation. One-third of the presentations were randomly pulled. Each sample was viewed and evaluated by two faculty members. If a difference of more than twenty points occurred, the presentation was viewed by a third faculty member. The scores were averaged for each rubric item and as a total for each presentation.
       
      Results of Paper Evaluations
       
      A total of eight papers were randomly selected from the twenty-four collected. Each paper was reviewed by two faculty members, using the rubric in Appendix I. Scores were within the twenty point difference range, and were averaged. These averages ranged from 61.4-82.4% (points converted to percentage for report). While lower than desired, there was an issue of possible inadequate directions to students.  The faculty member who assigned these papers is no longer on the faculty and the formats varied considerably from one paper to another. Faculty reviewers are very hopeful that the e-portfolio method proposed for future evaluation will eliminate these types of problems. This process will be reviewed and revised so that results are more accurate and reportable next year. Faculty members agree that increased attention on developmental writing skills would be beneficial to our students.
       
      Results of Presentation Evaluations
       
      Four presentations were randomly drawn from the video set of thirteen collected in the Spring, 05. Each was viewed and scored by two faculty members using the rubric. Scores on the first two viewed were (43, 41—42 M) and (45, 50, 42.5 M) or 70 %. Scores on the next two viewed were (32-12, 22 M) and 41, 34, 37.5 M) or 50%. Strengths noted were “rooted in theory”, “obvious connection between topic and examples given”. Weakness was in the use of technology. All reviewers noted frustration with the quality of the taping, the lack of vision of powerpoints being presented. The rubric was found to be ineffective and will be reviewed and revised.
       
   Learning Objectives: Values
   1. Students will demonstrate professional protocols of the workplace in regard to appropriate dress, deportment, sensitivity to diversity issues, as well as ethical and moral issues that may confront them.
       
      Assessment Tool – Internship and Field Experience Assessment rubric
       
      Supervisor’s evaluation of interns and students participating in field experiences were tabulated and aggregated. Mean scores were noted. The only mean below 90% is Critical Thinking (89%). Independent decision-making received 3-5 rankings as the lowest criterion. This is an important skill and it will be a focus of a newly developed senior-level course. Faculty members were in agreement that performance in Internships is a positive reflection of skills and knowledge student have learned in their program
   2. Students will demonstrate professional attitudes such as good work place habits, motivation, anticipating work demands, and ethical and social sensitivity, deemed essential for professional competence.
       
      Assessment Tool – Internship and Field Experience Assessment rubric
       
      Supervisor’s evaluation of interns and students participating in field experiences were tabulated and aggregated. Mean scores were noted. The only mean below 90% is Critical Thinking (89%). Independent decision-making received 3-5 rankings as the lowest criterion. This is an important skill and it will be a focus of a newly developed senior-level course. Faculty members were in agreement that performance in Internships is a positive reflection of skills and knowledge student have learned in their program.
    
3. Conclusions and Recommendations

   Family Studies faculty members have agreed to implement a developmental evaluation of student writing as per the WI/CD Committee’s recommendation. Utilizing the e-portfolio system available on campus, students will be asked to place their research papers from FSID 150, 340, and 402 in this electronic portfolio. Utilizing a standard rubric, faculty will randomly pull 20% of graduating seniors each Spring and assess their work posted in this portfolio. Analysis of both growth and final skill-level will be completed. Because the first class to complete this process will not have complete work posted until Spring, 09, no assessment will be done Spring, 06. Spring, 07 will pilot the process using 340 and 402 papers from that Senior pool. Spring, 08 will repeat that 2 paper review and Spring, 09 will begin the 3 paper review process.

   Family Studies faculty will actually view the original presentation of student group work during Spring, 2006 and the existing rubric will be used by both the instructor and two other faculty members. The two closest scores will be averaged and reported in the next annual report.

4. Use of Results

   Results will be used to formulate conclusions and recommendations following assessment activities.  Conclusions and recommendations will be implemented as results are analyzed.

5. Observations of the Process

   The following internal questions were discussed during a Family Studies retreat summer of 04.

   • Are current measures of student assessment reflective of the competencies we expect our graduates to have?
   • Are current methods of assessing students effective, consistent, and in keeping with standards in our disciplines?
   • Are there new or modified methods which will better assess what we want to learn about student outcomes?

   The responses to these questions include:

   1. See III above.
   2. The survey instrument developed in 2004-2005 will no longer be used.
   3. See III above.

   Family Studies faculty will continue to monitor assessment activities for efficiency, accuracy, and relevance to the established mission and objectives of the program. Data will continue to be gathered, analyzed, and used toward continued improvement activities.

   A revised Assessment Plan for Family Studies is attached that reflects the changes to the plan as outlined above.