University of Nebraska Kearney

Instructional Technology Graduate Program Assessment Report 2005: Indirect Measure: Alumni Survey 

Submitted Fall, 2005 

The indirect measure, which was the Alumni Survey was sent to 119 graduates of the program. Approximately 40 were returned by the US mail service for lack of valid addresses. 17 completed surveys were returned. This poor return rate will be addressed in future surveys. It was originally decided that we would wait for at least two years before we surveyed our graduates in order to give them time to see what skills they might be lacking. However, this time period also allowed us to lose contact with many of them. The form was rewritten, will be made electronic, and will be administered prior to graduation in the future. We will request a valid email address from them (not the UNK one that is currently used) and also tell them that we will follow up in two years in a second survey. This process should allow us to gather more data. For this report, the results of the 17 surveys will be provided. Seventeen Alumni surveys were returned. The responses were based on the following responses:

1. I have the ability to promote and implement the International Society for Technology in Education (ISTE) standards in Instructional Technology or in Educational Media;
   

   Mean response 3.29

2. I have the ability design instructional modules which provide content to my students.
   

   Mean response 4.38

3. I have the ability to design instructional modules which meet the ISTE NETS and/or ALA Media Standards
   

   Mean response 4.13

4. I am able to use a variety of software programs to design various instructional modules for my classrooms or training environments.
   

   Mean response 4.88

5. I am able to use a variety of software programs to develop various instructional modules for my classrooms or training environments.
   

   Mean response 4.88

6. I am able to use a variety of software programs to create various instructional modules for my classrooms or training environments.
   

   Mean response 4.76

7. I am able to use a variety of software programs to implement various instructional modules for my classrooms or training environments.
   
   Mean response 4.82
    
8. I am able to investigate or conduct current research into various contemporary issues in my area.
   

   Mean response 3.18

9. I am able to demonstrate my research analysis of the issues to my colleagues, school boards, school administration, parents, or similar stakeholder groups
   

   Mean response 3.13

10. The average responses to all the questions were as follows:
    

    Mean response 4.16

Analysis of Graduate Survey 

The overall response rate was poor. We have decided to conduct an initial survey during the semester students graduate. Then we will follow it up in two years. This should increase the number of responses. 17 out of 119 is not acceptable if we are going to gather valid data to improve the program. Having the survey web based will improve the return rate also. Because several students added comments to the survey, we have decided to modify the survey to allow narrative responses.

The scores apparently indicate two weakness areas in the program. One is the inclusion, or specifically the lack of discussion about the inclusion of ISTE Standards. Although students do meet or exceed the Standards, they apparently do not realize that they do. The program has never made much of an issue of ISTE Standards, although it has been developed around them. Therefore students simply do not relate to the ISTE Standards as they should. More effort needs to be expended by program faculty to tie in the content of the classes with the ISTE Standards and to inform the students of that connection.

The other low scoring area is the research class. Most of the respondents, due to when they graduated, were required to come to come to campus to take the research class. The specific class, which is an excellent research class, is not particularly well-suited for students who do not continue on and complete a field project or a thesis and it was not an online class. These two issues has been addressed by an online research class (TE 800) that has a different focus. It will be interesting to see how the students respond to those questions in future surveys.

Recommendations 

The program needs to conduct the Critical friends’ Survey. This has not been done and it is deemed important by the program faculty. It will be done this year.

It appeared that there were three areas in the Graduate Survey that needed additions. Two are comment sections and one is a reflective question on the portfolio project. The three questions which will be added are:

10. If I was required to submit a Graduation Portfolio, I found it useful and/or valuable. (eventually, the "If I was required" statement will be eliminated since all students must now complete the portfolio.)
11. If you were required to submit a Graduation Portfolio and there is any way that we can improve that process, please tell us what that is.
12. Please examine the program description that is attached and tell us what we can do to improve the program based on your experience in the program and your career using IT after you graduated. (e.g., what classes can we add, delete, modify, etc. and HOW or WHAT you would like to have in the program). Please include ANY other comments you might have.

Both the Graduate Survey and the Critical friends’ survey need to be conducted online. This will increase, quite dramatically, the response rate. As part of the Graduate Survey, we need to establish the need for a follow up survey in two year. We also need to obtain a “real” email address for our graduates and not just use the UNK email which naturally will not be in use in two years.