Submitted Fall, 2005
Methodology and Data
Based on The Sociology Curriculum Assessment Plan approved in 2003, two courses designated as courses for assessment which were taught during the 2004-2005 Academic Year. These two courses are the primary basis for the assessment report. Additional informal information was secured through an interview with a graduating senior.
The two courses are: 1) Sociology 370 Social Survey which was refocused and renamed as Sociology 370 Sociological Research during a curriculum revision process. The refocused Sociology 370 course allows for some flexibility in research methodology rather than being limited to the social survey methodology itself. 2) Sociology 468 Senior Seminar. This course, a capstone course, allows students to read sociological literature and engage in discussions focusing on sociological perspectives. Both of these courses were taught in the fall semester of 2004.
Students are required to write a sociological research paper during their course experience in SOC 370. Seven students’ papers were evaluated by two sociology faculty using the evaluation instrument designed for the research papers and included in the program’s assessment plan. In the SOC 468 seminar course, two faculty observers evaluated a number of skills expected from the students in the oral exam section of the course.
Each student in the SOC 370 course is required to participate in a sociological research project as developed in the course and submit a paper utilizing a sociological frame of reference. Students are expected to utilize the academic writing format of the American Sociological Association for writing the paper. The paper is also expected to include a literature review of the subject matter, data analysis where appropriate, statistical applications and incorporate theoretical thought while utilizing appropriate concepts.
Each student submits their paper which is then passed on to the director without the student names attached. The evaluation process involves two members of the sociology faculty who utilize an evaluation instrument to examine the degree to which each student paper meets the expected outcomes of the course objectives. Faculty members are selected who were not involved in teaching the course. After the evaluation process, the director utilizes the scores from the faculty evaluation process and calculates the percentage scored on a number of indicators supporting the objectives in the assessment document. The evaluation outcomes are presented below.
Objectives 3, 5 and 7 are most directly associated with the student paper.
The third assessment objective listed above: development of a substantive area within the discipline of sociology was measured by five indicators on the evaluation instrument. These included: 1) correctly applies the level of analysis, 2) correctly operationalizes sociological concepts, 3) demonstrates comprehension of a substantive area of sociological knowledge, 4) correctly applies theory and its assumptions and 5) summarizes the method used and correctly justified the techniques used in the research.
The result of the calculation indicates that 91% of the evaluator responses fell in superior to very good range of the scoring across these indicators. Fifty-one percent of these were considered superior while 40% were evaluated as very good while only 14% were evaluated as good or fair, none were poor.
The fifth assessment objective requires a methodologically viable research paper to be executed by the student. This fifth objective was assessed by the following indicators: 1) data tables and others are correctly constructed, 2) the control variable has been correctly utilized, 3) appropriate measures of association are used, 4) correctly interprets the data displayed in the tables and statistics, 5) demonstrates the appropriate discussion of methods, including identification of the sampling procedure, the salient characteristics of the sample, and a description of the design.
The result of the evaluation scores on this objective indicated that 92% of the scores rated in the superior to very good categories of the five level scale of evaluation. Only 1% of the scores fell into the good category while no scores were fair or poor.
The seventh objective, ethical issues in research is evaluated by three indicators: 1) design and methodologies conform to the ethical standards of the sociological profession, 2) subject selection and questionnaires conform to the standards of the profession and 3) ethical problems appropriate to the design and subjects are discussed in the paper.
The results of the calculation show that 97% of the papers were scored as superior or very good. In fact, 95% were scored as superior.
Understood throughout our evaluation of the students, are their writing and communication skills. While this does not appear as one of the separate objectives, it remains an understood skill underlying the objectives, although in an indirect manner. To assess the writing involved in these papers, four indicators were utilized. These are: 1) the thesis of the paper is well developed, 2) integrates the literature into a coherent whole, 3) correctly utilizes the ASA format for paper writing style and 4) the general overall quality of the writing.
Seventy-eight percent of the evaluator’s responses fell into the superior to very good scoring across these four items while 22% of the scoring indicated that the student’s writing was good or fair, most being judged good in this group. The lowest scoring was on the proper utilization of the ASA format in the papers.
The Senior Seminar Course: Sociology 468.
The first two objectives listed above were more directly measured in the Senior Seminar class. The Senior Seminar, SOC 468 was evaluated in the form of the oral exam in this particular semester.
Two sociology faculty members participated in observations of the oral exam held during the regular classroom context. The general theme for the oral exam focused on deviance in the context of social structure and culture in the U.S. The theme was ‘wilding’ and involved discussions of major corporate deviance as well as other social organizations. This was the first time we had occasion to evaluate the oral exam and instigated an observational evaluation device for this purpose in the form of questions used by the faculty evaluators that would reflect the student’s knowledge and use of sociological perspectives in the oral exam. Only one faculty member’s notes were available for this discussion and the evaluation remarks below reflect this deficiency.
The results reported by the lone observer, was that the experience was informative and instructive. The students responded to a series of questions prepared for the discussion by the instructor. The questions focused extensively on the theoretical, the cultural dimension of the phenomena under consideration, examples of this social activity, applications of the concepts and theoretical constructs involved, the implications for a society if this social activity is acceptable and policy implications and solutions in the context of this activity. All of the students engaged in the discussion with a fairly well balanced input across the class members to the questions from the instructor. Again, as was the case of the student papers, the overall general evaluation was judged to be superior to very good on the seven items used as indicators of a successful discussion.
The indicators utilized in evaluating the discussion were the following: 1) demonstrates an understanding of common sociological concepts, 2) student constructed reasonable social policy given information on social causation, 3) selects appropriate theory for the discussion, 4) identifies the assumptions for the theory, 5) describes the relationships between culture and social structure, 6) demonstrates how demographic and other social changes affect social structures and individuals and 7) demonstrates the effects of social institutions and their interactions on individuals.
During the course of the assessment process, we have utilized a Focus Group technique to gain insight and perspectives from students regarding their experiences in the sociology curriculum. During this assessment year, we did not have an ample number of graduating seniors in the particular semester that this is done so we do not have data from this technique. It will be utilized again when there are sufficient students in the second semester of the academic year.
Although not part of our formal assessment plan but utilized in years preceding the formal assessment pan, the chair or director would often interview students on a one-on-one basis. In the case of this spring, 2005, the director planned to interview two of the graduating seniors but only one student responded to the invitation. The discussion utilized a set of interview questions that had been used in other previous exit interviews of our graduating seniors. The questions follow a very similar pattern to that used in the Focus Group.
The student interviewed responded that they found several experiences valuable in preparing for their expected career. These included the internship and the field trip which had motivated them for more research. In addition the research methods and sociological inquiry were used in the internship and provided valuable insights and skills. They suggested that if they were to give advice to upcoming students that they need to think of getting a Master’s Degree and that they should focus on preparing to do this. The Sociology Club and the internship were mentioned as valuable as was learning how to deal with time problems.
In regard to the curriculum, it was suggested that a greater variety of courses spanning a range from Anthropology to Women’s Study courses would be interesting. More courses on settlements and minority groups involved in society would also have been welcomed. The student commented that they though that being involved in the settlement with other students on projects dealing with people would have been seen as very useful and would get students interacting with non-university personnel.
Summary of Findings and Responses
Based on the information collected during the 2004-2005 academic year from the courses and objectives evaluated through the assessment instruments we utilized during this time frame, the members of the sociology program have, to their credit, contributed to some very successful student outcomes. It does not appear to me that there are several areas where improvements can be contemplated. One is the use of the American Sociological Association’s academic writing style. Other areas could include more work on conceptualization of the content and the theoretical perspectives in the papers.
Follow-up Activities: Use of Results
The sociology faculty continue to work on several tasks that were identified in the last Focus Group that was conducted and reported in the 2003-2004 Assessment Report.
- These include the creation and offering of two Special Topics courses (Sociology of Human Sexuality and The Holocaust) which proved to be of interest to students and had good enrollments.
- Preparing more courses applicable to the intensive writing program (two were added).
- Seeking the greater utilization of internships by our students.
- Involving a higher percentage of our students in cooperative and integrating activities and developing a greater range of talents and skills among our students. The Sociology Club has been active in attempts to involve more of out students in cooperative and integrating activities which are beginning to have positive outcomes in the planning of activities during this academic year.
- In response to previous student and faculty concerns regarding the offering of the required sociology courses in the major, we have devised and will be publishing the rotation of the required sociology courses to students in the program.
- We have made adjustments and changed prerequisites in a number of our course offerings so that students will find it possible to fit a major into their degree within the shortened time they often have left after making a decision to major in sociology. This, hopefully, will improve the pre-advising capability of the students when they visit with their advisor.
- We have enhanced our various ‘emphasis’ offerings in the sociology major by adding a global and cultural emphasis in addition to the general sociology and a community emphasis.
Observation of the Process of Assessment
At the present time the sociology faculty are examining the assessment process and working on the fine-tuning of the assessment plan.
- We are working on making the process more ‘seamless’ in terms of the procedures and scheduling of the assessment activities.
- We will be making plans to have more coordination between the faculty teaching the SOC 370 and SOC 468 courses and the assessment committee.
- We will be looking more carefully at the selection of the reading for the Senior Seminar course so that the oral exam will provide coverage of the essential skills so that this will allow us to collect assessment data.
The sociology program has submitted a plan for the assessment of the General Studies offerings that we provide for the general liberal arts offerings of the University.