The English Language objectives for the General Studies Program include that a student should be able to:
- demonstrate the ability to form and support a coherent position on an issue,
- demonstrate the ability to write and speak in a formal manner appropriate to the audience,
- demonstrate the ability to write "expressive" as well as "transactional" language i.e., to develop and understand the role of voice in communication as well as the message itself.
As part of the departmental assessment plan the English Department allowed four sections (taught by an associate professor, graduate assistant, adjunct teacher, and a full time lecturer) of English courses to be assessed. The assessment used papers that were a scheduled class assignment and most papers had received revisions prior to the assessment. The papers were assessed using a rubric that the General Studies Director, the Faculty Assistant to the SVCAA and others created after consultation with members of the English Department. Several papers were read using the rubric and a reliability standard of having rated scores within one point of other ratings was established.
| |
1 |
2 |
3 |
| |
does not meet standards |
meets standards |
exceeds standards |
| CONTENT |
| Thesis |
No apparent thesis |
Thesis stated but is vague |
Clear and concise thesis |
| Evidence |
Insufficient or irrelevant evidence |
Evidence generally supportive |
High quality evidence |
| Originality |
Entirely of others' work |
Some creativity incorporated |
New insights incorporated |
| ORGANIZATION |
| Unity |
Ideas are irrelevant to thesis |
Occasionally strays |
Stays focused on thesis |
| Coherence |
Disconnected ideas, includes irrelevancies |
Flow leading to a conclusion, some irrelevancy |
Logical flow leading to a clear conclusion |
| Order/Transitions |
Telegraphic, no sequential logic |
generally sequential organization with weak transitions |
sequential organization with appropriate transitions |
| Paragraphing |
Distinct ideas unrelated to paragraph structure |
Each paragraph addresses an idea some digression allowed |
Each paragraph addresses a complete idea |
| Development |
Jumps from point to point without development of thesis |
Clear development at most every step |
Clear and logical development at every step |
| MECHANICS/USAGE |
| Punctuation, grammar, and spelling |
more than three errors |
3 or fewer errors |
No noticeable errors |
The papers were prepared for the reading by removing the name of the student and the instructor and assigning a number to each paper. The General Studies Graduate Assistant retained the master list. Graduate students from the Counseling and School Psychology Program were recruited to read the papers and were remunerated $100 for their services. Eight graduate students were trained in approximately 90 minutes on a Saturday morning. The 74 papers were each read by different readers. It was agreed that if their total ratings of the paper differed by more than one point, a third reader would read the paper and the most divergent rating would be tossed out. The agreement among the readers was so good that no third reading of any paper was needed. The reading and training was complete in approximately 5 hours. A summary of the data is presented in the Table below.
| |
READER 1
|
READER 2 |
| SCORE |
1
|
2
|
3
|
1
|
2
|
3
|
| Content |
13
|
44
|
17
|
13
|
48
|
13
|
| Organization |
16
|
47
|
11
|
21
|
43
|
10
|
| Mechanics |
42
|
28
|
4
|
36
|
38
|
0
|
For content, 82% of the students were rated as meets or exceeds standards. For organization, 78% (71% by the second reader) of the students were rated as meets or exceeds standards. For mechanics, 43% (51% by the second reader) of the students were rated as meeting or exceeding standards. As with Mathematics, the sample of students includes students who may ultimately earn poor grades in ENG 101 and so the data may underestimate performance of the students who pass. Even so, there is a marked difference between students' abilities in content and organization and their mechanics and usage.