Ismael at ICTlogy covers a presentation by Graham Attwell on The Future of Schooling. There are some interesting (and confirming) comments that Google is much more the virtual learning environment of choice than any learning management system. Ismael also asks some questions and then raises this point:
Raquel Xalabarder reads my mind and states that, outside of the educational system, you maybe need some assessment to give guarantees to an employer, to a customer e.g. a physicist’s patient.
A: Not that assessment is a thing to avoid, but it should be taken outside the learning process. On the other hand, self-assessment is reflection and thus becomes part of the learning process.
I agree that teaching and testing should be separate activities, as testing puts the teacher in a position of power and control, beyond what is healthy for learning. My suggestions from two years ago, still stand:
- Anyone who teaches is not allowed to test.
- Those who design the tests are answerable to those who learn and those who teach.
- Those who teach are only responsible to those who learn and are subjected to tests.
Your post is timely. This past Tuesday I took a midterm test in an instructional technology class. It took me two and a half hours to complete (several people in the class did not complete it). I wrote (by hand) a total of nine pages (plus what I did on scrap paper – and did not turn in). My hand ached afterwards.
Sadly, less than 10% of what appeared on the test would be applicable in a job, whether the job be non-profit, profit or governmental. In fact, if you took the dated ideas and concepts on which we were tested into your job, you would be laughed out of the office. And rightly so.
Especially frustrating is that this is a graduate level course. For an “educator” to insist that graduate students (or any student for that matter) memorize meaningless, unapplicable data, that will be forgotten in a matter of weeks, is absurd.
Sorry Harold. Didn’t mean to turn your fine blog into RantSpace. You just struck a chord with me (as you often do).
Timely indeed. The other day, while mapping out in my head what I would put on the midterm for my fourth year undergrad biochemistry course, I got onto a tangent wondering why so much of our university education revolves around assessment.
I pondered. I asked a few other profs whose opinions I respected, and I asked the students themselves. I wondered if maybe a pass, fail and possibly distinction system would work. You either understand enough of the material to be considered to have mastered it, or you haven’t. If you have excelled, that should be noted.
Other profs indicated that it is largely a method of determining who is going on and in what capacity. So, the distinction between C, B+, A and A+ might indicate suitability for further education or a profession.
What really surprised me though was the student response. I thought that I might get a rally of “testing is useless” and the like. What I got though was a lot of blank looks followed by comments that they don’t learn it until we test them and they have to have the same questions asked repeatedly before it really sinks in.
I followed this with some queries about what kinds of questions they would like on the midterm. I usually give problems that require thinking (but there is certainly some memorization given the nature of the subject). I was saddened to hear that they would rather have straight regurgitation questions, fill in the blanks and definitions than my usual thinking questions. They won’t get their vomit exam, but it did really put me off for a bit.
I encourage students to submit questions to me, and I promise to use some of them. Exam’s in less than 24 hours and only one student has submitted any.
I would like to offer the option of writing the final exam on their computers and submitting it electronically. I’ll explore that one. 20 years ago when I was an undergrad I could easily take notes all morning and manage a three hour exam, but I doubt I could now, and given the extensive computer use of current students, it may be silly to ask them to perform that way.
Got any good tech assessment suggestions?
Bob Hawkes taught me a rule of thumb years ago 1/3 of a test should be recall/regurgitation (low cognitive load), 1/3 should be short answer/ simple application (medium cognitive load) and 1/3 should be high level application (multi-step, or multi concept) – High cognitive load. This rule was used on all first Year Physics test/exams at the time. If you trust your students are doing the reading, and following along you might ditch the first third and make shorter tests that only cover the medium and high cognitive load questions.
Does Mount A have automatic exam grading software yet? I haven’t used it myself, but I’ve heard that if can be scarily good, and reading an essay to double check an automatic mark is appropriate is a lot simpler than reading it and marking it yourself.
I’ve been in “Contract graded” courses – where if you do all the assignments adequately you get an A, most assignemtns get a B, Core assignemtns get a C, and nothing gets a fail – + and – assigned by prof based on class work/whim. Every assignment get marked pass fail, and if you do the required ones you get the mark. This can work quite well.
Hi Cam,
I don’t think that Mt A has automatic exam grading software. We have WebCT that you can give the kids quizes and assignments on, but I haven’t heard of anything more intricate than that.
These courses called “contract graded”, are they just based on assignments? No testing per se? Hmmm.
Mt.A. Still has WebCT? Very odd – I thought Blackboard had killed it a year or two ago after they bought it.
Three years ago Dalhousie had Exam grading software. Give Toni Roberts over in Computer Services a call – see what he is supporting.
Google and Wikipedia have good info on setting up contract graded courses. The concept is at least as old as 1972. Contract grading usually doesn’t give marks for assignments – just pass fail – and the pass level is more like 70-80% not 50% – if people hand in crap they should expect to have to improve the assignment or do an alternative one.
I will get on Toni’s case. WebCT is still the only deal on campus, and he was quite chuffed to get what he referred to as BobCT dismantled (no mare aci server…) so us devoted Mac types have use the unbelievably clunky WebCT file transfer system.
Do you think it is possible to get students to do a final exam on a computer? I am going to have a final for 13 4th year students in the Dunn building and I thought it might be more real world if they could do it on their laptops. Obviously it would be open-computer, and even open-web, but that might be neat.
Most of the work my students submit, in the online writing and literature courses I teach, is via PDF, Word, or RTF files. But I use Blackboard’s assessment tool to create multiple-choice reading quizzes. The grade is recorded automatically, so it saves me a lot of time.
This term, I experimented with using the assessment tool for a mid-term exam. It allows different question types (e.g. multiple-choice, short answer, matching, etc.). And it will still grade the objective portion even if you have to intervene for the non-objective ones. It’s not a perfect tool, but it’s all I have at the moment. And I’m more-or-less happy with it.
Undoubtedly, teaching and testing should be separate activities. Even testing should be modified based on types of learners. Yesterday I was conducting an assessment center for sales officers (26-35 years) and I realized most participants fared well during role plays, group discussions, case study discussions but fared poorly during written tests.
While all the assessment activities were tests, only written tests were more formal where participants sat down with answer sheets and had a stopwatch monitoring time.
I feel adult learners (non academic professionals) have a negative mindset when it comes to formal tests while they are chilled taking the same test but in a more informal manner.
Roger Shank’s Student Bill of Rights spells it:
“For the use of students and teachers everywhere, and by way of summing up the real issues in education, I present the Student’s Bill of Rights:
1. Testing : No student should have to take a multiple-choice or fill-in-the-blank test.
2. Real-Life Skills: No student should be have to learn something that fails to relate to a skill that is likely to be required in life after school.
3. Memorization: No student should be required to memorize any information that is likely to be forgotten in six months.
4. Clarity of Goals: No student should be required to take a course, the results of which are not directly related to a goal held by the student, nor to engage in an activity without knowing what he can expect to gain from that activity.
5. Passivity: No student should be required to spend time passively watching or listening to anything unless there is a longer period of time devoted to allowing the student to participate in a corresponding active activity.
6. Arbitrary Standards: No student should be required to prepare his work in ways that are arbitrary or to jump through arbitrary hoops defined only by a particular teacher and not by the society at large.
7. Mastery: No student should be required to continue to study something he has already mastered.
8. Discovery: No student should be asked to learn anything unless there is the possibility of his being able to experiment in school with what he has learned.
9. Defined Curriculum: No student should be barred from engaging in activities that interest him within the framework of school because of breadth requirements imposed by the curriculum.
10. Freedom Of Thought: No student should be placed in a position of having to air his views on a subject if the opposing point of view is not presented and equally represented.”
Source: http://www.engines4ed.org/hyperbook/nodes/NODE-285-pg.html