Tie And Jeans

Refining Assessments

On the Monday and Tuesday of Thanksgiving week, our faculty met for 2 full days of assessment related PD. This wasn’t participatory bacon wrapped lesson planning. It was 16 hours of lecture and table notes, packaged curriculum and bullet points. While I  recognize the logistic realities that make giant scale whole group instruction like that an unavoidable reality, the format did not help the material.

Teachers often exhibit the same sorts of behaviors as their students would in similar situations. Middle school teachers if you sit amid a desk for six or eight hours 10 tend to lean a little snarky. But it’s interesting to note that snarky teachers (and students!) aren’t checked out, they’re just funneling their fidget impulse into a backchannel that yields immediate positive feedback from their peers.

In a really well-designed move the reflection for that for those PD days was put off for a couple weeks. So this morning, two weeks later by the calendar and a lifetime later according to the discontinuous internal teacher clock, were coming back into reflect on the experience and our thoughts on assessment. It’s a great move because I know all the snarky middle school teachers have been chewing and mulling over those ideas that entire time will come back together not as students watching the clock for the end of the day, but as educators ready  make better decisions for our classrooms and our students.

Do I have 16 hours worth of new wisdom? Probably not.  But I have a refinement of an earlier belief about how information access can reveal rich assessment, and a better sense of what practices teacher can iterate on to push them towards better assessments and stronger classrooms.  [The restatement ran long, so I’ll talk about the iterative practice angle tomorrow.]

I first started poking at this idea in a somnambulistic rant three years ago. That first form was just a floor, a cut off line for assessment strategies that have lost relevance in a information. This is a bit more nuanced, a floor and a ceiling for “kinda okay” assessment.

 At the minimum, an assessment must distinguish between a studied/prepared student working in information isolation (classic testing model) and an unprepared student with unfettered information access.

 A rich and well designed assessment reveals distinctions between an unprepared student with information access and a well prepared student with that same access.

I’ve had a number of uncomfortable conversations around that first principle, because it calls out many accepted teacher/classroom practices as wrong. It’s exactly the argument that teachers expect a tech person to make, full of bravado about how the internet has all the answers so students don’t need to learn anything. I honestly believe there’s more nuance than that, but it’s a fair cop.

We have tools that can graph any system of equations faster than any student, available on demand from any device. The floor criteria doesn’t say you can’t assess student’s ability to graph equations, rather that you also ask them to make judgments or extrapolations from that process beyond what Google or Wolfram Alpha yield.

I also want to point out that “prepared” included a lot of literacy skills that help students comprehend and tackle a given problem. My model for an unprepared student is a teacher from a different department in the same school. Without hearing a single minute of class experience, how well could I handle your history exam? If the class vocabulary is nothing more nuanced than the Wikipedia bullet points, then I have a pretty good chance of keeping up. If the question relies on meaningful work done throughout the term, then I’d need to churn through that cognitive backlog before even approaching the question.

That’s just the minimum we should expect from rich assessments, and I’ve struggled to articulate cross-curricular principles for truly great assessments. It’s easy enough to find specific examples, but those are great because the mode of assessment is so deeply tied to the subject matter. Build a line following robot from these components. Great, how would I do than in history?  Produce a museum exhibit from some subset of these items that tells a story about the historical individuals life, beliefs and society. Great, what does that look like for 7th grade Bio? Andrew Watt suggests trying to confirm some of Hooke’s observations with hand ground lenses and a USB microscope.  What about 12th grade Econ?  Dude, I don’t know yet!  We’re trying to build a Grimore for that stuff!

The criteria for rich assessments suggests that there needs to be skills brought into use during an assessment, rather than just information. The core must be some cognitive task that’s been practiced and refined through the duration of the course, which students have to apply in moderately novel context. Shawn is great at this stuff:  How much energy is in Mario’s fireball?  With information access but no skill practice, students will flounder and produce “naive” work. **  With practice and no information access, students produce shallow journeyman work, like an well structured AP Lit essay that doesn’t cite or analyze the text in question. When students have information access and practiced skills, there’s no ceiling to what they can accomplish.

The question I’m skirting in all of this is time. I can probably design a linear systems quiz that’s full of tricks and shortcuts so that a practiced student would be faster and more accurate than a naive google-bot.  What does that count as?  Building a robot from scratch is a nice idea, but that’s not a “exam” in any meaningful sense of the term. Time scale is what separates an assessment from a project, and I’m still unsure how that distinction changes the prompts and questions I’d use for either.

That’s not a perfect criteria, but it’s better than where I was before. Looking back at my teaching career, I’ve been really happy when I created assessments that managed to clear the floor criteria.  I think I’ve had a half dozen that found their legs and managed to reach the second criteria, and several of those were accidental creations. That’s a valuable sobering thought. Even with my best intentions, I can’t count on myself to create assessments with enough head room for well prepared internet-enabled students to truly shine. Since CMK, I’ve been using Gary’s good prompt guidelines to steer me though this process. Someday I hope to have a library of capstone assessments for all manner of subjects, each printed as a single sentence on a 3×5 card.

** Am I begging the question here? Can we qualitiativley identify naive work in respective disciplines? This is what I spend my time contemplating on mathmistakes.


Single Post Navigation

5 thoughts on “Refining Assessments

  1. I’m not sure the lenses have to be hand-ground, but a parallel form of observational tool. Isaac Newton stuck a bodkin in his eyesocket and then observed how changing the shape of the eyeball changed the shape and structure of his field of vision. But maybe asking students to study and observe one another’s eyes through large, hand-held lenses would be useful — not just because it would be replicating-ish a historical experiment, but because it would be discovering what it’s like to do direct observations of a living, active system. The mere act of observation, of course, changes the system.

    I like the idea of a capstone project as a sentence on a 3×5 card. At the same time, I’m discovering that I genuinely need the huge filing cabinets left in my Design Lab at the end of the summer, to hold all the detritus of project development. Later on, we’ll know how we did Project X in Grade Y… right now, we’re generating napkin-sketches and late-night conversations over burritos, trying to understand this Design Thinking eudaemon we’ve released through our efforts to build the grimoire.

  2. tieandjeans on said:

    I’m sure we discussed Stephenson’s Baroque Cycle at some point during the Maker Faire weekend. Even if I didn’t say the name out loud, I don’t think I can get too far into an natural philosophic midset without hitting that text. I’m willing to take most of his “young Newton” anecdotes as more or less historical (matches most of what I read in Glick’s Newton biography) up to the point where there’s a major alchemical fire. Any one of those exercises, from tracing the course of the sun on the wall of his bedroom, to tying intricate single thread nets to cradle smalls tones, would be a fine capstone project.

    One of the maker mindsets I try to cultivate in myself, and would love to find way to pass on to students, is the observation mode that he captures so well through his fictionalizations of Netwton (in the Baroque) and Turing (in Crypto).

    • My friend Scott says that “a picture is worth a thousand words, but a part is worth a thousand pictures.” When in Boy Scouts, teaching environmental science, I used to balance rocks on their pointy ends all over camp. The balanced stones, especially after I left, had a tendency to make scouts and scout leaders alike think ‘magical things’ were possible in that place. And being the guy who “made the magic” (how little they know), gave me ‘forest cred’ (instead of street cred), that I could then leverage to get across bigger ideas to them.

      I think that maybe you and I should make some of these things, like a net to catch a stone using a single string, and suspend them from the walls of our Lab spaces. The IDEO offices had a DC-10 wing hanging across their workspace, and the presence of extraordinary things in our rooms become symbols of the power of learning in those spaces. They’re as much talismans to the potentials of the Maker/Design Thinking movement, as they are actual tools. Sometimes the symbol is much more important than the real use of the tool.

  3. Pingback: Parts are Worth 1000 Pictures « Wanderings in the Labyrinth

  4. Pingback: Feedback First | Tie And Jeans

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: