The problem is the assessments

If you want to change the industrial school system to make it more equitable and better for more students, you’ve got to start with the assessments, says Chris Dede — my professor and someone who studies why many educational improvement efforts fail.

Here are my thoughts on his statement and a bit of brainstorming to take it further:

First off — why do we need to change our assessment model? What’s wrong with testing? I thought learning scientists love tests.

Yes, testing or quizzing can be a great study practice. For instance, if you want to memorize something, self-quizzing and other forms of retrieval practice are much better than re-reading notes or other passive study methods as documented in many research studies (Source 1).

But we’re not talking about testing/quizzing as a learning tool or as a way of assessing gaps (formative assessments). The assessments Chris Dede is talking about are referring to summative high stakes tests, which are used for grading. The major problem with summative high stakes tests seems to be that they are not capable of measuring all types of knowledge. I would argue that this type of testing encourages teachers, parents, administrators, students (e.g. everyone) to structure education around the temporary memorization of shallow content — instead of focusing on real learning.

So what is real learning?

That seems like a whole new post, but let’s say that real learning happens when a person changes and becomes someone new in relation to themselves and the world. I think of learning as the expansion or change of self. Most changes in life draw not only on the possession of knowledge but also skills like self confidence, teamwork, creativity, and other cognitive, inter- and intrapersonal skills. Here is an interesting definition of learning by Lave and Wegner who explain that all of this is embedded within a social or community context:

“Learning only partly — and often incidentally — implies becoming able to be involved in new activities, to perform new tasks and functions, to master new understandings. Activities, tasks, functions, and understandings do not exist in isolation; they are part of broader systems of relations in which they have meaning. These systems of relations arise out of and are reproduced and developed within social communities, which are in part systems of relations among persons. […] Learning thus implies becoming a different person with respect to the possibilities enabled by these systems of relations (Source 2: p. 53).”

And one more quote from the same book to cap it off:

“In contrast to learning as internalization, learning as increasing participation in communities of practice concerns the whole person acting in the world (Source 2: p. 49, my emphasis).”

So in summary, real learning is growing into someone new who can participate in life and communities in new ways. It’s not memorizing things, although that is an important prerequisite at times.

Changing assessments to encourage real learning

If we want to change the structure of industrial education we’ve got to change the system built around assessments. What are some options?

• Real-life projects and performances — the most obvious one is also often overlooked.
• User logs (in digital games and interfaces) — see what people do and what they’re capable of by analyzing their digital footprint/actions in class or games.
• Mastery demonstration — that moment when you can do the math problem perfectly nearly every time. Khan Academy, for instance, measures this. This is very similar to traditional testing in some areas.

There’s a lot of promise in all three options. And it’s worth pointing out that there are some contexts in which traditional testing still makes a lot of sense (but it’s not EVERY context, and that’s the point here).

A good question to ask, would be, if I were going to hire a person who could fill X role in our organization, how would I know they could fill that role well? In most cases, managers want employees to have demonstrated that skill previously in the same context or a similar one. With technology, this type of real-world assessment is becoming more possible and scalable.

For instance, imagine a language course where instead of sitting down to take a multiple choice exam, you get to enter a VR world and interact with people in a foreign country. The experience could be powered by actors using something like Mursion or in the future it could even be facilitated by AI.

This is the kind of assessment experience that gets me really excited about a future in which schools are structured around learning by doing and becoming for all students.

1. Brown, Peter C, Roediger, Henry L, & McDaniel, Mark A. (2015). To Learn, Retrieve. In Make It Stick (pp. 23–45). Cambridge, MA and London, England: Harvard University Press.
2. Lave, J., & Wenger, E. (1998). Situated learning: Legitimate peripheral participation. New York: Cambridge University Press.