How Rapid Prototyping in Schools can Fail.

How does your school solve problems, make changes, or figure out what works best? In my previous post I wrote about how important it is for schools to get used to the idea of conducting controlled experiments to generate new knowledge for how make decisions and solve problems. In this post, I am going to be talking about a design model that I’ve seen used in schools to facilitate the testing and revision of new ideas: The Rapid Prototyping model.

As the name suggests, rapid prototyping is a procedure consisting of the development of incomplete or draft versions of instructional initiatives, aka prototypes, and testing them in rapid succession to verify their effectiveness (Nixon & Lee, 2001). The theory is that by producing multiple iterations rather than one, and by emphasizing the process of revision, formative evaluation and testing early in the design process, an effective final product will likely emerge. While rapid prototyping is more of a problem-solving model than a scientific model, if the controlled component of experimental design is maintained during prototyping by creating two conditions, an experimental and a business-as-usual group, and only one variable is manipulated at a time, the Rapid Prototyping model can be a powerful grassroots tool for innovative schools.

Rapid Prototyping vs. Traditional Change Management

What have teachers traditionally done when they want to solve a problem? In my experience, teachers figure out ways to solve small problems by themselves, but for big problems they will usually go to their supervising principal. Since this can be an intimidating experience, and because principals are usually stretched thin, a lot of big problems never make it to principals’ desks. When problems are brought to a principal’s attention, either a top-down solution is formulated through introspection, or meetings are called to give the impression of collaboration and a solution is chosen by consensus.

I find it unlikely that the introspective, meeting-based approach that most schools use will lead to the best solutions. When principals and teachers sit around tables and think, think, think, the solutions that are generated are, at best, partially informed and inherently governed by instinct, bias, and belief. Instead of discussing what we believe will work, deploying it, and then having no way of measuring its success, we might be wise to design and deploy a variety of mini-experiments and test their effectiveness based on measurable criteria, keeping the initiatives that rise to the top. Essentially, I am describing the Rapid Prototyping model. When done well, I believe it can have a powerful impact on the culture of an organization, and is a more reliable method of generating effective and efficient learning solutions than the status quo.

Ideally, the rapid prototyping process should look something like this:

1. Prototype
   • Teachers identify a problem through active “problem-seeking”
   • A team of experts is formed and mock-up/prototypes are designed
2. Review
   • The prototypes are reviewed and then deployed for testing
   • Data is collected and the successes and failures of the prototypes are evaluated (compared to a control group)
3. Refine and Iterate
   • The prototypes are revised (preferably one element at a time) based on the feedback and redeployed for further testing.

To me, this is an exciting alternative to how schools are typically run. Rather than passively receiving top-down initiatives from ill-informed leaders only to watch the solutions decay and disappear with the leaders who championed them, teachers can be empowered to craft their own context-specific solutions with the guarantee that they will be rigorously tested and refined at all stages of development. 

When Rapid Prototyping Fails

Like all instructional design models, there is always a way to totally screw up the rapid prototyping process to the point that it becomes nothing more than a formality and leads to flawed solutions. For the rest of this post, I’d like to describe some of the common pitfalls that I’ve seen firsthand with a rapid prototyping approach, and what we can do as intentional educators to preserve its integrity.

Lack of knowledge or expertise

While the Rapid Prototyping model provides the iterative cycle that describes the generic process for creating and refining school initiatives, it doesn’t include the context-specific recommendations for how to do each of the steps. Rapid prototyping can fail to generate effective solutions when members of the design team do not possess the a) expertise for what a good initial prototype for the project might look like, nor the b) specific knowledge for how to refine the prototypes to make them better. For example, let’s say that a school is trying to adopt a model for teaching math. A successful design team would readily have multiple options already in mind, and a bit of research would produce a few more. Once a couple of selected prototypes are deployed for initial testing, a successful design team would assign experts in math instruction to review the success of the prototype against the objectives, and their expert knowledge would enable them to recognize multiple approaches to refining the prototypes in ways that novices simply cannot. Since one cannot think well about nothing (no, one can’t just Google it), a design team without deep wells of organized knowledge about the problem is doomed to failure from the start.

Lack of research skills or opposition to research

While a design team of teachers without knowledge of the problem at hand is seriously disadvantaged when using the Rapid Prototyping model compared to a more knowledgable design team, a design team can develop their expertise through research on the problem. I’m talking about using a research database and reading peer-reviewed articles in scholarly journals. I described my evolution towards becoming a more research-informed educator in a recent podcast I was featured in, The Research Question. Unfortunately, I’ve been met by opposition to this approach in the past. The main arguments against using research are that a) teachers cannot possibly learn to be research literate (absurd!), b) that research is constantly changing so we’ll never know what’s true and what’s not (short answer, that’s science, but there are, of course, things that have an overwhelming evidence base to the point that they are unlikely to be challenged), and that c) experience and intuition, aka craft knowledge, trumps what those elites say in academia (maybe, except that so much of the knowledge that science produces is counterintuitive and flies in the face of our experiences). For a rapid prototyping approach to work in school settings, those doing the prototyping must draw from the research literature, including choosing between others’ ready-made solutions, to expand their understanding of the problem they’re meant to tackle.

Taking a non-scientific approach

Deploying a prototype can look different depending on the nature of the problem and the context, but the part that tends to be forgotten is the controlled part. Whenever we test something in our schools, we should do our best to organize a business-as-usual group, and we should change only one variable at a time. To continue the example of prototyping in order to adopt or construct a math instructional model, one class section could be given XtraMath to see if it would be a efficient way of improving math fact fluency, while another class section continues learning math facts with the usual method for a pre-determined amount of time. If we roll out XtraMath for all the grades all at once and keep changing the conditions for those involved in the experiment, how can we expect to learn anything from the results?

Risk-adverse or time-constrained faculty

Rapid Prototyping can fail when teachers feel uncomfortable taking risks. Without building a culture that places value on scientific experimentation, which can produce failed experiments or inconclusive results, teachers may prefer instead to go through a more traditional top-down problem-solving process. Equally as debilitating to the rapid prototyping approach is when time for prototyping simply doesn’t exist. Teachers should feel excited to contribute creative solutions, so prototyping shouldn’t be added on to our workloads without taking something else away.

Lack of buy-in from leadership

Leaders that decide to adopt a rapid prototyping approach to school improvement have to walk the walk. When a deployed prototype is well-intentioned, but ultimately fails, it has to be understood by the administrator that this is a normal part of the rapid prototyping process. Administrators can ensure that rapid prototyping succeeds by carving out time in teachers’ timetables to allow them to do this sort of work – No adding without subtracting! –  and use professional development sessions to improve the prototyping methodology. Finally, administrators should model the prototyping process by carrying out their own experiments. If teachers do not have a model for what effective prototyping looks like, we shouldn’t expect that they will be successful at replicating the process themselves.

---

Rapid prototyping is a process for school improvement in which teachers propose instructional initiatives and test them. Even the most well-thought-out experiments can fail, which is both exciting and scary. Still, when teachers leverage their collective knowledge and expertise, integrating research and theory in the process, and receive the support they need from administrators, I think the Rapid Prototyping model can be a highly effective way of designing both instructional and non-instructional solutions for a school.

How does your school solve problems and generate knowledge? Does it look anything like this?

– Zach Groshell @mrzachg

References

Nixon, E. K., & Lee, D. (2001). Rapid Prototyping in the Instructional Design Process Classic Instructional. Performance Improvement Quarterly, 14(3), 95–116.