JSTOR Labs Methods: the Wall of Ignorance

JSTOR Labs works on a wide, wide variety of projects, from incubation projects like Constellate and JSTOR Access in Prison to discovery projects like the Future of Scholarly Meetings to Cultural History Baseball Cards.  One unifying feature across all of these projects is that they are all categorized by how little we, the team of people responsible for developing the products, know. We might not know whether what we’re building is possible, or if it’s a good idea, or who might best benefit from it, or if it’s a feature, product or business, or if ITHAKA is the best organization to develop it, or what partners we need, or … we could keep going.  There’s so much we don’t know. It can get quite overwhelming.

We have written in the past about the methods we use to learn these things: prototyping, user research, rapid iterations, etc., methods borrowed from or inspired by design thinking and lean startup methodologies. Over the past few years, we’ve hit upon a new method that we haven’t seen mentioned elsewhere, and we’re eager to share it.  We are pleased to introduce you to … The Wall of Ignorance.

Our First Wall of Ignorance  
In early 2022, the project team for the JSTOR Access in Prison Initiative met in Ann Arbor.  We had received a grant to scale our pilot projects to every higher education in prison program, and we had just hired the team that would be dedicated to making it happen. As we began to make plans for achieving this ambitious goal, we had what felt like infinite questions swirling in our minds about the future of JSTOR Access in Prison.  Some of us in the room were used to working in uncertainty, but the sheer volume felt daunting. We decided that it was better to be ignorant together than alone. So we made a collective activity out of it. First, we gave everyone in the room a stack of sticky notes and had everyone brainstorm on their own, silently responses to this prompt (one item per sticky note):

 “What do you wish you knew that you don’t, about this project? Which uncertainties worry you? Which questions, if you had an answer to them, would make your job easier?”

After five or ten minutes of brainstorming (ignorant-storming? Question-storming?), we took turns reading our stickies to everyone else in the room, and we put them all up on one big wall. Finally, we “affinity diagrammed” them (that is, we grouped the stickies  by similarity, putting similar or duplicate ideas together). This is how the wall looked when we were done:

Just the act of sharing these questions and getting them out in the open was helpful. In many cases, we found that others in the room had the same worry or question, and there was comfort in knowing we weren’t alone.  In others, someone had questions that we didn’t, helping us more fully understand the scale and scope of the problem.  

But what really helped was what we did next: turn the wall into an ignorance-smashing roadmap. We could answer some of the questions quickly, with just a phone call to the right person or a query to a database. That week, we used a kanban board to knock out as many of these as we could.  Other questions were just too big to be answered quickly – in those cases, we could prioritize and sequence the questions and devise tests to get to answers as lean-ly as possible.  When we built a project roadmap at this same workshop, we included a swimlane dedicated to What We Will Be Learning which sat right alongside the technical and outreach work we had to do.

Since then, we have used the Wall of Ignorance and related activities on a number of other projects, from expanding access to JSTOR in secondary schools to universal transfer explorer.  Each time, we are amazed how quickly it helps us explore a complicated territory.

Your Own Wall of Ignorance: What to Expect
What kinds of questions have we encountered and might you expect if you build your own Wall of Ignorance? For us, we are always impressed with how broad-ranging the questions are, especially when the room is populated with our usual diverse team including engineers, designers, product owners and stakeholders. Usually questions fall into a few categories:

User questions, like:

• Is X group or Y more likely to need or use our product?
• Can we find a clear enough way to present X data in the UI?
• What % of use in mobile vs tablet vs desktop will we have?
• Might there be someone who needs this product even more than the people we were originally making it for?
• Where can we recruit the people we need to test with?

Tech questions, like:

• Will the data we can source be high enough quality?
• Should/can we work with X or Y platform/framework?
• For which pieces of our product should we write code from scratch vs. using something pre-existing that’s less customizable?
• Which features should we prioritize building?
• Given that this is a prototype and we don’t know if it will be a permanent official product, how should we balance dev speed with other longer-term concerns like maintainability, documentation, load balancing, and resilience/redundancies?

Business questions, like:

• What is the size of the market?
• If we build something awesome, what % of that market will pay for it?
• Is X group or Y more likely to buy our product?
• Should we try working with a new partner?
• Why isn’t product adoption as high as we’d expected?
• What should the price point be?
• How should we market this?
• What are other players in the market doing?

There may also be questions that are more general, or uncategorizable, like “Is it time to pivot? How do we know when it is?”

When to Use a Wall of Ignorance
In your own work, you might find the most value from this method when you have an overwhelming number of questions about how to proceed in a particular project. It’s probably more likely to be useful when you’re working under conditions of uncertainty, early in your work on a particular problem or in a particular market. The method also requires group psychological safety; of course, it has to be safe to admit you don’t know everything and in fact, don’t know a whole heck of a lot of things. Other than that interpersonal safety, all you need to create a wall of ignorance is about an hour of time and real or virtual sticky notes.

Benefits
From this activity, we get a shared understanding of what we know and don’t know. Occasionally, someone asks a question that someone else in the room already knows the answer to, or knows how to get it quickly.

It makes it easy to identify the riskiest assumptions and unknowns for the product, so we know where to focus our attention.

It helps us create an experiment- and research-focused roadmap; we understand which questions are vital to get answers to and can think about approaches to find those answers.

Finally, we’re able to retro with these questions 6 months or a year down the line. It’s edifying to realize in hindsight whether you were asking the wrong questions or the right ones. This reflection lets us hone our innovation instincts.

It’s not easy to work on projects with evolving requirements under conditions of extreme uncertainty, but we have found this shared understanding of all our questions - and the prioritized answer-finding that follows - turn our ignorance into progress. Let’s be ignorant together! We’d love to hear how it goes if you try making your own Wall.