Menu
Client
Sheba Medical Center ARC innovation lab
Industry
Healthcare
Services
Design Sprint, Rapid Prototyping, UX/UI
By Ram Almog,
CEO
Israel’s Sheba Medical Center ranks 9th in the world in Newsweek’s prestigious list.
It’s a huge facility and it’s innovation lab runs more than 40 Machine learning research projects.
When the Covid-19 outbreak started, we knew we wanted to take part in the effort, make ourselves useful.
Sheba was in the midst of a war, and as part of their strategy, they installed status dashboards in the Corona departments, showing them the current patients stats and available resources.
A lot of data was coming in and the Machine Learning team was working on prediction models that will help the doctors assess possible deterioration of patients based on different measurements, background illness and the progress of the disease. Sheba’s innovation center wanted to use the situation and the change of atmosphere to get a better understanding of how these types of models should be integrated. She wanted to come out with a great process that will ensure cooperation and real impact for these type of projects.
We had 2 big barriers:
What we offered is to help them find out how to integrate these predictive models into the doctor’s day to day routine. This wasn’t a simple UX task as doctors are very busy, and overloaded and are very suspicious of technologies that try to be predictive. Also, machine learning algorithms need training and we needed doctors to cooperate and give us input.
To overcome our challenges, it was clear to me and my team that we need to do a few shifts in the process.
First, we made one on one interviews with as many people as we could get.
Especially senior doctors that were willing to give us their mind. To prepare everybody for the interview and sprint, we sent the interviewees a questionnaire, asking them about the challenges and their long term goal.
Based on the interviews, we prepared the user journey map:
and some of the main challenges we got from the users, reframed in an “How Might We” question format.
Next, we split the workshop into 3 zoom sessions, 3 hours each: Define the challenge, Create Multiple solutions, Curate and select a solution. The rest of the sprint was carried out internally with scheduled feedback from Sheba’s team.
Let’s dive deep into the process:
The first session was focused on defining the challenge. It’s all about alignment! Making sure we are all on the same page and fully understand what we need to solve. This was a remote sprint so we used Mural to help us communicate and interact. On Mural, we use a preset template with dedicated spaces to all exercises. We also prepared a personal space for each participant so we can apply the sprint’s “Together Alone” principle.
The format we use for this is the “How might we” questions, for example: “How might we generate trust” or “How might we get feedback on our predictions”.
Reframing challenges into this format helps us refine them and standardize them into solvable challenges.
For the expert interview we diverted a bit from the classic expert interview.
we went through the map we drew before the workshop. While I was presenting, I asked the participants to write down on notes, the challenges they identify. The format we use for this is the “How might we” questions, for example: “How might we generate trust” or “How might we get feedback on our predictions”.
Reframing challenges into this format helps us refine them and standardize them into solvable challenges. After we wrote the challenges, we voted on the most important challenges and got the top 6 challenges that we are facing.
Usually, we ask participants to look 2 years ahead. This time, looking at the time pressure and the uncertainty of the reality around us, we asked the participants to look 1 year ahead and think about the most optimistic scenario. Each participant wrote her own 1 year goal on a virtual sticky in her personal space. When time was up we copied the goals to one place and each participant read her goal with a short explanation.
We had a short virtual vote (one vote for each participant and when done, Nathalie (the decider for this sprint) chose the following goal:
“In one year we will have a standard protocol to integrate AI based tools into the clinical process enabling us to try new (developed) models in less than a month and make real clinical impact”
We now had a better context, and a wider challenge – coming up with a standard integration protocol.
This also meant that we probably needed to interview doctors from different wards in the hospital not only corona related medical staff.
After all of this positive thinking, we have to start looking at the things that may block us. I asked the participants to look at the long term goal and think of the things that will stand in our way. I asked them to phrase it as a question, starting with the words “Can We”. These will become the sprint questions. The things we want to make sure we can solve by showing a prototype to our users. Each participant wrote their top “can we questions” and then we voted and decided on the top four.
Usually, we stick to only three but sometimes you just have more.
At the end of the previous session, I have asked all participants to research and bring a few sample solutions that may be relevant to our challenge. I have asked them to look at the “Can We” questions and the “How might we” and find samples from different domains, solving simslar challenges.
Each participant wrote the name of the solution and the big idea behind it, and got a few minutes to present it. This way, before going into creating our own challenges, we got some inspiration and ideas.
Interestingly enough, we looked at netflix for getting user feedback, we looked at digital banking for trust issues and Monday project management for a sense of control.
By Roi Weinberg,
Product Designer
This step is part of smaller steps.
The first step was the note taking exercise.
In this step we want to get our brains running. I asked the participants to copy the long term goal, the sprint questions and anything interesting on the board into their pre-prepared A4 papers.
I do believe that the writing process makes you think about it, and forces you to go back and inhale the alignment we got. I asked the team to start and write some notes to themselves and some ideas for a solution.
When done nothing, I asked the team to start forming up an idea of a solution.
I showed them some samples, but they had to do it on their own.
This is sometimes a little bit awkward, but I re-assured them this is natural and that the process can be trusted.
After having a basic concept, we made the team iterate. They folded a paper into 8 parts and I gave them one minute each time to think of a different way they could approach their basic solution or alter it.
We do that 8 times and this makes people go beyond their initial ideas into optional solutions.
All participants spent 45-minutes creating a 3 part sketch. We were drawing on A4 with sharpies and uploaded the results to the mural when done. Then, we added a catchy title and notes on each solution to better explain the concept and make it more self explanatory. By the end of this session we had 7 different solutions.
In this session we go over the solutions, understand the core idea and the details and select one or two solutions as our prototype base.
Each participant went through all the different solutions and used a red circle to point out what they found interesting. Whenever questions arose, the team added a note next to that part of the sketch.
By the time we were over, we had a hit map showing us points of interest.
Actually, the most important thing in this exercise is that it made us read carefully the solutions while getting a clean first impression.
Each participant went through all the different solutions and used a red circle to point out what they found interesting. Whenever questions arose, the team added a note next to that part of the sketch.
By the time we were over, we had a hit map showing us points of interest.
Actually, the most important thing in this exercise is that it made us read carefully the solutions while getting a clean first impression.
Each participant presented her solution. Normally, the facilitator would do that, but as we did not have enough time to prepare and unlike face to face sprint, it’s much more difficult to understand and present another person’s solution, we decided that solution presentations will be done by the solution creators.
Each participant got one Mural vote to select a winning concept. Once voting was done, each participant explained the reasoning behind their vote.
Now it was time to make a final call. Nathalie had a short discussion and finally voted on a winning solution to work from and asked us to combine it with another complementary concept.
In this exercise we asked participants to take 6 notes and outline the actual screens that we should prototype.
Each participant wrote 6 notes representing 6 steps
in the user test flow. We all presented our suggestions and voted.
The decider decided on the winning flow. She added one additional step to fine tune it.
This ended our joint workshop with the Sheba team. Now it was on us to deliver.
In many cases, we prefer to do this with our clients.
This time, we worked on our own and asked for help when we needed it.
Ram and I used the user test flow and the selected concept, and layed out wireframes that outlined the prototype functionality. We used Figma as this is the easiest way to do this kind of remote collaborative design work. Once done, we presented it to the team and got some feedback.
Now that we had all of the information and the basic design, I took over and created a real looking prototype.
I worked on creating a real looking prototype to be presented the day after to the doctors.
We had a short hand shake with the team to make sure we are all on the same page and we were ready for user testing.
It was quite hard to get the doctors’ attention at this time, but the team really helped, and we decided to take our time and did the user interviews over two days (we usually do it in one day).
We interviewed six doctors from different departments and specialities.
All of them were specialized doctors with a lot of experience. So we got very useful feedback from them.
All of them really liked our solution and stated they would be really disappointed if we decide not to do it.
This is a great indication that we are offering something of value to our users.
The doctors really liked the timeline view with our predictive information.
However, we got strong rejection with regards to the system recommendation. The doctors were really concerned about getting treatment recommendations from a machine. They noted that these are very complex issues and therefore suggested recommendations should be more restrained and limited with very restrictive language.
They also said that we should minimize and simplify the feedback they are asked to give as part of the flow
as they will be too busy to cooperate. To sum it up,
we got positive answers to our sprint questions and affirmation on our solution. It was a very exciting experience and this is just the first step with this exciting project. The Sheba team were awesome, we learned a lot and hopefully we will make a good impact on the way AI is used in the medical Sphere.
By the end of this Audit call, you will have a clear understanding of the next steps you can take to step up your product.
Product Leaders (Startup CEO’s, Product Managers) who are currently dealing with complex problems, creating a new product or significantly improving an exiting one
