Over the past four months, our team conducted three rounds of tree testing to improve our website’s navigation. As I mentioned in my previous blog post, more than 90 students, faculty, and staff participated in these online exercises. Now that the tree tests are complete, I’m excited to share our findings and plans for future usability studies.
Goals
For each test, we aimed to recruit at least five students, five faculty, and five staff members. We emailed various groups and invited them to take the test online. To ensure that we were getting genuine results, we only invited participants to complete a single test. If a participant was contacted during Round 1, for example, we did not contact that person again for subsequent rounds.
For each test, we determined all the clearest paths a person could take to get to the information they were searching for. When a participant follows one of these clear paths, it’s considered a success. We aimed for an 80% success rate for each task and test overall. If a success rate for a task was below 80%, that told us where we needed to improve for the next round of testing. If a task’s success rate was above 80%, that told us where things are working. With each round of testing, we hoped to improve success rates across the board.
Round 1
For the first round of tree testing, we wanted to establish a performance and participation baseline for all future exercises. To do this, we designed Tree Test A: a menu system with the exact same options, wording, and organization as our website’s current navigation menus. Commonly referred to as the “Mega Menu,” this navigation system contains a total of 116 links divided into seven categories:
- Admissions
- Academics
- Students
- Community
- Research
- About
- Quick Links
| Test | Dates | Students | Faculty | Staff | Total Participants |
|---|---|---|---|---|---|
| Tree Test A | March 31 – April 13, 2022 | 8 | 7 | 15 | 30 |
Round 2
After we analyzed the results from Round 1 and looked for areas of improvement, we designed two hypothetical navigation menus for the second round of testing: Tree Test B and Tree Test C. Each test took a different approach to solve the issues we identified from Round 1.
The menu for Tree Test B contained 81 links divided into three categories:
- New Students
- Current Students
- Our University
In Tree Test C, the menu contained 67 links divided into three slightly different categories:
- Students
- Faculty & Staff
- Our University
| Test | Dates | Students | Faculty | Staff | Total Participants |
|---|---|---|---|---|---|
| Tree Test B | May 10 – 30, 2022 | 5 | 5 | 6 | 16 |
| Tree Test C | May 11 – 30, 2022 | 5 | 6 | 5 | 16 |
Round 3
For the next round, we analyzed the strengths and weaknesses of the tests from Round 2 and designed Tree Test D. This was the simplest menu we tested, with 58 links divided into three categories:
- Students
- Faculty & Staff
- Our University
We planned to conclude all tree testing before beginning Phase 2 of the redesign project, so we were pleased to have finished six weeks ahead of schedule.
| Test | Dates | Students | Faculty | Staff | Total Participants |
|---|---|---|---|---|---|
| Tree Test D | June 6 – 20, 2022 | 24 | 5 | 6 | 35 |
Results
Just as we expected, many participants in Tree Test A felt overwhelmed by the amount of options and had a hard time finding what they were looking for. This was consistent with the feedback we received during stakeholder interviews in the past: the current mega menu is too complicated to be helpful.
When we began exploring simpler menus in Round 2, people were more likely to find what they were looking for. We gave participants the option to leave feedback, and many people left positive comments about how much easier the menus were to use. Tree Test C ended up being the menu with the highest overall success rate and the first test to meet our goal of an 80% success rate. It had room for improvement, however, so we moved forward with creating Tree Test D.
Although Tree Test D had a slightly lower overall success rate than Test C, it resulted in the highest success rates by audience. Students, faculty, and staff each had the easiest time finding information with this version of the test. The feedback was also overwhelmingly positive.
| Task | Test A | Test B | Test C | Test D |
|---|---|---|---|---|
| 1: Find a list of majors and programs | 98% | 94% | 88% | 86% |
| 2: Find tuition and fees | 83% | 100% | 94% | 97% |
| 3: Find the registration deadline | 73% | 69% | 88% | 89% |
| 4: Find a professor’s phone number | 33% | 75% | 81% | 80% |
| 5: Find information on career planning and job placement | 87% | 100% | 100% | 89% |
| Test Overall | 75% | 88% | 90% | 88% |
| Audience | Test A | Test B | Test C | Test D |
|---|---|---|---|---|
| Students | 80% | 84% | 84% | 85% |
| Faculty | 71% | 84% | 90% | 92% |
| Staff | 73% | 93% | 96% | 97% |
Next Steps
Since we met our success rate goals with Tree Test D, we’ve concluded tree testing. We’ll use the results to help inform our decisions as we build a menu system for the redesigned site. We now know that having fewer options that are organized in a familiar way is most helpful to people.
Next, we’re going to begin preference tests to determine what words and phrases make the most sense to our audiences.
This post was written by Ashi Loftus, UI/UX Designer.