Finding Roebling


Finding Roebling
What: Finding Roebling is an AR exhibit that tells the story of Washington Roebling, the RPI alumni who built the Brooklyn Bridge.

Where: It was exhibited at the Folsom Library at Rensselaer Polytechnic Institute, presented at the Post-Digital Library Symposium, and featured in the Times Union.

Why: To promote cultural heritage and share a historian's passion by engaging visitors in an AR experience.

How: Using Unity 3D, Cinema 4D, and Folsom Library archives.

Development Team: Archivist, Immersive Technologies Professor, and a Washington Roebling expert.

My Role: I was hired as the lead developer and UX designer.

When: Development was from August-October 2017, exhibiting from October-December 2017.

Library display case
Early Development
Finding the Core Problems
In the first meeting with the Folsom Library Director I learned that they were looking to create an engaging AR exhibit about Washington Roebling, the RPI alumni who built the Brooklyn Bridge. I also found out that they wanted to have it done by Alumni weekend, only a couple months away. My first step was to develop research questions to better understand the problem space.

Exhibit case
Exploring Solutions
Visitors were watched and found to consist of a large range of ages. Most were students, but occasionally faculty members also visited. Some visitors looked at the previous exhibit that was still up. It was noted that most walked up close to the case and slowly went from case to case in a circle.

After researching the tradeoffs of multiple AR headsets, and testing the Seebright AR headset, the the Microsoft Hololens was chosen because of its tracking, high-contrast display, and comfort (despite the limited field of view).

By researching traditional exhibit design, the team found a useful set of guidelines written by the Smithsonian for designing interactive exhibits. Despite being from 2002, it is still extremely useful because it focuses on the people experiencing the exhibit and keeping them engaged.

By researching interactive exhibits, theme park attractions, and theatrical productions the team found the immersive adventure experience called 5-wits. The founder has a TED talk where he explains the critical elements of the experiences.

The team learned more about the information the exhibit could potentially convey and how the historian saw Washington as a person. Erica Wagner, the Washington Roebling expert, mentioned that her interest in Roebling wasn't limited to one event, but expanded to multiple facets of his entire life. We decided it was important to show the different faces of Roebling to best communicate his story.

The major time constraint we had was needing to finish by the Alumni Weekend event in October. Another constraint was found after talking to library staff; certain nearby cases couldn't be moved because of fire safety codes. We also needed to consider that exhibit was going to be in a library environment, we couldn't have anything too loud or disruptive. The space has two big windows, lights directly above it, and glare prone glass. This may be an issue because the Hololens has an additive display, so the darkest color it can show the user is not projecting any light on it (which just shows the world behind the visor).

Library display case
Mapping It Out
Once enough resources were collected, we began brainstorming and developed an initial mockup of the experience based around effectively communicating the info to a wide range of users:
brainstorming whiteboard
3D Wireframe
Testing and Iterating
After developing the experience with Unity 3D, I began user testing it with students and faculty on-site at the library using talk aloud protocol and rigorous note taking.

Iteration 1
First iteration of AR exhibit case

Observation: Users didn't intuitively understand the walkway concept. Most users didn't know to stand on it initially. Some interpreted it as a wall they shouldn't step on, others become distracted by it and missed important content that appeared on the exhibit case.

Iteration: We decided to remove the walkway and replaced it with instructions at the start about how each side had different content. The instructions were text based and used a miniature model of the exhibit case to show where people needed to stand.

Observation: Some users thought the floating footprint signs meant something was happening and waited to be shown something.

Iteration: The floating signs were removed and replaced by arrows that pointed to the special stepping spots only if the users didn't step on them after 15 seconds. The teleportation idea didn't communicate either. It was replaced by an immediate reaction similar to stepping on a button. In addition audio cues were added to communicate getting close to a button, stepping on a button, and stepping off a button.

Image tracking markers
Observation: Users didn't understand why they needed to look at a tracking marker to initially align the virtual and real worlds.

Iteration: This observation was expected and helped us confirm our predictions. Because this was an early iteration, and it's best to test early and often, it was ok for some features to not be entirely implemented. In the next iteration we replaced the traditional marker with an image of a portal with the words "Start" over it. It aims to convey that they're entering the experience.

Iteration 2
New instruction text
Observation: The text instructions still didn't communicate well.

Iteration: Upon seeing the instructions at the start of the exhibit, most users immediately looked for the AR content before it was actually there. In the next iteration the instructions appear during the experience instead of frontloading it. This also helps with directing users from side to side in the intended order.
Arrow pointing down at footprints
Observation: Users saw the arrows but didn't look down to see the footprint circles.

Iteration: I expected the arrows above each circle would lead users to notice them, but it didn't work consistently. The main issue with the footprint symbol circles is that the Hololens' field of view cuts them off because they're too low to the ground, making them hard to notice. I revisited the findings from the observational study and developed a new interaction system around that. Rather than buttons, the new iteration simply detects if a user is close enough to a specific edge on the exhibit case to trigger that content on that side of the exhibit.

Observation: The environment could change and make it challenging to demo the app.

Iteration: If it's too sunny the large windows in the library can make it difficult for users to see the AR content on the Hololens' additive display. This was handled by angling some of the content lower to counteract glare. Another challenge is quickly restarting the app when repeatedly testing it with users. This was solved by triggering the experience to reset if the Hololens was set down and not moved at all for a few seconds.

Observation: Users really enjoyed the 3d models and additional media.

Iteration: In the next iteration I replaced some of the normal informational text with audio clips from a voice actor impersonating Washington Roebling. I also updated the models and lighting in the scene because so many users enjoyed walking around to look at them.

Library display case
Final Product & Impact
Final Product
Many more iterations were carried out that tested features such as gaze selection, hand gestures, and attention directing arrows. The project concluded after showcasing the exhibit for the historian's book tour. Here is a video of the final product.

Erica, the historian, was blown away by the exhibit when she came to RPI. She spent the longest time using it of anyone who tried it and was completely enthralled. Erica said she was excited that her ideas could be shared in such a powerful new medium, and thanked us for our work.

Later I showed the project at the Post-Digital Library Symposium. I was one of the speakers presenting work that redefined libraries' role in the digital age. Afterwards I was interviewed by the Times Union and an article was written about the exhibit.

Overall the exhibit was very successful and I look forward to pursuing similar projects in the future.