home. / work. / The voice of art

Overview

72% of Brazilians have never been to museums nor art institutes because they do not relate to or understand art. For the launch of Watson in Brazil, IBM collaborated with Ogilvy to create “The Voice of Art” project, an interactive guide that allowed people to have conversations with art pieces housed in Pinacoteca, São Paulo’s oldest museum, known for its innovative vein. Initially for two months, Pinacoteca’s visitors could use an exclusive app to literally talk to seven art pieces of the collection, asking them whatever they wanted, the way they wanted; the response was given by Watson through its Brazilian voice in real time and natural language, explaining subjects such as the history behind the piece, art techniques and styling, and even the relationship with current topics. It was a creative and unprecedented idea that not only brought museum visitors’ experience to a new level and generated awareness to Watson: it also made Brazilians become more open to and interested in art.

The Challenge

• Design a solution for a broad audience aged from 6 to 70 years old.
• Introduce the app in the context of museum visitation without interfering with the regular flow of visitors.
• Trainning the watson as many question as possible, even distinguinching the age of the person who’s asking
• Design a mobile app to help visitors and guide them to have the best experience

Project Schedule

We‘ve developed an action plan that splitted the work into:
1.User Research
2.Insights and ideas for the app
3.Prototyping
4.Tests
5.Iteration for final solution

User Research

The biggest challenge of this project was designing a product that could serve a broad audience from 6 to 70 years old, since we’re talking about all visitors of the Pinacoteca museum. To begin with, we did a qualitative research by interviewing 25 people, a considerable amount based on the extensive age range.

Learnings

The goal was to understand users’ behavior regarding the visiting experience at the museum, by mapping their journey and needs. Furthermore, we’ve studied their behavior while interacting with daily apps so that we could design an experience as familiar as possible.

Users aged 40 to 70 years old feel a great barrier to go to the museum, believing that understanding art is very difficult. They feel more comfortable when some help is offered such as audio guides. But it needs to be offered at the beginning of the visit, or else they just give up.

For the younger audience, such as children from 6 to 12 years old, the visit is an exploration experience. Visiting the museum is not difficult to them, even without being able to fully understand art. But they do get bored with pictures and they prefer interactive art pieces they can touch or understand its meaning more easily.

About the behavior inside the museum, even with very different age groups, a common preference among adults are the indications on where to start the visit, making them feel a little less lost.
Children’s visits followed the order of the rooms, even without a logical order of the exhibition.

In Brazil the most popular apps are for chatting, social networking, location guides, bank services, streaming and gaming.
Only the most distant age groups, such as children from 6 to 12 years old and elderly from 50 to 70 years old, have a big difference between them.

Some Data

The museum splits its audience into 2 types: Educative Audience and Total Audience. Educative Audience refers to students and Total Audience is the common visitor.

In 2016, one year before the project, the numbers were:
Educative Audience: +42K people
Total Audience: +325K people

Weekends are the most visited days in the museum, especially on Saturdays. Virtually 4X more visitors than on weekdays, including Educative Audience visitings.

This data helped us to plan the days and hours for the prototype tests to approach reality.

Design Sprint

We’ve mapped the key insights:
• The solution should be offered to public at the entrance of the museum, specifying the purpose to help understanding the art pieces in a simple way and free of charge.
• Location guidelines are welcome within the museum and they need to be didactic as it is common that the audience has never been there.
• Make clear to the audience that they could ask art pieces anything in any way, just like a common conversation.
• Make clear they could talk to the art pieces alone, without the help of anyone.
• The audience is used to apps with specific tasks and few distractions on the interface because they are easier to use, like WhatsApp, Youtube, Skype and Netflix.

The Solution

The strategy was putting all Watson’s cognitive power into a mobile app available to visitors of the Pinacoteca Museum. This application uses several Watson APIs to:
• Recognize visitors’ questions
• Decode the intent of those questions
• Deliver specific answers
Other challenge of this project was designing a product that should be easy to handle and at the same time simplified all the complexities behind watson. How to turn algorithms into voice, being accessible and relevant to the age range of museum users.

In addition, we’ve incorporated a system at the museum building that recognizes the visitors’ presence in front of the art pieces to begin the interaction in the app without visually interfering in the art pieces.

Prototype

This phase was divided into two fronts.
1 – Design of app user experience: the experience while interacting with the art pieces should be intuitive and easy to talk to and to explore.
2 – Rethink the experience in the physical location: mapping of messages and instructions so the audience could find the art pieces and be guided by the museum.

APP EXPERIENCE

App user experience
Considering all learnings in the research process, we developed the wireframe with 4 main intentions:

1. Create a great onboarding process
2. Locate and recognize the art piece in the museum
3. Facilitate interaction with the art piece
4. Evaluate Watson’s answers to improve its cognition

A great onboarding was crucial since our public needed a didactic experience, so we addressed the app’s functions in a quick and easy way.

Art Piece Location

(user location inside the museum X art pieces locations)

Art Piece Recognition

When approaching the art pieces, a screen opened up to quickly recognize the artwork.

Questions Recognition

The main goal was to encourage the user to talk to the art piece, so we’ve highlighted the question button right on the interface and we did some animations for each stage (stand by, listening, thinking, speaking …). The main screen also had easy access to map and help section.

Mapping messages

We’ve made a map of messages that guided the audience inside the museum. These messages were sent to the device by proximity through beacons. But, if users passed through the same beacon again, the instructions were not repeated a second time.

We’ve classified the beacons into 3 functions:

Location indicator beacons: These were activated within 20 meters of a room containing an interactive art piece. Users listened to (or read) the instructions on where they were and which art pieces were nearby.

Art pieces beacons: These beacons were activated within 5 meters from the artwork. An introductory text was launched inviting users to talk to it.

Gateway and exit beacons: These beacons were activated 2 meters away from the sensor and thanked for the visit.

Test

We’ve tested and iterated the app each week for 2 months. The message map and the quality of Watson responses were also tested (this helped improving the understanding of artificial intelligence).

The main lessons we learned about APP

• People struggle to interpret maps and blueprints, because they are much more visual. Telling them the floor number, the room name and showing a picture of the art piece was enough to locate themselves.
• In Brazil, people hardly go alone to museums. They like to talk to someone during visitation to share their perceptions, so the app shouldn’t become a barrier between them.
• They prefer not to follow scripts. They like to tour as they please.
• They prefer to choose when to start the conversation and when to stop it at any time.

The main lessons ]about the beacons

• Visitors don’t follow ordered directions and as they may be coming from different sides, informations like “turn right, turn left” are very confusing in this context.
• People like to be reminded when they are already in the room that there are cognitive art pieces there. They chose to enter this room and the information comes in form of a “tip” without disturbing their visitation.
• But they don’t need to be reminded every time they enter a room that there are cognitive works. They understood the message on the first time.
• It’s positive when the work presents itself to them at first, but to keep repeating the same information is annoying.

Final Solution

The main change was that the app wouldn’t be based on a map any longer. So by taking off the museum map, the app became much easier to use. We also continued to prioritize one action at a time (onboarding, locating the artwork, talking with the artwork, evaluate the answer).

Finding and recognizing the art piece in the museum

We changed the app in order to work accordingly with the floor the user were at. For instance, if the person was on the first floor, the app showed which galleries were there and which cognitive art pieces they would find. The user could also swipe the screen to see all galleries on that floor.

Talking to the art piece

On the question screen, the “question button” gained more relevance. We removed other buttons since tests’ results proved it to be unnecessary. We added a tab so users could access all art pieces whenever they wanted. If they weren’t close enough to the art piece, the button was disabled.

By the end of each question, we added a questionnaire directing Watson’s training team to improve quality on responses. This team was composed by IBM members and museum curators.

Results

Impact

$3.3 million in earned media value

More than 50 local and international PR stories, including a 4 minute segment on Brazil’s leading TV news program; highest organic reach (+1M) and engagement rate (37% – our average is 3%) in the history of IBM Brazil’s Facebook page; +6M views on YouTube and Facebook, being +280K organic views.
But more than that, it has already delivered business results: 4 new client opportunities for IBM including another big museum, proving that the results were positive in different spheres.

Awards