WatchOwl

AI | Data Visualization | Social Media Analytics

WatchOwl is a web based app that uses social media data to identify potential threats and the spread of fake news

Project Background

The data science team has developed a series of machine learning and deep learning algorithms to detect deceptive news and information for social media data. The team was struggling with selling the technology to the clients, since the technology is complicated and hard to understand. I was brought in the team to design an application that can demonstrate how the technology can be used. Part of the goal was to identify solid needs and strong use cases of this cutting edge technology that can resonate with the potential clients.

My Roles and Contributions

I was the lead user experience designer working with a team of data scientists and software engineers.
My main contribution was finding and selling the real need for the emerging technology. 2/3 of the analysts said they would use the tool because the extra info I provided in the tool; 3/4 of the analysts said the app will greatly increase the productivity when looking at social media data.

The Prototype Video

Lisa is a cyber analyst who monitors and analyzes a wide variety of data sources. Social media data is one of the data sources that it is often overlooked, because the amount of data is overwhelmingly large and incredibly noisy. It's impossible track down everything manually and identify whether the posts are trustworthy or not at the same time.
The video demonstrates how Lisa uses WatchOwl to identify the suspicious news, understand how it is spread, and who spreads it.
‍

The Narratives

In this session, I introduce the purpose of the main views and the tasks that Lisa needs to accomplish.

Dashboard View

Goal: Get a summary of what's going on

Tasks: Find the point of interest and apply filters
‍

Detailed View

Goal: Inspect individual tweets

Tasks: Find suspicious tweets; find the suspicious authors and identify their network; understand what keywords people are talking about overtimes

The Source Tweet View

Goal: Investigate the source of the suspicious tweet and understand how it's spread.
‍
Task: Identify the pattern of the user demographics.

About the Models

Goal: Help the user build the appropriate trust in the models
‍
Task: Read the overview of the model and understand the model limitations.

*The model card's content structure is based on the work done by Google.

The Approach

1. Understand the technology

The challenges for me: at the time, I was new to the AI field. There were a lot of new concepts and terms I needed to pick up to fully understand the technology.

2. Frame the problem and user needs

The challenges for me: Finding the real need and identifying the target user wasn't easy. It required a lot of research and networking with people, as well as deep understanding of the clients' mission and business.

3. Mapping user flow with goals

The workflow is guided by user needs and Ben Shneiderman’s mantra in data visualization: overview first, zoom and filter, then details-on-demand.

4. Design iterations

I worked very closely with the data scientists and got their feedback early to make sure the models can do what I designed. I suggested some changes to the model in order to accommodate the user's needs. For instance, the idea of giving a summary of each model, remove the jargon, and group some classifications to reduce cognitive load. On the other hand, I also gathered qualitative feedback from the users to improve the app. For instance, I found the user really cares about the accuracy of the model. However, the models don't have that kind of data. After discussing with the team, we added confidence score to each classification.

The dashboard iterations

One of the layouts I tried.
Problem: Too many labels and numbers

Picked the one using more visualizations to help summarize the situation.
‍Problem: Still too many labels for reaction

I had a conversation with the team and gave suggestions on how to group things. They agreed. The users like this solution a lot.

More iterations

App Flow

Visualizations
All the visualizations in the app were carefully picked using a lot of research and user consideration. I'd love to talk more about it in person.

AI & UX
I learned a lot about machine learning, deep learning and Natural Language Processing. AI & UX is an emerging topic that requires in depth study. In particular, I am very interested in Explainable AI, transparency, and trust in AI systems.

Product Thinking
I also learned the best practices for framing a problem and use cases with the business in mind.

What I have learned