OVERVIEW
Challenge
How could we help creative workers to get rid of the tedious routine discussion due to the convoluted process and limited results?
My Role
Team Leader, UX Designer, Programmer
Timeline
March 2019 - June 2018 (4 months)
Solution
We present Innomate, an interactive visualization system that facilitates creative meeting based on semantic analysis and knowledge graph. It aims to structure and visualize the intrinsic logic of the discussion and help team members to summarize it as well as find more inspirations.
Team members
Chaoran CHEN, Yixiang ZHANG, Ziyi Liu, Jingyuan KANG, Shangyi BAI
Mentor
Xiaohua SUN
Our Approach
01 Research
Literature review
Competitive Analysis
Persona
Journey map
02 Design
Ideation
Scenario
System map
Software development
Hardware development
03 Evaluate
User study
Discussion
My Contribution
During the research phase, I moderated several brainstorming and took part in the literature review on human robot collaboration.
During the design phase, I was mainly responsible for the software development. I also took part in refining our system mape and hardware development.
Finally, I moderated several usability testing sessions and analyzed the user feedback. I then took part in the discussion of design implications from our study and our potential future work with my teammate.
Outcome
Research
As designers, we often suffer from low productivity in discussion. Starting with this problem, we searched some related work and conducted some quick interview with our classmates to find the character and design opportunities in this area.
Background Survey
Previous research proposes that people in face-to-face brainstorming meetings are less efficient at generating ideas than when working alone, which is so-called productivity loss.
On the other hand, creative workers, who have strong personality and emphasize the ultimate result, often have to put tremendous effort into discussion and cooperation with each other. Thus they suffer a lot from the inefficient meeting, which is due to less structured ideas as well as lack of valid references and quantitative metrics to evaluate the design process.
Persona
In order to more clearly understand our target user - people who often do creative work collaboratively, we made a brief summary of their character and design consideration.
To ameliorate this situation, we proposed "Innomate" - an intelligent discussion visualization system based on semantic analysis and knowledge map. Our final goal aims to let intelligent systems understand the discussion of innovative context and pass the results of machine analysis to creative workers through visualization.
Design
Senario
The discussion begins.
User wakes up Inno.
When creative workers are concentrating on discussions, Inno will grab the key words and grow a knowledge map.
When the discussion is deadlocked...
Inno will join the discussion and give relevant references to inspire users.
When the discussion is over, Inno will send the output of the discussion to the user's mobile side for review and query.
System Map
To solve such problem, we design Innomate to assist in structuring the logic of the discussion and finishing a lot of repetitive work. As shown in Figure, it consists of the hardware part and the visualization part. For the hardware part, the robot will be waken up by voice interaction. Through vocal recognition, it can extract sound feature and give feedback by both voice and facial expression, which can give relevant information and extra inspirations.
For the visualization part, we use Google's BERT to convert voice into text. For semantic analysis, we use TF-IDF algorithm and word2vec technique to decompose the text into structured text and the core words will be analyzed and labeled in the corpus. These corpora come from authoritative design forums and websites. Based on the domain knowledge in these websites, we can give meaning to core words. These meanings include the tags, class, related words, and types. After the core words are labeled, they can be connected with other core words.
If the mentioned keywords are in the built domain knowledge network, we can directly assign the word attribute label, level, relevance, and link them with the nodes already displayed in the knowledge graph to update the visualization. If the word is not in the domain knowledge network, we will calculate the distance between the word and nodes in the existing graph based on the corpus, and gather around the existing nodes as floating nodes.
In the visualization scheme, we use a force directed graph to show the knowledge graph. We use the size of the node as the level of the core word. The larger the radius, the higher the level of the word and the wider the semantic range. If the relationship between two things is relatively close or similar according to domain knowledge, the force connecting the two circles will be larger, and their distance will be closer. In addition, we use the gradient of color to encode the frequency of the reference in the discussion. If a word is mentioned frequently, the circle representing the word will appear darker. Finally, we added a method to voice interaction. Depending on the type of these words, the user can use the voice command to highlight all the circles representing the same type.
Hardware Making
We use rhino to make the model and 3D printed it. Here are the making processing. The code was written in Arduino.
Evaluate
We conducted a preliminary user study to evaluate our initial design. We recruited three students(age: 20-22) from College of Design and Innovation in Tongji University. The participants are asked to discuss some topics related with design field. One researcher observed the whole process.
When the discussion began, the participants woke up Innomate by calling the name 'Inno'. When they were concentrating on discussions, Innomate grabbed the keywords and generated a dynamic knowledge graph. When the discussion were deadlocked, it joined in the discussion and gave relevant references to inspire participants. When the discussion was over, it sent the output of the discussion to the participants' mobile phone for further review or query. We found that the discussion process went smoother than using the traditional design method.
Discussion
We present initial design of Innomate, an interactive visualization system. We observe that our initial design help creative workers to gain a high-efficient cooperation and enjoy a better emotional experience instead of facing free rider problems or solely expressing own opinions. Designers are able to express their ideas in depth and no need worrying about omitting a fleeting inspirations during discussion. While it still have some limitations in the accuracy of semantic analysis and the feedback of facial expression of the robot. Future work includes improving the accuracy of the semantic analysis and refine the facial expression system. We will extend it by adding facial recognition and real-time speech translation as well.