DIAG

In Designing Intelligence Augmentation Group (DIAG), we do research on the intersection of Human-Computer Interaction and Artificial Intelligence. Our focus lies in the study and development of hybrid intelligence systems, which integrate the power of both human and machine intelligence. Our objective is to address complex computational and social problems through the design of novel and effective human-AI interactions for these systems.

DIAG is currently a hybrid lab, with members working remotely or in person from Incheon and Hyunpoong, fostering cross-location collaboration.

JOIN US

We are looking for motivated and well-qualified students who are passionate about designing and implementing human-AI hybrid intelligent systems to address real world problems.

If you are interested in joining our group and doing research in the areas of Human-Computer Interaction, Human-AI Interaction, Crowdsourcing, Human Computation, or any other related domains, please send your CV to jeansong@yonsei.ac.kr.

People

Professor

Jean Y. Song
Assistant Professor

Graduate Students

Chanwoo Park
Integrated
Ph.D. student

Dokyun Lee
Integrated
Ph.D. student

Yeonsun Yang
Integrated
Ph.D. student

SangEun Seo
M.S. Student

Taehun Kim
M.S. Student

Haeun Jang
M.S. Student

Undergraduate Students

Ahyeon Shin
Undergraduate
Research Intern

Alumni

Mincheol Kang, Undergrad Intern (DGIST), Spring 2024 - Summer 2024
Jiheon Kang, Undergrad Intern (DGIST), Spring 2024 - Summer 2024
Yibin Moon, Undergrad Intern (DGIST), Spring 2024
Nayoung Kim, Undergrad Intern (DGIST), Winter 2023 - Spring 2024
Jaeyoung Choi, Undergrad Intern (DGIST), Winter 2023 - Spring 2024
Suyeon Shin, Undergrad Intern (DGIST), Summer 2023 - Spring 2024
Sungmin Ju, Undergrad Intern (DGIST), Summer 2023 - Spring 2024
Jeongeon Park, Researcher, Fall 2023 - Spring 2024
Sunoo Kim, Undergrad Intern (DGIST), Winter 2024 - Spring 2024
Youngjin Park, Undergrad Intern (DGIST), Winter 2024
Dain Kim, Undergrad Intern (DGIST), Winter 2023 - Fall 2023
Sihwan Seok, Undergrad Intern (DGIST), Winter 2023 - Fall 2023
Subeen Jo, Undergad Intern (DGIST), Winter 2023 - Fall 2023
Minseong Kim, Undergrad Intern (DGIST), Spring 2023 - Summer 2023
Huidam Woo, Undergrad Intern (DGIST), Spring 2023
Chan Lee, Undergrad Intern (DGIST), Spring 2023
Ahyeon Shin, Undergrad Intern (DGIST), Winter 2023
Sangeun Seo, Undergrad Intern (UNIST), Summer 2022, Winter 2023
Chanyu Moon, Undergrad Intern (DGIST), Winter 2023
Gunuk Nam, Undergrad Intern (Handong Global University), Winter 2023
Nayoung Kim, Undergrad Intern (DGIST), Winter 2022 - Winter 2023
Hojin Jin, Undergrad Intern (DGIST), Winter 2022 - Fall 2022
Hyeonho Kwon, Undergrad Intern (DGIST), Winter 2022 - Fall 2022
Giwon Lee, Undergrad Intern (DGIST), Winter 2022 - Fall 2022
Giwa Osaruiyobo Henrietta, Graduate Intern (DGIST), Summer 2022

Publications

Conference and Journal Papers

Sera Lee*, Dae R. Jeong*, Junyoung Choi, Jaeheon Kwak, Seoyun Son, Jean Y. Song, and Insik Shin. "Serenus: Alleviating Low-Battery Anxiety Through Real-time Accurate and User-Friendly Energy Consumption Prediction of Mobile Applications." In Proceedings of The ACM Symposium on User Interface Software and Technology (UIST 2024). (* Equal contribution)
Dokyun Lee, Sangeun Seo, Chanwoo Park, Sunjun Kim, Buru Chang, and Jean Y. Song. "Exploring Intervention Techniques to Alleviate Negative Emotions during Video Content Moderation Tasks as a Worker-centered Task Design." In Proceedings of the ACM Conference on Designing Interactive Systems (DIS 2024).
Yeonsun Yang, Ahyeon Shin, Nayoung Kim, Huidam Woo, John Joon Young Chung, and Jean Y. Song. "Find the Bot!: Gamifying Facial Emotion Recognition for Both Human Training and Machine Learning Data Collection." In Proceedings of the ACM/SIGCHI Conference on Human Factors in Computing Systems (CHI 2024).
Sunjae Lee*, Minwoo Jeong*, Daye Song, Junyoung Choi, Seoyun Son, Jean Y. Song†, Insik Shin†. "FLUID-IoT : Flexible and Granular Access Control in Shared IoT Environments via-UI-Level Control Distribution." In Proceedings of the ACM/SIGCHI Conference on Human Factors in Computing Systems (CHI 2024). (* Equal contribution, †Co-corresponding authors)
Dongyoon Han*, Junsuk Choe*, Seonghyeok Chun, John Chung, Minsuk Chang, Sangdoo Yun, Jean Y. Song, Seong Joon Oh. "Neglected Free Lunch - Learning Image Classifiers Using Annotation Byproducts." In Proceedings of the International Conference on Computer Vision (ICCV 2023). (* Equal contribution)
Jean Y. Song*, Sangwook Lee*, Jisoo Lee, Mina Kim, and Juho Kim. "ModSandbox: Facilitating Online Community Moderation Through Error Prediction and Improvement of Automated Rules." In Proceedings of the ACM/SIGCHI Conference on Human Factors in Computing Systems (CHI 2023). (* Equal contribution)
Seoyun Son, Junyoung Choi, Sunjae Lee, Jean Y. Song, and Insik Shin. "It is Okay to be Distracted: How Real-time Transcriptions Facilitate Online Meeting with Distraction." In Proceedings of the ACM/SIGCHI Conference on Human Factors in Computing Systems (CHI 2023).
Sunjae Lee, Hoyoung Kim, Sijung Kim, Sangwook Lee, Hyosu Kim, Jean Y. Song, Steven Y. Ko, Sangeun Oh, and Insik Shin. "A-Mash: Providing Single-app Illusion for Multi-app Use through User-centric UI Mashup." In Proceedings of the International Conference On Mobile Computing And Networking (MobiCom 2022).
Yoonjoo Lee, John Joon Young Chung, Taesoo Kim, Jean Y. Song, and Juho Kim. "Promptiverse: Scalable Generation of Scaffolding Prompts through Human-AI Knowledge Graph Annotation." In Proceedings of the ACM/SIGCHI Conference on Human Factors in Computing Systems (CHI 2022).
Sunjae Lee, Hayeon Lee, Hoyoung Kim, Sangmin Lee, Jeong Woon Choi, Yuseung Lee, Seono Lee, Ahyeon Kim, Jean Y. Song, Sangeun Oh, Steven Y. Ko, and Insik Shin. "FLUID-XP: Flexible User Interface Distribution forCross-Platform Experience." In Proceedings of the International Conference On Mobile Computing And Networking (MobiCom 2021).
Zhefan Ye, Jean Y. Song, Zhiqiang Sui, Stephen Hart, Jorge Vilchis, Arbor, Walter S. Lasecki, and Odest C. Jenkins. "Human-in-the-loop Pose Estimation via Shared Autonomy." In Proceedings of the ACM International Conference on Intelligent User Interfaces (IUI 2021). Best Paper Honorable Mention
Stephan J. Lemmer, Jean Y. Song, and Jason J. Corso. "Crowdsourcing More Effective Initializations for Single-target Trackers Through Automatic Re-querying." In Proceedings of the ACM/SIGCHI Conference on Human Factors in Computing Systems (CHI 2021).
Yoonjoo Lee, John Joon Young Chung, Jean Y. Song, Minsuk Chang, and Juho Kim. "Personalizing Ambience and Illusionary Presence: How People Use "Study with Me" Videos to Create Effective Studying Environments." In Proceedings of the ACM/SIGCHI Conference on Human Factors in Computing Systems (CHI 2021).
Jean Y. Song, John Joon Young Chung, David F. Fouhey, and Walter S. Lasecki. "C-Reference: Improving 2D to 3D Object Pose Estimation Accuracy via Crowdsourced Joint Object Estimation." In Proceedings of the ACM International Conference on Computer Supported Cooperative Work and Social Computing (CSCW 2020).
Divya Ramesh, Anthony Z. Liu, Andres J. Echeverria, Jean Y. Song, Nicholas R. Waytowich, and Walter S. Lasecki. "Yesterday’s Reward is Today’s Punishment: Contrast Effects in Human Feedback to Reinforcement Learning Agents." In Proceedings of the International Conference on Autonomous Agents and Multiagent Systems (AAMAS 2020). Pragnesh Jay Modi Best Student Paper
Yan Chen, Maulishree Pandey, Jean Y. Song, Walter S. Lasecki, and Steve Oney. "Improving Crowd-Supported GUI Testing with Structural Guidance." In Proceedings of the ACM/SIGCHI Conference on Human Factors in Computing Systems (CHI 2020).
John Joon Young Chung, Jean Y. Song, Sindhu Kutty, Sungsoo (Ray) Hong, Juho Kim, and Walter S. Lasecki. "Efficient Elicitation Approaches to Estimate Collective Crowd Answers." In Proceedings of the ACM International Conference on Computer Supported Cooperative Work and Social Computing (CSCW 2019). Austin, TX. Best Paper Honorable Mention
Jean Y. Song, Raymond Fok, Juho Kim, and Walter S. Lasecki. "FourEyes: Leveraging Tool Diversity as a Means to Improve Aggregate Accuracy in Crowdsourcing." In ACM Transactions on Interactive Intelligent Systems, Volume 19, Issue 1, Article No. 3 (TiiS 2019).
Jean Y. Song, Stephan J. Lemmer, Michael Xieyang Liu, Shiyan Yan, Juho Kim, Jason J. Corso, and Walter S. Lasecki. "Popup: Reconstructing 3D Video Using Particle Filtering to Aggregate Crowd Responses." In Proceedings of the ACM International Conference on Intelligent User Interfaces (IUI 2019). Los Angeles, CA.
Jean Y. Song, Raymond Fok, Alan Lundgard, Fan Yang, Juho Kim, and Walter S. Lasecki. "Two Tools are Better Than One: Tool Diversity as a Means of Improving Aggregate Crowd Performance." In Proceedings of the ACM International Conference on Intelligent User Interfaces (IUI 2018). Tokyo, Japan. Best Student Paper Honorable Mention

Posters, Demos, and Workshop Papers

Jeongeon Park, Bryan Min, Jean Y. Song, Xiaojuan Ma, and Juho Kim. "How do multiple LLM-powered conversational agents assist sensemaking and decision-making in an unfamiliar domain?" In Proceedings of the ACM/SIGCHI Conference on Human Factors in Computing Systems (CHI 2024).
Hyungyu Shin, Nabila Sindi, Yoonjoo Lee, Jaeryoung Ka, Jean Y. Song, and Juho Kim. "XDesign: Integrating Interface Design into Explainable AI Education." In Proceedings of the ACM Technical Symposium on Computer Science Education (SIGCSE TS 2022).
Andrew M. Vernier, Jean Y. Song, Edward Sun, Allison Kench, and Walter S. Lasecki. "Towards Universal Evaluation of Image Annotation Interfaces." In Proceedings of the ACM Symposium on User Interface Software and Technology (UIST 2019). New Orleans, LA.
Jean Y. Song, Raymond Fok, Fan Yang, Kyle Wang, Alan Lundgard, and Walter S. Lasecki. "Tool Diversity as a Means of Improving Aggregate Crowd Performance on Image Segmentation Tasks." In HCOMP Workshop on Human Computation for Image and Video Analysis (HCOMP GroupSight 2017). Quebec City, Quebec. 2017.
Sai R. Gouravajhala, Jean Y. Song, Jinyeong Yim, Raymond Fok, Yanda Huang, Fan Yang, Kyle Wang, Yilei An, and Walter S. Lasecki. "Towards Hybrid Intelligence for Robotics." In Collective Intelligence Conference (CI 2017). New York, NY.

Projects

Our research projects include but are not limited to the following list. We are open to discuss new project ideas in the topic of Human-Computer Interaction, Human-AI Interaction, Crowdsourcing, Human Computation, or any other related domains. Please feel free to contact us if you have exciting ideas.

Facilitating Online Community Moderation using AI Techniques

Communities on social platforms have a group of users who volunteer to moderate their communities, called online moderators. They respond to the behavior of community members that violate rules and work to improve overall interaction experiences between community members. This project aims to build human-AI interaction tools to support moderators to more easily and transparently moderate their online communities.

Improving Facial Emotion Recognition for both Human and AI

This project focuses on enhancing Facial Emotion Recognition(FER) for individuals who struggle with recognizing facial expressions, as well as for current models with low FER performance. To address these challenges, we developed a gamified application called "Find the Bot!" that re-labels FER datasets and trains individuals with difficulties in recognizing facial emotions simultaneously. Our vision is to improve FER abilities for individuals with weak FER, leading to enhanced relationship quality, social acceptance, and increased social productivity, while also contributing to the development of reliable multi-labeling for facial expression datasts.

Reducing Negative Emotions for Crowd Workers Exposed to Disturbing Content

Often times crowd workers who do content moderation or data annotations are exposed to harmful content in their daily routines. This project explores UI intervention techniques that can prevent crowd workers from getting too much negative emotional impacts when conducting these tasks. Through this study, we aim to find ways to maintain the quality and cost of the crowdsourced work while protecting the emotions and mental health of crowd workers.

Continual Learning from a Data Science Perspective

Implementing a model created in a controlled environment into the real world requires solving the problem of Catastrophic Forgetting. To solve that problem, we seek to draw inspiration from the Human Complementary Learning Systems (CLS) from a data science perspective. Through this research, we hope that the model created in a controlled environment can continuously learn in the real world, and through this, the controlled model can be applied to daily life.

Augmenting QnA Dataset for Technical Documents with Crowdsourcing

This project proposes a new sentence structuring method and crowdsourcing interface to augment rich Korean dataset in text documents specialized in technology and science. Our novel authoring tool enables to efficiently generating various question and answer pairs for technical documentation by breaking down sentences into elements that can be recombined to form new sentences.

Contact

We welcome collaborations!
If you have any questions or would like to hear more about our research, please feel free to shoot us an email: jeansong@yonsei.ac.kr