We developed a research that focuses on human-AI interactions, employing a crowd-based methodology to collect and assess the reactions and perceptions of a human audience to a dialogue between a human and an artificial intelligent agent. The study is conducted through a live streaming platform where human streamers broadcast interviews to a custom-made GPT voice interface. The questions extracted from the dialogues were categorized based on emotional and cognitive criteria.
Our method covers thematic, emotional, and sentiment analyses of the comments platform users shared during the interview. This work aims to contribute to Human-Computer Interaction (HCI) and Human-Centered AI, emphasizing the need for a paradigm shift in AI research from focusing on technological development to considering its impact on human beings.
This is the overview of the work:
We analyzed the audience’s reaction using a crowdsourcing method to gather data from user comments posted on the platform in real time as responses to the ongoing interviews. The interviews focused on thoughtfully crafted questions derived from the fundamental notion of trust. Our study, therefore, aims to better understand human factors in human-AI interactions by directly observing users in an interactive digital setting.
Our research evaluates the user’s interaction with responses to the generative AI model OpenAI GPT3.5. Volunteer streamers from a popular streaming platform interviewed the AI model and involved in real time the audience from their community in the interaction. The communication between the streamers and the AI was oral (without any visual clues or avatar representing the AI), using a web application delivering Speech-To-Text and Text-To-Speech services between streamers and the API of the model. At the same time, the audience comments were typed into the chat UI. Our system recorded interview utterances and live-users comments.
The details are published in the paper:
Giudici, M., Liguori, F., Tocchetti, A., Brambilla, M. (2024). Unveiling Human-AI Interaction and Subjective Perceptions About Artificial Intelligent Agents. In: International Conference on Web Engineering. ICWE 2024. Lecture Notes in Computer Science, vol 14629. Springer. https://doi.org/10.1007/978-3-031-62362-2_36
