Programme

Prof. Dr. Giulio Jacucci is Professor at the Department of Computer Science at the University of Helsinki. He has been Professor at the Aalto University, Department of Design 2009-2010. His research over the years focussed on mobile and ubiquitous computing, including public displays, mobile group media, and augmented reality, and exploratory search. Currently the research focuses on task based entity recommendation, behaviour change and wellbeing,  affective interaction in XR.   He co-authored two patents in the area of information seeking and on modular screens and is co-founder of several research spin offs , recently www.formulator.care applying AI to mental health. 

Algorithmic technologies, concepts, and practices as socio-technical constructs emerge and proliferate through language in society. Discourses around Artificial Intelligence (AI) shape our collective imagination, affect technological development, and influence policymaking. What can we learn from critically examining wide-ranging discourses around AI, using a mix of qualitative methods and Natural Language Processing (NLP), both among actors who influence its development and the publics who are affected by it? In this chapter, we examine the "language of algorithms" to make sense of AI Watch reports and stakeholder responses to the proposed AI Act in the European Union. Linguistic devices such as metaphors, metonymy, and personification reveal how we conceptualize, narrate, contest, or attribute agency to AI systems. Saying that AI is “trustworthy”, “biased”, or “transforming society” are discursive acts that implicitly attribute a sense of agency to technology rather than the human actors involved in its creation. Critically examining such AI discourses reveals how language affects attitudes, influences practices and policies, and shapes future imaginaries around AI.

Collecting data is one of the bottlenecks of Human-Computer Interaction (HCI) research. Motivated by this, we explore the potential of large language models (LLMs) in generating synthetic user research data.We use OpenAI’s GPT-3 model to generate open-ended questionnaire responses about experiencing video games as art, a topic not tractable with traditional computational user models. We test whether synthetic responses can be distinguished from real responses, analyze errors of synthetic data, and investigate content similarities between synthetic and real data. We conclude that GPT-3 can, in this context, yield believable accounts of HCI experiences. Given the low cost and high speed of LLM data generation, synthetic data should be useful in ideating and piloting new experiments, although any findings must obviously always be validated with real data. The results also raise concerns: if employed by malicious users of crowdsourcing services, LLMs may make crowdsourcing of self-report data fundamentally unreliable.

The COVID-19 pandemic exemplified the complexity of the field of crisis communication, with multiple channels and streams of information and misinformation causing new challenges for the authorities and general public alike. This complexity requires better addressing the situated and interrelated aspects of sensemaking practices and platforms, and how different disciplines and organisations collaborate during a crisis to turn ambiguity into resilience, and complexity into comprehension. We use design research and participatory design methodology to draw on learnings from the Finnish context and response to COVID-19 and other crises. These insights are then used to create design principles that bridge crisis informatics theory with HCI knowledge to create speculative, diegetic artefacts, which embody new practices and platforms that can be used to encourage collaborative sensemaking to tackle complex, large-scale crises and therefore have a positive impact on the resilience of the society.

While user interfaces (UIs) display elements such as images and text in a grid-based layout, UI types differ significantly in the number of elements and how they are displayed. For example, webpage designs rely heavily on images and text, whereas desktop UIs tend to feature numerous small images. To examine how such differences affect the way users look at UIs, we collected and analyzed a large eye-tracking-based dataset, UEyes (62 participants and 1,980 UI screenshots), covering four major UI types: webpage, desktop UI, mobile UI, and poster. We analyze its differences in biases related to such factors as color, location, and gaze direction. We also compare state-of-the-art predictive models and propose improvements for better capturing typical tendencies across UI types. Both the dataset and the models are publicly available.

Text-to-image generation (TTIG) models, a recent addition to creative AI, can generate images based on a text description. These models have begun to rival the work of professional creatives, and sparked discussions on the future of creative work, loss of jobs, and copyright issues, amongst other important implications. To support the sustainable adoption of TTIG, we must provide rich, reliable and transparent insights into how professionals perceive, adopt and use TTIG. Crucially though, the public debate is shallow, narrow and lacking transparency, while academic work has focused on studying the use of TTIG in a general artist population, but not on the perceptions and attitudes of professionals in a specific industry. In this paper, we contribute a qualitative, exploratory interview study on TTIG in the Finnish videogame industry. Through a Template Analysis on semi-structured interviews with 14 game professionals, we reveal 12 overarching themes, structured into 49 sub-themes on professionals’ perception, adoption and use of TTIG systems in games industry practice. Experiencing (yet another) change of roles and creative processes, our participants’ reflections can inform discussions within the industry, be used by policymakers to inform urgently needed legislation, and support researchers in games, HCI and AI to support the sustainable, professional use of TTIG to benefit people and games as cultural artefacts.

We propose Unified Model of Saliency and Scanpaths (UMSS)-a model that learns to predict multi-duration saliency and scanpaths (i.e. sequences of eye fixations) on information visualisations. Although scanpaths provide rich information about the importance of different visualisation elements during the visual exploration process, prior work has been limited to predicting aggregated attention statistics, such as visual saliency. We present in-depth analyses of gaze behaviour for different information visualisation elements (e.g. Title, Label, Data) on the popular MASSVIS dataset. We show that while, overall, gaze patterns are surprisingly consistent across visualisations and viewers, there are also structural differences in gaze dynamics for different elements. Informed by our analyses, UMSS first predicts multi-duration element-level saliency maps, then probabilistically samples scanpaths from them. Extensive experiments on MASSVIS show that our method consistently outperforms state-of-the-art methods with respect to several, widely used scanpath and saliency evaluation metrics. Our method achieves a relative improvement in sequence score of 11.5% for scanpath prediction, and a relative improvement in Pearson correlation coefficient of up to 23.6 These results are auspicious and point towards richer user models and simulations of visual attention on visualisations without the need for any eye tracking equipment.

I will present my past and ongoing research on more-than-human design. The talk will mainly focus on how information technologies can support the noticing of nature. I will present cases of citizen science and crowdsourcing a library of chemical tests of soil samples. At-home chemical tests are increasingly used to measure the health of people and environments. The procedure often involves sending the test to interpretation by an expert. However, there is a middle ground where the skills of professional biologists and chemists are made accessible through digital services. There is also an opportunity to develop technologies that support interpretation. I have identified key challenges for interdisciplinary bridging biochemists and communities of DIY biochemical testing. These challenges are based on ethnographic and co-design studies of citizens interested in sustainability. These citizens conducted soil chromatography, a chemical test method to assess the composition of soils. The research showed that citizens could conduct the tests reliably, while it was much more challenging to access the resources needed to interpret the tests in a meaningful way.

Algorithmic decision-making and big data systems are increasingly being used to provide innovative and essential services in the public sector. Such public services that utilize AI entail many related risks and responsibilities for citizens and public sector providers. Furthermore, the distinct regulatory demands and responsibilities of public sector services require crucial consideration of inclusiveness and civic empowerment. In this empirical study, we examine practitioners’  attitudes, practices and challenges of implementing inclusive AI services in the public sector that can empower greater civic agency. We conducted in-depth interviews with ten practitioners responsible for managing, developing or designing AI-enabled public services across three big public organizations in Finland in domains relating to the municipality, taxes, and social insurance. The results show that the discussion on inclusion and civic empowerment is just in its beginning in the public sector. Practitioners perceive the concept of inclusion as devising accessible public services for all members of society. Civic empowerment was understood as 1) institutional transparency, 2) civic participation in shaping the services, and 3) easing the use of the services for their users. The research suggests two distinct socio-cultural constructs emerging among practitioners that may influence (or hinder) how civic empowerment is manifested in such services: risk-averse and expert culture. The contributions of the study are twofold. First, we describe the practitioners’ perspectives on empowerment and inclusion in regard to public sector AI. Further, we recognize how expert and risk-averse cultures among practitioners explain their actions and restraints in devising public sector AI services.

There have been significant advances in simulation models predicting human behavior across various interactive tasks. One issue remains, however: identifying the parameter values that best describe an individual user. These parameters often express personal cognitive and physiological characteristics, and inferring their exact values has significant effects on individual-level predictions. Still, the high complexity of simulation models usually causes parameter inference to consume prohibitively large amounts of time, as much as days per user. We investigated amortized inference for its potential to reduce inference time dramatically, to mere tens of milliseconds. Its principle is to pre-train a neural proxy model for probabilistic inference, using synthetic data simulated from a range of parameter combinations. From examining the efficiency and prediction performance of amortized inference in three challenging cases that involve real-world data (menu search, point-and-click, and touchscreen typing), the paper demonstrates that an amortized-inference approach permits analyzing large-scale datasets by means of simulation models. It also addresses emerging opportunities and challenges in applying amortized inference in HCI.

As technological innovation continues to shape our world at an accelerating pace, policy makers struggle to keep up with the unintended consequences of these new technologies. To address this policy-novelty gap, Responsible Research Innovation (RRI) has been proposed to drive science and technology innovation towards socially desirable goals. This work suggests a more active HCI's position in the materialisation of pluralistic future visions and emphasizes the engagement between policy design and HCI for more agile and responsive evaluation environments. It calls for both fields to engage in questioning which and how futures are constructed, who they are benefiting, and how the findings of these interventions are interpreted towards other futures.

Human Augmentation Technologies improve human capabilities using technology. In this study, we investigate the placebo effect of Augmentation Technologies. Thirty naïve participants were told to be augmented with a cognitive augmentation technology or no augmentation system while conducting a Columbia Card Task. In this risk-taking measure, participants flip win and loss cards. The sham augmentation system consisted of a brain-computer interface allegedly coordinated to play non-audible sounds that increase cognitive functions. However, no sounds were played throughout all conditions. We show a placebo effect in human augmentation, where a sustained belief of improvement after using the sham system remains and an increase in risk-taking conditional on heightened expectancy using Bayesian statistical modeling. Furthermore, we identify differences in event-related potentials in the electroencephalogram that occur during the sham condition when flipping loss cards. Finally, we integrate our findings into theories of human augmentation and discuss implications for the future assessment of augmentation technologies.

Choosing colors is a pivotal but challenging component of graphic design. The paper presents an intelligent interaction technique supporting designers’ creativity in color design. It fills a gap in the literature by proposing an integrated technique for color exploration, assignment, and refinement: CoColor. Our design goals were 1) let designers focus on color choice by freeing them from pixel-level editing and 2) support rapid flow between low- and high-level decisions. Our interaction technique utilizes three steps – choice of focus, choice of suitable colors, and the colors’ application to designs – wherein the choices are interlinked and computer- assisted, thus supporting divergent and convergent thinking. It considers color harmony, visual saliency, and elementary accessibility requirements. The technique was incorporated into the popular design tool Figma and evaluated in a study with 16 designers. Participants explored the coloring options more easily with CoColor and considered it helpful

Although menu selection has been extensively studied in HCI, most existing studies have focused on sighted users, leaving blind users’ menu selection under-studied. In this paper, we propose a computational model that can simulate blind users’ menu selection performance and strategies, including the way they use techniques like swiping, gliding, and direct touch. We assume that selection behavior emerges as an adaptation to the user’s memory of item positions based on experience and feedback from the screen reader. A key aspect of our model is a model of long-term memory, predicting how a user recalls and forgets item position based on previous menu selections. We compare simulation results predicted by our model against data obtained in an empirical study with ten blind users. The model correctly simulated the efect of the menu length and menu arrangement on selection time, the action composition, and the menu selection strategy of the users.

Manipulative online behaviors such as trolling are becoming increasingly problematic on social media due to their capacity to obfuscate meanings, to influence democratic decision making and to disseminate harmful information. However, trolling is difficult to identify as it involves deception and indirect forms of manipulation. We discuss how analyzing the dynamics of turn-by-turn computer-mediated interactions allows a deeper understanding of trolling in context. We show how Machine Learning (ML) utilizing a theory-based frame for analyzing interaction can achieve good results in trolling identification for text-based asynchronic online conversations.

The demo allows attendees to interactively experience recent research prototypes aiming to facilitate designers' creative and problem-solving capabilities in user interface (UI) design.Empirical work on designers suggests that UI design is challenging, partially because of the presence of very large design spaces, multiple and ill-defined objectives, design fixation and biases, as well as multiple requirements that need to to kept in mind. At the exhibition, members of the lab provide live demonstrations of six computational features, with a special focus on plug-ins created for Figma, a popular UI design tool. The demos draw from the group's latest research published at HCI conferences. They demonstrate how to interactively exploit machine learning methods ranging from deep nets to Bayesian inference and NLP. We also present our design approach and provide a summary of findings from empirical evaluations with designers. 

Designing for temperature experiences have attracted increasing attention in the field of Human-Computer Interaction (HCI) in the past decade. Thermal feedback is used in various contexts such as virtual reality, meditation and mindfulness practice, for affective computing or guidance tasks. As an output modality, the sensory experience of heat and cold holds significant promise due to its richness, emotional resonance, and ability to grab attention. This interactive demo will showcase a physical/virtual experience of temperatures in VR using a wearable system on the hands.

Ultrasonic sources are inaudible to humans, and while digital signal processing techniques are available to bring ultrasonic signals into the audible range, there are currently no systems which also simultaneously permit the listener to localise the sources through spatial hearing. Therefore, we describe a method whereby an in-situ listener with normal binaural hearing can localise ultrasonic sources in real-time; opening-up new applications, such as the monitoring of certain forms of wild life in their habitats and man-made systems. In this work, an array of ultrasonic microphones is mounted to headphones, and the spatial parameters of the ultrasonic sound-field are extracted. A pitch-shifted signal is then rendered to the headphones with spatial properties dictated by the estimated parameters. The processing provides the listener with the spatial cues that would normally occur if the acoustic wave produced by the source were to arrive at the listener having already been pitch-shifted. The results show that the localisation accuracy delivered by the proof-of-concept device implemented here is almost as good as with audible sources, as tested both in the laboratory and under conditions in the field. 

The Trust System (TS) in VRChat is designed to measure and indicate users’ trustworthiness in order to reduce toxicity in the platform. We gathered and analyzed data from “r/VRChat,” to see how users perceive this tool. We found that users interpret the system differently and the ability of the TS indicating trustworthiness is in question among users. In addition, although it purports to contribute to keeping users away from toxicity, in practice, the trust ranks might add grounds based on which people form stereotypes, discrimination, and further conflicts. Via discussion, we reveal that trust ranks might enable toxic meritocracy. The prevalence of toxicity in social VR is not just caused by people but might also by system designs. Finally, we discuss design implications for prosocial norms in social VR.

Exploring innovative ideas for interactive software has its challenges. A new process called the User-Centred Design (UCD) Sprint process has been suggested to support teams in exploring users’ needs and the future usage of the software developed with active involvement of users. The course that we run at CHI'23 will introduce the UCD Sprint process, and participants will practice two steps from the UCD Sprint: the user group analysis and stating user experience goals. The course targets at researchers and developers  interested in exploring their innovative ideas through a user-centred step-by-step process. 

Explainability is a crucial aspect of models which ensures their reliable use by both engineers and end-users. However, explainability depends on the user and the model's usage context, making it an important dimension for user personalization. In this article, we explore personalization of opaque-box image classifiers using an interactive hyperparameter tuning approach, in which the user iteratively rates the quality of explanations for a selected set of query images. Using a multi-objective Bayesian optimization (MOBO) algorithm, we optimize for both, the classifier's accuracy and the perceived explainability ratings. In our user study, we found Pareto-optimal parameters for each participant, that could significantly improve explainability ratings of queried images while minimally impacting classifier accuracy. Furthermore, this improved explainability with tuned hyperparameters generalized to held-out validation images, with the extent of generalization being dependent on the variance within the queried images, and the similarity between the query and validation images. This MOBO-based method has the potential to be used in general to jointly optimize any machine learning objective along with any human-centric objective. The Pareto front produced after the interactive hyperparameter tuning can be useful during deployment, allowing for desired trade-offs between the objectives (if any) to be chosen by selecting the appropriate parameters. Additionally, user studies like ours can assess if commonly assumed trade-offs, such as accuracy versus explainability, exist in a given context.

The public sector is increasingly embracing algorithmic decision-making and data-centric infrastructures to offer innovative digital services to citizens. As public AI services become more prevalent and affect citizens’ lived experiences, we must critically question their social, political, and ethical implications on rights, risks, and responsibilities for providers and recipients, particularly the most vulnerable in society. This project engages conceptual, empirical, and instrumental research to assess wide ranging and conflicting perspectives and practices among experts, providers and citizens. The European Commission’s proposed Artificial Intelligence Act has raised vigorous deliberations regarding the implications of  implementing this regulatory framework across the EU. The diverse and contested discourses among AI experts, regulators, public actors, and citizen advocates, offer a timely window of opportunity to critically examine their implications, while promoting citizen participation and civic agency in shaping the AI Act and its governance in Finland and the EU. In our research we examine discourses through the theoretical lens of Critical Discourse Analysis to gain insights on diverse values, narratives, and positions, while Natural Language Processing methods are used to contextually examine salient topics. We also conduct in-depth case studies of public AI services in Finland to highlight the key practices and challenges for incorporating AI in the public sector, while ensuring trust, accountability and governance. We explore citizen perspectives, agency, and imaginaries on digital citizenship. An interdisciplinary approach, allows us to critically assess the transformation of discourses, civic agency, and democratization of public AI services and design frameworks for stakeholder participation in AI governance. 

The poster will present ongoing research around “eHealth for home dialysis”. Our research focuses on human-centred design of eHealth solutions for home care by involving both patients’ and clinicians’ perspectives. The study utilizes a range of human-centered design methods and contributes to the design and customization of eHealth solutions for improved user experience. 

Choosing suitable colors is a fundamental requirement in design. While a considerable body of literature exists on normative accounts of color choice, such as those from color theory, little work has examined actual experiences and practices. Addressing this gap, the paper presents findings from interviews with 12 designers. These revealed that designers experience color design as a challenge and not as routine. They struggle to overcome design fixation, in both early stages (e.g., with mood boards) and later choices (with hi-fi prototypes etc.). Interviewees reported that color design must address multiple objectives, some of them involving matters beyond normative accounts' scope, such as the ``message'' conveyed by colors and their effect on end users.  To generate candidate solutions, designers do exploit normative theories, but they also deliberately try ``unorthodox'' solutions, to learn from them.  Practitioners turn to design samples for inspiration and seek feedback from others. The study's findings offer a rich account of color-design practice and suggest that research into computer-assisted color design must exceed color picking. 

The role of cognitive ability in the use of computers has received little attention, even though computer use is pervasive and success in using them affects thriving. Ease-of-use is a prime goal in user interface design, and empirical studies linking task performance to background factors suggest that computer applications require skills and knowledge. However, it remains an open question to what extent they require general, computer-independent cognitive ability. The purpose of our study is to understand whether individual differences in cognitive abilities, such as verbal comprehension, perceptual reasoning, working memory, processing speed, attention control, and executive functioning have any significant effects on our abilities to use computers. We test our hypotheses experimentally in a within-subjects design (N=88), where each participant completes a diverse set of 18 computerised tasks representative of everyday usage. Cognitive abilities are estimated with the Wechsler Adult Intelligence Scale (WAIS-IV), accompanied by computerised tests for measuring executive functioning. Our results indicate that there are large individual differences in task performance rates and cognitive abilities: most of this variation is explained by demographic factors such as age; but we also find that a significant proportion is explained by cognitive functioning. The analysis is on-going, and the results are yet to be published.

"As the future demands more user-agent social interaction, artificial agents are increasingly appearing in VR. Facial cues can communicate the emotion of the actor and can elicit immediate behavioral reactions from the perceiver [4]. With advancements in VR technology, highly detailed emotional expressions can be integrated onto virtual agents to turn them into affective agents. These agents can utilize touch to enhance the emotional experience in VR. We investigated the impact of haptics in interpersonal interaction with embodied affective agents in immersive VR. These agents can show emotional facial expressions and accept touch from or initiate touch with the user. Our objectives were to identify effective haptic technologies for perceiving touch in affective VR and to gain a detailed understanding of touch perception and expression during interaction with embodied affective agents. Three studies are presented to support these objectives. The investigation explored a comprehensive understanding of the effect of cross-modal integration of visual and tactile stimuli during user-agent social interaction in VR. The results indicate that mechanical pressure-based haptic actuators are more effective than vibrotactile actuators for affective interaction in VR. The virtual agents' emotional expressions influenced users' emotions, as well as how they perceived and expressed touch. These findings can be valuable for researchers and practitioners interested in enhancing the emotional adaptability of HCI systems with haptic technology."

eHealth, any digital health services, platforms, applications, or tools delivered electronically, is an efficient and cost-effective digital solution to deliver Health services. eHealth enables people to have unlimited access to healthcare services without considering temporal and geographical boundaries. The DigiIN and NoreHealth projects aim to identify the challenges and opportunities in digitalization of health services and create solutions which will ensure that the social welfare and healthcare sector’s digital services are available and accessible to everyone. These projects are trying to minimize digital divide, prevent marginalization among vulnerable people, achieve a socially sustainable society, where everyone is treated equally and increase eHealth use for everyone.  To meet these goals several qualitative and quantitative research have been done in different countries, Finland, Sweden, Norway and Estonia. This poster will present outcomes of these two projects regarding older adults, who are 65 years old and above in Finland, Aalto University. In addition, future plans and expected outcomes will be presented.

Although eye-tracking has been widely used in information visualisation researcs, the ways in which viewers look at visualisations remains poorly understood. Since eye-tracking is time-consuming and expensive, the eye-tracking datasets are always with small data scale.  Meanwhile, the large-scale crowd-sourcing datasets collected in natural scenes have paved the way to more promising computational attention models.  However, information visualisations are fundamentally different from natural scenes:They usually contain more text, e.g. axis labels or legend, as well as larger areas with a uniform color and little texture.  Therefore, a large-scale visualisation dataset is fundamental to deeply understand human attention behaviours on visualisations. Another reason is that, eye trackers only provide information about users’ past and current focus of attention.   They cannot predict information on which locations a user is likely to visually attend to in the future.   In addition, eye tracking based approaches are limited to post-hoc optimization of information visualisations to users’ attention, i.e. after a particular visualisation has been prototypically implemented.  These limitations do not only pose significant technical barriers that impede progress in using attention analysis in information visualisation research but also force designers to engage in a laborious and inefficient iterative refinement process for information visualisations. The main goals of this project is threefold: 1) to understand human attention behaviours on information visualisations without the need for special purpose eye tracking equipment; 2) to automatically quantify spatio-temporal human visual attention by proposing computational attention models and 3) to integrate the quantification of visual attention directly into the visualisation design process.

Consensus-building is an essential process for the success of codesign projects. To build consensus, stakeholders need to discuss conflicting needs and viewpoints, converge their ideas toward shared interests, and grow their willingness to commit to group decisions. However, managing group discussions is challenging in large co-design projects with multiple stakeholders. In this paper, we investigate the interaction design of a chatbot that can mediate consensus-building conversationally. By interacting with individual stakeholders, the chatbot collects ideas to satisfy conflicting needs and engages stakeholders to consider others’ viewpoints, without having stakeholders directly interact with each other. Results from an empirical study in an educational setting (N = 12) suggest that the approach can increase stakeholders’ commitment to group decisions and maintain the effect even on the group decisions that conflict with personal interests. We conclude that chatbots can facilitate consensus-building in small-to-medium-sized projects, but more work is needed to scale up to larger projects.