Human Centered AI
Research in Human-centered AI
The dominant paradigm for interacting with computers now involves new media and multimodal input on mobile devices—such as speech, images, gestures, gaze, writing, multi-touch, bio-signals and a multitude of sensors. These new interfaces provide better support for human performance than keyboards of the past, and they are proliferating rapidly on everything from smart watches to automobiles to robots.
Our group is developing new “deeply human-centered” systems at the boundary of HCI and AI that can identify human emotional, cognitive, and health status, and then develop more personalised and adaptive interfaces based on this information for health, education, and other areas.
Group-level affect prediction
Lead investigators: Dr Abhinav Dhall, Professor Jianfei Cai and Dr Garima Sharma
Most of the research in affective computing is focused on a single subject in a sample but available online data from social events contains samples with multiple subjects. This project investigates methods of modelling a group of subjects for predicting the perceived affect. Exploring multiple modalities for a fusion-based system can benefit with the presence of context. Aspects of data curation, labelling and automatic methods are being worked upon currently. The resulting VGAF dataset is one of the first audio-visual group-level affect labelled datasets. Experiments on the dataset show the importance of background-based context on bottom-up affect prediction.
Multimodal deepfakes analysis
Investigators: Dr Abhinav Dhall and Dr Kalin Stefanov
The progress in deep learning techniques has led to an unwanted side effect of video manipulation for fake video generation. Popularly known as deepfakes, these videos can cause a lot of damage due to easy propagation of fake news through them. The goal of this project is to create deepfake detection techniques, which can work for newer manipulation techniques. We are developing multimodal techniques based on human implicit signals and audio-visual content analysis to detect and localise deepfake videos.
Video demo
Monitoring health and wellbeing of seniors using unintrusive sensors
Investigators: Professor Ingrid Zukerman, Dr Mor Vered, Dr Mahsa Salehi and Yueyi Ge
Caring for the elderly is a growing challenge for Australian society. Collaborating with a partner in the aged care sector, this project addresses concerns regarding the wellbeing of seniors living at home by modelling their daily routines to detect significant changes due to functional decline or illness. If successful, the resulting models will be integrated into a personalised in-home monitoring system that also detects instantaneous adverse events, such as falls, and decides how to communicate with seniors and carers to deliver information and alerts. The effectiveness of these technologies will be evaluated on a diverse population of seniors and carers.
Eye Gaze Prediction and Labelling
Investigators: Dr Abhinav Dhall, Dr Munawar Hayat and Shreya Ghosh
Eye gaze analysis plays an important role in the field of computer vision and Human-Computer Interaction (HCI). Even with significant progress in the last few years, automatic gaze analysis still remains challenging due to the individuality of eyes, eye-head interplay, occlusion, image quality, and illumination conditions. There are several open questions including what are the important cues to interpret gaze direction in an unconstrained environment without prior knowledge and how to encode them in real-time. The main aim of this project is to develop automatic methods that advance the algorithm for gaze estimation with limited ground truth annotation. To achieve this, deep learning-based pipelines are explored for various automatic labelling methods from the perspective of real-world applications. Earlier studies in this domain have been restricted to controlled environments due to several challenges involved in moving from the controlled environment to real-world scenarios. Moreover, the project of gaze estimation requires a lot of labelled data. The manual labelling of these data is a huge burden as well as an error-prone process. This project explores a generic pipeline that considers all of the above-mentioned limitations. It is a step forward for designing a real-world gaze analysis tool that can propagate to other domains including AR, VR, and HCI.