Augmented Human-Computer Interaction – AHCI

Research Activity

The research activities of the Augmented HCI research group will focus on the design and testing of intelligent virtual and augmented reality environments and advanced human-machine interfaces, tailored for specific tasks and users. In doing so, the group aims to take advantage of Artificial Intelligence (AI) techniques to improve the user experience and to optimize the AI ​​itself. In fact, AI can help in adapting the interfaces to individual users/scenarios and can enable the design of bio-feedback powered virtual environments. However, the AI itself can be improved by the human component, whose experience and intuition can direct the learning phase, which in turn can be drastically reduced by using VR environments as a training ground.
In more details, the research activities envisaged are:

  • Precision human-machine interaction: use of AI techniques to identify the optimal tailoring of the interface to a specific user based on her/his personal preferences, emotional state, current situation, tasks to be performed;
  • Virtual/Augmented reality with AI-powered sensing & actuation: use of AI techniques to improve the user’s perception within the virtual environment and to optimize the visual and sound feedback that the environment returns; a possible application is bio-feedback, in which a bodily function is monitored, processed and then reproduced in the virtual environment to encourage the user to implement control strategies;
  • Human-AI collaboration, with “human-in-the-loop” approaches in the learning phase: explore a model in which the human component contributes to the resolution of computationally complex problems using its experience and intuition; through a heuristic selection of samples, which requires a dedicated interface to enable this intervention, the human component can guide the AI ​​reducing the complexity of the learning phase;
  • Virtual reality as a training ground for AI algorithms: VR can be used as simulation environment for the training of automata – robots, drones and diagnostic tools – before they are tested in the real world. The same approach is applicable to AI. To goal is to reduce the learning time of AI thanks to the use of highly realistic virtual simulations “faster than real-time”, which can be repeatedly instantiated to parallelize the training.

 

Goals

The main goal is to introduce artificial intelligence techniques into interaction design, pursuing the definition of an “augmented” human-machine interaction paradigm. The combination of interaction models and techniques and ad hoc artificial intelligence algorithms has to: i) enable mechanisms multi-modal natural interaction (gestural, vocal, bio-feedback, etc.) in augmented, virtual and mixed reality environments; ii) allow the learning of user interaction patterns and therefore the design of highly personalized interaction solutions based on user preferences or any cognitive and/or motor disabilities.

 

Application Fields

Cultural heritage, for enabling personalized access in augmented/mixed reality environments;
Health, for enabling assessment and cognitive training as well as bio-feedback powered interfaces in virtual reality environments;
Smart factory, for providing augmented assistance to the production activities.

Back To Top