Overview of Albert Gatt's Working Group Presentation at INESC-ID, Lisbon
Albert Gatt from the University of Malta presented the thematic focus, activities, and future plans of Working Group 1 on Grounded Multi-Modal Reasoning and Generation at a meeting in Lisbon. The presentation outlined initiatives and collaborations, emphasizing discussions on multimodal models, symbol representations, commonsense knowledge, and more.
Uploaded on Sep 19, 2024 | 0 Views
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. Download presentation by click this link. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
E N D
Presentation Transcript
Albert Gatt University of Malta 3rd Management Committee and Working Group Meetings Working Group Presentations Working Group 1 INESC-ID, Lisbon - December, 11th 2020
WG1 - Overview Grounded multi-modal reasoning and generation Chair: Co-Chair: Mehul Bhatt, rebro University, Sweden Albert Gatt, University of Malta Currently, around 21 members 2
Outline 1. Thematic focus of the WG 1. Activities to date 1. Future plans 3
WG Thematic Focus Discussed during the meeting in Copenhagen (January 2020) 1. Explanation and transparency of multimodal models 2. Complementarity vs Redundancy 3. Relationship between Symbolic and Subsymbolic representations 4. The role of Commonsense and World Knowledge in multimodal models 5. Data sources 6. Situatedness 4
Initiatives by WG1 members Round-table discussion on Multimodal Rhetoric and Visuo-auditory media. 5
Initiatives by WG1 members Short-term scientific mission: Prof Francois Portet (U. Grenoble) visited UM in October 2020. Outcomes: Intensive discussions with the UM NLP group on grounding beyond the visual modalities, leading to a review document/technical report (in progress). Design of a M.Sc. thesis project on grounding and generation from video data. Now underway, jointly supervised by UM and Grenoble. 6
Initiatives by WG1 members Ongoing collaboration between UM, Uni. Heidelberg and U. v. Amsterdam. Focus: visual grounding of large-scale pretrained multimodal transformers. 7
Where do we go from here? At the Copehagen meeting, two initiatives in particular were highlighted: 1. Review of the state of the art in multimodal grounding and reasoning. 2. Shared task The above need to be discussed at the Lisbon WG meeting. The aim is to formulate a concrete program for a full WG meeting in early 2021. 8
