Search Results for author:
Found 4 papers, 3 papers with code
ScreenAI: A Vision-Language Model for UI and Infographics Understanding
At the heart of this mixture is a novel screen annotation task in which the model has to identify the type and location of UI elements.
Ranked #3 on
Visual Question Answering (VQA)
on InfographicVQA
(using extra training data)
Towards Better Semantic Understanding of Mobile Interfaces
Improving the accessibility and automation capabilities of mobile devices can have a significant positive impact on the daily lives of countless users.
ScreenQA: Large-Scale Question-Answer Pairs over Mobile App Screenshots
We present a new task and dataset, ScreenQA, for screen content understanding via question answering.
PhotoChat: A Human-Human Dialogue Dataset with Photo Sharing Behavior for Joint Image-Text Modeling
Based on this dataset, we propose two tasks to facilitate research on image-text modeling: a photo-sharing intent prediction task that predicts whether one intends to share a photo in the next conversation turn, and a photo retrieval task that retrieves the most relevant photo according to the dialogue context.
Ranked #5 on
Image Retrieval
on PhotoChat
