Imagine your phone understanding your screen, not just displaying it. That's the promise of on-device AI, and it's getting closer thanks to a massive new dataset called MobileViews. Current screen assistants, like those that summarize webpages or help users with disabilities, often rely on cloud-based AI, raising privacy concerns. To address this, the trend is shifting towards smaller, on-device AI models, but these models need vast amounts of data to truly understand the complexities of mobile interfaces. That's where MobileViews comes in. Researchers have created the largest public dataset of mobile GUIs, containing over 600,000 screenshot and view hierarchy pairs from more than 20,000 Android apps. This dwarfs previous datasets like Rico, collected back in 2017. Gathering this data presented some serious challenges. The team developed a clever system using an LLM-enhanced crawler to automatically navigate through apps, mimicking real-world usage. They even used SoC clusters, essentially mini-data centers, to run hundreds of Android instances simultaneously, speeding up the collection process. MobileViews’ size and diversity are key advantages. It captures a far broader range of modern app interfaces than older datasets, making it much more relevant to today's mobile landscape. The researchers tested MobileViews by training state-of-the-art multimodal LLMs (the same technology behind things like GPT-4) and found significant improvements in several key areas, such as predicting which UI elements are tappable and understanding the relationships between different elements on the screen. The research also reveals an ongoing challenge: the quality of labels describing UI elements. Many components lack labels or have poorly written ones, making it difficult for AI to understand their function. While improvements have been made over the years, there’s still a long way to go. This research not only provides a valuable resource for developers building better on-device AI experiences but also highlights the need for developers to improve how they label their apps, making them more accessible to both AI and humans, especially those with disabilities. MobileViews is a significant step towards a future where our phones can truly understand what they’re showing us, opening up exciting possibilities for more intuitive and helpful mobile experiences.