VizPilot: Automated Onboarding for SVG-based Composite Visualizations using Multimodal LLMs

Abstract

Composite visualizations integrate multiple visualizations in a meaningful manner and show strong expressiveness in representing complex datasets. They have been widely proposed by visualization developers in different applications. However, they often impose a high initial cognitive load on novice users, due to their intrinsic composite designs. Existing visualization onboarding approaches have attempted to mitigate this issue, but they are typically platform-dependent, require substantial manual authoring effort, and struggle with the structural complexity of composite visualizations, making them unable to be used for general composite visualizations. In this paper, we present VizPilot, a novel automated visualization onboarding approach that reverse-engineers composite visualization structure to enable an interactive onboarding experience directly from raw visualization artifacts. The proposed approach consists of two primary modules: the Composite Visualization Analyzer and the Onboarding Interface. Leveraging Multimodal Large Language Models (MLLMs), the Analyzer employs a two-stage pipeline: the Semantic Inference stage decomposes the visualization into individual visual components and extracts structured knowledge using a five-category taxonomy to generate atomic explanation units, and the Semantic Mapping stage anchors these explanations to precise SVG elements via hierarchical DOM traversal, enabling reliable highlighting and interactions. We implement VizPilot as a browser extension. To initiate the workflow, a visualization developer provides a brief description and optional interaction source code, and VizPilot then automatically extracts the SVG and generates a bitmap image to drive the multimodal reasoning. The interactive Onboarding Interface then presents the structured outputs from the Composite Visualization Analyzer, supporting both Narrative Scrollytelling, providing guided, step-by-step explanations with synchronized highlighting, and Free Exploration for on-demand inquiry by users. We extensively evaluate VizPilot through a comparative analysis of different input modalities to assess pipeline robustness, a usage scenario to illustrate reduced authoring effort, and a user study to confirm its effect in reducing users' cognitive load. The overall results demonstrate the effectiveness and usability of VizPilot in facilitating easy onboarding of composite visualizations.