Back2Struct: Making Structured Images Editable Again

Abstract

Structured images — diagrams, charts, and flowcharts — are inherently symbolic and can be compactly represented in an editable format, yet in practice they are shared as raster images and are no longer graphically editable. We present Back2Struct, which "makes structured images editable again" by directly recovering vector graphics code (SVG / XML) from image representations. Given an image of a structured graphic, Back2Struct predicts semantically object-level SVG/XML that explicitly encodes text, shapes, topology, and layout — rather than performing low-level pixel vectorization — so the code can be imported straight into tools like PowerPoint to edit, restyle, and reuse content while preserving structural fidelity. Beyond supervised fine-tuning on ground-truth SVG token sequences, we further optimize Back2Struct with reward-based learning using a composite reward that jointly encourages SVG/XML compilability, length consistency with the reference code, and structural/semantic similarity to the input. Experiments show Back2Struct improves accuracy, editability, validity, and user alignment over strong baselines.

Highlights

Three contributions toward editable structured-image generation.

Method

Object-level SVG generation, warm-started by SFT and refined by GRPO.

Figure not generated yet — run bash convert_figs.sh to render framework.png.

StructHub Dataset

84K paired image–code examples across three sources, with a 1,000-sample held-out benchmark.

The benchmark is split by ground-truth SVG length into easy (≤2,048 tok), medium (2,048–4,096), and hard (>4,096) tiers.

Figure not generated yet — run bash convert_figs.sh to render dataset_overview.png.

Figure not generated yet — run bash convert_figs.sh to render data_samples.png.

Results

On StructHub, Back2Struct is the strongest open/trained model — and on renderable outputs it rivals frontier closed models many times its size.

Back2Struct is the strongest open / trained model on every tier and metric — render success 52.3%→71.5% overall — with its largest margins on the Hard tier.

On compiled outputs, Back2Struct achieves the best GPTScore, SSIM, and LPIPS — ahead of GPT-5 and Gemini-2.5-Pro, at a fraction of the size.

Figure not generated yet — run bash convert_figs.sh to render performance.png.

Qualitative Samples

Figure not generated yet — run bash convert_figs.sh to render qualitative.png.

Figure not generated yet — run bash convert_figs.sh to render text2svg.png.

BibTeX

@misc{yan2026back2structmakingstructuredimages,
      title={Back2Struct: Making Structured Images Editable Again},
      author={Pengyu Yan and Yixin Wu and Yunjie Tian and David Doermann},
      year={2026},
      eprint={2609.37016},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2609.37016},
}