Accès ouvert · CC BY

Retrieval Free Context Augmentation for Image Captioning: Learning from Self-Generated Beam Hypotheses

Article scientifique 2026 Sciences mathématiques et statistiques Anglais

Résumé

Abstract Vision-language models (VLMs) generate image captions by grounding visual perception in semantically coherent natural language, a process whose quality is largely determined by the supervision signal used during training rather than by model capacity alone. Existing Retrieval-Augmented Generation (RAG), approaches rely exclusively on fixed human-annotated captions, leaving unex-ploited the rich, diverse hypotheses that the model itself generates during inference. To address this limitation, we introduce CAG-Cap, a retrieval-free caption-refinement framework inspired by the offline caching principle of 1 Cache-Augmented Generation (CAG). Unlike conventional CAG, where cached knowledge is injected into the inference context, CAG-Cap uses cached information exclusively as auxiliary sequence-level supervision during training, with no additional inference-time parameters or retrieval operations. CAG-Cap operates in two stages. In Stage A, a baseline captioner comprising a frozen SigLIP 2-B/16 vision encoder and a four-layer Transformer decoder is trained under next-token prediction to convergence, a caption cache is then constructed by running beam search over the training set and retaining the top-k hypotheses per image as self-generated pseudo-labels. In Stage B, the model is fine-tuned under a dual-loss objective that combines ground-truth negative log-likelihood (NLL) supervision with a CAG loss, defined as the mean NLL over all k cached hypotheses, weighted by a tunable coefficient λ CAG. Experiments on MS COCO 2017 and Flickr30k demonstrate the effectiveness of CAG-Cap. The optimal configuration on MS COCO 2017 achieves a CIDEr of 2.2106, a BLEU-4 of 0.3902, and a SPICE of 0.4115, on Flickr30k, it attains a BLEU-4 of 0.3218 and a METEOR of 0.4990. CAG-Cap establishes a principled, retrieval-free alternative to RAG for training-time caption supervision, offering a scalable pathway toward self-improving VLMs. The source code is accessible: https://github.com/abimouloud/CAG-Cap

Citer ce document

Hani, A., Tagougui, N., ABIMOULOUD, M. L., Taleb‐Ahmed, A., & Kherallah, M. (2026). Retrieval Free Context Augmentation for Image Captioning: Learning from Self-Generated Beam Hypotheses. Research Square. https://doi.org/10.21203/rs.3.rs-11184261/v1

Exporter : BibTeX · RIS (Zotero, Mendeley, EndNote)

Accès au document

Texte intégral archivé sur SPHAERO, en accès ouvert : lisible gratuitement en ligne après connexion. Le document reste aussi disponible à sa source. Se connecter

Voir l'article sur le site de la revue

Licence et provenance

Licence : CC BY

Notice moissonnée depuis OpenAlex le 16/11/2026. Le document reste hébergé par sa source.
Voir le document à la source →

Statistiques

Consultations (hors robots) : 1

Téléchargements (hors robots) : 0

Méthode de comptage