Prototipo de descripción de figuras geométricas para no videntes : descripción de figuras geométricas (tikz svg) usando un LLM.

Abstract

This document presents the Curricular Integration Project that aims to develop a service to generate detailed and contextualized descriptions of figures in SVG or TikZ format, present in academic and scientific documents in LaTeX format, using LLM models for automatic generation. The goal is to facilitate the understanding, recreation, and interpretation of these visual elements for everyone, including people with a certain degree of visual impairment, thus contributing to inclusive learning. The first chapter presents the motivation and research of the problem that is intended to be solved through the use of technologies and software solutions. It introduces the general objective of creating a service that uses Large Language Models (LLMs), particularly the Meta AI Llama 2 model, along with techniques like Retrieval-Augmented Generation (RAG) to generate contextualized descriptions of figures in formats like SVG or TikZ. Chapter two begins with a study of the State of the Art in the use of LLMs for figure description, presenting different projects and research that serve as the foundation for the development of this Curricular Integration Project. It also defines all the hardware and software considerations required to implement the service and describes the system architecture and development phases, including the design, coding, and implementation of the description generator service. Finally, chapter three summarizes and analyzes the results obtained from the description generator service, establishing test scenarios and validating its functionality through a closed survey. It also sets out the conclusions and recommendations for the Curricular Integration Project, considering aspects for future work on this topic.