Dries Buytaert Automates Alt-Text Generation for 9,000 Images Using AI

Dries Buytaert, the founder of Drupal, has developed an AI-powered system to automate alt-text generation for 9,000 images on his website. This move aims to improve web accessibility. With billions of images online lacking proper descriptions, Dries identified a similar issue on his site and built a structured process to generate and update alt-text efficiently. His system involves checking if an image has alt-text, using AI to generate descriptions when needed, and updating the website's metadata accordingly through a custom-built API.

Initially, Dries experimented with local Large Language Models (LLMs) using PyTorch but found their quality insufficient for accessibility purposes. After comparing various options, he opted for cloud-based AI models such as OpenAI's ChatGPT-4o and Anthropic's Claude-3 Sonnet, which provided significantly better descriptions. To streamline integration, he leveraged Simon Willison's 'llm' tool, which supports both local and cloud-based models, and enhanced AI-generated descriptions with additional contextual metadata, such as location details.

To implement AI-generated alt-text at scale, Dries automated the metadata update process through API calls while ensuring transparency by labelling AI-generated descriptions separately from human-written ones. This approach allows for future refinements as AI models evolve. While Dries acknowledges the challenge of fully trusting AI-generated content, he prioritizes progress over perfection and encourages other developers to adopt similar accessibility improvements. Please read the full blog post on his website for a detailed project breakdown.