AI Products & Retrieval Augmented Generation (RAG) resources

This is a collection of AI Products Development & Retrieval Augmented Generation (RAG) resources, either informative blogposts or tutorials, along with books, tweets, youtube videos, short courses, etc, that I have found useful at some point during my career.

Blogposts

• I'm sorry, but those are vanity evals By Izzy Miller, Hex Team, April, 2025

• A Field Guide to Rapidly Improving AI Products By Hamel Husain, March, 2025

• Evaluating RAG-enhanced LLM systems beyond model capabilities By Juan Pablo Mesa Lopez (Myself), December, 2024

• Integrating ML and Generative AI into your business: start with the problem, not the solution By Juan Pablo Mesa Lopez (Myself), December, 2024

• Building AI Products—Part I: Back-end Architecture By Phil Calçado, December, 2024

• Fine-Tuning in Sentence Transformers 3 By Juan Pablo Mesa Lopez (Myself), Aurelio, August, 2024

• Evaluating Chunking Strategies for Retrieval By Chroma DB Team, July, 2024

• Improving Retrieval with LLM-as-a-Judge By Jo Kristian Bergum, Vespa, April, 2024

• Evaluating Retrieval Augmented Generation using RAGAS By Superlinked Team, May, 2024

• An evaluation of RAG Retrieval Chunking Methods By Superlinked Team, April, 2024

• Your AI Product Needs Evals By Hamel Husain, March, 2024

• 12 RAG Pain Points and Proposed Solutions By Wenqi Glantz, January, 2024

• A Practical Guide to RAG Pipeline Evaluation (Part 1: Retrieval) By Yi Zhang and Pasquale Antonante, December, 2023

• Patterns for Building LLM-based Systems & Products By Eugene Yan, July, 2023

• Building RAG-based LLM Applications for Production By Goku Mohandas and Philipp Moritz, October, 2023

Practical guide to building successful LLM products

By Eugene Yan, Bryan Bischof, Charles Frye, Hamel Husain, Jason Liu, and Shreya Shankar

• Strategic: What We Learned from a Year of Building with LLMs (Part III) June, 2024
• Operational: What We Learned from a Year of Building with LLMs (Part II) June, 2024
• Tactical: What We Learned from a Year of Building with LLMs (Part I) May, 2024

Blogs from Jason Liu

• Systematically Improving Your RAG May, 2024
• Stop using LGTM@Few as a metric (Better RAG) February, 2024
• Levels of Complexity: RAG Applications February, 2024
• How to build a terrible RAG system January, 2024
• RAG is more than just embedding search September, 2023

Books

• AI Engineering by Chip Huyen, December, 2024
• Prompt Engineering for LLMs by John Berryman and Albert Ziegler, November, 2024
• LLM Engineer's Handbook by Paul Iusztin and Maxime Labonne, October, 2024

Tutorials

• The Definitive Guide to LLM Evaluation By Arize Team

Tweets