RAG4Tickets: AI-Powered Ticket Resolution via Retrieval-Augmented Generation on JIRA and GitHub Data

summary

Video file (mp4)

The gist

The gist: The proposed Retrieval-Augmented Generation (RAG) framework integrates Sentence-Transformers, FAISS, and an LLM to deliver context-aware ticket resolution recommendations by embedding

In short

The research proposes a Retrieval-Augmented Generation (RAG) framework to help software teams resolve recurring issues faster. It integrates historical JIRA tickets and GitHub PR data using Sentence-Transformers and FAISS for semantic search. This system retrieves relevant past cases, which an LLM then synthesizes into grounded, explainable resolution suggestions, significantly improving accuracy and reducing resolution time.

Key concepts

Retrieval-Augmented Generation (RAG)
A framework that enhances a Large Language Model by first retrieving relevant external information before generating an answer. In this context, it means the system searches historical JIRA tickets and code metadata to find similar past problems before asking the LLM to suggest a fix, ensuring the suggestions are based on real data.
Sentence-Transformers
These are models used to convert text (like ticket descriptions or comments) into numerical vectors called embeddings. They capture the meaning of sentences semantically, allowing the system to understand that two different pieces of text discussing similar problems are conceptually close, even if they use different words.
FAISS
A library used for efficient similarity search over large sets of vectors (embeddings). It allows the system to quickly find past JIRA tickets or PRs that are semantically similar to a new ticket by performing Approximate Nearest Neighbor (ANN) searches, making the knowledge retrieval process very fast.

Terminology used across episodes

This episode discusses

The paper

RAG4Tickets: AI-Powered Ticket Resolution via Retrieval-Augmented Generation on JIRA and GitHub Data · Read on arXiv

Mohammad Baqar

Cisco Systems Inc

Modern software teams frequently encounter delays in resolving recurring or related issues due to fragmented knowledge scattered across JIRA tickets, developer discussions, and GitHub pull requests (PRs). To address this challenge, we propose a Retrieval-Augmented Generation (RAG) framework that integrates Sentence-Transformers for semantic embeddings with FAISS-based vector search to deliver context-aware ticket resolution recommendations. The approach embeds historical JIRA tickets, user comments, and linked PR metadata to retrieve semantically similar past cases, which are then synthesized by a Large Language Model (LLM) into grounded and explainable resolution suggestions. The framework contributes a unified pipeline linking JIRA and GitHub data, an embedding and FAISS indexing strategy for heterogeneous software artifacts, and a resolution generation module guided by retrieved evidence. Experimental evaluation using precision, recall, resolution time reduction, and developer acceptance metrics shows that the proposed system significantly improves resolution accuracy, fix quality, and knowledge reuse in modern DevOps environments.

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: Today's paper: "RAG4Tickets: AI-Powered Ticket Resolution via Retrieval-Augmented Generation on JIRA and GitHub Data".

Jane: The gist: The proposed Retrieval-Augmented Generation (RAG) framework integrates Sentence-Transformers, FAISS,

Tom: First, who's behind it and why it matters.

Paper summary: Tom: So, wrapping up this discussion on RAG4Tickets: AI-Powered Ticket Resolution via Retrieval-Augmented Generation on JIRA and GitHub Data. The authors are Mohammad Baqar from Cisco Systems Inc, CA, USA.

Jane: They essentially propose a unified pipeline for ingesting and linking JIRA tickets, developer discussions, and GitHub pull requests into an integrated system for AI-assisted ticket triage and resolution generation.

Tom: It’s about using that RAG framework to deliver context-aware ticket resolution recommendations based on past cases found in the data.

Jane: So the implication is that this helps software teams manage their knowledge better by providing a mechanism to ground AI suggestions in historical, specific evidence.

Tom: This means less time spent searching through disparate systems and more time actually fixing bugs and building new features.

Tom: The paper title RAG4Tickets: AI-Powered Ticket Resolution via Retrieval-Augmented Generation on JIRA and GitHub Data points toward a future where AI isn't just classifying issues, but actively reasoning based on the evidence it finds.

Jane: It suggests that combining semantic retrieval with an LLM is a practical way to make those LLMs useful in operational settings by grounding their answers in specific, historical evidence.

Tom: So this framework is about using the JIRA and GitHub data to improve how software teams resolve technical problems.

Jane: That’s what RAG4Tickets: AI-Powered Ticket Resolution via Retrieval-Augmented Generation on JIRA and GitHub Data does for the software development community.

Conclusion: Tom: So we're wrapping up on RAG4Tickets, that paper by Baqar from Cisco Systems Inc. It’s about taking all our messy JIRA tickets and GitHub code history and using a Retrieval-Augmented Generation framework to help AI actually suggest how to fix things.

Jane: It’s really interesting because it’s not just throwing an LLM at a ticket and hoping for the best; it's actively retrieving specific past examples to ground the answer in real data.

Lu: The authors are focusing on unifying JIRA and GitHub data through this pipeline, which is a pretty big move for how we connect our development processes.

Meng: From where I’m sitting, I’m curious about how much effort goes into keeping that connection unified when you're dealing with so many different systems.

Lalam: My literature review shows that the core idea here is embedding historical tickets and PR metadata so the LLM can pull relevant context for resolution suggestions.

Tom: Exactly, Lalam, and it’s about making sure those suggestions aren't just guesses but are based on what has actually worked before. It links the JIRA world to the GitHub world directly.

Jane: So what does this mean for a team that spends all day searching through old tickets and trying to piece together a solution from scattered comments?

Lu: It means a unified pipeline, which is key, and they use Sentence-Transformers for embeddings and then FAISS for fast vector search across millions of those historical artifacts.

Meng: Fast search is good, but what about the accuracy when you’re searching across both text descriptions in JIRA and the actual code diffs in PRs? That sounds like a tough matching problem.

Tom: They actually tackle that by partitioning the embeddings by artifact type, so they can target their search space better for tickets versus pull requests.

Jane: And then they use a decision pipeline where that retrieved evidence guides the final response generated by the LLM. That’s how you get those grounded suggestions.

Lalam: The evaluation metrics they use include Recall at k and Mean Reciprocal Rank, which measures how well the system finds and ranks those relevant past cases.

Tom: They showed a forty-five percent improvement in average resolution time in their case study, which is a solid number when you’re talking about real developer productivity.

Jane: It seems like the authors are showing how retrieval and generation can be effectively fused to emulate human-like diagnostic behavior in DevOps workflows. That’s the big picture here.

Lu: They also looked at limitations, specifically factual reliability where the LLM sometimes hallucinates if it goes too far outside what it finds in those retrieved documents.

Meng: That’s a fair point; hallucinations are always a risk when you rely on an LLM to synthesize things, even with good retrieval. But they plan to add a verification layer and reinforcement learning with human feedback later on to fix that.

Tom: It sounds like the paper proves the concept is viable for improving accuracy and knowledge reuse in modern DevOps environments right now.

Jane: It really shows how this RAG framework can help mitigate cognitive overload for developers dealing with technical debt across multiple platforms. So, where do we go from here with this kind of system?

More episodes

← Home