Microsoft has developed a graph-based method for retrieval-augmented generation (RAG) called GraphRAG, which allows users to answer questions about private or unseen datasets. You can now get GraphRAG on GitHub.
The tool provides more systematic information extraction and complete response production compared to conventional RAG methodologies. The solution accelerator that comes with the GraphRAG code repository offers an intuitive API experience that is hosted on Azure and can be deployed without any coding knowledge.
GraphRAG automatically extracts a knowledge graph from any set of text documents using a large language model (LLM). This graph-based data index finds “communities” of densely connected nodes in a hierarchical manner, allowing it to report on the semantic structure of the data before user queries.
Without requiring knowledge of specific questions beforehand, each community summary provides an overview of a dataset by describing its entities and their relationships.
Recent studies showed that GraphRAG can respond to “global questions” that cover the whole dataset, an area in which crude RAG methods frequently fall short.
GraphRAG’s community summaries provide more thorough and varied responses since they take into account all input texts. By aggregating community reports up to the LLM context window size, this method applies a map-reduce technique. It then maps the question across each group to generate community answers, which are then reduced into a final global answer.
Also Read: Microsoft’s Suleyman Sparks Debate on AI Training Using Internet Content
GraphRAG performs better than naive RAG in comprehensiveness and variety, with a 70–80% win rate, according to comparative experiments conducted with GPT-4. At lower token costs, it outperformed hierarchical source-text summarization as well. These outcomes demonstrate how well GraphRAG may produce comprehensive and diverse responses from huge datasets.
Potential uses for GraphRAG include a wide range of industries needing in-depth data insights. The goal of releasing GraphRAG and its solution accelerator to the public is to enable users who require global data understanding to have access to graph-based RAG techniques.
Also Read: Microsoft Unveils ‘Skeleton Key’ Attack Exploiting Generative AI Systems
