Meet GraphRAG: Microsoft’s New Graph-Based AI Method for Superior Data Insights

Microsoft introduces GraphRAG, a graph-based AI method for retrieval-augmented generation, now available on GitHub. This tool enhances data retrieval and question answering for private or unseen datasets, offering a systematic and complete response production.

Microsoft has developed a graph-based method for retrieval-augmented generation (RAG) called GraphRAG, which allows users to answer questions about private or unseen datasets. You can now get GraphRAG on GitHub.

The tool provides more systematic information extraction and complete response production compared to conventional RAG methodologies. The solution accelerator that comes with the GraphRAG code repository offers an intuitive API experience that is hosted on Azure and can be deployed without any coding knowledge.

GraphRAG automatically extracts a knowledge graph from any set of text documents using a large language model (LLM). This graph-based data index finds “communities” of densely connected nodes in a hierarchical manner, allowing it to report on the semantic structure of the data before user queries.

Without requiring knowledge of specific questions beforehand, each community summary provides an overview of a dataset by describing its entities and their relationships.

Recent studies showed that GraphRAG can respond to “global questions” that cover the whole dataset, an area in which crude RAG methods frequently fall short.

A comparison of naive RAG and GraphRAG responses to a global question about a news dataset(opens in new tab) indicates that GraphRAG outperformed naïve RAG in terms of comprehensiveness, diversity, and empowerment.

GraphRAG’s community summaries provide more thorough and varied responses since they take into account all input texts. By aggregating community reports up to the LLM context window size, this method applies a map-reduce technique. It then maps the question across each group to generate community answers, which are then reduced into a final global answer.

Also Read: Microsoft’s Suleyman Sparks Debate on AI Training Using Internet Content

GraphRAG performs better than naive RAG in comprehensiveness and variety, with a 70–80% win rate, according to comparative experiments conducted with GPT-4. At lower token costs, it outperformed hierarchical source-text summarization as well. These outcomes demonstrate how well GraphRAG may produce comprehensive and diverse responses from huge datasets.

Potential uses for GraphRAG include a wide range of industries needing in-depth data insights. The goal of releasing GraphRAG and its solution accelerator to the public is to enable users who require global data understanding to have access to graph-based RAG techniques.

Also Read: Microsoft Unveils ‘Skeleton Key’ Attack Exploiting Generative AI Systems

This post was last modified on July 9, 2024 5:12 am

Kumud Sahni Pruthi

A postgraduate in Science with an inclination towards education and technology. She always looks for ways to help people improve their lives by putting complex things into simple words through her writing.