The Graph Review: The "Google of Blockchains"?

Libraries are vast collections of thousands of neatly arranged books that follow a pattern to help us find our desired books easily. The books are first separated based on genre and then alphabetically. Famous authors and subjects have dedicated shelves, and an extensive library also features a map to navigate different aisles. Finally, a librarian is also available to process requests and provide personal assistance. 

Imagine the chaos if none of such guidelines were present, with everything dumped in a massive pile at your mercy. Finding a book would be like looking for a needle in a haystack. Therefore, the available digital data is only valuable in the presence of adequate means to find and sort through it.

In data management, we call this concept indexing, a process of creating and maintaining digital data structures that allow for efficient and quick retrieval of specific data from a larger data set. We live in the digital age. Today, the use of data is deeply rooted even in the most trivial aspects of our day-to-day life. 

Most of us must realize our unparalleled dependence on data management tools provided by centralized, profit-oriented entities and their reigning monopolies. This review will cover everything you need to know about The Graph, a decentralized indexing protocol with a model to end the current data monopoly with blockchain technology.

Data Indexing

In computer science, we deal with loads of unstructured data, akin to finding information from a book without an index. Indexing is the process of organizing and mapping the data to organize it in a structured format so that users and programs can retrieve relevant data from it efficiently. Indexing is essential for:

• Improving data retrieval
• Reducing query latency
• Ensuring data integrity
• Optimizing data sorting 
• Enabling text search
• Data analysis

We call the process of data retrieval querying, done using a computer-readable querying language like SQL (Structured Query Language). Without indexing, the network would need to shuffle through the entire dataset for every little query request, consuming time and computational resources and making querying expensive. Internet users send billions of queries every second, so indexing is the cornerstone of modern network systems.

Indexing in Web3

Indexing blockchain data, especially for vast networks like Ethereum, is indispensable. The importance of indexing is evident if we observe how a blockchain records and stores data. Blockchains store a linear and immutable history of data divided into pockets of data called blocks, with every new block linked to the one before it.

A naive approach to querying some data on the blockchain is to begin with the genesis block, look for the data, and move on to the next block until the query is satisfied. Imagine having to carry out this process for every query, taking more and more time as new blocks join the chain with time.

Ingestion Services

Ingestion services collect large volumes of data, transforming and transporting them from multiple sources to the requested storage or processing environment. They collect data from various sources in several formats and ensure it is cleaned and formatted for efficient access. In essence, ingestion services provide a bridge that ensures data is seamlessly and efficiently moved and prepared for analysis, supporting various applications, including big data analytics, machine learning, and business intelligence.

Amazon Web Services (AWS) is a globally operating data ingestion service giant, providing a broad and deep set of cloud services to millions of customers and multiple data centers scattered around the globe. AWS offers over 200 fully-featured services, including computing, storage, machine learning, analytics, etc.

An Overlooked Piece of the Decentralization Puzzle

Decentralized applications (dApps) run on the blockchain, securing its permissionless and immutable characteristics. While dApps run in a decentralized environment, they sometimes rely on centralized services like AWS (Amazon Web Services) for various aspects of their operation. This dependence manifests in hosting websites, backend services, databases, or APIs on centralized servers. Here are a few examples:

• Website hosting - While the dApp smart contract runs on the blockchain, the user interface required to access the dApp without running a node is often hosted on traditional web servers. dApps may use centralized APIs and backend services for additional functionality.
• Data storage - Some blockchain products like NFTs are a hybrid of on and off-chain data. The ownership history for an NFT may live on the blockchain, while its metadata (the underlying asset, like an image) might be stored on AWS’s storage services like Amazon S3. 
• Computational Resources - Some complex computational tasks (for instance, interest and risk calculations on a lending dApp) might get offloaded outside the chain to ensure faster processing.

Blockchain explorers like Etherscan are another classic example of a centralized ingestion service. Such blockchain explorers download the raw blockchain data and organize and refine it to present a real-time blockchain snapshot.

While efficient, these services could be better for building truly decentralized applications, as they require trust in the provided data. Another alternative dApps may adopt is to implement a proprietary ingestion service baked into their protocol. Again, while efficient, it creates a lot of redundant and expensive work. Therefore, dApp developers are facing a dilemma: either build a performant but dependent service or build a decentralized but expensive and inefficient one.

The Ethos of The Graph

The Graph network seeks to decentralize internet application API and query layers. Instead of relying on a centralized service like AWS, a diverse set of indexers serve the users who only pay for the queries they request. 

This way, the network deploys the laws of supply and demand to create an open market for indexed data by incentivizing participants to provide, curate, and consume data in a decentralized and permissionless setting, eliminating the need for a centralized party. It eliminates single points of failure and ensures liveness by deploying its native GRT token such that there is always an incentive to service the user's queries. The Graph protocol positions itself right at the center of the developer’s dilemma.

The Graph Protocol

Consider an example of a lending and borrowing dApp operating in the Ethereum network that allows users to lend, borrow, stake, swap, and yield farm multiple cryptocurrencies with the help of numerous smart contracts. The dApp features an interactive UI, letting users access these features from a single screen. Now, a user must retrieve the financial activity of a bunch of addresses for an analysis. They would do so in one of the following ways:

Option 1 - Build your own server

One approach is to build a new server and download the blockchain data on the local database to run the queries. The obvious obstacle is the steep resource requirements behind maintaining such vast amounts of data and the single points of failure.

Option 2 - Use a Blockchain Explorer

One could always refer to a blockchain explorer, a vast library of structured blockchain data since the genesis block. However, there are a few pitfalls here as well:

• Blockchain explorers are not designed to conduct intricate research like the one described above. One would have to jump multiple pages for each smart contract, transaction, and address, consuming time.
• Blockchain explorers like Etherscan are a centralized service. The underlying blockchain protocol does not guarantee the integrity of its data. Therefore, this approach is not trustless.

Option 3 - The Graph protocol

The Graph is a decentralized protocol that indexes blockchain data and enables its efficient querying by creating data subgraphs - indexed data APIs in the protocol. The protocol can index data from networks like IPFS and Ethereum using the GraphQL query language. Advantages of using the Graph:

• Unified Queries: With The Graph, developers can create a subgraph that indexes and organizes data from multiple smart contracts into a unified, easily queryable API. Users and applications can retrieve comprehensive, correlated data with a single query.
• Data Transformation: The Graph allows for transforming raw blockchain data into a more readable and usable format. It can handle complex data structures, relationships, and computations to deliver ready-to-use data.
• Real-Time Data Access: The Graph enables real-time data access, making it suitable for applications that require up-to-the-minute data for decision-making, analytics, or user interfaces.

In complex scenarios like the DeFi application described above, The Graph offers significant advantages regarding data accessibility, usability, and real-time analytics. It simplifies the process of retrieving and interpreting blockchain data, making it a preferred choice for applications that require complex, correlated, and real-time data access beyond the capabilities of a traditional blockchain explorer.

The Graph Network Overview

The Graph protocol enables developers to build serverless applications with their entire stack running on public infrastructure, extending the use of blockchain networks from processing smart contracts to also as the front-end interface to fetch blockchain data. With the Graph, dApps operate without ever accessing a centralized service like AWS, and thus, they achieve complete stack decentralization. 

To ensure an efficient and accurate supply of indexed data from a public infrastructure, the Graph Network relies on an economic model of incentives and punishments powered by GRT, an ERC-20 token native to the protocol.

How The Graph Protocol Works

The primary job of the Graph protocol is to index blockchain data APIs, called subgraphs. The dApp developers create and enable its querying by the users. The protocol learns what and how to index blockchain data using subgraph manifests. Developers include indexing descriptions and details within the manifests to guide the protocol.

Once a developer completes writing a manifest, they use a Graph Command Line Interface (CLI), a tool for building and deploying to The Graph to store the manifest definition on IPFS. Indexers pick these up as a signal to start indexing data for that subgraph. The indexed subgraph is now queriable, paving the way for more subgraphs. 

Here is a stepwise breakdown of the workflow cycle:

1. The cycle begins with dApps using smart contracts to process interactions between addresses in a blockchain that emits one or more events.
2. Graph nodes(indexers) constantly scan the blockchain for new blocks, looking for events described in subgraph manifests.
3. If the Graph node finds matching events in a new block, it runs the mapping handlers provided in the subgraph. The mapping module creates or updates the data entities the Graph node stores in response to blockchain events.
4. The recorded events are now indexed and ready for Graph query requests made with GraphQL.
5. A dApp or an end user queries the Graph node for the indexed blockchain data using the node’s GraphQL endpoint. The node uses indexing capabilities to fetch the requested data from the indexed subgraphs. 
6. The dApp displays the results received from the Graph node in a rich UI for end-users, which they use as data to issue new interactions on the blockchain, creating new data that needs indexing. The cycle repeats.

Roles in the Graph Ecosystem 

The Graph protocol involves several roles, each contributing to the functioning and security of the network. These roles are crucial for indexing, querying, and governing the protocol. Here are the primary roles:

Indexers

Indexers run the Graph Nodes that index data from a specific blockchain network and serve queries on that data. They stake GRT tokens as collateral to provide indexing and query processing services. 

Indexer Economic Model:

The indexers earn rewards for their services from two primary streams:

• Query fee rebates - The subgraph consumers pay for queries to the indexers in GRT tokens.
• Indexing rewards - Indexers earn rewards for indexing subgraphs in the Graph protocol. The protocol issues new GRT tokens at 3% annually and distributes them to indexers as indexing rewards.

Indexers stake at least 100,000 GRT to earn the right to index subgraphs in the protocol. This stake is slashed if the indexer behaves maliciously by supplying incorrect data or incorrectly indexing subgraphs.

The Query Market serves as an API that efficiently serves data required by the user or the application in the front end. Unlike traditional ingestion services, the query market is a decentralized network of Graph indexers competing to serve query requests most efficiently, priced by supply and demand.

Consumers

Consumers are the end users (individuals or applications) that query data from the Graph Network. The consumers rely on the availability and accuracy of the indexed data. If served incorrectly, it may affect how applications run and what on-chain decisions individuals make.

 Consumer Economic Model:

The consumers have the incentive of access to reliable, efficient, and quick access to indexed blockchain data from an on-chain, permissionless, and decentralized source without the overheads of running a full node and extracting data manually.

Consumers pay for metered access to the indexer’s services in GRT tokens, meaning they only pay for the number and the complexity of the queries they make. In essence, consumers are only charged for the specific resources spent by the protocol in serving their requests, ensuring fair pricing, resource optimization, and sustainability for indexers.

Delegators

Indexers who run a full Graph node can pool GRT tokens from others who wish to participate in the network’s security without operating a full node. These individuals are delegators. They stake GRT to existing indexers and earn some of the query fees and the indexing rewards.

Delegators may select indexers based on specific key characteristics like:

• The indexer’s performance, like slashing history and past uptime.
• The portion of rewards and indexer’s pledge to its delegators.

Curators

The curators play an essential role in the Graph ecosystem. The indexers index subgraphs, which end users pay to query and generate revenue for the Graph ecosystem. The ecosystem generates more revenue when it serves more queries. It serves more queries when it indexes efficient subgraphs that are in high demand.

Creating subgraphs in the Graph protocol is completely permissionless, so anyone can create any subgraph, regardless of its efficiency and use. Therefore, finding the most ideal subgraphs is essential for the system’s efficiency and ensuring optimum resource pricing, which is the job of Curators.

Curators use their knowledge of Web3, blockchain technology, and data management to assess subgraphs and signal the indexers which ones to index to generate the most revenue. The Graph Network rewards Curators when one of their signalled subgraphs is queried.

Curator Economic Model

The Graph Network wants indexers to find the subgraphs with the highest revenue potential as quickly as possible. Therefore, it incentivizes the Curators to signal subgraphs early, using a Bonding Curve to price their stake and reward.

Each subgraph is divided into curation shares. The Curators stake GRT to buy curation shares, which becomes linearly expensive with increasing demand for the shares. Therefore, Curators who can assess a subgraph’s value potential early can buy those shares for cheap. The act of buying curation shares is called signalling. Indexers index the subgraphs with the highest signals.

Curators are not subject to slashing for signalling poor subgraphs, but they incur a deposit tax to deter them from bad behavior. Moreover, the open signalling market creates an economic incentive to buy shares of undervalued subgraphs so that they may profit from selling them later.

Fishermen and Arbitrators

Fishermen are network participants who check indexer query responses and may dispute them. They are rewarded if the dispute concludes in their favor and the corresponding indexer is slashed. At the time of writing, the slashing penalty is set at 2.5% of the indexer’s stake. Half of this is rewarded to the fishermen while the other half is burned.

Arbitrators are the entities called to settle disputes in the network. They are set via protocol governance. The Fishermen and Arbitrator model keeps the network in check and maximizes utility and reliability.

The Graph Explorer

The Graph Explorer is a web interface to interact with the various aspects of The Graph Protocol. As users, you can find and query all the subgraphs listed in the market. As Curators, you may stake GRT to signal your or other listed subgraphs for an economic incentive. The ‘Participants’ tab overviews the indexer activity, their staked amount, and delegation details. The ‘Network’ section lets you analyze network metrics in more detail. In conclusion, the Graph Explorer is the front end for network participants and users to access the Graph Prtocool.

Subgraph Studio

The Subgraph Studio is a development environment provided by The Graph for developers to build, test, and deploy subgraphs. Subgraphs are open APIs indexing blockchain data, making it easily accessible and queryable. The Subgraph Studio offers tools and features that streamline creating and managing subgraphs. 

Key Functionalities of Subgraph Studio:

1. Subgraph Creation:
   • Development Tools - Provides tools for writing and testing the code that defines how blockchain data is indexed and queried.
   • Schema Definition - Allows developers to define the GraphQL schema, specifying the types of data to be indexed and how they can be queried.
2. Testing and Simulation:
   • Local Testing - Enables developers to test subgraphs locally, simulating the indexing and querying of blockchain data to ensure accuracy and efficiency.
   • Query Simulation - Allows the testing of GraphQL queries to ensure that the subgraph returns the expected data.
3. Deployment:
   • Publishing - Offers features for deploying subgraphs to The Graph network, making them publicly accessible for queries.
   • Version Management - Supports managing different versions of a subgraph, allowing updates and improvements.
4. Monitoring and Analytics:
   • Performance Monitoring - Provides analytics and insights into the performance of deployed subgraphs, including query volume, latency, and other metrics.
   • Error Tracking - Helps identify and diagnose errors or issues with the subgraph’s indexing or querying.
5. Collaboration and Sharing:
   • Collaboration Tools - This may offer features for collaborative development, allowing multiple developers to work on a subgraph together.
   • Sharing - Enables sharing subgraphs with others, promoting discovery and usage.

GRT Tokenomics

GRT is an ERC-20 token with an initial supply of 10 billion. GRT is subject to 3% annual inflation, disturbed to indexers as the indexing reward. GRT is also subject to burning:

• 1.25% of the indexer’s stake is burned as a dispute-slashing penalty.
• All unclaimed rewards from the rebate pool are also burned.

These burning schemes keep the network in check while also contracting the supply of GRT tokens, which may lead to increased demand.

The GRT vesting and distribution schedules range between 6 months to 10 years. Here is the GRT vesting schedule, as provided by the Graph Blog:

Supported Networks

One straightforward source of demand for GRT tokens is the increasing support of the subgraph studio to more blockchain networks. Apart from Ethereum, the Subgraph Studio is also available on several EVM networks like Optimism, Polygon, zkSync, and Arbitrum One, and many non-EVM networks like Cosmos Hub, Fantom, Moonbeam, Base, and Avalanche. Increased network support entails more GRT spent creating and querying subgraphs, creating more demand for the tokens. You can check the full list on the Graph's Supported Networks page.

Roadmap

The R&D roadmap of the Graph Protocol focuses on some critical areas of development:

1. Data and APIs - Improving the indexing performance and protocol reliability to ensure good indexing uptime, speed, and scalability is one of the protocol’s key development focus. The team is working on standardizing data extraction and developing multi-chain data extraction integrations to improve the efficiency of the network.
2. SNARK Force - The team is working on using zk proofs to lower trust assumptions required to use the Graph to 1 of N network participants.
3. Protocol Economics - The team is also exploring how the Graph can leverage rollup designs to scale protocol logic and reduce costs by deploying a multi-layered design.

Improving Network Efficiency With AI

In December 2021, Semiotic Labs joined The Graph Network as its fourth core developer. Semiotic has since been researching AI's potential in improving The Graph Protocol's utility. The open data market allows network participants to allocate resources and independently price their indexing services in the query market. Semiotic realized that the network participants can leverage AI to make dynamic decisions based on supply and demand that maximize protocol returns while also ensuring the network operates sustainably.

The Semiotoc team has developed two AI tools to push this goal -  AutoAgora and Allocation Optimizer.

AutoAgora

Indexers in the Graph Protocol use a domain-specific language called Agora to place their price bids for the various GraphQL queries. Indexers stake GRT to obtain indexing rights and spend computational resources in indexing the subgraphs required to serve the queries, so they want adequate compensation for their efforts. Conversely, the users want the indexers to offer the best possible price.

To help optimize the query market economy, Semitic developed AutoAgora. It is an AI tool consisting of various modules that scan query requests to determine their volume (the resources the indexer would spend processing the request). The module helps AutoAgora dynamically adjust the indexer price bids for optimum output.

AutoAgora is trained in mathematical models that use Gaussian distribution to determine query pricing. It liberates the indexer from constantly adjusting their price bids manually with automation.

Allocation Optimizer

The Allocation Optimizer is an AI-based tool that solves the optimization problems for the indexers to extract optimum value from their resources. 

The Graph defines the optimization problem as follows:

The indexers earn GRT by serving query requests, which requires them to index the subgraphs detailed in the query. Indexing a subgraph is very resource-intensive, so indexers can’t simply index as many subgraphs as possible. Indexers find the most profitable subgraphs to index by following Curator signals. If the Curators only signal subgraphs that generate the highest query fees, the remaining subgraphs would remain unindexed; this is the optimization problem.

The Graph Protocol combats the optimization problem with indexing rewards. With indexing rewards, the protocol can regulate the net rewards for indexing subgraphs such that all kinds of subgraphs are indexed. The Allocation Optimizer allows indexers to automate the dynamic management of their resources to maximize indexer returns and protocol sustainability.

The Future of AI in the Graph

The team behind Graph believes that AI may benefit from the Graph Protocol in the future. It argues that Large Language Models (LLMs) like ChatGPT require vast volumes of data to train them. Traditionally, corporations with immense monetary muscle operate data silos to maintain an edge for their AI protocols.

The open market of Graph Networks opens these data silos and encourages permissionless and openly verifiable participation. With the Graph, smaller AI initiatives will also have access to rich and affordable data so that they can go head-to-head with projects that have a head start in the AI industry.

You can learn about some of the other crypto projects leveraging Artificial Intelligence in our Top AI Blockchain Projects article. 

The Graph Review: Conclusion 

The Graph protocol has emerged as a vital component in the Web3 ecosystem, enabling efficient and decentralized access to blockchain data. This article has provided an in-depth analysis of the protocol's economics and workflow, demystifying its complexities for general readers. As the demand for decentralized applications (dApps) and blockchain-based solutions continues to grow, the Graph protocol is poised to play an increasingly important role in the future of the decentralized internet.

The potential of GRT tokens, the native utility token of the Graph protocol, is intrinsically linked to the success and adoption of the protocol itself. As more developers and projects leverage the Graph for their data needs, the demand for GRT tokens will likely increase. However, it is essential to remember that the cryptocurrency market is inherently volatile, and any investment should be made with caution and thorough research. Ultimately, the Graph protocol's innovative approach to data indexing and querying holds immense promise for the future of the blockchain industry and the broader Web3 landscape.