Subgraph composition

[Jannis]

Rationale / Use Cases

Subgraph composition—or, more specifically, query-time subgraph composition—is the ability to compose and extend subgraph schemas and traverse relationships across subgraph boundaries in queries.

This ability unlocks several use cases:

1. Linking entities across subgraphs.
2. Extending entities defined in other subgraphs by adding new fields.
3. Break down data silos by composing subgraphs and defining richer schemas without indexing the same data over and over again.

Requirements

To be added.

Proposed User Experience

The following descriptions assume the following:

1. There is an ethereum/mainnet subgraph with an Address entity type.
2. We're building a DAO subgraph with proposals.

Compose subgraphs by linking foreign entities

In order to link to foreign entities, a developer would first import these entity types from the other subgraph.

Let's say the DAO subgraph contains a Proposal type that has a proposer field that should link to an Ethereum address (think: Ethereum accounts or contracts) and a transaction field that should link to an Ethereum transaction. The developer would then write the DAO subgraph schema as follows:

# These definitions are implicitly added by Graph Node:
#
# input NamedTypeImport {
#   name: String!
#   as: String!
# }
# union TypeImport = String | NamedTypeImport
# input SubgraphImportById { id: String! }
# input SubgraphImportByName { name: String! }
# union SubgraphImport = SubgraphImportById | SubgraphImportByName
# directive import {
#   types: [TypeImport!]!
#   from: SubgraphImport!
# }

@import(
  types: ["Address", { name: "Transaction", as: "EthereumTransaction" }]
  from: { name: "ethereum/mainnet" }
)

type Proposal @entity {
  id: ID!
  proposer: Address! # Address is actually an interface in `ethereum/mainnet` but that's supported
  transaction: EthereumTransaction!
}

This would then allow queries that follow the references to addresses and transactions, like

{
  proposals { 
    proposer { balance address }
    transaction { hash block { number } }
  }
}

Extend foreign types

Extending foreign types from another subgraph involves a few steps:

1. Import the foreign entity types.
2. Extend these entity types with custom fields.
3. Manage (e.g. create) extended entities in subgraph mappings, where only the extension fields are accessible.

Let's say the DAO subgraph wants to extend the Ethereum Account and Contract types to include the proposals created by that account in the context of the DAO. To achieve this, the developer would write the following schema:

@import(
  types: ["Address", "Account", "Contract"]
  from: { name: "ethereum/mainnet" }
)

type Proposal @entity {
  id: ID!
  proposer: Address!
}

extend type Account {
  proposals: [Proposal!]! @derivedFrom(field: "proposal")
}

extend type Contract {
  proposals: [Proposal!]! @derivedFrom(field: "proposal")
}

This makes queries like the following possible, where you can go "back" from accounts and contracts to proposal entities, despite the Account and Contract types originally being defined in the ethereum/mainnet subgraph.

{
  accounts {
    address
    proposals {
      id
      proposer {
        address
    }
  }
}

An open question here is whether it is possible to extend foreign interfaces. For example, Account and Contract both implement the Address interface in the ethereum/mainnet subgraph and so it would be great if the proposals field could be added to this interface as well.

When types are extended with non-derived fields, like

extend type Account {
  proposals: [Proposal!]!
}

the code generation in Graph CLI will generate a local Account type that can be used in subgraph mappings. This type will only have an id and proposals field. The subgraph mappings then has to create local instances of this local Account extension type to populate the proposals.

At query time, Graph Node then looks up both local Account and foreign Account instances and merges them on the id fields. The mapping that creates the local Account instance could look as follows:

import { Account } from '...'

export function handleNewProposal(newProposal: NewProposal): void {
   let account = Account.load(newProposal.params.proposer.toHexString())
   if (account == null) {
    account = new Account(newProposal.params.proposer.toHexString())
    account.proposals = []
  }
  let proposals = account.proposals
  proposals.push(newProposal.params.proposalID)
  account.proposals = proposals
  account.save()
}

Next steps

The following parts of the plan still need to be filled in:

• Proposed Implementation
  • Graph Node will have to resolve @import directives to pull foreign schemas
  • Graph Node will have to verify whether imported types exist in the foreign schemas
  • Graph Node will have to merge subgraph schemas before generating the GraphQL API, adding @subgraphId directives to types according to where they come from
  • Graph Node will have to take @subgraphId directives into account when querying
  • Graph Node will have to split up queries for extended types up and merge results from both subgraphs
  • Graph CLI will have to validate @import directives (e.g. their arguments)
• Proposed Documentation Updates
• Proposed Tests / Acceptance Criteria
• Tasks

[leoyvens]

One thing that still needs to be specified is how to do something like importing extensions from different subgraphs and combining them into your own extension.