🧑‍💻2024/08/08
Review of recent & in-progress subgraph DX enhancements (+ a few ideas)
Presentation feedback
Please submit your feedback in this form.
Q4 2023 -> Q2 2024
Aggregations and timeseries
Timeseries and aggregations enable your subgraph to track statistics like daily average price, hourly total transfers, etc.
This feature introduces two new types of subgraph entity. Timeseries entities record data points with timestamps. Aggregation entities perform pre-declared calculations on the Timeseries data points on an hourly or daily basis, then store the results for easy access via GraphQL.
Timeseries entities are defined with @entity(timeseries: true)
in schema.graphql
, while aggregation entities are defined with @aggregation
.
type Data @entity(timeseries: true) {
id: Int8!
timestamp: Timestamp!
price: BigDecimal!
}
type Stats @aggregation(intervals: ["hour", "day"], source: "Data") {
id: Int8!
timestamp: Timestamp!
sum: BigDecimal! @aggregate(fn: "sum", arg: "price")
}
Every timeseries entity must have a unique ID of the int8 type, a timestamp of the Timestamp type, and include data that will be used for calculation by aggregation entities.
These Timeseries entities can be saved in regular trigger handlers, and act as the “raw data” for the Aggregation entities.
Every aggregation entity defines the source from which it will gather data (which must be a Timeseries entity), sets the intervals (e.g., hour, day), and specifies the aggregation function it will use (e.g., sum, count, min, max, first, last).
Aggregation entities are automatically calculated on the basis of the specified source at the end of the required interval.
Example GraphQL query
{
stats(interval: "hour", where: { timestamp_gt: 1704085200 }) {
id
timestamp
sum
}
}
Learn more about aggregations here.
Indexed argument filtering
Topic filters, also known as indexed argument filters, allow developers to precisely filter blockchain events based on the values of their indexed arguments.
When a smart contract emits an event, any arguments that are marked as indexed can be used as filters in a subgraph's manifest. This allows the subgraph to listen selectively for events that match these indexed arguments.
The Ethereum Virtual Machine (EVM) allows up to three indexed arguments per event.
The first indexed argument corresponds to
topic1
, the second totopic2
, and third totopic3
Token.sol
// SPDX-License-Identifier: MIT
pragma solidity ^0.8.0;
contract Token {
// Event declaration with indexed parameters for addresses
event Transfer(address indexed from, address indexed to, uint256 value);
// Function to simulate transferring tokens
function transfer(address to, uint256 value) public {
// Emitting the Transfer event with from, to, and value
emit Transfer(msg.sender, to, value);
}
}
Topic filters are defined directly within the event handler configuration in the subgraph manifest.
subgraph.yaml
with Topic 1 and Topic 2 filters for Transfer
event
eventHandlers:
- event: Transfer(indexed address,indexed address,uint256)
handler: handleDirectedTransfer
topic1: ['0xAddressA'] # Sender Address, can include >1 input address (comma separated)
topic2: ['0xAddressB'] # Receiver Address, can include >1 input address (comma separated)
In this configuration:
topic1
is configured to filterTransfer
events where0xAddressA
is the sender.topic2
is configured to filterTransfer
events where0xAddressB
is the receiver.The subgraph will only index transactions that occur directly from
0xAddressA
to0xAddressB
.
Declarative eth_calls
Allows eth_calls
to be executed ahead of time, enabling graph-node
to execute them in parallel.
Declared eth_calls
can access the event.address
of the underlying event as well as all the event.params
.
subgraph.yaml
with event.address
eventHandlers:
event: Swap(indexed address,indexed address,int256,int256,uint160,uint128,int24)
handler: handleSwap
calls:
global0X128: Pool[event.address].feeGrowthGlobal0X128()
global1X128: Pool[event.address].feeGrowthGlobal1X128()
Details for the example above:
global0X128
is the label of a declaredeth_call
. So isglobal1X128
.Pool[event.address].feeGrowthGlobal0X128()
is the actualeth_call
that will be executed, which is in the form ofContract[address].function(arguments)
The
address
andarguments
can be replaced with variables that will be available when the handler is executed.
subgraph.yaml
with event.params
calls:
- ERC20DecimalsToken0: ERC20[event.params.token0].decimals()
Q3 2024
In progress work in E&N's Graph Node team
Subgraph as dataSource/reuse existing entities
Functionality that allows subgraphs to query and leverage data from other subgraphs. This can be achieved by creating a separate blockstream that fetches triggers from subgraph database tables. New entities will be triggers.
subgraph.yaml
specVersion: 1.1.x
description: 'Example declaring subgraph(s) as dataSources'
repository: https://github.com/graphprotocol/graph-tooling
schema:
file: ./schema.graphql
dataSources:
- kind: subgraph
name: Foo
network: mainnet
source:
id: 'Qmblahblahblah'
startBlock: 123456
mapping:
kind: subgraph/entities
apiVersion: 0.0.y
language: wasm/assemblyscript
entityHandlers:
- entity: FooSpecificEntity
handler: handleFooSpecificEntity
- entity: SomeOtherFooEntity
handler: handleSomeOtherFooEntity
file: ./src/fooMappings.ts
IPFS robustness improvements
Why? Enhanced scalability and reliability.
Migrate to IPFS Gateway API
Separate IPFS file hosting
i.e. required subgraph definition files vs optional IPFS File Data Sources
subgraph.yaml
schema.graphql
src/mappings.ts
kind: file/ipfs
Ideas
Early stage concepts that may never reach production
Plug-in system
Generic structure
subgraph.yaml
specVersion: x.y.z
description: Subgraph with Foo and Bar plug-ins
repository: https://github.com/graphprotocol/graph-tooling
schema:
file: ./schema.graphql
indexerHints:
prune: auto
plugIns:
- Foo
- Bar
schema.graphql
type NFT @entity(immutable: true) {
id: Bytes!
owner: Bytes
fooPluginAttr: @foo(fn: "baz", arg: "id")
}
type User @entity(immutable: true) {
id: Bytes!
nfts: [NFT!]
barPluginAttr: @bar(fn: "lorem ipsum", arg: "nfts")
}
ENS
Primary name reverse lookup
subgraph.yaml
specVersion: x.y.z
description: Gravatar for Ethereum + ENS
repository: https://github.com/graphprotocol/graph-tooling
schema:
file: ./schema.graphql
plugIns:
- ENS
dataSources:
- kind: ethereum/contract
name: Gravity
network: mainnet
source:
address: '0x2E645469f354BB4F5c8a05B3b30A929361cf77eC'
abi: Gravity
mapping:
kind: ethereum/events
apiVersion: 0.0.6
language: wasm/assemblyscript
entities:
- Gravatar
abis:
- name: Gravity
file: ./artifacts/contracts/Gravity.sol/GravatarRegistry.json
eventHandlers:
- event: NewGravatar(uint256,address,string,string)
handler: handleNewGravatar
- event: UpdatedGravatar(uint256,address,string,string)
handler: handleUpdatedGravatar
file: ./src/mapping.ts
schema.graphql
type Gravatar @entity{
id: ID!
owner: Bytes!
displayName: String!
imageUrl: String!
ownerEnsName: String @ens(fn: "name", arg: "owner")
}
Oracle-powered price feeds
Chronicle Protocol
subgraph.yaml
specVersion: x.y.z
description: 'Information about Ethereum Mainnet Blocks'
repository: https://github.com/graphprotocol/graph-tooling
schema:
file: ./schema.graphql
plugIns:
- Chronicle
dataSources:
- kind: ethereum/contract
name: Contract
network: mainnet
source:
# We don't realy need a contract, just use Gravatar to have something
# here
address: '0x2E645469f354BB4F5c8a05B3b30A929361cf77eC'
abi: Contract
startBlock: 0
mapping:
kind: ethereum/events
apiVersion: 0.0.7
language: wasm/assemblyscript
entities:
- Block
abis:
- name: Contract
file: ./abis/Contract.json
blockHandlers:
- handler: handleBlock
file: ./src/mappings/blocks.ts
schema.graphql
type Block @entity(timeseries: true) {
# Timeseries must have an id of type Int8
id: Int8!
# Timeseries must have a timestamp field
timestamp: Timestamp!
# The block hash
hash: Bytes!
# The block number
number: Int!
# Price in USD
price: BigInt @chronicle(oracle: "0x46ef0071b1E2fF6B42d36e5A177EA43Ae5917f4E", arg: "number")
}
type Stats @aggregation(intervals: ["hour", "day"], source: "Block") {
# The id; it is the id of one of the data points that were aggregated into
# this bucket, but which one is undefined and should not be relied on
id: Int8!
# The timestamp of the bucket is always the timestamp of the beginning of
# the interval
timestamp: Timestamp!
# The aggregates
count: Int! @aggregate(fn: "count")
maxPrice: BigInt @aggregate(fn: "max", arg: "price")
}
src/mappings/blocks.ts
import { ethereum } from '@graphprotocol/graph-ts';
import { Block } from '../../generated/schema';
export function handleBlock(block: ethereum.Block): void {
// The id for timeseries is autogenerated; even if we set it to a real
// value, it would be silently overwritten
let blockEntity = new Block('auto');
let number = block.number.toI32();
blockEntity.hash = block.hash;
blockEntity.number = number;
blockEntity.timestamp = block.timestamp.toI32();
blockEntity.save();
}
query.graphql
stats(interval: "day",
current: ignore,
where: {
timestamp_gte: 1234567890 }) {
id
timestamp
count
maxPrice
}
Chainlink AggregatorV3Interface
subgraph.yaml
specVersion: x.y.z
description: 'Information about Ethereum Mainnet Blocks'
repository: https://github.com/graphprotocol/graph-tooling
schema:
file: ./schema.graphql
plugIns:
- AggregatorV3Interface
dataSources:
- kind: ethereum/contract
name: Contract
network: mainnet
source:
# We don't realy need a contract, just use Gravatar to have something
# here
address: '0x2E645469f354BB4F5c8a05B3b30A929361cf77eC'
abi: Contract
startBlock: 0
mapping:
kind: ethereum/events
apiVersion: 0.0.7
language: wasm/assemblyscript
entities:
- Block
abis:
- name: Contract
file: ./abis/Contract.json
blockHandlers:
- handler: handleBlock
file: ./src/mappings/blocks.ts
schema.graphql
type Block @entity(timeseries: true) {
# Timeseries must have an id of type Int8
id: Int8!
# Timeseries must have a timestamp field
timestamp: Timestamp!
# The block hash
hash: Bytes!
# The block number
number: Int!
# Price in USD
price: BigInt @aggregatorv3interface(feed: "0x694AA1769357215DE4FAC081bf1f309aDC325306", arg: "number")
}
type Stats @aggregation(intervals: ["hour", "day"], source: "Block") {
# The id; it is the id of one of the data points that were aggregated into
# this bucket, but which one is undefined and should not be relied on
id: Int8!
# The timestamp of the bucket is always the timestamp of the beginning of
# the interval
timestamp: Timestamp!
# The aggregates
count: Int! @aggregate(fn: "count")
maxPrice: BigInt @aggregate(fn: "max", arg: "price")
}
src/mappings/blocks.ts
import { ethereum } from '@graphprotocol/graph-ts';
import { Block } from '../../generated/schema';
export function handleBlock(block: ethereum.Block): void {
// The id for timeseries is autogenerated; even if we set it to a real
// value, it would be silently overwritten
let blockEntity = new Block('auto');
let number = block.number.toI32();
blockEntity.hash = block.hash;
blockEntity.number = number;
blockEntity.timestamp = block.timestamp.toI32();
blockEntity.save();
}
query.graphql
stats(interval: "day",
current: ignore,
where: {
timestamp_gte: 1234567890 }) {
id
timestamp
count
maxPrice
}
Last updated