TLDR
Cryo, by Paradigm, is a tool that is as cool as its name suggests. If you’re venturing into blockchain data, whether you’re a researcher, developer, or just a curious explorer, cryo
is about to become your new best friend. This guide is designed to walk you through what cryo
is, how it works, and how to harness its power to fetch blockchain data quickly.
At its core, cryo
is a command-line interface (CLI) tool, but don’t let the simplicity of its interface fool you. This tool packs a powerful punch, making an easy and flexible way to extract blockchain data into various user-friendly formats. Whether you need your data in Parquet, CSV, JSON, or piped directly into a Python data frame, cryo
has got you covered.
So, whether you’re planning to build complex applications, conduct in-depth research, or satisfy your curiosity about blockchain operations, cryo
will make the process simple and fast. Today, let’s learn how to use cryo
coupled with a high-performance Chainstack Global Node.
Here you can find the cryo repository.
cryo
worksUnderstanding the magic under the hood is always important to utilize a full tool’s potential. This section will give you a glimpse into the inner workings of cryo
, explaining its data extraction process, how it handles data schemas and formatting, and the range of blockchain networks it supports.
cryo
’s primary tool is the JSON-RPC protocol, a widely used standard that allows for communication with a blockchain node.
When you run a cryo
command, it sends out JSON-RPC requests to a blockchain node. These requests ask for specific pieces of data like blocks, transactions, or logs. The node responds with raw data, which cryo
then meticulously processes. It’s not just about fetching data; cryo
transforms this data into structured and readable formats like CSV, JSON, or Parquet files. This transformation makes it incredibly straightforward to use this data in various applications or analyses.
The speed and efficiency of cryo
are standout features, largely attributed to its development in Rust. Renowned for its performance and memory safety, Rust enables cryo
to handle blockchain data with exceptional speed and efficiency. This results in rapid data processing, even when dealing with the large datasets typical in blockchain networks. Rust’s prowess in concurrency further amplifies cryo's
ability to manage multiple data extraction tasks simultaneously, ensuring swift and smooth operation. In short, cryo
leverages Rust’s strengths to offer a fast, reliable, and efficient data extraction experience.
The blockchain world is vast and diverse, and cryo
is built to navigate this diversity. It’s compatible with various blockchain networks, making it a versatile tool for users interested in different ecosystems.
Primarily, cryo
is compatible with Ethereum and supports EVM-based networks. This wide range of compatibility is possible because cryo
utilizes ethers.rs for JSON-RPC requests, allowing it to interact with any chain compatible with ethers-rs. This versatility makes cryo
a valuable asset, whether you’re getting into the bustling world of Ethereum or exploring the unique landscapes of its various Layer 2 solutions and sidechains.
Getting cryo
up and running involves a few straightforward steps. This section will guide you through the prerequisites, the installation process, and setting up essential environment variables. We’ll also set up a Chainstack global endpoint.
Before installing cryo
ensure that your system meets the following requirements:
cryo
is built in Rust, so you must install Rust on your machine. If you haven’t installed Rust, you can do so via rustup, the recommended way to install the Rust programming language.cryo
can be installed either directly from the source or via crates.io
. Here’s how you can do it:
Method 1: Install from source
Clone the cryo
repository:
Navigate to the cryo
directory:
Install cryo
using Cargo:
Method 2: Install from crates.io
Run the install command:
Installing from source has been the most reliable method so far.
cryo
Diving into cryo
begins with understanding its basic and help commands and the variety of data types it can extract. This section will cover the foundational aspects of using cryo
, including the essential commands and options that make it a versatile tool for blockchain data extraction.
cryo
offers several commands to help you navigate its functionalities:
cryo help
: This is your go-to command for any assistance. It provides an overview of all available commands and options in cryo
. Whenever in doubt, just type cryo help
in your terminal.cryo help syntax
: Blockchain data queries can sometimes get complex. The cryo help syntax
command is designed to help you understand how to effectively specify block ranges, transaction hashes, and other query parameters.cryo help datasets
: This command displays all the available datasets that cryo
can extract. Datasets include blocks
, transactions
, logs
, and many others, each serving a specific type of data extraction.cryo help DATASET(S)
: Use this command for detailed information about a specific dataset. It helps you understand the nuances of each dataset, what data it includes, and how it can be used. For instance, if you want to know more about the logs
dataset, you should use cryo help logs
.cryo
can extract various types of blockchain data, each with its own set of applicable options:
To further refine your data extraction, cryo
provides a range of options:
-include-columns
: Specify which columns to include in your output. For instance, if you’re only interested in certain aspects of a transaction, like gas prices and transaction hashes, this option allows you to focus on just those columns.-exclude-columns
: Conversely, if there are columns you want to omit from your output, this option lets you exclude them, streamlining your dataset.-blocks
: A crucial option for specifying the range of blocks you are interested in. You can define a single block, a range, or multiple ranges.-contract
: This option lets you specify a particular contract address when dealing with log-related data.cryo
also includes various other options for output format (--csv
, --json
), sorting (--sort
), and filtering based on transaction parameters. Combining these data types and options gives you a powerful toolkit to customize your data extraction precisely to your needs.
cryo
with a custom RPCTo use cryo
you’ll need a custom RPC (Remote Procedure Call) endpoint. This section will explain what a custom RPC endpoint is and how to use it with cryo
.
An RPC endpoint in the context of blockchain is a server interface that allows you to interact with the blockchain network. It’s like a gateway through which you send requests (like fetching data) and receive responses. So, how do you choose an RPC?
Cryo is a high-performing tool that can send many requests per second, and the Chainstack global node is ideal for this tool.
Chainstack global nodes are geo-load-balanced nodes enabling intelligent routing of requests to the nearest server, reducing latency and delivering maximum performance.
By proactively monitoring node status, global nodes adapt to network conditions in real time providing instant failover to another node during network interruptions on a global scale.
Follow these steps to deploy an Ethereum node:
To follow this guide, deploy a global Ethereum node.
Once you deploy the node, you’ll have access to an RPC endpoint, which will look like this:
Now, we are ready to start fetching some blockchain data.
cryo
To use a custom RPC endpoint with cryo
, you can use the --rpc
flag followed by the URL of your RPC endpoint or add it as an environment variable. Here’s how to do it:
In your cryo
command, add the -rpc
flag followed by your custom RPC URL. For example:
If you want to use it as an environment variable, export it as a variable named ETH_RPC_URL
by running this in the console:
If you add the endpoints as an environment variable, you do not need to add the --rpc
flag when running a command.
At this point, you are ready to get your hands dirty with blockchain data; let’s explore a few kinds of datasets you can get from cryo
and you can fine-tune the requests.
Block information is a fundamental kind of data needed to analyze the blockchain’s state at specific times, which is essential for historical analysis, auditing, and verifying transaction integrity. It is also somewhat resource-intensive, especially with a wide block range.
The basic block data command syntax is:
Let’s explore some cryo
commands to work with blocks.
The --dry
flag in cryo
is useful for previewing the structure and content of the data you plan to extract without actually executing the data extraction.
Using the --dry
command with cryo provides a snapshot of the parameters, source details, output configuration, and the data schema for the requested dataset. This feature is highly beneficial for confirming the data fields and format before running a full data extraction process. It helps in understanding the range of data (like block numbers and types of data points), source information (like network and RPC URL), and how the data will be output (such as the file format and chunk size). This preemptive insight allows users to adjust their query parameters and output settings as needed, ensuring they get the data they need in the desired format. This is especially valuable for large-scale data operations where efficiency and precision are critical.
Run it with:
The output will be similar to the following:
Let’s briefly analyze the response and see what customizations we can make.
From the dry run above, we can see a few details:
schema for blocks
displays what kind of data will be extracted.We can use a few extra flags to customize this:
This command will limit the RPS to 30, return a JSON file, and only keep the block number, block hash, timestamp, and chain ID. Running it will return the following:
Once you verify that’s the data you need, run the same command without the --dry
flag to actually send the requests, you’ll get an output similar to this:
As you can see, it gives us the details of the operation, and you’ll find a JSON file in your output path with the following structure:
Because of the rate limit, it took 34 seconds to complete the request; the good news is that Chainstack does not have any rate limit starting from a paid plan, which is recommended if you need really high-performance nodes.
The following is the result of the same request using a premium Chainstack Global Node with unlimited RPS:
As you can see, this is an enormous improvement, almost 90% faster, and this is the power of this Rust-based tool.
This is the Gist of this powerful tool, and you can follow the same principle for the other datasets, but we’ll explore a couple more.
cryo
Event logs on the blockchain offer invaluable insights into contract interactions and transactions and are one of the most sought-after data types. cryo
simplifies the extraction of these logs, offering two primary approaches:
cryo
comes with built-in capabilities to extract standard event logs like ERC-20 and ERC-721 Transfer
events. This functionality is accessible without needing intricate parameters, significantly streamlining the process.
For example, extracting ERC-20 Transfer
events can be achieved with a straightforward command:
This command will retrieve theTransfer
events from the latest block in JSON format.
You can specify additional parameters for more targeted data retrieval, such as a specific token address or a range of blocks. For instance, to extract Transfer
events of the APE token over a range of 500 blocks, the command would be:
The output is structured as a JSON file like the following:
You can do the same for ERC-721 Transfers
.
cryo
also allows users to extract custom event logs. This feature is useful for analyzing non-standard events or those specific to a particular smart contract.
For example, to fetch custom events, you can use the logs
dataset command with specific topics or event signatures. To extract the same Transfer
events as above using a custom approach, the command would be:
Alternatively, you can directly use the event signature:
Consider extracting data about new token pairs created on SushiSwap V2 for a more complex example. The following command accomplishes this for a specific block:
The extracted transactions are sorted by block number and log index by default. For more examples and custom event extraction scenarios, refer to the cryo
documentation.
The versatility of cryo
extends beyond event logs, encompassing various data types inherent to blockchain technology. Each dataset can be customized and extracted based on user-specific requirements, enabling various analysis and research possibilities. Post extraction, the data can be seamlessly integrated into different frameworks or tools for further processing and analysis, offering a comprehensive solution for blockchain data retrieval.
In this guide, we’ve journeyed through the remarkable capabilities of cryo
, a tool that stands out in blockchain data extraction. Cryo offers a seamless and efficient way to access blockchain data, from its intuitive command-line interface to its powerful Rust-based engine.
Whether you’re getting into complex application development, embarking on in-depth research, or simply exploring the blockchain landscape, cryo
proves to be an indispensable ally. Its ability to interact with various blockchain networks and its flexibility in data formatting and extraction ensures that your blockchain data needs are met with precision and ease.
Integrating cryo
with high-performance RPC endpoints like Chainstack global nodes further elevates its efficiency, providing lightning-fast data retrieval and enhanced reliability. This synergy enables you to harness the full potential of blockchain data, unlocking insights and opportunities that were previously challenging to access.
TLDR
Cryo, by Paradigm, is a tool that is as cool as its name suggests. If you’re venturing into blockchain data, whether you’re a researcher, developer, or just a curious explorer, cryo
is about to become your new best friend. This guide is designed to walk you through what cryo
is, how it works, and how to harness its power to fetch blockchain data quickly.
At its core, cryo
is a command-line interface (CLI) tool, but don’t let the simplicity of its interface fool you. This tool packs a powerful punch, making an easy and flexible way to extract blockchain data into various user-friendly formats. Whether you need your data in Parquet, CSV, JSON, or piped directly into a Python data frame, cryo
has got you covered.
So, whether you’re planning to build complex applications, conduct in-depth research, or satisfy your curiosity about blockchain operations, cryo
will make the process simple and fast. Today, let’s learn how to use cryo
coupled with a high-performance Chainstack Global Node.
Here you can find the cryo repository.
cryo
worksUnderstanding the magic under the hood is always important to utilize a full tool’s potential. This section will give you a glimpse into the inner workings of cryo
, explaining its data extraction process, how it handles data schemas and formatting, and the range of blockchain networks it supports.
cryo
’s primary tool is the JSON-RPC protocol, a widely used standard that allows for communication with a blockchain node.
When you run a cryo
command, it sends out JSON-RPC requests to a blockchain node. These requests ask for specific pieces of data like blocks, transactions, or logs. The node responds with raw data, which cryo
then meticulously processes. It’s not just about fetching data; cryo
transforms this data into structured and readable formats like CSV, JSON, or Parquet files. This transformation makes it incredibly straightforward to use this data in various applications or analyses.
The speed and efficiency of cryo
are standout features, largely attributed to its development in Rust. Renowned for its performance and memory safety, Rust enables cryo
to handle blockchain data with exceptional speed and efficiency. This results in rapid data processing, even when dealing with the large datasets typical in blockchain networks. Rust’s prowess in concurrency further amplifies cryo's
ability to manage multiple data extraction tasks simultaneously, ensuring swift and smooth operation. In short, cryo
leverages Rust’s strengths to offer a fast, reliable, and efficient data extraction experience.
The blockchain world is vast and diverse, and cryo
is built to navigate this diversity. It’s compatible with various blockchain networks, making it a versatile tool for users interested in different ecosystems.
Primarily, cryo
is compatible with Ethereum and supports EVM-based networks. This wide range of compatibility is possible because cryo
utilizes ethers.rs for JSON-RPC requests, allowing it to interact with any chain compatible with ethers-rs. This versatility makes cryo
a valuable asset, whether you’re getting into the bustling world of Ethereum or exploring the unique landscapes of its various Layer 2 solutions and sidechains.
Getting cryo
up and running involves a few straightforward steps. This section will guide you through the prerequisites, the installation process, and setting up essential environment variables. We’ll also set up a Chainstack global endpoint.
Before installing cryo
ensure that your system meets the following requirements:
cryo
is built in Rust, so you must install Rust on your machine. If you haven’t installed Rust, you can do so via rustup, the recommended way to install the Rust programming language.cryo
can be installed either directly from the source or via crates.io
. Here’s how you can do it:
Method 1: Install from source
Clone the cryo
repository:
Navigate to the cryo
directory:
Install cryo
using Cargo:
Method 2: Install from crates.io
Run the install command:
Installing from source has been the most reliable method so far.
cryo
Diving into cryo
begins with understanding its basic and help commands and the variety of data types it can extract. This section will cover the foundational aspects of using cryo
, including the essential commands and options that make it a versatile tool for blockchain data extraction.
cryo
offers several commands to help you navigate its functionalities:
cryo help
: This is your go-to command for any assistance. It provides an overview of all available commands and options in cryo
. Whenever in doubt, just type cryo help
in your terminal.cryo help syntax
: Blockchain data queries can sometimes get complex. The cryo help syntax
command is designed to help you understand how to effectively specify block ranges, transaction hashes, and other query parameters.cryo help datasets
: This command displays all the available datasets that cryo
can extract. Datasets include blocks
, transactions
, logs
, and many others, each serving a specific type of data extraction.cryo help DATASET(S)
: Use this command for detailed information about a specific dataset. It helps you understand the nuances of each dataset, what data it includes, and how it can be used. For instance, if you want to know more about the logs
dataset, you should use cryo help logs
.cryo
can extract various types of blockchain data, each with its own set of applicable options:
To further refine your data extraction, cryo
provides a range of options:
-include-columns
: Specify which columns to include in your output. For instance, if you’re only interested in certain aspects of a transaction, like gas prices and transaction hashes, this option allows you to focus on just those columns.-exclude-columns
: Conversely, if there are columns you want to omit from your output, this option lets you exclude them, streamlining your dataset.-blocks
: A crucial option for specifying the range of blocks you are interested in. You can define a single block, a range, or multiple ranges.-contract
: This option lets you specify a particular contract address when dealing with log-related data.cryo
also includes various other options for output format (--csv
, --json
), sorting (--sort
), and filtering based on transaction parameters. Combining these data types and options gives you a powerful toolkit to customize your data extraction precisely to your needs.
cryo
with a custom RPCTo use cryo
you’ll need a custom RPC (Remote Procedure Call) endpoint. This section will explain what a custom RPC endpoint is and how to use it with cryo
.
An RPC endpoint in the context of blockchain is a server interface that allows you to interact with the blockchain network. It’s like a gateway through which you send requests (like fetching data) and receive responses. So, how do you choose an RPC?
Cryo is a high-performing tool that can send many requests per second, and the Chainstack global node is ideal for this tool.
Chainstack global nodes are geo-load-balanced nodes enabling intelligent routing of requests to the nearest server, reducing latency and delivering maximum performance.
By proactively monitoring node status, global nodes adapt to network conditions in real time providing instant failover to another node during network interruptions on a global scale.
Follow these steps to deploy an Ethereum node:
To follow this guide, deploy a global Ethereum node.
Once you deploy the node, you’ll have access to an RPC endpoint, which will look like this:
Now, we are ready to start fetching some blockchain data.
cryo
To use a custom RPC endpoint with cryo
, you can use the --rpc
flag followed by the URL of your RPC endpoint or add it as an environment variable. Here’s how to do it:
In your cryo
command, add the -rpc
flag followed by your custom RPC URL. For example:
If you want to use it as an environment variable, export it as a variable named ETH_RPC_URL
by running this in the console:
If you add the endpoints as an environment variable, you do not need to add the --rpc
flag when running a command.
At this point, you are ready to get your hands dirty with blockchain data; let’s explore a few kinds of datasets you can get from cryo
and you can fine-tune the requests.
Block information is a fundamental kind of data needed to analyze the blockchain’s state at specific times, which is essential for historical analysis, auditing, and verifying transaction integrity. It is also somewhat resource-intensive, especially with a wide block range.
The basic block data command syntax is:
Let’s explore some cryo
commands to work with blocks.
The --dry
flag in cryo
is useful for previewing the structure and content of the data you plan to extract without actually executing the data extraction.
Using the --dry
command with cryo provides a snapshot of the parameters, source details, output configuration, and the data schema for the requested dataset. This feature is highly beneficial for confirming the data fields and format before running a full data extraction process. It helps in understanding the range of data (like block numbers and types of data points), source information (like network and RPC URL), and how the data will be output (such as the file format and chunk size). This preemptive insight allows users to adjust their query parameters and output settings as needed, ensuring they get the data they need in the desired format. This is especially valuable for large-scale data operations where efficiency and precision are critical.
Run it with:
The output will be similar to the following:
Let’s briefly analyze the response and see what customizations we can make.
From the dry run above, we can see a few details:
schema for blocks
displays what kind of data will be extracted.We can use a few extra flags to customize this:
This command will limit the RPS to 30, return a JSON file, and only keep the block number, block hash, timestamp, and chain ID. Running it will return the following:
Once you verify that’s the data you need, run the same command without the --dry
flag to actually send the requests, you’ll get an output similar to this:
As you can see, it gives us the details of the operation, and you’ll find a JSON file in your output path with the following structure:
Because of the rate limit, it took 34 seconds to complete the request; the good news is that Chainstack does not have any rate limit starting from a paid plan, which is recommended if you need really high-performance nodes.
The following is the result of the same request using a premium Chainstack Global Node with unlimited RPS:
As you can see, this is an enormous improvement, almost 90% faster, and this is the power of this Rust-based tool.
This is the Gist of this powerful tool, and you can follow the same principle for the other datasets, but we’ll explore a couple more.
cryo
Event logs on the blockchain offer invaluable insights into contract interactions and transactions and are one of the most sought-after data types. cryo
simplifies the extraction of these logs, offering two primary approaches:
cryo
comes with built-in capabilities to extract standard event logs like ERC-20 and ERC-721 Transfer
events. This functionality is accessible without needing intricate parameters, significantly streamlining the process.
For example, extracting ERC-20 Transfer
events can be achieved with a straightforward command:
This command will retrieve theTransfer
events from the latest block in JSON format.
You can specify additional parameters for more targeted data retrieval, such as a specific token address or a range of blocks. For instance, to extract Transfer
events of the APE token over a range of 500 blocks, the command would be:
The output is structured as a JSON file like the following:
You can do the same for ERC-721 Transfers
.
cryo
also allows users to extract custom event logs. This feature is useful for analyzing non-standard events or those specific to a particular smart contract.
For example, to fetch custom events, you can use the logs
dataset command with specific topics or event signatures. To extract the same Transfer
events as above using a custom approach, the command would be:
Alternatively, you can directly use the event signature:
Consider extracting data about new token pairs created on SushiSwap V2 for a more complex example. The following command accomplishes this for a specific block:
The extracted transactions are sorted by block number and log index by default. For more examples and custom event extraction scenarios, refer to the cryo
documentation.
The versatility of cryo
extends beyond event logs, encompassing various data types inherent to blockchain technology. Each dataset can be customized and extracted based on user-specific requirements, enabling various analysis and research possibilities. Post extraction, the data can be seamlessly integrated into different frameworks or tools for further processing and analysis, offering a comprehensive solution for blockchain data retrieval.
In this guide, we’ve journeyed through the remarkable capabilities of cryo
, a tool that stands out in blockchain data extraction. Cryo offers a seamless and efficient way to access blockchain data, from its intuitive command-line interface to its powerful Rust-based engine.
Whether you’re getting into complex application development, embarking on in-depth research, or simply exploring the blockchain landscape, cryo
proves to be an indispensable ally. Its ability to interact with various blockchain networks and its flexibility in data formatting and extraction ensures that your blockchain data needs are met with precision and ease.
Integrating cryo
with high-performance RPC endpoints like Chainstack global nodes further elevates its efficiency, providing lightning-fast data retrieval and enhanced reliability. This synergy enables you to harness the full potential of blockchain data, unlocking insights and opportunities that were previously challenging to access.