TLDR
In blockchain data exploration, we previously introduced you to cryo
, Paradigm’s powerful command-line interface tool. As you might recall, this tool is a beacon for developers, researchers, and blockchain enthusiasts, optimizing the process of extracting data from various blockchain networks. Our initial journey through cryo
revealed its data formatting efficiency and seamless integration with Chainstack Global Nodes.
Learn how to use cryo
and how it works with cryo: Your gateway to blockchain data.
Now, we embark on a sequel, bridging cryo
with the world of Python. This guide will show you how to use the Python wrapper made for the cryo
CLI by covering setup, basic usage, and data extraction and manipulation using common Python libraries.
Python is known for its simplicity and data manipulation and analysis capability. The Python wrapper allows you to couple cryo
’s Rust-based efficiency for data extraction with Python’s data manipulation capabilities. This integration enhances the analytical power at your fingertips, allowing you to leverage Python’s rich library ecosystem for in-depth data analysis, visualization, and machine learning.
This section will lay the groundwork for integrating the cryo
tool with Python. This process involves ensuring that your system has the necessary tools and libraries and installing the Python wrapper for cryo
.
Before diving into the installation process, ensure your environment is primed for the task. The following prerequisites are essential:
Follow these steps to deploy an Ethereum node:
To follow this guide, deploy a Standard Ethereum node, which will default to a Global Node.
Once you deploy the node, you’ll have access to an RPC endpoint, which will look like this:
Create a .env
file in your root directory and place the endpoint in it.
cryo
to work, the Python integration is a lightweight wrapper for the cryo
CLI, so you’ll still need to meet the app’s requirements.Install Rust following the rustup instructions.
Python Environment: Ensure that you have Python installed on your system and create a new virtual environment in your project’s directory; you can run the following:
Then activate the virtual environment with:
Required Libraries: cryo_python
depends on several libraries, make sure to install the following libraries,
Note that the python-dotenv web3 matplotlib
libraries are not strictly required to run cryo_python
, but we’ll use them along the guide.
With the prerequisites in place, let’s move on to the installation steps:
Clone the cryo Repository: Use git to clone the cryo
repository from GitHub. If you don’t have git installed, you can download it from git.
Navigate to the Python Directory:
Build cryo_python
:
maturin
build command:
Install the Python Wrapper:
.whl
file generated by maturin. It will be located in the target/wheels
directory.<PATH_TO_WHEEL_FILE>
with the actual path to the .whl
file generated, it will look like this:
Your current draft provides a solid foundation. To enhance it, we can add more context and details based on the source files, particularly focusing on the functionality and technical nuances of cryo.collect()
and cryo.freeze()
. Here’s an improved version:
cryo_python
cryo_python
serves as a lightweight wrapper for the cryo
CLI offers a seamless Python interface to the powerful CLI commands. With cryo_python
users can access two principal functions that mirror their CLI counterparts:
cryo.collect()
extracts blockchain data and returns it as a Python-friendly data frame, enabling direct use within scripts for real-time analysis and manipulation.cryo.freeze()
fetches data and saves it to a file, facilitating subsequent use or long-term storage.Explore the source files for cryo.collect() and cryo.freeze() in the GitHub repository.
cryo.collect()
Main AspectsAsynchronous Support: cryo.collect()
includes both async_collect
and collect
methods, designed to operate asynchronously. This feature is vital for efficiently handling large datasets or high-throughput tasks, ensuring optimal resource utilization and performance.
Multiple Output Formats: cryo.collect()
allows you to organize data in various Python-friendly formats for diverse scenarios:
cryo.freeze()
Main Aspectscryo.freeze()
can handle single and multiple data types, showcasing its versatility in accommodating various data collection needs.cryo.collect()
, cryo.freeze()
also parses additional keyword arguments (*kwargs
), enhancing the customization possibilities in data collection and storage.Having grasped the basics of cryo_python
, let’s get into practical examples to demonstrate its usage. Throughout this guide, we’ll consistently retrieve the RPC endpoint from a .env
file.
Ensure you have your RPC endpoint details in a .env
file for these examples.
cryo.collect
basic exampleStart by creating a file named main.py
and paste the following code:
Here’s an explanation of how it works and what it does:
Environment Setup:
os
for environment variable management, cryo
for accessing blockchain data, and load_dotenv
from the dotenv
package to load environment variables from a .env
file.load_dotenv()
, which reads the .env
file and sets the variables.Accessing Ethereum RPC Endpoint:
ETH_RPC
variable, which contains the URL to an Ethereum RPC endpoint, is fetched from the environment variables using os.getenv("ETH_RPC")
.Data Collection with cryo.collect
:
cryo.collect
function has specific parameters to fetch data from the Ethereum blockchain.datatype
: Set to "blocks"
, indicating that the function should fetch data about blockchain blocks.blocks
: Specifies the range of blocks to fetch data for (in this case, from block 18734050
to 18735050
).rpc
: The Ethereum RPC endpoint URL, passed as eth_rpc
.output_format
: Set to "pandas"
, indicating that the data should be returned as a Pandas DataFrame.hex
: The boolean parameter set to True
will return the data already converted to hexadecimal.Output:
data
, a Pandas DataFrame.data
, showing the fetched blockchain data.The result of this script is a detailed listing of data for the specified range of Ethereum blocks. The DataFrame columns represent each block’s attributes, such as block_hash
, author
, block_number
, gas_used
, extra_data
, timestamp
, base_fee_per_gas
, and chain_id
.
Here is an example of the output in the console:
Running this Python script is the equivalent of running this command from the cryo
CLI directly:
Please note that Chainstack endpoints on the Developer plan are limited to 30 RPS, so you might need to add rate limiting to your code; starting from the Growth plan, there is no rate limit.
To manage rate limits, cryo.collect
can be adjusted using the requests_per_second
parameter:
cryo.freeze
basic exampleThe principle of cryo.freeze
is quite similar to cryo.collect
. In a new file, paste this code:
This script uses cryo.freeze
to fetch and save the same block data as a JSON file in the specified directory. The logic and syntax closely follow the cryo
CLI. The result is a JSON file containing data for the blocks in the root/blocks_data/
directory.
Since both cryo.freeze
and cryo.collect
are just wrappers around the CLI; you can use the same commands. Let’s explore a few more examples.
cryo
This section will guide you in using cryo_python
to retrieve ERC-20 token balances from specified addresses and contracts. We’ll get the balance of the APECoin token in the Binance address in a range of 10,000 blocks.
Start by creating a new Python file and paste the following code:
Executing this script will generate a JSON file containing the ERC-20 balance data structured as follows:
This structure, erc20_balances
efficiently organizes ERC-20 balances by block, offering a clear and accessible format for data analysis.
Check the cryo
documentation to find what other datasets you can fetch.
Having explored the basic functionality of cryo_python
, let’s now get into a more advanced application by integrating it with essential Python libraries for data manipulation and visualization.
In this example, we’ll fetch Ethereum blockchain data and visualize the top block authors using cryo_python
, pandas
, and matplotlib
.
In a Python file, paste the following:
Here’s a step-by-step breakdown of what this script does:
os
, time
, pandas
, matplotlib.pyplot
, and Web3
, along with cryo
. Then, we define constants for the RPC URL, the number of blocks to look back on, and the number of top authors to display.cryo.collect
to get data on these blocks and returns it as a pandas DataFrame. We track the time taken for this operation, offering insights into the performance of our data retrieval process.matplotlib
to create a bar chart, showcasing the top authors based on the number of blocks mined.main
function, we initialize a Web3 instance, connect to the Ethereum node, fetch the block data, and, if successful, visualize the top authors. We handle potential errors, such as missing environment variables or connection issues, to ensure robustness.This example demonstrates how to effectively combine cryo
with other Python tools to fetch, process, and visualize Ethereum blockchain data, providing valuable insights into blockchain activity.
Here is an example of the console output and chart. The console will output something like the following:
And the chart will look like this:
The next example we’ll work on will use the same erc20_balances
dataset used in one of the previous examples. This time, we’ll fetch and visualize how much WETH is in theWETH-USDT
pool from Uniswap V2.
In a new file, paste the following code:
Here’s a step-by-step explanation of what’s going on:
Fetch Block Range:
Fetch ERC-20 Balances:
cryo.collect
function is called, and the data is returned in a pandas DataFrame format.Data Conversion and Cleaning:
None
values to avoid errors during conversion.Summarizing Data:
Data Visualization:
matplotlib
. The x-axis represents block numbers, and the y-axis represents the balance in Ether.Remember to adapt the request per second.
Here is an example of the result:
Graph for the balance change over a day:
As you can see, we can use cryo
to fetch data and manipulate it with Python, a very powerful combo.
The integration of cryo
with Python is a significant advancement for blockchain data analysis. It combines cryo
’s efficient data extraction capabilities with Python’s powerful data processing and visualization tools. This synergy, coupled with high-performance Chainstack Global Nodes, enables users to easily extract, analyze, and visualize blockchain data, making it an invaluable resource for developers, researchers, and enthusiasts in the blockchain community. The practical examples demonstrate this integration’s real-world utility, highlighting its potential to yield insightful and actionable information from complex blockchain datasets. In essence, cryo
and Python offer an effective and accessible platform for in-depth blockchain data exploration.
TLDR
In blockchain data exploration, we previously introduced you to cryo
, Paradigm’s powerful command-line interface tool. As you might recall, this tool is a beacon for developers, researchers, and blockchain enthusiasts, optimizing the process of extracting data from various blockchain networks. Our initial journey through cryo
revealed its data formatting efficiency and seamless integration with Chainstack Global Nodes.
Learn how to use cryo
and how it works with cryo: Your gateway to blockchain data.
Now, we embark on a sequel, bridging cryo
with the world of Python. This guide will show you how to use the Python wrapper made for the cryo
CLI by covering setup, basic usage, and data extraction and manipulation using common Python libraries.
Python is known for its simplicity and data manipulation and analysis capability. The Python wrapper allows you to couple cryo
’s Rust-based efficiency for data extraction with Python’s data manipulation capabilities. This integration enhances the analytical power at your fingertips, allowing you to leverage Python’s rich library ecosystem for in-depth data analysis, visualization, and machine learning.
This section will lay the groundwork for integrating the cryo
tool with Python. This process involves ensuring that your system has the necessary tools and libraries and installing the Python wrapper for cryo
.
Before diving into the installation process, ensure your environment is primed for the task. The following prerequisites are essential:
Follow these steps to deploy an Ethereum node:
To follow this guide, deploy a Standard Ethereum node, which will default to a Global Node.
Once you deploy the node, you’ll have access to an RPC endpoint, which will look like this:
Create a .env
file in your root directory and place the endpoint in it.
cryo
to work, the Python integration is a lightweight wrapper for the cryo
CLI, so you’ll still need to meet the app’s requirements.Install Rust following the rustup instructions.
Python Environment: Ensure that you have Python installed on your system and create a new virtual environment in your project’s directory; you can run the following:
Then activate the virtual environment with:
Required Libraries: cryo_python
depends on several libraries, make sure to install the following libraries,
Note that the python-dotenv web3 matplotlib
libraries are not strictly required to run cryo_python
, but we’ll use them along the guide.
With the prerequisites in place, let’s move on to the installation steps:
Clone the cryo Repository: Use git to clone the cryo
repository from GitHub. If you don’t have git installed, you can download it from git.
Navigate to the Python Directory:
Build cryo_python
:
maturin
build command:
Install the Python Wrapper:
.whl
file generated by maturin. It will be located in the target/wheels
directory.<PATH_TO_WHEEL_FILE>
with the actual path to the .whl
file generated, it will look like this:
Your current draft provides a solid foundation. To enhance it, we can add more context and details based on the source files, particularly focusing on the functionality and technical nuances of cryo.collect()
and cryo.freeze()
. Here’s an improved version:
cryo_python
cryo_python
serves as a lightweight wrapper for the cryo
CLI offers a seamless Python interface to the powerful CLI commands. With cryo_python
users can access two principal functions that mirror their CLI counterparts:
cryo.collect()
extracts blockchain data and returns it as a Python-friendly data frame, enabling direct use within scripts for real-time analysis and manipulation.cryo.freeze()
fetches data and saves it to a file, facilitating subsequent use or long-term storage.Explore the source files for cryo.collect() and cryo.freeze() in the GitHub repository.
cryo.collect()
Main AspectsAsynchronous Support: cryo.collect()
includes both async_collect
and collect
methods, designed to operate asynchronously. This feature is vital for efficiently handling large datasets or high-throughput tasks, ensuring optimal resource utilization and performance.
Multiple Output Formats: cryo.collect()
allows you to organize data in various Python-friendly formats for diverse scenarios:
cryo.freeze()
Main Aspectscryo.freeze()
can handle single and multiple data types, showcasing its versatility in accommodating various data collection needs.cryo.collect()
, cryo.freeze()
also parses additional keyword arguments (*kwargs
), enhancing the customization possibilities in data collection and storage.Having grasped the basics of cryo_python
, let’s get into practical examples to demonstrate its usage. Throughout this guide, we’ll consistently retrieve the RPC endpoint from a .env
file.
Ensure you have your RPC endpoint details in a .env
file for these examples.
cryo.collect
basic exampleStart by creating a file named main.py
and paste the following code:
Here’s an explanation of how it works and what it does:
Environment Setup:
os
for environment variable management, cryo
for accessing blockchain data, and load_dotenv
from the dotenv
package to load environment variables from a .env
file.load_dotenv()
, which reads the .env
file and sets the variables.Accessing Ethereum RPC Endpoint:
ETH_RPC
variable, which contains the URL to an Ethereum RPC endpoint, is fetched from the environment variables using os.getenv("ETH_RPC")
.Data Collection with cryo.collect
:
cryo.collect
function has specific parameters to fetch data from the Ethereum blockchain.datatype
: Set to "blocks"
, indicating that the function should fetch data about blockchain blocks.blocks
: Specifies the range of blocks to fetch data for (in this case, from block 18734050
to 18735050
).rpc
: The Ethereum RPC endpoint URL, passed as eth_rpc
.output_format
: Set to "pandas"
, indicating that the data should be returned as a Pandas DataFrame.hex
: The boolean parameter set to True
will return the data already converted to hexadecimal.Output:
data
, a Pandas DataFrame.data
, showing the fetched blockchain data.The result of this script is a detailed listing of data for the specified range of Ethereum blocks. The DataFrame columns represent each block’s attributes, such as block_hash
, author
, block_number
, gas_used
, extra_data
, timestamp
, base_fee_per_gas
, and chain_id
.
Here is an example of the output in the console:
Running this Python script is the equivalent of running this command from the cryo
CLI directly:
Please note that Chainstack endpoints on the Developer plan are limited to 30 RPS, so you might need to add rate limiting to your code; starting from the Growth plan, there is no rate limit.
To manage rate limits, cryo.collect
can be adjusted using the requests_per_second
parameter:
cryo.freeze
basic exampleThe principle of cryo.freeze
is quite similar to cryo.collect
. In a new file, paste this code:
This script uses cryo.freeze
to fetch and save the same block data as a JSON file in the specified directory. The logic and syntax closely follow the cryo
CLI. The result is a JSON file containing data for the blocks in the root/blocks_data/
directory.
Since both cryo.freeze
and cryo.collect
are just wrappers around the CLI; you can use the same commands. Let’s explore a few more examples.
cryo
This section will guide you in using cryo_python
to retrieve ERC-20 token balances from specified addresses and contracts. We’ll get the balance of the APECoin token in the Binance address in a range of 10,000 blocks.
Start by creating a new Python file and paste the following code:
Executing this script will generate a JSON file containing the ERC-20 balance data structured as follows:
This structure, erc20_balances
efficiently organizes ERC-20 balances by block, offering a clear and accessible format for data analysis.
Check the cryo
documentation to find what other datasets you can fetch.
Having explored the basic functionality of cryo_python
, let’s now get into a more advanced application by integrating it with essential Python libraries for data manipulation and visualization.
In this example, we’ll fetch Ethereum blockchain data and visualize the top block authors using cryo_python
, pandas
, and matplotlib
.
In a Python file, paste the following:
Here’s a step-by-step breakdown of what this script does:
os
, time
, pandas
, matplotlib.pyplot
, and Web3
, along with cryo
. Then, we define constants for the RPC URL, the number of blocks to look back on, and the number of top authors to display.cryo.collect
to get data on these blocks and returns it as a pandas DataFrame. We track the time taken for this operation, offering insights into the performance of our data retrieval process.matplotlib
to create a bar chart, showcasing the top authors based on the number of blocks mined.main
function, we initialize a Web3 instance, connect to the Ethereum node, fetch the block data, and, if successful, visualize the top authors. We handle potential errors, such as missing environment variables or connection issues, to ensure robustness.This example demonstrates how to effectively combine cryo
with other Python tools to fetch, process, and visualize Ethereum blockchain data, providing valuable insights into blockchain activity.
Here is an example of the console output and chart. The console will output something like the following:
And the chart will look like this:
The next example we’ll work on will use the same erc20_balances
dataset used in one of the previous examples. This time, we’ll fetch and visualize how much WETH is in theWETH-USDT
pool from Uniswap V2.
In a new file, paste the following code:
Here’s a step-by-step explanation of what’s going on:
Fetch Block Range:
Fetch ERC-20 Balances:
cryo.collect
function is called, and the data is returned in a pandas DataFrame format.Data Conversion and Cleaning:
None
values to avoid errors during conversion.Summarizing Data:
Data Visualization:
matplotlib
. The x-axis represents block numbers, and the y-axis represents the balance in Ether.Remember to adapt the request per second.
Here is an example of the result:
Graph for the balance change over a day:
As you can see, we can use cryo
to fetch data and manipulate it with Python, a very powerful combo.
The integration of cryo
with Python is a significant advancement for blockchain data analysis. It combines cryo
’s efficient data extraction capabilities with Python’s powerful data processing and visualization tools. This synergy, coupled with high-performance Chainstack Global Nodes, enables users to easily extract, analyze, and visualize blockchain data, making it an invaluable resource for developers, researchers, and enthusiasts in the blockchain community. The practical examples demonstrate this integration’s real-world utility, highlighting its potential to yield insightful and actionable information from complex blockchain datasets. In essence, cryo
and Python offer an effective and accessible platform for in-depth blockchain data exploration.