Chainstack Compare: Revolutionizing RPC node performance analysis in blockchains

TLDR:

• Chainstack Compare emphasizes data-fetching efficiency (blocks processed per second) to give developers a deeper insight into RPC node performance.
• By leveraging Python’s concurrency features, it surpasses traditional metrics that mainly focus on latency and requests per second.
• Benchmarks demonstrate dramatic gains in fetching large amounts of data – crucial for data-intensive Web3 applications.
• The tool’s focus on real-world conditions, rate limits, and hardware considerations makes it a practical and more accurate solution for evaluating and selecting RPC providers.

​

Main article

In the evolving world of blockchain, the efficiency of RPC nodes plays an important role in powering data-hungry decentralized applications (DApps). However, traditional metrics for evaluating these nodes often fail to provide a comprehensive view of their capabilities, especially regarding data fetching efficiency—a critical aspect for any blockchain application. This gap in assessment tools led to the development of Chainstack Compare, a cutting-edge Node Performance Tool designed to address this challenge.

Chainstack Compare offers a unique approach to measuring RPC node performance, focusing on the efficiency of data fetching from blockchain networks. This tool analyzes how EVM-compatible RPC nodes handle real-time blockchain data retrieval, going beyond the standard metrics of latency and requests per second.

In this article, we’ll jump into the world of RPC nodes, uncover the limitations of traditional performance metrics, and introduce the innovative methodology behind Chainstack Compare.

​

The need for a specialized tool—Chainstack Compare

Traditional methods to test RPC nodes’ performance primarily focus on measuring latency and the number of requests a node can handle per second. While these metrics are useful, they must fully capture the node’s ability to handle complex data-fetching tasks essential for blockchain applications. This limitation becomes particularly evident in applications where the goal is fetching and handling large amounts of data.

​

Importance of data fetching performance

The true test of an RPC node’s performance lies in its ability to handle requests efficiently and its capacity to manage and fetch substantial volumes of data. This capability is particularly crucial for DApps that act as data aggregators, API providers, or some institutional users that need to ingest data directly from the source, where the sheer volume and complexity of the data being processed are significantly higher.

Traditional performance metrics fall short for such applications. They need to assess how well a node can handle data-fetching operations adequately.

​

Introducing Chainstack Compare

Chainstack Compare innovatively bridges the gap in traditional performance analysis by introducing a pivotal metric: the rate of blocks processed per second, BPS. This metric goes beyond surface-level assessments, offering a deeper insight into a node’s operational efficiency in real-world conditions.

Chainstack Compare stands as a crucial asset for developers. It empowers them with a more refined and accurate evaluation of RPC node performance, specifically tailored to the needs of data-intensive DApps.

​

How Chainstack Compare works

In this article section, we will get into the technical workings of Chainstack Compare, focusing on the underlying methodology and logic that powers its node performance tests. This exploration will explain Chainstack Compare’s approach to evaluating RPC nodes, particularly in fetching and processing blockchain data.

​

Overview of the testing approach behind Chainstack Compare

Chainstack Compare uses a comprehensive testing approach to assess the performance of EVM-compatible RPC nodes. The core of this evaluation lies in its ability to fetch and process large amounts of blockchain data, specifically focusing on retrieving the latest 127 blocks. This task concerns quantity and replicating realistic scenarios that an RPC node might encounter in a live blockchain environment.

​

Why we chose the path of Chainstack Compare

In the journey to create a better way of testing RPC nodes for data ingestion performance, we quickly learned that there’s more to it than just the nodes themselves. The type of architecture and programming language plays a big role, too. This led to a couple of key decisions:

• Using Python for better data fetching: We all know fetching data loads can be tedious, especially with those endless for loops. That’s why we decided to use Python’s concurrency features. It’s not just about speeding things up; it’s about making the process smoother and more manageable. Sure, languages like Rust might offer more efficiency, but many of us are familiar with Python. It’s user-friendly and doesn’t scare away beginners.
• Going big on data collection: We wanted to mimic a real scenario by fetching a hefty amount of data across various blocks. It’s like a test drive for data ingestion apps that need to pull info from blockchains. This way, we get a clearer picture of how well an RPC node performs under pressure, similar to what developers face in the real world.
• Looking beyond just fetching data: It’s not only about how fast or how much data you can fetch. We also looked into the limitations, like rate limits providers set up. Chainstack Compare doesn’t just tell you how effective an RPC node is; it also shows if any rate limits were triggered and how it affected the fetching process, including the percentage of failed attempts.

By taking this route, we’d like to shed some light on the whole process, making it easier for developers to understand what works best when dealing with blockchain data. It’s about finding that sweet spot between technical efficiency and practical usability.

​

Technical logic behind Chainstack Compare

At the heart of Chainstack Compare’s testing process is a Python-based system that leverages asynchronous programming and multithreading to optimize data retrieval and processing. This approach allows the tool to efficiently handle multiple tasks concurrently, significantly enhancing the speed and accuracy of data fetching.

1. Asynchronous and multithreaded processing: Chainstack Compare employs asynchronous programming alongside a ThreadPoolExecutor for multithreading. This combination enables the tool to initiate and manage multiple block data fetch operations simultaneously, minimizing wait times and maximizing the throughput of data processing.
2. Testing various RPC methods: Chainstack Compare tests how an RPC node performs with various RPC methods. At launch, it will include eth_getblockByNumber, eth_call, and debug_traceBlockByNumber.
3. Performance metrics calculation: Post data retrieval, Chainstack Compare calculates key metrics such as blocks per second, whether all the data was processed correctly, and if any rate limit was triggered. These metrics offer insights into the node’s efficiency in processing blockchain data, responsiveness, and capability to handle high-demand scenarios typical in blockchain applications.

​

Comparative analysis of fetching methods: Python concurrency vs. traditional JavaScript

To highlight the effectiveness of Chainstack Compare’s methodology, we conducted benchmarks using Viem, a JavaScript Web3 library. We explored traditional JavaScript methods versus Python’s concurrency approach, which is foundational to Chainstack Compare.

​

Traditional methods in JavaScript with Viem

• Regular looping: Using Viem with a standard for loop in JavaScript, fetching 100 blocks showed a blocks per second (BPS) rate of approximately 3.1; in this case, the RPC node used does not make any difference. This approach clearly shows the limitations of synchronous processing in data-intensive tasks.
• Asynchronous concurrency: Implementing an asynchronous approach with Viem yielded a significant improvement, peaking at 47.68 BPS and fetching 100 blocks. Despite this enhancement, JavaScript’s model of async concurrency, lacking true multi-threading, revealed an upper limit in performance; with this approach, the RPC node makes a difference, but not substantial, as it would not exceed 47/48 BPS.

Due to those results, I decided to use concurrency in Python as it could yield better performance.

​

Python concurrency: Elevating performance

After exploring traditional JavaScript fetching methods with Viem, we turned our focus to the concurrency features of Python to assess their impact on performance. The results shed light on Python’s significant advantages in efficiently managing large-scale data fetching tasks.

​

Benchmark results using Python

Our benchmarks were specifically designed to fetch varying numbers of blocks, aiming to understand how the volume of data and the configuration of concurrent workers affect performance. For these tests, we utilized 60 workers to optimize task execution. Here’s what we discovered:

• Fetching 10 blocks: Completed in 0.764 seconds, resulting in a rate of 13.09 blocks per second (BPS). This baseline test demonstrates Python’s capability to quickly process smaller data sets, benefiting from the efficient management of concurrent workers.
• Fetching 100 blocks: The processing time was significantly reduced compared to the JavaScript approach to 1.367 seconds, pushing the rate to 73.14 BPS. Compared to the slower JavaScript approach, this marked improvement underscores Python’s efficiency in scaling with data volume and worker allocation.
• Fetching 250 blocks: This test achieved a performance of 108.76 BPS within just 2.299 seconds, highlighting Python’s exceptional ability to manage intermediate data sets efficiently, leveraging the optimal number of workers.
• Fetching 500 blocks: As the data volume increased, the processing time of 3.143 seconds for a rate of 159.07 BPS exemplified Python’s robustness in handling large-scale data-fetching tasks, with worker count playing a key role in maximizing efficiency.
• Fetching 1000 blocks: In our most comprehensive test, the process was completed in 5.682 seconds, achieving a groundbreaking rate of 175.99 BPS. This outcome showcased Python’s unparalleled scalability and performance, even with massive data sets, attributed to the strategic use of 60 workers.

​

Benchmark overview

Our benchmark tests clearly show that two main factors impact the efficiency of data processing tasks: the volume of data being processed (measured in blocks) and how many workers are running concurrently.

The hardware used in these tests—specifically, the CPU’s multi-core design and fast memory access significantly reduces the overhead of managing and switching between multiple concurrent tasks, enhancing the throughput of block-fetching operations.

This finding highlights the need to consider the hardware to optimize data processing tasks.

Solutions like Chainstack Compare can illuminate how RPC nodes distribute data and the varying performance levels across providers by considering the setup of concurrent workers and making the most of certain hardware features.

​

Overview of Chainstack Compare

The deployed version of Chainstack Compare runs on more basic hardware. Currently, a machine running 2 vCPU and up to 2 GB of RAM. It highlights the performance difference between RPC nodes but displays less impressive BPS figures.

In evaluating the performance of an RPC node, the ability to rapidly fetch data is crucial but not the only consideration. This becomes particularly relevant in scenarios where a provider imposes stringent rate-limiting. To address this, Chainstack Compare adopts a comprehensive approach to testing RPC nodes that is both realistic and practical, focusing on several key aspects:

• Ensuring users understand the tool’s architecture and how its integration with specific hardware impacts data retrieval capabilities.
• Fetching a specific quantity of actual data to provide tangible performance metrics, in this case, a certain number of blocks in the past.
• Measuring how fast an RPC node can retrieve data from the blockchain, quantified by the number of blocks per second it can process and return to the user.
• Managing rate-limiting errors and presenting users with data that includes the proportion of unsuccessful requests.

This approach allows Chainstack Compare to offer nuanced insights into RPC node performance, considering the complex interplay between software capabilities, hardware limitations, and provider restrictions.

​

Conclusion

Chainstack Compare offers a deep dive into RPC node performance that traditional metrics often overlook. Throughout this article, we’ve explored its unique approach to evaluating data fetching efficiency, a critical component for the smooth operation of DApps.

By integrating Python concurrency features, Chainstack Compare highlights the limitations of conventional methods and sets a new standard in node performance analysis. This tool is a testament to the importance of precision and adaptability in the blockchain domain, empowering developers with the insights needed to optimize DApps for the challenges of tomorrow.