The QUIC protocol, introduced in version 0.13 of the Solana validator, plays a crucial role in maintaining a stable data path between nodes. Its primary purpose is to channel critical transaction and block-related data, ensuring efficient communication across the network. Both peer-to-peer (P2P) validator nodes and remote procedure call (RPC) validator nodes utilize the QUIC protocol to enhance speed and reliability. For P2P nodes, QUIC is activated after node information is exchanged via the Gossip Protocol’s ContactInfo sharing, at which point the faster endpoint is enabled. This article explores the mechanics of the QUIC protocol and its significance in blockchain node communications.

Core Functions of the QUIC Protocol

At its core, QUIC facilitates a stable communication channel between nodes, making it an essential protocol for data-heavy operations such as block transfers and complex transactions. It generally operates on port 8001 and offers substantial performance benefits over traditional TCP, especially in handling large volumes of data. By enabling QUIC, validator nodes can synchronize faster, reducing the likelihood of lag behind other nodes.

However, the protocol also has its limitations. If QUIC is disabled, nodes must rely solely on the Gossip Protocol for synchronization, leading to slower performance. On the flip side, if Gossip is disabled and only QUIC is enabled, the node won’t receive essential network information like ContactInfo and vote transactions. Thus, an optimal setup often involves leveraging both QUIC and Gossip for comprehensive and high-performance communication.

QUIC vs. Gossip Protocol

The QUIC protocol excels at accelerating the exchange of blocks and transactions between validator nodes. However, the Gossip Protocol remains essential for exchanging supplementary data, such as ContactInfo and vote transactions. These two protocols work in tandem: QUIC handles the bulk of large data transfers, while Gossip ensures the smooth dissemination of smaller, yet critical, network information. Therefore, enabling both protocols allows a node to benefit from fast synchronization without losing access to the broader network context provided by Gossip.

SWQoS (Stake-Weighted Quality of Service)

Stake-Weighted Quality of Service (SWQoS) is a critical component of the QUIC protocol, determining how network resources are allocated based on a node’s stake in the network. This concept prioritizes high-stake nodes by reserving resources for them, ensuring they receive reliable service even in times of network congestion.
Two key parameters define connection limits in QUIC:

• MAX_STAKED_CONNECTIONS: 2,000 for staked nodes.

• MAX_UNSTAKED_CONNECTIONS: 500 for unstaked nodes.

It is important to note that SWQoS doesn’t manage the number of connections but the number of streams per connection. This distinction is essential, as streams carry the data, and the protocol ensures that higher-stake nodes can utilize more streams, thus optimizing their bandwidth.

Deeper Dive on QUIC

Now, lets explore the main components of QUIC.

Stream Allocation and Throttling in QUIC

The QUIC protocol handles data streams as a resource, distributing them based on a node’s stake. A total pool of 100,000 streams is available, and these are allocated according to each peer’s stake. This allocation mechanism ensures that staked nodes, which contribute more to the network, receive a proportionally larger share of available streams.

For example, in the code (lines 294-302), the number of streams is assigned based on the peer node’s stake, ensuring that RPC nodes marked as staked (using the “--staked-mapped-id” flag) receive the correct allocation. However, careful configuration of the “--staked-mapped-id” flag is necessary, as improper use could result in a single node dominating the traffic, disrupting network balance.

In the code snippet above, per connection, a receiver buffer is calculated based on the stake of the connected peer. It is calculated by multiplying a ratio with a DATA_PACKET_SIZE i.e. 1232 bytes payload, with a minimum of ~157 KB window per receiver which is mostly used for unstaked and low staked peers, and a maximum of  ~631 KB. In QUIC this receive window size is shared between multiple QUIC streams upon a single QUIC connection.

As seen from the image above, blue transactions belong to the node with max stake in the network so more transactions land in the leader mempool/TPU area. Similarly, followed by the green, yellow, and red stakes. The left side of the image is an abstract version of SWQoS and the right side is independent of SWQoS and based on priority fee transaction inclusion.

Buffering and Window Size Calculations

Buffering plays a crucial role in optimizing QUIC’s performance. Each node’s receiver buffer is calculated based on its stake, which determines the window size it can utilize for incoming data. For example, a low-staked node might receive a window of approximately 157 KB, while a high-staked node could receive up to 631 KB.

This window size is shared across multiple streams within a single QUIC connection, ensuring efficient use of resources. The QUIC protocol dynamically adjusts these buffer sizes, allowing it to balance traffic between high- and low-staked nodes effectively.

Throttling and Load Management 

To prevent any single node from consuming excessive resources, QUIC employs a throttling mechanism. The Solana validator protocol monitors network load and throttles connections based on stream type, ensuring fair distribution of resources. This is achieved through the use of an exponential moving average (EMA), which updates every 5 ms over 10 intervals, providing an accurate measure of the network load over a 50 ms window.

For unstaked nodes, streams are limited to 20% of the network’s capacity, regardless of congestion. Meanwhile, staked nodes share the remaining 80%, with their allocation dynamically adjusted based on network conditions. This ensures that even during congestion, staked nodes can operate efficiently without overwhelming the network.

Flow is as follows:

1. Initial parameters regarding the streams are passed, they are configurations, some are fixed constants, and others are dynamically calculated during the runtime.

2. When a staked connection opens a stream, the load is added to the pool of available capacity. And EMA is updated if conditions are met.

3. When an unstaked node tries to open a stream, it shares the fixed quota and not the available load pool like the staked node logic in point 2.

4. If the peer is trying to overload the system then a backoff sleep time is implemented to ease off the load, aka throttling. If there is no congestion, the staked nodes can have more than their share, but unstaked nodes are still constrained by the fixed quota irrespective of network load.

5. Also, at regular intervals, the throttle counter is reset.

   
   

Congestion Handling

In periods of network congestion, QUIC’s load management becomes critical. If a peer node attempts to open too many streams or overburden the system, a backoff mechanism is triggered, forcing the node to “sleep” for a short period before it can reattempt communication. This throttling helps alleviate strain on the network while maintaining service quality for all participants.

Additionally, when congestion subsides, staked nodes can utilize more streams than initially allocated, as long as they do not exceed their fair share. In contrast, unstaked nodes remain bound to their fixed quota, regardless of network load, ensuring they do not monopolize resources.

Conclusion

The QUIC protocol provides a powerful and efficient method for handling node communications in blockchain networks, particularly in environments where data transfers and block synchronization are paramount. By incorporating the Stake-Weighted Quality of Service (SWQoS) model, QUIC ensures that nodes are prioritized based on their contribution to the network, promoting fairness and stability.

To maintain an optimal balance between performance and network integrity, it is critical to enable both QUIC and the Gossip Protocol. This combination ensures fast, reliable synchronization without sacrificing access to essential network information. As the Solana network continues to evolve, the QUIC protocol will remain a cornerstone of efficient and secure node communication.