Introduction
s2n-quic is a Rust implementation of the IETF QUIC protocol.
User Guide
If you are wanting to use s2n-quic in your application, you can find documentation on how to do that in the user guide.
Developer Guide
If you are wanting to contribute and/or learn about implementation details of s2n-quic, please read the developer guide.
License
s2n-quic source code and documentation is released under the Apache 2.0 License.
Installation
Contributing Guidelines
Thank you for your interest in contributing to our project. Whether it's a bug report, new feature, correction, or additional documentation, we greatly value feedback and contributions from our community.
Please read through this document before submitting any issues or pull requests to ensure we have all the necessary information to effectively respond to your bug report or contribution.
Reporting Bugs/Feature Requests
We welcome you to use the GitHub issue tracker to report bugs or suggest features.
When filing an issue, please check existing open, or recently closed, issues to make sure somebody else hasn't already reported the issue. Please try to include as much information as you can. Details like these are incredibly useful:
- A reproducible test case or series of steps
- The version of our code being used
- Any modifications you've made relevant to the bug
- Anything unusual about your environment or deployment
Contributing via Pull Requests
Contributions via pull requests are much appreciated. Before sending us a pull request, please ensure that:
- You are working against the latest source on the main branch.
- You check existing open, and recently merged, pull requests to make sure someone else hasn't addressed the problem already.
- You open an issue to discuss any significant work - we would hate for your time to be wasted.
To send us a pull request, please:
- Fork the repository.
- Modify the source; please focus on the specific change you are contributing. If you also reformat all the code, it will be hard for us to focus on your change.
- Ensure local tests pass.
- We follow the Conventional Commits guidance for commit messages and pull request titles.
- Send us a pull request and answer the default questions in the pull request interface.
- Pay attention to any automated CI failures reported in the pull request, and stay involved in the conversation.
GitHub provides additional document on forking a repository and creating a pull request.
Finding contributions to work on
Looking at the existing issues is a great way to find something to contribute on. As our projects, by default, use the default GitHub issue labels (enhancement/bug/duplicate/help wanted/invalid/question/wontfix), looking at any 'help wanted' issues is a great place to start.
Code of Conduct
This project has adopted the Amazon Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact opensource-codeofconduct@amazon.com with any additional questions or comments.
Security issue notifications
If you discover a potential security issue in s2n-quic we ask that you notify AWS Security via our vulnerability reporting page. Please do not create a public github issue.
If you package or distribute s2n-quic, or use s2n-quic as part of a large multi-user service, you may be eligible for pre-notification of future s2n-quic releases. Please contact s2n-pre-notification@amazon.com.
Licensing
s2n-quic source code and documentation is released under the Apache 2.0 License. We will ask you to confirm the licensing of your contribution.
We may ask you to sign a Contributor License Agreement (CLA) for larger changes.
Setup
Prerequisites
- GCC / clang (some CC). Installation of these items depends on your package manager.
- Install rustup
# Install components to test and analyze code
rustup component add rustfmt clippy rust-analysis
# Install the nightly toolchain for testing
rustup toolchain install nightly
s2n-quic Continuous Integration
s2n-quic executes a comprehensive suite of tests, linters, benchmarks, simulations and other checks on each pull request and merge into main. Information about these checks is provided below.
Tests
s2n-quic defines many tests that can be executed with cargo test. These tests, described below, are executed across a variety of operating systems, architectures, and Rust versions.
Unit Tests
Unit tests validate the expected behavior of individual components of s2n-quic. Typically, unit tests will be located in a tests module at the end of the file being tested. When there are a significant number of unit test functions for given component, the tests module may be located in a separate file.
Integration Tests
s2n-quic integration tests use the public API to validate the end-to-end behavior of the library under specific scenarios and configurations. Integration tests are located in the top-level s2n-quic crate, in the tests module.
Snapshot Tests
Snapshot tests use insta to assert complex output remains consistent with expected references values. In s2n-quic, snapshot tests are typically constructed in one of two ways:
- using the
insta::assert_debug_snapshotmacro to compare theDebugrepresentation of an instance to the snapshot - using the
event::testing::Publisher::snapshot()event publisher to assert against a snapshot of events emitted by thes2n-quicevent framework
Property Tests
Property tests assert that specific properties of the output of functions and components are upheld under a variety of inputs. s2n-quic uses the Bolero property-testing framework to execute property-testing with multiple fuzzing engines as well as the Kani Rust Verifier. For more details on how Bolero and Kani are used together in s2n-quic, see the following blog posts from the Kani Rust Verifier Blog:
- From Fuzzing to Proof: Using Kani with the Bolero Property-Testing Framework
- How s2n-quic uses Kani to inspire confidence
Fuzz Tests
Fuzz tests provide large amounts of varied input data to assert s2n-quic behaves as expected regardless of the input. Fuzz testing in s2n-quic comes in three flavors:
- Component-level fuzz testing using the Bolero property-testing framework. These tests generate a corpus of inputs that is included in the
s2n-quicrepository to allow the fuzz tests to be replayed when executingcargo test. - End-to-end QUIC protocol-level fuzzing using quic-attack.
quic-attackis a collection of features that collectively turns2n-quicinto an online QUIC protocol fuzzer. It allows incoming and outgoing datagrams and packets, as well as the port number on incoming datagrams, to be intercepted and manipulated. - UDP protocol-level fuzzing using udp-attack.
udp-attackgenerates random UDP packets and transmits them tos2n-quicto catch issues with packet handling.
Concurrency Permutation Tests
loom is used to validate the behavior of concurrent code in s2n-quic. loom executes concurrent code using a simulation of the operating system scheduler and Rust memory model to evaluate concurrent code under all possible thread interleavings.
Interoperability
The quic-interop-runner defines a suite of test cases that ensure compatibility between QUIC implementations. s2n-quic publishes a report with the results.
Compliance
s2n-quic annotates source code with inline references to requirements in IETF RFC specifications. Duvet is used to generate a report, which makes it easy to track compliance with each requirement.
Simulations
A Monte Carlo simulation tool is used to execute thousands of randomized simulations of s2n-quic that vary one or more network variables, such as bandwidth, jitter, and round trip time. The report output provides a visual representation of the relationship between the input variables and overall performance.
A loss recovery simulation tool plots the growth of the congestion window over time under various simulated loss scenarios and publishes the results in a report to visualize changes to the congestion control algorithm.
Performance & Efficiency Profiling
Flame graphs are generated and published in a report to visualize stack traces produced under a variety of data transfer scenarios.
dhat performs heap profiling and publishes the results in a report.
Clippy
clippy is a rust linter which catches common mistakes.
Rustfmt
rustfmt ensures code is consistently formatted.
Miri
Miri detects undefined behavior and memory leaks.
Code Coverage
LLVM source-based code coverage measures how much of s2n-quic code is executed by tests. s2n-quic publishes a report with the results.
Kani
s2n-quic uses the Kani Rust Verifier tool for verifying properties throughout various places in the codebase. The Kani test harnesses are written using bolero, which is also capable of running them as concrete tests.
Getting started
First, you will need make sure you install Rust and Kani on your system.
Install Rust
The easiest way to install Rust is via rustup. Otherwise, check your system's package manager for recommended installation methods.
Install Kani
Kani is installed with cargo:
$ cargo install kani-verifier
$ cargo-kani setup
Running Kani proofs
After installing Rust and Kani, you can run the s2n-quic proof harnesses. These are currently all located in the s2n-quic-core crate:
$ cd quic/s2n-quic-core
$ cargo kani --tests
Listing Kani proofs
You can find all of the kani harnesses by searching for kani::proof
$ cd quic/s2n-quic-core
$ grep -Rn 'kani::proof' .
< LIST OF LOCATIONS WITH KANI PROOFS >
Async client hello callback
#[tokio::main]
async fn main() -> Result<(), Box<dyn Error>> {
let client = Client::builder()
.with_tls(CERT_PEM)?
.with_io("0.0.0.0:0")?
.start()?;
let addr: SocketAddr = "127.0.0.1:4433".parse()?;
let connect = Connect::new(addr).with_server_name("localhost");
let mut connection = client.connect(connect).await?;
// ensure the connection doesn't time out with inactivity
connection.keep_alive(true)?;
// open a new stream and split the receiving and sending sides
let stream = connection.open_bidirectional_stream().await?;
let (mut receive_stream, mut send_stream) = stream.split();
// spawn a task that copies responses from the server to stdout
tokio::spawn(async move {
let mut stdout = tokio::io::stdout();
let _ = tokio::io::copy(&mut receive_stream, &mut stdout).await;
});
// copy data from stdin and send it to the server
let mut stdin = tokio::io::stdin();
tokio::io::copy(&mut stdin, &mut send_stream).await?;
Ok(())
}
// Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
// SPDX-License-Identifier: Apache-2.0
use moka::sync::Cache;
use rand::{distributions::WeightedIndex, prelude::*};
use s2n_quic::{
provider::tls::s2n_tls::{
callbacks::{ClientHelloCallback, ConfigResolver, ConnectionFuture},
config::Config,
connection::Connection,
error::Error as S2nError,
},
Server,
};
use std::{error::Error, fmt::Display, pin::Pin, sync::Arc, time::Duration};
use tokio::{fs, sync::OnceCell};
/// NOTE: this certificate is to be used for demonstration purposes only!
pub static CERT_PEM_PATH: &str = concat!(
env!("CARGO_MANIFEST_DIR"),
"/../../quic/s2n-quic-core/certs/cert.pem"
);
/// NOTE: this certificate is to be used for demonstration purposes only!
pub static KEY_PEM_PATH: &str = concat!(
env!("CARGO_MANIFEST_DIR"),
"/../../quic/s2n-quic-core/certs/key.pem"
);
type Sni = String;
// A Config cache associated with as SNI (server name indication).
//
// Implements ClientHelloCallback, loading the certificates asynchronously,
// and caching the s2n_tls::config::Config for subsequent calls with the
// same SNI.
//
// An SNI, indicates which hostname the client is attempting to connect to.
// Some deployments could require configuring the s2n_tls::config::Config
// based on the SNI (certificate).
struct ConfigCache {
cache: Cache<Sni, Arc<OnceCell<Config>>>,
}
impl ConfigCache {
fn new() -> Self {
ConfigCache {
// store Config for up to 100 unique SNI
cache: Cache::new(100),
}
}
}
impl ClientHelloCallback for ConfigCache {
fn on_client_hello(
&self,
connection: &mut Connection,
) -> Result<Option<Pin<Box<dyn ConnectionFuture>>>, S2nError> {
let sni = connection
.server_name()
.ok_or_else(|| S2nError::application(Box::new(CustomError)))?
.to_string();
let once_cell_config = self
.cache
.get_with(sni.clone(), || Arc::new(OnceCell::new()));
if let Some(config) = once_cell_config.get() {
eprintln!("Config already cached for SNI: {}", sni);
connection.set_config(config.clone())?;
// return `None` if the Config is already in the cache
return Ok(None);
}
// simulate failure 75% of times and success 25% of the times
let choices = [true, false];
let weights = [3, 1];
let dist = WeightedIndex::new(weights).unwrap();
let fut = async move {
let fut = once_cell_config.get_or_try_init(|| async {
let simulated_network_call_failed = choices[dist.sample(&mut thread_rng())];
if simulated_network_call_failed {
eprintln!("simulated network call failed");
return Err(S2nError::application(Box::new(CustomError)));
}
eprintln!("resolving certificate for SNI: {}", sni);
// load the cert and key file asynchronously.
let (cert, key) = {
// the SNI can be used to load the appropriate cert file
let _sni = sni;
let cert = fs::read_to_string(CERT_PEM_PATH)
.await
.map_err(|_| S2nError::application(Box::new(CustomError)))?;
let key = fs::read_to_string(KEY_PEM_PATH)
.await
.map_err(|_| S2nError::application(Box::new(CustomError)))?;
(cert, key)
};
// sleep(async tokio task which doesn't block thread) to mimic delay
tokio::time::sleep(Duration::from_secs(3)).await;
let config = s2n_quic::provider::tls::s2n_tls::Server::builder()
.with_certificate(cert, key)?
.build()?
.into();
Ok(config)
});
fut.await.cloned()
};
// return `Some(ConnectionFuture)` if the Config wasn't found in the
// cache and we need to load it asynchronously
Ok(Some(Box::pin(ConfigResolver::new(fut))))
}
}
#[tokio::main]
async fn main() -> Result<(), Box<dyn Error>> {
let tls = s2n_quic::provider::tls::s2n_tls::Server::builder()
.with_client_hello_handler(ConfigCache::new())?
.build()?;
let mut server = Server::builder()
.with_tls(tls)?
.with_io("127.0.0.1:4433")?
.start()?;
while let Some(mut connection) = server.accept().await {
// spawn a new task for the connection
tokio::spawn(async move {
eprintln!("Connection accepted from {:?}", connection.remote_addr());
while let Ok(Some(mut stream)) = connection.accept_bidirectional_stream().await {
// spawn a new task for the stream
tokio::spawn(async move {
eprintln!("Stream opened from {:?}", stream.connection().remote_addr());
// echo any data back to the stream
while let Ok(Some(data)) = stream.receive().await {
stream.send(data).await.expect("stream should be open");
}
});
}
});
}
Ok(())
}
#[derive(Debug)]
struct CustomError;
impl Display for CustomError {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
write!(f, "custom error")?;
Ok(())
}
}
impl Error for CustomError {}
Custom Congestion Controller
This folder contains an example of implementing and configuring a custom congestion controller in s2n-quic. s2n-quic includes CUBIC and BBRv2 congestion controller implementations, but you may
implement the CongestionController trait, found in congestion_controller.rs, to provide your own.
Set-up
The CongestionController trait is considered unstable and may be subject to change in a future release. In order to build it you must
add this line to your Cargo.toml file:
[dependencies]
s2n-quic = { version = "1", features = ["unstable-congestion-controller"]}
Example
#[tokio::main]
async fn main() -> Result<(), Box<dyn Error>> {
// Build an `s2n_quic::Server`
let mut server = Server::builder()
.with_tls((CERT_PEM, KEY_PEM))?
.with_io("127.0.0.1:4433")?
// Specify the custom congestion controller endpoint defined in `custom-congestion-controller/src/lib.rs`
.with_congestion_controller(MyCongestionControllerEndpoint::default())?
.start()?;
while let Some(mut connection) = server.accept().await {
// spawn a new task for the connection
tokio::spawn(async move {
eprintln!("Connection accepted from {:?}", connection.remote_addr());
while let Ok(Some(mut stream)) = connection.accept_bidirectional_stream().await {
// spawn a new task for the stream
tokio::spawn(async move {
eprintln!("Stream opened from {:?}", stream.connection().remote_addr());
// echo any data back to the stream
while let Ok(Some(data)) = stream.receive().await {
stream.send(data).await.expect("stream should be open");
}
});
}
});
}
Ok(())
}