TCP Fast Open

TCP Fast Open, or TFO for short, is an extension to the TCP protocol that allow a small amount of data to be sent alongside the initial SYN packet, when a TCP connection is being established. This can save round-trips and reduce response time in certain cases.

TFO cannot always be used, however, because it changes how TCP behave: The receiving end might see this data duplicated due to retransmission of the SYN packet. For this reason, TFO must only be used when the data in the initial packet will be processed idempotently. Whether this is the case, depends on the protocol and the application.

One important use case where idempotency is guaranteed, is the TLS Client Hello message. This is the first message that a client sends to a server, to initiate the TLS handshake. This saves a round-trip when establishing TLS connections. Since version 4.1.61.Final, the SslHandler in Netty will automatically take advantage of TFO when available. Specifically, server-side and client-side TFO is only supported on the epoll transport, and it requires support to be enabled in the Linux kernel. Since version 4.1.67.Final, TCP FastOpen will also be supported for client-side connections on the kqueue transport.

First, TFO needs to be supported and enabled in the operating system. On MacOS, this is enabled by default. On Linux, this is controlled with the /proc/sys/net/ipv4/tcp_fastopen file. The file may contain one of the following values:

0. TFO is not enabled.
1. TFO is enabled for outgoing connections (clients).
2. TFO is enabled for incoming connections (servers).
3. TFO is enabled for both clients and servers.

The setting can be changed by writing the derised configuration value to the file as the root user.

The second step is to enable TFO in Netty. This is done in different ways for servers and clients.

For servers, you set the ChannelOption.TCP_FASTOPEN as an option on the ServerBootstrap:

ServerBootstrap sb = ...;
sb.option(ChannelOption.TCP_FASTOPEN, maxPendingFastOpen);

And that's all it takes. The option specifies how many fast-open requests can be pending on the socket at any one time. This limits the system resources that can be tied up in fast-open payloads for connections that haven't been established. See the Passive Open appendix to RFC 7413 for reference.

For clients, the situation is a little more involved. You first set the ChannelOption.TCP_FASTOPEN_CONNECT option to true on Bootstrap:

Bootstrap cb = ...;
cb.option(ChannelOption.TCP_FASTOPEN_CONNECT, true);

Then, you also have to decide what data is allowed to be sent alongside the SYN packet, taking into consideration that it may arrive multiple times due to retransmission. This data then needs to be placed in the channels outbound buffer before the channel is connected. To obtain a channel before it's connected, we call register. Here is an example of what this can look like:

Bootstrap cb = new Bootstrap();
cb.option(ChannelOption.TCP_FASTOPEN_CONNECT, true);
// ...set handler, etc...
Channel channel = cb.register().sync().channel(); // Get unconnected channel.

ByteBuf fastOpenData = ...;
ByteBuf normalData = ...;

channel.write(fastOpenData);           // Write TFO data.
channel.connect(remoteAddress).sync(); // Establish connection (flushes TFO data).
channel.write(normalData);             // TCP connection works like normal now.

An important aspect about the above: we call connect() on the channel produced by register(). We cannot use the Bootstrap.connect() method, because that will create another new channel with its own outbound buffer.