io.netty.handler.codec.http

Class HttpObjectDecoder

java.lang.Object

io.netty.channel.ChannelHandlerAdapter

io.netty.channel.ChannelInboundHandlerAdapter

io.netty.handler.codec.ByteToMessageDecoder

io.netty.handler.codec.http.HttpObjectDecoder

All Implemented Interfaces:

ChannelHandler, ChannelInboundHandler

Direct Known Subclasses:

HttpRequestDecoder, HttpResponseDecoder, RtspDecoder, RtspObjectDecoder

public abstract class HttpObjectDecoder
extends ByteToMessageDecoder

Decodes ByteBufs into HttpMessages and HttpContents.

Parameters that prevents excessive memory consumption

Name	Default value	Meaning
maxInitialLineLength	4096	The maximum length of the initial line (e.g. "GET / HTTP/1.0" or "HTTP/1.0 200 OK") If the length of the initial line exceeds this value, a TooLongHttpLineException will be raised.
maxHeaderSize	8192	The maximum length of all headers. If the sum of the length of each header exceeds this value, a TooLongHttpHeaderException will be raised.
maxChunkSize	8192	The maximum length of the content or each chunk. If the content length (or the length of each chunk) exceeds this value, the content or chunk will be split into multiple HttpContents whose length is maxChunkSize at maximum.

Parameters that control parsing behavior

Name	Default value	Meaning
allowDuplicateContentLengths	false	When set to false, will reject any messages that contain multiple Content-Length header fields. When set to true, will allow multiple Content-Length headers only if they are all the same decimal value. The duplicated field-values will be replaced with a single valid Content-Length field. See RFC 7230, Section 3.3.2.
allowPartialChunks	true	If the length of a chunk exceeds the ByteBufs readable bytes and allowPartialChunks is set to true, the chunk will be split into multiple HttpContents. Otherwise, if the chunk size does not exceed maxChunkSize and allowPartialChunks is set to false, the ByteBuf is not decoded into an HttpContent until the readable bytes are greater or equal to the chunk size.

Chunked Content

If the content of an HTTP message is greater than maxChunkSize or the transfer encoding of the HTTP message is 'chunked', this decoder generates one HttpMessage instance and its following HttpContents per single HTTP message to avoid excessive memory consumption. For example, the following HTTP message:

 GET / HTTP/1.1
 Transfer-Encoding: chunked

 1a
 abcdefghijklmnopqrstuvwxyz
 10
 1234567890abcdef
 0
 Content-MD5: ...
 [blank line]
 

triggers HttpRequestDecoder to generate 3 objects:

1. An HttpRequest,
2. The first HttpContent whose content is 'abcdefghijklmnopqrstuvwxyz',
3. The second LastHttpContent whose content is '1234567890abcdef', which marks the end of the content.

If you prefer not to handle HttpContents by yourself for your convenience, insert HttpObjectAggregator after this decoder in the ChannelPipeline. However, please note that your server might not be as memory efficient as without the aggregator.

Extensibility

Please note that this decoder is designed to be extended to implement a protocol derived from HTTP, such as RTSP and ICAP. To implement the decoder of such a derived protocol, extend this class and implement all abstract methods properly.

Header Validation

It is recommended to always enable header validation.

Without header validation, your system can become vulnerable to CWE-113: Improper Neutralization of CRLF Sequences in HTTP Headers ('HTTP Response Splitting') .

This recommendation stands even when both peers in the HTTP exchange are trusted, as it helps with defence-in-depth.

Nested Class Summary

Nested classes/interfaces inherited from class io.netty.handler.codec.ByteToMessageDecoder

ByteToMessageDecoder.Cumulator

Nested classes/interfaces inherited from interface io.netty.channel.ChannelHandler

ChannelHandler.Sharable

Field Summary

Fields

Modifier and Type	Field and Description
static boolean	DEFAULT_ALLOW_DUPLICATE_CONTENT_LENGTHS
static boolean	DEFAULT_ALLOW_PARTIAL_CHUNKS
static boolean	DEFAULT_CHUNKED_SUPPORTED
static int	DEFAULT_INITIAL_BUFFER_SIZE
static int	DEFAULT_MAX_CHUNK_SIZE
static int	DEFAULT_MAX_HEADER_SIZE
static int	DEFAULT_MAX_INITIAL_LINE_LENGTH
static boolean	DEFAULT_VALIDATE_HEADERS
protected HttpHeadersFactory	headersFactory
protected HttpHeadersFactory	trailersFactory
protected boolean	validateHeaders Deprecated.

Fields inherited from class io.netty.handler.codec.ByteToMessageDecoder

COMPOSITE_CUMULATOR, MERGE_CUMULATOR

Constructor Summary

Constructors

Modifier	Constructor and Description
protected	HttpObjectDecoder() Creates a new instance with the default maxInitialLineLength (4096), maxHeaderSize (8192), and maxChunkSize (8192).
protected	HttpObjectDecoder(HttpDecoderConfig config) Creates a new instance with the specified configuration.
protected	HttpObjectDecoder(int maxInitialLineLength, int maxHeaderSize, int maxChunkSize, boolean chunkedSupported) Deprecated. Use HttpObjectDecoder(HttpDecoderConfig) instead.
protected	HttpObjectDecoder(int maxInitialLineLength, int maxHeaderSize, int maxChunkSize, boolean chunkedSupported, boolean validateHeaders) Deprecated. Use HttpObjectDecoder(HttpDecoderConfig) instead.
protected	HttpObjectDecoder(int maxInitialLineLength, int maxHeaderSize, int maxChunkSize, boolean chunkedSupported, boolean validateHeaders, int initialBufferSize) Deprecated. Use HttpObjectDecoder(HttpDecoderConfig) instead.
protected	HttpObjectDecoder(int maxInitialLineLength, int maxHeaderSize, int maxChunkSize, boolean chunkedSupported, boolean validateHeaders, int initialBufferSize, boolean allowDuplicateContentLengths) Deprecated. Use HttpObjectDecoder(HttpDecoderConfig) instead.
protected	HttpObjectDecoder(int maxInitialLineLength, int maxHeaderSize, int maxChunkSize, boolean chunkedSupported, boolean validateHeaders, int initialBufferSize, boolean allowDuplicateContentLengths, boolean allowPartialChunks) Deprecated. Use HttpObjectDecoder(HttpDecoderConfig) instead.

Method Summary

Modifier and Type	Method and Description
protected abstract HttpMessage	createInvalidMessage()
protected abstract HttpMessage	createMessage(String[] initialLine)
protected void	decode(ChannelHandlerContext ctx, ByteBuf buffer, List<Object> out) Decode the from one ByteBuf to an other.
protected void	decodeLast(ChannelHandlerContext ctx, ByteBuf in, List<Object> out) Is called one last time when the ChannelHandlerContext goes in-active.
protected void	handlerRemoved0(ChannelHandlerContext ctx) Gets called after the ByteToMessageDecoder was removed from the actual context and it doesn't handle events anymore.
protected void	handleTransferEncodingChunkedWithContentLength(HttpMessage message) Invoked when a message with both a "Transfer-Encoding: chunked" and a "Content-Length" header field is detected.
protected boolean	isContentAlwaysEmpty(HttpMessage msg)
protected abstract boolean	isDecodingRequest()
protected boolean	isSwitchingToNonHttp1Protocol(HttpResponse msg) Returns true if the server switched to a different protocol than HTTP/1.0 or HTTP/1.1, e.g.
protected boolean	isValidating(HttpHeadersFactory headersFactory)
void	reset() Resets the state of the decoder so that it is ready to decode a new message.
protected String	splitFirstWordInitialLine(byte[] asciiContent, int start, int length)
protected AsciiString	splitHeaderName(byte[] sb, int start, int length)
protected String	splitSecondWordInitialLine(byte[] asciiContent, int start, int length)
protected String	splitThirdWordInitialLine(byte[] asciiContent, int start, int length)
void	userEventTriggered(ChannelHandlerContext ctx, Object evt) Calls ChannelHandlerContext.fireUserEventTriggered(Object) to forward to the next ChannelInboundHandler in the ChannelPipeline.

Methods inherited from class io.netty.handler.codec.ByteToMessageDecoder

actualReadableBytes, callDecode, channelInactive, channelRead, channelReadComplete, discardSomeReadBytes, handlerRemoved, internalBuffer, isSingleDecode, setCumulator, setDiscardAfterReads, setSingleDecode

Methods inherited from class io.netty.channel.ChannelInboundHandlerAdapter

channelActive, channelRegistered, channelUnregistered, channelWritabilityChanged, exceptionCaught

Methods inherited from class io.netty.channel.ChannelHandlerAdapter

ensureNotSharable, handlerAdded, isSharable

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface io.netty.channel.ChannelHandler

handlerAdded

Field Detail

DEFAULT_MAX_INITIAL_LINE_LENGTH

public static final int DEFAULT_MAX_INITIAL_LINE_LENGTH

See Also:

Constant Field Values

DEFAULT_MAX_HEADER_SIZE

public static final int DEFAULT_MAX_HEADER_SIZE

See Also:

Constant Field Values

DEFAULT_CHUNKED_SUPPORTED

public static final boolean DEFAULT_CHUNKED_SUPPORTED

See Also:

Constant Field Values

DEFAULT_ALLOW_PARTIAL_CHUNKS

public static final boolean DEFAULT_ALLOW_PARTIAL_CHUNKS

See Also:

Constant Field Values

DEFAULT_MAX_CHUNK_SIZE

public static final int DEFAULT_MAX_CHUNK_SIZE

See Also:

Constant Field Values

DEFAULT_VALIDATE_HEADERS

public static final boolean DEFAULT_VALIDATE_HEADERS

See Also:

Constant Field Values

DEFAULT_INITIAL_BUFFER_SIZE

public static final int DEFAULT_INITIAL_BUFFER_SIZE

See Also:

Constant Field Values

DEFAULT_ALLOW_DUPLICATE_CONTENT_LENGTHS

public static final boolean DEFAULT_ALLOW_DUPLICATE_CONTENT_LENGTHS

See Also:

Constant Field Values

validateHeaders

@Deprecated
protected final boolean validateHeaders

Deprecated.

This field is no longer used. It is only kept around for backwards compatibility purpose.

headersFactory

protected final HttpHeadersFactory headersFactory

trailersFactory

protected final HttpHeadersFactory trailersFactory

Constructor Detail

HttpObjectDecoder

protected HttpObjectDecoder()

Creates a new instance with the default maxInitialLineLength (4096), maxHeaderSize (8192), and maxChunkSize (8192).

HttpObjectDecoder

@Deprecated
protected HttpObjectDecoder(int maxInitialLineLength,
                                        int maxHeaderSize,
                                        int maxChunkSize,
                                        boolean chunkedSupported)

Deprecated. Use HttpObjectDecoder(HttpDecoderConfig) instead.

Creates a new instance with the specified parameters.

HttpObjectDecoder

@Deprecated
protected HttpObjectDecoder(int maxInitialLineLength,
                                        int maxHeaderSize,
                                        int maxChunkSize,
                                        boolean chunkedSupported,
                                        boolean validateHeaders)

Deprecated. Use HttpObjectDecoder(HttpDecoderConfig) instead.

Creates a new instance with the specified parameters.

HttpObjectDecoder

@Deprecated
protected HttpObjectDecoder(int maxInitialLineLength,
                                        int maxHeaderSize,
                                        int maxChunkSize,
                                        boolean chunkedSupported,
                                        boolean validateHeaders,
                                        int initialBufferSize)

Deprecated. Use HttpObjectDecoder(HttpDecoderConfig) instead.

Creates a new instance with the specified parameters.

HttpObjectDecoder

@Deprecated
protected HttpObjectDecoder(int maxInitialLineLength,
                                        int maxHeaderSize,
                                        int maxChunkSize,
                                        boolean chunkedSupported,
                                        boolean validateHeaders,
                                        int initialBufferSize,
                                        boolean allowDuplicateContentLengths)

Deprecated. Use HttpObjectDecoder(HttpDecoderConfig) instead.

Creates a new instance with the specified parameters.

HttpObjectDecoder

@Deprecated
protected HttpObjectDecoder(int maxInitialLineLength,
                                        int maxHeaderSize,
                                        int maxChunkSize,
                                        boolean chunkedSupported,
                                        boolean validateHeaders,
                                        int initialBufferSize,
                                        boolean allowDuplicateContentLengths,
                                        boolean allowPartialChunks)

Deprecated. Use HttpObjectDecoder(HttpDecoderConfig) instead.

Creates a new instance with the specified parameters.

HttpObjectDecoder

protected HttpObjectDecoder(HttpDecoderConfig config)

Creates a new instance with the specified configuration.

Method Detail

handlerRemoved0

protected void handlerRemoved0(ChannelHandlerContext ctx)
                        throws Exception

Description copied from class: ByteToMessageDecoder

Gets called after the ByteToMessageDecoder was removed from the actual context and it doesn't handle events anymore.

Overrides:

handlerRemoved0 in class ByteToMessageDecoder

Throws:

Exception

isValidating

protected boolean isValidating(HttpHeadersFactory headersFactory)

decode

protected void decode(ChannelHandlerContext ctx,
                      ByteBuf buffer,
                      List<Object> out)
               throws Exception

Description copied from class: ByteToMessageDecoder

Decode the from one ByteBuf to an other. This method will be called till either the input ByteBuf has nothing to read when return from this method or till nothing was read from the input ByteBuf.

Specified by:

decode in class ByteToMessageDecoder

Parameters:

ctx - the ChannelHandlerContext which this ByteToMessageDecoder belongs to

buffer - the ByteBuf from which to read data

out - the List to which decoded messages should be added

Throws:

Exception - is thrown if an error occurs

decodeLast

protected void decodeLast(ChannelHandlerContext ctx,
                          ByteBuf in,
                          List<Object> out)
                   throws Exception

Description copied from class: ByteToMessageDecoder

Is called one last time when the ChannelHandlerContext goes in-active. Which means the ByteToMessageDecoder.channelInactive(ChannelHandlerContext) was triggered. By default, this will just call ByteToMessageDecoder.decode(ChannelHandlerContext, ByteBuf, List) but sub-classes may override this for some special cleanup operation.

Overrides:

decodeLast in class ByteToMessageDecoder

Throws:

Exception

userEventTriggered

public void userEventTriggered(ChannelHandlerContext ctx,
                               Object evt)
                        throws Exception

Description copied from class: ChannelInboundHandlerAdapter

Calls ChannelHandlerContext.fireUserEventTriggered(Object) to forward to the next ChannelInboundHandler in the ChannelPipeline. Sub-classes may override this method to change behavior.

Specified by:

userEventTriggered in interface ChannelInboundHandler

Overrides:

userEventTriggered in class ByteToMessageDecoder

Throws:

Exception

isContentAlwaysEmpty

protected boolean isContentAlwaysEmpty(HttpMessage msg)

isSwitchingToNonHttp1Protocol

protected boolean isSwitchingToNonHttp1Protocol(HttpResponse msg)

Returns true if the server switched to a different protocol than HTTP/1.0 or HTTP/1.1, e.g. HTTP/2 or Websocket. Returns false if the upgrade happened in a different layer, e.g. upgrade from HTTP/1.1 to HTTP/1.1 over TLS.

reset

public void reset()

Resets the state of the decoder so that it is ready to decode a new message. This method is useful for handling a rejected request with Expect: 100-continue header.

handleTransferEncodingChunkedWithContentLength

protected void handleTransferEncodingChunkedWithContentLength(HttpMessage message)

Invoked when a message with both a "Transfer-Encoding: chunked" and a "Content-Length" header field is detected. The default behavior is to remove the Content-Length field, but this method could be overridden to change the behavior (to, e.g., throw an exception and produce an invalid message).

See: https://tools.ietf.org/html/rfc7230#section-3.3.3

     If a message is received with both a Transfer-Encoding and a
     Content-Length header field, the Transfer-Encoding overrides the
     Content-Length.  Such a message might indicate an attempt to
     perform request smuggling (Section 9.5) or response splitting
     (Section 9.4) and ought to be handled as an error.  A sender MUST
     remove the received Content-Length field prior to forwarding such
     a message downstream.
 

Also see: https://github.com/apache/tomcat/blob/b693d7c1981fa7f51e58bc8c8e72e3fe80b7b773/ java/org/apache/coyote/http11/Http11Processor.java#L747-L755 https://github.com/nginx/nginx/blob/0ad4393e30c119d250415cb769e3d8bc8dce5186/ src/http/ngx_http_request.c#L1946-L1953

isDecodingRequest

protected abstract boolean isDecodingRequest()

createMessage

protected abstract HttpMessage createMessage(String[] initialLine)
                                      throws Exception

Throws:

Exception

createInvalidMessage

protected abstract HttpMessage createInvalidMessage()

splitFirstWordInitialLine

protected String splitFirstWordInitialLine(byte[] asciiContent,
                                           int start,
                                           int length)

splitSecondWordInitialLine

protected String splitSecondWordInitialLine(byte[] asciiContent,
                                            int start,
                                            int length)

splitThirdWordInitialLine

protected String splitThirdWordInitialLine(byte[] asciiContent,
                                           int start,
                                           int length)

splitHeaderName

protected AsciiString splitHeaderName(byte[] sb,
                                      int start,
                                      int length)