#(#databuffers)8. Data Buffers and Codecs

预计阅读时间: 13 分钟

8. 数据缓冲区和编解码器

Java NIO provides ByteBuffer but many libraries build their own byte buffer API on top, especially for network operations where reusing buffers and/or using direct buffers is beneficial for performance. For example Netty has the ByteBuf hierarchy, Undertow uses XNIO, Jetty uses pooled byte buffers with a callback to be released, and so on. The spring-core module provides a set of abstractions to work with various byte buffer APIs as follows:
Java NIO 提供了 ByteBuffer ，但许多库在其之上构建了自己的字节缓冲区 API，尤其是在网络操作中，重用缓冲区和/或使用直接缓冲区对性能有益。例如，Netty 有 ByteBuf 层次结构，Undertow 使用 XNIO，Jetty 使用带有释放回调的池化字节缓冲区，等等。 spring-core 模块提供了一套抽象，用于与各种字节缓冲区 API 一起工作，如下所示：

• DataBufferFactory abstracts the creation of a data buffer.
  DataBufferFactory 抽象化数据缓冲区的创建。

• DataBuffer represents a byte buffer, which may be pooled.
  DataBuffer 代表一个字节缓冲区，可能被池化。

• DataBufferUtils offers utility methods for data buffers.
  DataBufferUtils 提供数据缓冲区的实用方法。

• Codecs decode or encode data buffer streams into higher level objects.
  编解码器将数据缓冲流解码或编码成高级对象。

#(#databuffers-factory)8.1.DataBufferFactory

DataBufferFactory is used to create data buffers in one of two ways:
DataBufferFactory 用于以两种方式之一创建数据缓冲区：

1. Allocate a new data buffer, optionally specifying capacity upfront, if known, which is more efficient even though implementations of DataBuffer can grow and shrink on demand.
   分配一个新的数据缓冲区，可选地提前指定容量（如果已知），这虽然使得 DataBuffer 的实现可以按需增长和缩小，但效率更高。

2. Wrap an existing byte or java.nio.ByteBuffer, which decorates the given data with a DataBuffer implementation and that does not involve allocation.
   包装现有的 byte 或 java.nio.ByteBuffer ，该装饰器使用 DataBuffer 实现对给定数据的装饰，且不涉及分配。

Note that WebFlux applications do not create a DataBufferFactory directly but instead access it through the ServerHttpResponse or the ClientHttpRequest on the client side. The type of factory depends on the underlying client or server, e.g. NettyDataBufferFactory for Reactor Netty, DefaultDataBufferFactory for others.
请注意，WebFlux 应用程序不会直接创建 DataBufferFactory ，而是通过客户端的 ServerHttpResponse 或 ClientHttpRequest 来访问它。工厂的类型取决于底层客户端或服务器，例如 NettyDataBufferFactory 用于 Reactor Netty， DefaultDataBufferFactory 用于其他情况。

#(#databuffers-buffer)8.2.DataBuffer

The DataBuffer interface offers similar operations as java.nio.ByteBuffer but also brings a few additional benefits some of which are inspired by the Netty ByteBuf. Below is a partial list of benefits:
DataBuffer 接口提供了与 java.nio.ByteBuffer 相似的操作，同时也带来了一些额外的优势，其中一些灵感来源于 Netty ByteBuf 。以下是部分优势的列表：

• Read and write with independent positions, i.e. not requiring a call to flip() to alternate between read and write.

• Capacity expanded on demand as with java.lang.StringBuilder.

• Pooled buffers and reference counting via PooledDataBuffer.

• View a buffer as java.nio.ByteBuffer, InputStream, or OutputStream.

• Determine the index, or the last index, for a given byte.

#(#databuffers-buffer-pooled)8.3.PooledDataBuffer

As explained in the Javadoc for ByteBuffer, byte buffers can be direct or non-direct. Direct buffers may reside outside the Java heap which eliminates the need for copying for native I/O operations.
如 Javadoc 中所述，ByteBuffer 可以是直接或非直接缓冲区。直接缓冲区可能位于 Java 堆之外，从而消除了在本地 I/O 操作中复制的需求。
That makes direct buffers particularly useful for receiving and sending data over a socket, but they’re also more expensive to create and release, which leads to the idea of pooling buffers.
这使得直接缓冲区在通过套接字接收和发送数据时特别有用，但它们的创建和释放成本也更高，这导致了缓冲池的概念。

PooledDataBuffer is an extension of DataBuffer that helps with reference counting which is essential for byte buffer pooling. How does it work? When a PooledDataBuffer is allocated the reference count is at 1. Calls to retain() increment the count, while calls to release() decrement it. As long as the count is above 0, the buffer is guaranteed not to be released. When the count is decreased to 0, the pooled buffer can be released, which in practice could mean the reserved memory for the buffer is returned to the memory pool.
PooledDataBuffer 是 DataBuffer 的扩展，有助于引用计数，这对于字节缓冲区池是必不可少的。它是如何工作的？当一个 PooledDataBuffer 被分配时，引用计数为 1。对 retain() 的调用会增加计数，而对 release() 的调用会减少它。只要计数大于 0，就可以保证缓冲区不会被释放。当计数减少到 0 时，池化的缓冲区可以被释放，这在实践中可能意味着为缓冲区保留的内存被返回到内存池。

Note that instead of operating on PooledDataBuffer directly, in most cases it’s better to use the convenience methods in DataBufferUtils that apply release or retain to a DataBuffer only if it is an instance of PooledDataBuffer.
请注意，在大多数情况下，直接操作 PooledDataBuffer 不如使用 DataBufferUtils 中的便利方法，这些方法仅在 DataBuffer 是 PooledDataBuffer 的实例时才应用释放或保留。

#(#databuffers-utils)8.4.DataBufferUtils

DataBufferUtils offers a number of utility methods to operate on data buffers:
DataBufferUtils 提供了多种实用方法来操作数据缓冲区：

• Join a stream of data buffers into a single buffer possibly with zero copy, e.g. via composite buffers, if that’s supported by the underlying byte buffer API.
  将数据缓冲区流合并成一个单独的缓冲区，可能实现零拷贝，例如通过复合缓冲区，如果底层字节缓冲区 API 支持这样做。

• Turn InputStream or NIO Channel into Flux<DataBuffer>, and vice versa a Publisher<DataBuffer> into OutputStream or NIO Channel.
  将 InputStream 或 NIO Channel 转换为 Flux<DataBuffer> ，反之亦然，将 a Publisher<DataBuffer> 转换为 OutputStream 或 NIO Channel 。

• Methods to release or retain a DataBuffer if the buffer is an instance of PooledDataBuffer.
  方法用于释放或保留一个 DataBuffer 如果缓冲区是 PooledDataBuffer 的实例。

• Skip or take from a stream of bytes until a specific byte count.
  跳过或从字节流中取出，直到达到特定的字节数。

#(#codecs)8.5. Codecs 8.5. 编解码器

The org.springframework.core.codec package provides the following strategy interfaces:
The org.springframework.core.codec package provides the following strategy interfaces: 该 org.springframework.core.codec 包提供以下策略接口：

• Encoder to encode Publisher<T> into a stream of data buffers.
  将 Publisher<T> 编码为数据缓冲区流。

• Decoder to decode Publisher<DataBuffer> into a stream of higher level objects.
  Decoder 解码 Publisher<DataBuffer> 成一系列更高级的对象流。

The spring-core module provides byte, ByteBuffer, DataBuffer, Resource, and String encoder and decoder implementations. The spring-web module adds Jackson JSON, Jackson Smile, JAXB2, Protocol Buffers and other encoders and decoders. See Codecs in the WebFlux section.
The spring-core 模块提供了 byte 、 ByteBuffer 、 DataBuffer 、 Resource 和 String 编码器和解码器实现。 spring-web 模块增加了 Jackson JSON、Jackson Smile、JAXB2、Protocol Buffers 等编码器和解码器。请参阅 WebFlux 部分的编解码器。

#(#databuffers-using)8.6. UsingDataBuffer 8.6. 使用DataBuffer

When working with data buffers, special care must be taken to ensure buffers are released since they may be pooled. We’ll use codecs to illustrate how that works but the concepts apply more generally. Let’s see what codecs must do internally to manage data buffers.
处理数据缓冲区时，必须特别注意释放缓冲区，因为它们可能会被池化。我们将使用编解码器来说明这是如何工作的，但这个概念更普遍适用。让我们看看编解码器在内部必须做什么来管理数据缓冲区。

A Decoder is the last to read input data buffers, before creating higher level objects, and therefore it must release them as follows:
一个 Decoder 是最后一个读取输入数据缓冲区的，在创建高级对象之前，因此它必须按照以下方式释放它们：

1. If a Decoder simply reads each input buffer and is ready to release it immediately, it can do so via DataBufferUtils.release(dataBuffer).
   如果 Decoder 仅仅读取每个输入缓冲区并立即准备释放它，它可以通过 DataBufferUtils.release(dataBuffer) 这样做。

2. If a Decoder is using Flux or Mono operators such as flatMap, reduce, and others that prefetch and cache data items internally, or is using operators such as filter, skip, and others that leave out items, then doOnDiscard(PooledDataBuffer.class, DataBufferUtils::release) must be added to the composition chain to ensure such buffers are released prior to being discarded, possibly also as a result of an error or cancellation signal.
   如果 Decoder 正在使用 Flux 或 Mono 运算符，如 flatMap 、 reduce 等内部预取和缓存数据项的运算符，或者正在使用 filter 、 skip 等省略项的运算符，那么必须在组成链中添加 doOnDiscard(PooledDataBuffer.class, DataBufferUtils::release) 以确保在丢弃之前释放此类缓冲区，可能还可能是由于错误或取消信号的结果。

3. If a Decoder holds on to one or more data buffers in any other way, it must ensure they are released when fully read, or in case of an error or cancellation signals that take place before the cached data buffers have been read and released.
   如果 Decoder 以任何方式保留一个或多个数据缓冲区，它必须确保在完全读取后、或在发生错误或取消信号之前（在缓存的数据缓冲区被读取和释放之前），释放这些缓冲区。

Note that DataBufferUtils#join offers a safe and efficient way to aggregate a data buffer stream into a single data buffer. Likewise skipUntilByteCount and takeUntilByteCount are additional safe methods for decoders to use.
请注意， DataBufferUtils#join 提供了一种安全高效地将数据缓冲流聚合为单个数据缓冲区的方法。同样， skipUntilByteCount 和 takeUntilByteCount 是解码器使用的其他安全方法。

An Encoder allocates data buffers that others must read (and release). So an Encoder doesn’t have much to do. However an Encoder must take care to release a data buffer if a serialization error occurs while populating the buffer with data. For example:
一个 Encoder 分配数据缓冲区，其他必须读取（并释放）。因此，一个 Encoder 没有太多事情要做。然而，一个 Encoder 必须注意，如果在用数据填充缓冲区时发生序列化错误，必须释放数据缓冲区。例如：

The consumer of an Encoder is responsible for releasing the data buffers it receives. In a WebFlux application, the output of the Encoder is used to write to the HTTP server response, or to the client HTTP request, in which case releasing the data buffers is the responsibility of the code writing to the server response, or to the client request.
消费者负责释放它接收到的数据缓冲区。在 WebFlux 应用程序中， Encoder 的输出用于写入 HTTP 服务器响应，或客户端 HTTP 请求，在这种情况下，释放数据缓冲区的责任在于写入服务器响应或客户端请求的代码。

Note that when running on Netty, there are debugging options for troubleshooting buffer leaks.
请注意，在 Netty 上运行时，有调试选项用于解决缓冲区泄漏问题。