Detailed explanation of using Buffer to encode and decode binary data in Node.js

JavaScript is very good at handling strings, but because it was originally designed to handle HTML documents, it is not very good at handling binary data. JavaScript has no byte type, no structured types, not even byte arrays, only numbers and strings. (Original text: JavaScript doesn't have a byte type — it just has numbers — or structured types, or http://skylitecellars.com/ even byte arrays: It just has strings.)

Because Node is based on JavaScript , it can naturally handle text protocols like HTTP, but you can also use it to interact with databases, handle image or file uploads, etc. You can imagine how difficult it would be to do these things just using strings. Earlier, Node processed binary data by encoding bytes into text characters, but this approach later proved to be unfeasible, wasteful of resources, slow, inflexible, and difficult to maintain.

Node has a binary buffer implementation Buffer. This pseudo-class (pseudo-class) provides a series of APIs for processing binary data, simplifying tasks that require processing binary data. The length of the buffer is determined by the length of the byte data, and you can randomly set and get the byte data in the buffer.

Note: There is a special feature of the Buffer class. The memory occupied by the byte data in the buffer is not allocated on the JavaScrp

It VM memory heap, which means that these objects will not Processed by JavaScript's garbage collection algorithm, it is replaced by a permanent memory address that will not be modified, which also avoids CPU waste caused by memory copying of buffer contents.

Create a buffer

You can create a buffer using a UTF-8 string, like this:

var buf = new Buffer(‘Hello World!');

You can also create a buffer using a string with a specified encoding:

var buf = new Buffer('8b76fde713ce', 'base64');

Acceptable character encodings and identifiers are as follows:

1.ascii - ASCI, only applicable to the ASCII character set.
2.utf8 - UTF-8, this variable-width encoding is suitable for any character in the Unicode character set. It has become the preferred encoding in the Web world and is also the default encoding type of Node.
3.base64——Base64, this encoding is based on 64 printable ASCII characters to represent binary data. Base64 is usually used to embed binary data in character documents that can be converted into strings and can be complete when needed. Lossless conversion back to original binary format.

If there is no data to initialize the buffer, you can create an empty buffer with the specified capacity:

var buf = new Buffer(1024); // 创建一个1024字节的缓冲

Get and set buffer data

After creating or receiving a buffer object , you may want to view or modify its content, you can access a certain byte of the buffer through the [] operator:

var buf = new Buffer('my buffer content');
// 访问缓冲内第10个字节
console.log(buf[10]); // -> 99

Note: When you (use the buffer capacity size) to create an initialized buffer When doing this, be sure to note that the buffered data is not initialized to 0, but is random data.

var buf = new Buffer(1024);
console.log(buf[100]); // -> 5 (某个随机值)

You can modify the data at any location in the buffer like this:

buf[99] = 125; // 把第100个字节的值设置为125

Note: In some cases, some buffer operations will not cause errors, such as:

1. The maximum value of bytes in the buffer is 255. If a byte is assigned a number greater than 256, it will be modulo 256, and then the result will be assigned to this byte.
2. If a certain byte in the buffer is assigned a value of 256, its actual value will be 0 (Translator's Note: Actually repeated with the first one, 256%256=0)
3. If you use A floating point number is assigned to a certain byte in the buffer, such as 100.7. The actual value will be the integer part of the floating point number - 100
4. If you try to assign a value to a location that exceeds the buffer capacity, the assignment operation will fail. The buffer is not modified in any way.

You can use the length attribute to get the length of the buffer:

var buf = new Buffer(100);
console.log(buf.length); // -> 100

You can also use the buffer length to iterate over the contents of the buffer to read or set each byte:

var buf = new Buffer(100);
for(var i = 0; i < buf.length; i++) {
    buf[i] = i;
}

The above code creates a new buffer containing 100 bytes and sets each byte in the buffer from 0 to 99.

Split buffer data

Once you create or receive a buffer, you may need to extract part of the buffer data. You can split the existing buffer by specifying the starting position to create another A smaller buffer:

var buffer = new Buffer("this is the content of my buffer");

var smallerBuffer = buffer.slice(8, 19);
console.log(smallerBuffer.toString()); // -> "the content"

Note that when splitting a buffer, no new memory is allocated or copied. The new buffer uses the memory of the parent buffer. It is just a certain piece of data in the parent buffer (by starting position). This passage contains several meanings.

First of all, if your program modifies the contents of the parent buffer, these modifications will also affect the related child buffers. Because the parent buffer and the child buffer are different JavaScript objects, it is easy to ignore this problem and cause some Potential bugs.

Secondly, when you create a smaller child buffer from the parent buffer in this way, the parent buffer object will still be retained after the operation is completed and will not be garbage collected. If you don't pay attention, It is easy to cause memory leaks.

Note: If you are worried about memory leaks, you can use the copy method instead of the slice operation. Copy will be introduced below.

Copy buffer data

You can use copy to copy part of the buffer to another buffer like this:

var buffer1 = new Buffer("this is the content of my buffer");
var buffer2 = new Buffer(11);
var targetStart = 0;
var sourceStart = 8;
var sourceEnd = 19;
buffer1.copy(buffer2, targetStart, sourceStart, sourceEnd);
console.log(buffer2.toString()); // -> "the content"

The above code copies the 9th to 20th bits of the source buffer Bytes to the beginning of the target buffer.

Decoding buffered data

The buffered data can be converted into a UTF-8 string like this:

var str = buf.toString();

还可以通过指定编码类型来将缓冲数据解码成任何编码类型的数据。比如，你想把一个缓冲解码成base64字符串，可以这么做：

var b64Str = buf.toString("base64");

使用toString函数，你还可以把一个UTF-8字符串转码成base64字符串：

var utf8String = 'my string';
var buf = new Buffer(utf8String);
var base64String = buf.toString('base64')

小结

有时候，你不得不跟二进制数据打交道，但是原生JavaScript又没有明确的方式来做这件事，于是Node提供了Buffer类，封装了一些针对连续内存块的操作。你可以在两个缓冲之间切分或复制内存数据。

你也可以把一个缓冲转换成某种编码的字符串，或者反过来，把一个字符串转化成缓冲，来访问或处理每个bit。

更多Node.js中使用Buffer编码、解码二进制数据详解相关文章请关注PHP中文网！