Coding algorithm of utf8 reprint_basic knowledge
unicode字符集是我们世界上最完善最全面的字符集,几乎包含了世界上所有的字符。其实可以这么理解,unicode字符集是一张巨大的表格,把世界上各种语言的字符和标点符号都编排到里面,然后按照一定的顺序给每个字符排号(很遗憾的是对于中文来说,这个顺序不是按照汉语拼音的顺序)。有了这张巨大的表格,世界上大部分字符就都有一个unicode内码(整数)来对应。计算机就通过记录字符的unicode码来表示这个字符,然后交给操作系统,操作系统通过unicode码到字符字体点阵的映射把这个内码转换成字体点阵显示到我们的屏幕上。
utf8是我们常用的编码方式,在web开发中使用utf8编码能完全解决字符集问题。其实utf8是unicode字符集的一种物理实现,它描述了如何高效的存储unicode的内码(就是上面说的字符在字符集的顺序码),RFC2044文档(http: //www.ietf.org/rfc/rfc2044.txt?number=2044)描述了如何从一个内码转换成utf8格式的算法。英文不好没关系,看这个转换表就会马上明白了:
UCS-4 range (hex.) UTF-8 octet sequence (binary)
0000 0000-0000 007F 0xxxxxxx
0000 0080-0000 07FF 110xxxxx 10xxxxxx
0000 0800-0000 FFFF 1110xxxx 10xxxxxx 10xxxxxx
0001 0000-001F FFFF 11110xxx 10xxxxxx 10xxxxxx 10xxxxxx
0020 0000-03FF FFFF 111110xx 10xxxxxx 10xxxxxx 10xxxxxx 10xxxxxx
0400 0000-7FFF FFFF 1111110x 10xxxxxx ... 10xxxxxx
上面的表格左边是16进制表示的unicode内码,最后一行的16进制数“7FFF FFFF”是utf8所能表示的内码的最大值,换成10进制是这样的一个数:2147483647(够大吧:))[对不起,此文最初这儿写错了,已改正]。上面表格中右边一列就是utf8的二进制格式,转换规则可谓一目了然。我直接给出算法吧(js代码):
function toUtf8(code)
{
var iByte=0;
var i=0;
result="";
while(code>0x7f)
{
iByte=code%0x40;
code=(code-iByte)/0x40;
result="%"+(iByte|0x80).toString(16).toUpperCase()+result;
i++;
}
prefix=[0x0,0xc0,0xe0,0xf0,0xf8,0xfc];
if(i>prefix.length)
{
i=5;
}
result=""+(code|prefix[i]).toString(16).toUpperCase()+result;
return result;
}
例如字符"汉"的unicode是6C49,把这个unicode字符表示为一个大整数,然后转变成多字节编码110110001001001:
观察这个整数的二进制码序列(110,110001,001001)
从后往前取
如果这个二进制序列只有后7位(小于128,也就是ascii字符)则直接取后7位二进制数形成一个utf8字符。
上面的字符“汉”二进制序列大于7位,所以取后6位(1001001),加10形成一个utf8字节(10 001001 ,16进制89)。
剩下的二进制序列(110,110001)从后向前取6位,加10形成一个utf8字节(10 110001,16进制B1)。
剩下的二进制序列(110)从后向前取6位,由于不足6位,将这个数和1110000相或,得到字符11100110,16进制E6
最后,就得到了utf8编码,16进制表示为E6B189
【Application Areas】
Although most of this standard algorithm has been implemented by development tool providers or libraries, we still need to implement this algorithm ourselves at some point:
character, the server uses the above algorithm to convert the unicode serial number after u into utf8 character Use the above algorithm combined with escape to directly implement the encodeURI function on the client (this solution is recommended)
Implement json-rpc service
in the form of "uXXXX". So we need to convert the server-side characters into json. For php, there are now two open source projects
JSON-PHP and PHP-JSON. .

Hot AI Tools

Undresser.AI Undress
AI-powered app for creating realistic nude photos

AI Clothes Remover
Online AI tool for removing clothes from photos.

Undress AI Tool
Undress images for free

Clothoff.io
AI clothes remover

Video Face Swap
Swap faces in any video effortlessly with our completely free AI face swap tool!

Hot Article

Hot Tools

Notepad++7.3.1
Easy-to-use and free code editor

SublimeText3 Chinese version
Chinese version, very easy to use

Zend Studio 13.0.1
Powerful PHP integrated development environment

Dreamweaver CS6
Visual web development tools

SublimeText3 Mac version
God-level code editing software (SublimeText3)

Hot Topics

Frequently Asked Questions and Solutions for Front-end Thermal Paper Ticket Printing In Front-end Development, Ticket Printing is a common requirement. However, many developers are implementing...

JavaScript is the cornerstone of modern web development, and its main functions include event-driven programming, dynamic content generation and asynchronous programming. 1) Event-driven programming allows web pages to change dynamically according to user operations. 2) Dynamic content generation allows page content to be adjusted according to conditions. 3) Asynchronous programming ensures that the user interface is not blocked. JavaScript is widely used in web interaction, single-page application and server-side development, greatly improving the flexibility of user experience and cross-platform development.

There is no absolute salary for Python and JavaScript developers, depending on skills and industry needs. 1. Python may be paid more in data science and machine learning. 2. JavaScript has great demand in front-end and full-stack development, and its salary is also considerable. 3. Influencing factors include experience, geographical location, company size and specific skills.

Learning JavaScript is not difficult, but it is challenging. 1) Understand basic concepts such as variables, data types, functions, etc. 2) Master asynchronous programming and implement it through event loops. 3) Use DOM operations and Promise to handle asynchronous requests. 4) Avoid common mistakes and use debugging techniques. 5) Optimize performance and follow best practices.

Discussion on the realization of parallax scrolling and element animation effects in this article will explore how to achieve similar to Shiseido official website (https://www.shiseido.co.jp/sb/wonderland/)...

How to merge array elements with the same ID into one object in JavaScript? When processing data, we often encounter the need to have the same ID...

The latest trends in JavaScript include the rise of TypeScript, the popularity of modern frameworks and libraries, and the application of WebAssembly. Future prospects cover more powerful type systems, the development of server-side JavaScript, the expansion of artificial intelligence and machine learning, and the potential of IoT and edge computing.

In-depth discussion of the root causes of the difference in console.log output. This article will analyze the differences in the output results of console.log function in a piece of code and explain the reasons behind it. �...
