Language is the expression symbol used by people to communicate and communicate. Each language has its own symbols, expressions and rules. As far as programming language is concerned, it is also composed of specific symbols, specific expressions and rules. The function of language is communication, whether it is natural language or programming language. The difference between them is that natural language is a tool for communication between people, while programming language is a communication channel between people and machines.
As far as the PHP language is concerned, it is also a set of agreed instructions that comply with certain rules. After programmers implement their ideas in PHP language, these PHP instructions are converted into C language through PHP's virtual machine (to be precise, it should be PHP's language engine Zend). (can be understood as a lower-level instruction set) instructions, and the C language will be converted into assembly language. Finally, the assembly language will be converted into machine code for execution according to the rules of the processor. This is a process of constant concretization and refinement of higher-level abstractions.
The transformation from one language to another is called compilation, and the two languages can be called source language and target language respectively. This compilation process occurs when the target language is lower level (or lower level) than the source language.
The compilation process of language conversion is completed by the compiler. The encoder is usually divided into a series of processes: lexical analysis, syntax analysis, semantic analysis, intermediate code generation, code optimization, target code generation, etc. The function of the previous stages (lexical analysis, syntax analysis and semantic analysis) is to analyze the source program, which we can call the front end of the compiler. The function of the following stages (intermediate code generation, code optimization and target code generation) is to construct the target program, which we can call the back-end of the compiler. A language is called a compiled language, generally because there is a translation process before the program is executed. The key point is that an equivalent program with a completely different form is generated. The reason why PHP is called an interpreted language is because there is no such program to generate it.
What it generates is the intermediate code Opcode, which is just an internal data structure of PHP.
For example, let’s write a simple program
<?php echo "Hello World!"; $a = 1 + 1; echo $a; ?>
1.Scanning(Lexing) ,将PHP代码转换为语言片段(Tokens) 2.Parsing, 将Tokens转换成简单而有意义的表达式 3.Compilation, 将表达式编译成Opocdes 4.Execution, 顺次执行Opcodes,每次一条,从而实现PHP脚本的功能。
Note 2: Some current caches, such as APC, can enable PHP to cache Opcodes. In this way, every time a request comes, there is no need to repeat the first three steps, which can greatly improve the execution speed of PHP.
So what is Lexing? Students who have studied the principles of compilation should have an understanding of the lexical analysis steps in the principles of compilation. Lex is a basis table for lexical analysis.
For PHP, Flex was used at the beginning, and later changed to re2c. MySQL used Flex for lexical analysis. In addition, Lex is the standard lexical analyzer for UNIX systems, etc. These tools all read in an input string stream representing the lexical analyzer rules, and then output the lexical analyzer source code implemented in C language. Here we only introduce PHP’s current lexical analyzer, re2c. The Zend/zend_language_scanner.l file in the source code directory is the re2c rule file. If you need to modify the rule file, you need to install re2c to recompile and generate a new rule file. Zend/zend_language_scanner.c will be input based on Zend/zend_language_scanner.l. The PHP code performs lexical analysis to obtain "words" one by one.
Starting from PHP 4.2, a function called token_get_all is provided. This function can Scanning a piece of PHP code into Tokens;
We use the following code to use the token_get_all function to process the PHP code we mentioned at the beginning.
"; $phpcode = <<$token) { $tokens[$key][0] = token_name($token[0]); } print_r($tokens); ?>
Note: In order to facilitate understanding and viewing, I used the token_name function to change the parser code name into a symbol name description.
If some children want to see the original version, you can remove the comments on lines 10 and 11 in the above code.
For a detailed list of interpreter codenames, please see: http://www.php.net/manual/zh/tokens.php
The result obtained is as follows:
Array ( [0] => Array ( [0] => T_OPEN_TAG [1] => 1 ) [1] => Array ( [0] => T_WHITESPACE [1] => [2] => 2 ) [2] => Array ( [0] => T_ECHO [1] => echo [2] => 2 ) [3] => Array ( [0] => T_WHITESPACE [1] => [2] => 2 ) [4] => Array ( [0] => T_CONSTANT_ENCAPSED_STRING [1] => "Hello World!" [2] => 2 ) [5] => [6] => Array ( [0] => T_WHITESPACE [1] => [2] => 2 ) [7] => [8] => Array ( [0] => T_WHITESPACE [1] => [2] => 3 ) [9] => Array ( [0] => T_LNUMBER [1] => 1 [2] => 3 ) [10] => Array ( [0] => T_WHITESPACE [1] => [2] => 3 ) [11] => [12] => Array ( [0] => T_WHITESPACE [1] => [2] => 3 ) [13] => Array ( [0] => T_LNUMBER [1] => 1 [2] => 3 ) [14] => [15] => Array ( [0] => T_WHITESPACE [1] => [2] => 3 ) [16] => Array ( [0] => T_ECHO [1] => echo [2] => 4 ) [17] => Array ( [0] => T_WHITESPACE [1] => [2] => 4 ) [18] => [19] => Array ( [0] => T_WHITESPACE [1] => [2] => 4 ) [20] => Array ( [0] => T_CLOSE_TAG [1] => ?> [2] => 5 ) )
Analyzing this return result, we can find that the strings, characters, and spaces in the source code will be returned unchanged.
Every character in the source code will appear in the corresponding order.
Others, such as tags, operators, and statements, will be converted into a three-part
1. Token ID interpreter code (That is, changing the corresponding code of Token inside Zend, such as T_ECHO, T_STRING)
2. Original content in the source code
3. Which line is this word in the source code?
Next, is the Parsing stage. Parsing will first discard excess spaces in the Tokens Array,
然后将剩余的Tokens转换成一个一个的简单的表达式
1.echo a constant string 2.add two numbers together 3.store the result of the prior expression to a variable 4.echo a variable
Bison是一种通用目的的分析器生成器。它将LALR(1)上下文无关文法的描述转化成分析该文法的C程序。 使用它可以生成解释器,编译器,协议实现等多种程序。 Bison向上兼容Yacc,所有书写正确的Yacc语法都应该可以不加修改地在Bison下工作。 它不但与Yacc兼容还具有许多Yacc不具备的特性。
Bison分析器文件是定义了名为yyparse并且实现了某个语法的函数的C代码。 这个函数并不是一个可以完成所有的语法分析任务的C程序。 除此这外我们还必须提供额外的一些函数: 如词法分析器、分析器报告错误时调用的错误报告函数等等。 我们知道一个完整的C程序必须以名为main的函数开头,如果我们要生成一个可执行文件,并且要运行语法解析器, 那么我们就需要有main函数,并且在某个地方直接或间接调用yyparse,否则语法分析器永远都不会运行。
在PHP源码中,词法分析器的最终是调用re2c规则定义的lex_scan函数,而提供给Bison的函数则为zendlex。 而yyparse被zendparse代替。
在PHP实现内部,opcode由如下的结构体表如下:
struct _zend_op { opcode_handler_t handler; // 执行该opcode时调用的处理函数 znode result; znode op1; znode op2; ulong extended_value; uint lineno; zend_uchar opcode; // opcode代码 };
和CPU的指令类似,有一个标示指令的opcode字段,以及这个opcode所操作的操作数。
PHP不像汇编那么底层, 在脚本实际执行的时候可能还需要其他更多的信息,extended_value字段就保存了这类信息。
其中的result域则是保存该指令执行完成后的结果。
PHP脚本编译为opcode保存在op_array中,其内部存储的结构如下:
struct _zend_op_array { /* Common elements */ zend_uchar type; char *function_name; // 如果是用户定义的函数则,这里将保存函数的名字 zend_class_entry *scope; zend_uint fn_flags; union _zend_function *prototype; zend_uint num_args; zend_uint required_num_args; zend_arg_info *arg_info; zend_bool pass_rest_by_reference; unsigned char return_reference; /* END of common elements */ zend_bool done_pass_two; zend_uint *refcount; zend_op *opcodes; // opcode数组 zend_uint last,size; zend_compiled_variable *vars; int last_var,size_var; // ... }
ZEND_API void execute(zend_op_array *op_array TSRMLS_DC) { // ... 循环执行op_array中的opcode或者执行其他op_array中的opcode }
前面提到每条opcode都有一个opcode_handler_t的函数指针字段,用于执行该opcode。
PHP有三种方式来进行opcode的处理:CALL,SWITCH和GOTO。
PHP默认使用CALL的方式,也就是函数调用的方式, 由于opcode执行是每个PHP程序频繁需要进行的操作,
可以使用SWITCH或者GOTO的方式来分发, 通常GOTO的效率相对会高一些,
不过效率是否提高依赖于不同的CPU。
在我们上面的例子中,我们的PHP代码会被Parsing成:* ZEND_ECHO 'Hello World%21' * ZEND_ADD ~0 1 1 * ZEND_ASSIGN !0 ~0 * ZEND_ECHO !0 * ZEND_RETURN 1
a)op_type : 为IS_CONST, IS_TMP_VAR, IS_VAR, IS_UNUSED, or IS_CV b)u,一个联合体,根据op_type的不同,分别用不同的类型保存了这个操作数的值(const)或者左值(var)