php:与进程的并发。角与 shmop 的进程间通信

PHPz
发布: 2024-08-20 06:34:37
原创
439 人浏览过

php 不是一种开发人员通常会考虑内存之类的语言。我们只是围绕变量和函数进行处理,然后让内部为我们找出所有“内存的东西”。让我们改变这一点。

在本系列的第一部分中,我们构建了一个 php 脚本,该脚本能够通过分叉子进程同时运行多个任务。它工作得很好,但有一个明显的、未解决的问题:这些子进程无法将数据发送回父进程。

在本期中,我们将通过使用 shmop(php 的“共享内存操作”)来解决该问题。

php: concurrency with processes. pt. interprocess communication with shmop
shmop!

内容

  • 关于共享内存的非常简短且极其可选的解释
  • shmop 的基本天桥
  • 实际实施
  • 使用 shmop_open 打开内存块
  • 使用 shmop_write 来...写入
  • 使用 shmop_read 读取内存
  • 使用 shmop_delete 进行清理
  • 处理错误

对共享内存的非常简短且极其可选的解释

当一个新进程启动时,操作系统会为其分配一块内存以供使用。进程无法读取或写入不属于自己的内存,因为这将是一场安全噩梦。完全合理。

然而,在处理本系列第一部分中使用 pcntl_fork 创建的进程时,这给我们带来了一个问题,因为这意味着子进程没有简单的方法可以相互通信或与父进程通信。子进程在创建时将获得其父进程内存的副本,因此在 fork 之前分配的所有变量都可以访问,但对这些变量的任何更改都将仅限于子进程。不同的记忆等等。如果我们希望子进程能够写入父进程可以读取的变量,我们就会遇到问题。

对此有许多解决方案,所有这些解决方案都归入“进程间通信”或 iPC 的一般类别下。我们将用于 php 脚本的是“共享内存”。

顾名思义,共享内存是任意数量的进程可以访问的内存块。共享内存块由(希望)唯一的键来标识。任何知道密钥是什么的进程都可以访问该内存块。这使得子进程可以向其父进程报告;子进程将数据写入共享内存块,退出后,父进程将读取共享数据。这是一个优雅的解决方案。

当然,在执行此操作时,我们需要避免一些问题:我们需要确保标识共享内存块的密钥是唯一的,并且我们需要强制共享内存通信只能进行一次避免多个进程同时尝试写入同一个块并造成混乱的方法。我们将在实现中涵盖所有这些。

shmop 的基本天桥

php 有丰富而强大的 api 来处理共享内存。手册中指出“Shmop 是一组易于使用的函数,允许 PHP 读取、写入、创建和删除 Unix 共享内存段”,并且......这没有错。

我们来看看使用共享内存的核心步骤:

创建唯一的key:所有共享内存都由一个key来标识。任何知道共享内存块的键是什么的进程都可以访问它。传统上,该密钥是通过从文件系统生成数据(即从现有文件的索引节点构建的值)来创建的,因为文件系统是所有进程共有的。我们将为此使用 ftok。

使用键分配内存块:进程可以使用共享内存块的键通过 shmop_open“打开”它。如果共享内存块不存在,则创建它。 open函数的返回值是可用于读写的指针。如果你以前使用过 fopen 和 fwrite,这个过程应该很熟悉。

将数据写入内存块:写入共享内存的接口与fwrite非常相似。使用指针,并将要写入内存的字符串作为参数传递。执行此操作的函数称为 shmop_write。

从内存块读取数据:从共享内存读取数据是通过 shmop_read 完成的,同样使用 shmop_open 中的指针。返回值是一个字符串。

使用键删除内存块:在不再需要共享内存后删除它很重要。这是通过 shmop_delete 完成的。

实际执行

让我们从一个例子开始。下面的代码可以工作,如果你足够不好奇或者是 tl;dr 类型,你可以复制粘贴修改它,但对于其他人,我们将回顾所有 shmop 步骤并解释它们的作用以及它们如何工作。

// the file used by ftok. can be any file.
$shmop_file = "/usr/bin/php8.3";

for($i = 0; $i < 4; $i++) {
    // create the fork
    $pid = pcntl_fork();

    // an error has ocurred
    if($pid === -1) {
        echo "error".PHP_EOL;
    }

    // child process
    else if(!$pid) {

        // create a random 'word' for this child to write to shared memory 
        $random_word = join(array_map(fn($n) => range('a', 'z')[rand(0, 25)], range(1,5)));

        // write to shmop
        $shm_key = ftok($shmop_file, $i);
        $shm_id = shmop_open($shm_key, 'n', 0755, 1024);
        shmop_write($shm_id, $random_word, 0);

        print "child $i wrote '$random_word' to shmop".PHP_EOL;

        // terminate the child process
        exit(0);
    }
}

// wait for all child processes to finish
while(($pid = pcntl_waitpid(0, $status)) != -1) {
    echo "pid $pid finished".PHP_EOL;
}

// read all our shared memories
for($i = 0; $i < 4; $i++) {

    // recreate the shm key
    $shm_key = ftok($shmop_file, $i);

    // read from the shared memory
    $shm_id = shmop_open($shm_key, 'a', 0755, 1024);
    $shmop_contents = shmop_read($shm_id, 0, 1024);

    print "reading '$shmop_contents' from child $i".PHP_EOL;

    // delete the shared memory so the shm key can be reused in future runs of this script
    shmop_delete($shm_id);
}
登录后复制

使用 ftok 创建共享内存密钥

正如我们上面介绍的,所有共享内存块都由唯一的整数键标识,在我们开始分配内存的任务之前,我们已经创建了该键。

老实说,我们可以使用任何我们想要的整数,只要它是唯一的,但是普遍接受的规范方法是使用 ftok 使用文件系统中的现有文件作为整数来创建整数参考点。

这样做的理由非常简单。进程彼此之间一无所知,这使得它们很难共享共同商定的值。不过,系统上所有进程确实所共有的少数几个共同点之一就是文件系统。因此,ftok。

除了现有文件的路径之外,ftok 还接受一个project_id 参数。根据文档,这是一个“单字符串”,使用其他编程语言的人会称之为“字符”。项目id的目的是防止创建共享内存时发生冲突。如果两个不同供应商的两个项目都决定使用 /etc/passwd 作为 ftok 的参数,那么混乱就会随之而来。

让我们看一个相当简单的例子:

$shm_key = ftok('/usr/bin/php8.3', 'j');
print "shm_key = $shm_key";
登录后复制

在这里,我们将完整路径传递给我们知道系统上存在的文件,并提供一个字符的project_id“j”。如果我们运行这个,打印语句将输出如下内容:

shm_key = 855706266
登录后复制

这是一个很好的整数,可以用来创建我们的共享内存!

如果您在系统上运行此代码,即使您使用了相同的参数,您几乎肯定会得到不同的返回值。这是因为,在底层,ftok 使用文件的 inode,而这在不同的系统中是不同的。

如果由于某种原因,我们传递给 ftok 一个存在的文件,我们会收到警告。

PHP Warning:  ftok(): ftok() failed - No such file or directory in <our script> on line <the line>
shm_key = -1
登录后复制

请注意,这只是一个警告,ftok 会向前冲并给我们一个 -1 的值,这将导致以后出现问题。小心点。

现在,让我们重新审视第 20 行对 ftok 的调用:

$shm_key = ftok($shmop_file, $i);
登录后复制

这里我们向 ftok 传递了我们在 $shm_key 中设置的文件的路径,在本例中为 /usr/bin/php8.3,我们知道系统上存在一个文件。

对于我们的project_id,我们使用$i,我们正在循环的数组的索引。我们这样做是为了让每个子进程都有自己的共享内存块来存储其结果。请记住,如果多个进程尝试写入共享内存,就会发生不好的事情。在这里使用索引可以帮助我们避免这种情况。

使用 shmop_open 打开内存块

如果您曾经使用 php 的 fopen 和 fwrite 等工具进行过文件访问,那么使用 shmop 将会非常熟悉。

让我们从使用 shmop_open 打开共享内存块开始:

$shm_id = shmop_open($shm_key, 'n', 0755, 1024);
登录后复制

这个函数有四个参数:

  • key:我们使用 ftok 创建的唯一密钥。
  • 模式:我们想要的访问类型。当使用 fopen 打开文件时,我们使用 r 表示“读取”或 w 表示写入等模式。 shmop_open 的模式与此类似,但也有区别。我们将回顾以下所有内容。
  • permissions:共享内存块的读/写/执行权限,以八进制表示。处理共享内存的接口与文件访问非常相似,其中包括权限。如果您对八进制表示法没有信心,可以使用文件权限计算器。我们在此示例中使用 0755,但您可能需要收紧它。
  • size:内存块的大小(以字节为单位)。在示例中,我们分配了 1 MB,这显然是多余的。但是,请注意,如果我们覆盖共享内存块,该值将被截断。如果我们尝试从内存块中读取比其大小更多的字节,则会发生致命错误。如果我们不确定要写入内存的数据的确切大小,最好高估我们需要的内存块有多大。

这里的一个重要注意事项是,如果该键上的内存块尚不存在,则调用 shmop_open 将创建一个新的内存块。这与 fopen 的行为类似,但对于 shmop_open,此行为取决于我们传递的“模式”参数。

如示例所示,shmop_open 返回一个可用于访问的指针:读或写,具体取决于用于打开内存块的模式。

php: concurrency with processes. pt. interprocess communication with shmop
007 are bad permissions for a spy

a little bit more about that 'mode' argument

the mode argument that we pass to shmop_open determines how we can access our shared memory block. there are four options, all covered in the official documentation, but for the sake of simplicity, we'll only look at the two we need for our purposes.

  • n : the 'n' stands for 'new' and is used when we want to create a new shared memory block. there does exist a mode c for 'create', but we are choosing to use n here because this mode will fail if we try to open a memory block that already exists. that's a safety feature! in fact, the docs state that using n to create a new shared memory block is 'useful' for 'security purposes'. the pointer returned from shmop_open using mode n is writeable.
  • a : this is for 'access'; ie. reading. do not confuse this with fopen's a mode, which is for 'append'. memory blocks opened with mode a are read-only.

if we look at the example, we can see that when we open the shared memory block in the child process to write our data, we use the n mode. this creates the new memory block in a safe way and returns a pointer that we can write to.

using shmop_write to... write.

once our child process has created a new shared memory block and received a pointer to it, it can write whatever it wants there using shmop_write.

in our example, doing this looks like:

shmop_write($shm_id, $random_word, 0);
登录后复制

the shmop_write function takes three arguments:

  • the pointer: the pointer returned from shmop_open. note that shmop_open must be called with a mode that allows writing (n in our example), otherwise attempts to use shmop_write will fail.
  • the value to write: the string to write to the shared memory block.
  • the offset: the number of bytes in memory to offset the start point of the write by. using the offset can allow us to append to a value already in the shared memory block, but doing this means keeping track of bytes written and can become unmanageable pretty quickly. in our example, we use the offset 0; we start writing at the beginning of the memory block.

shmop_write returns, as an integer, the number of bytes written.

a short note about shmop_close

if you've done file access using fopen, you're probably (hopefully!) in the habit of calling fclose when you're done writing.

we do not do that with shmop.

there is a shmop_close function, but it has been deprecated since php 8.0 and does nothing (other than throw a deprecation warning, that is). the standard practice with shmop is to just leave the pointer 'open' after we're done writing. we'll delete it later.

reading from shared memory

once all the child processes have written their data to their respective shared memory blocks an exited, all that remains is for the parent process to read that data. the strategy for this is:

  • recreate the key of the shared memory block
  • use the key to open the shared memory in 'access only' mode
  • read the data into a variable

let's look again at the example we have for reading shared memory.

// read all our shared memories
for($i = 0; $i < 4; $i++) {

    // recreate the shm key
    $shm_key = ftok($shmop_file, $i);

    // read from the shared memory
    $shm_id = shmop_open($shm_key, 'a', 0755, 1024);
    $shmop_contents = shmop_read($shm_id, 0, 1024);

    print "reading '$shmop_contents' from child $i".PHP_EOL;

    // delete the shared memory so the shm key can be reused in future runs of this script
    shmop_delete($shm_id);
}
登录后复制

recreating the shmop key

when we made the key to create our shared memory blocks, we used ftok with two arguments: the path an existing file in the filesystem, and a 'project id'. for the project id, we used the index of the array we looped over to fork multiple children.

we can use the exact same strategy to recreate the keys for reading. as long as we input the same two arguments into ftok, we get the same value back.

opening the shared memory

we open the shared memory block for reading almost exactly the same way as we did above for writing. the only difference is the mode.

for reading, we use the a mode. this stands for 'access', and gives us a read-only pointer to our shared memory block.

reading from the shared memory block

once we have a pointer to our shared memory block, we can read from it using shmop_read.

shmop_read takes three arguments:

  • the pointer we got from shmop_open.
  • the offset, in bytes. since we are reading the entirety of the memory block, starting at the beginning, this is 0 in our example (and will probably be for most real-life uses, as well)
  • the number of bytes to read. in most cases, the smart thing here is to just read the entire size of the block, in our example 1024 bytes.

the return type is a string. if there are errors reading, we get a boolean false.

deleting shared memory blocks

once we are done reading our shared memory, we can delete it.

this is an important step. unlike variables in our script, the memory we assigned with shmop will persist after our program has exited, hogging resources. we do not want to litter our system with blocks of unused, reserved memory, piling up higher and higher with each successive run of our script!

freeing up shared memory blocks is done with shmop_delete. this function takes one argument: the pointer we created with shmop_open, and returns a boolean true on success.

note that shmop_delete destroys the memory block and frees up the space for other applications to use. we should only call it when we're completely done with using the memory.

handling errors

the example we've been going over doesn't really do any error handling. this is a decision borne out of a desire for brevity, not delusional optimism. in real applications we should certainly do some error testing!

we used a path to a file as an argument for ftok; we should test that it exists. shmop_write will throw a value error if our memory block is opened read-only or we overwrite its size. that should be handled. if there's a problem reading data, shmop_read will return false. test for that.

php: concurrency with processes. pt. interprocess communication with shmop
i am asking you to do some error handling

fixing 'already exists' errors with shmop_open

if we open a shared memory block and then the script terminates before we call shmop_delete, the memory block still exists. if we then try to open that memory block again with shmop_open using the n mode, we will get the error:

PHP Warning:  shmop_open(): Unable to attach or create shared memory segment "File exists" in /path/to/my/script on line <line number>
登录后复制

if our script is well-designed, this shouldn't happen. but, while developing and testing we may create these orphaned memory blocks. let's go over how to delete them.

the first step is to get the key of the memory block as a hex number. we do this by calling ftok as normal, and then converting the returned integer from base ten to base-16 like so:

$shm_key = ftok($shmop_file, $i);
$shm_key_hex = "0x".base_convert($shm_key, 10, 16);
登录后复制

we do this because linux comes with a number of 'interprocess communication' tools that we can use to manage shared memory blocks, and they all use hexadecimal numbers for their keys.

the first command line tool we'll use is ipcs. we're going to use this to confirm that the shared memory block we want to delete does, in fact, exist.

the ipcs command, when run without arguments, will output all interprocess communication channels, including all shared memory blocks. we'll narrow down that output by using grep with the hexadecimal key we created above. for instance, if our shared memory block's key in hexadecimal is 0x33010024, we could do this:

ipcs | grep "0x33010024"
登录后复制

if we get a line of output, the memory block exists. if nothing is returned, it does not.

once we've confirm that a shared memory block exists, we can remove it with ipcrm

ipcrm --shmem-key 0x33010024
登录后复制

knowing how to inspect and clean up (without resorting to a restart) shared memory allows us to develop and experiment without turning our ram into a ghost town of abandoned blocks.

wrapping up

achieving concurrency in php using fork and shared memory does take some effort and knowledge (and the official manual is scant help). but it does work and, if you've made it through this article and the first installment on pcntl_fork, you should have a good base from which to start.

? this post originally appeared in the grant horwood technical blog

以上是php:与进程的并发。角与 shmop 的进程间通信的详细内容。更多信息请关注PHP中文网其他相关文章!

来源:dev.to
本站声明
本文内容由网友自发贡献,版权归原作者所有,本站不承担相应法律责任。如您发现有涉嫌抄袭侵权的内容,请联系admin@php.cn
热门教程
更多>
最新下载
更多>
网站特效
网站源码
网站素材
前端模板
关于我们 免责声明 Sitemap
PHP中文网:公益在线PHP培训,帮助PHP学习者快速成长!