采集一个网站的数据时，返回的是以chunked编码，gzip压缩的文档，该网站的服务器显示是IIS，。。。
解码chunked没问题，但是解压gzip压缩文档时，偶尔会失败，这样就影响我提取下一组请求连接了。。。
解压10组左右，就会出现解压失败的情况。。
这是解压前的数据：
解压后的数据：
显然在最后一组，解压失败了。。
这是尝试用过的三组方法：
private function _deCompressData() { if($this->is_gzip) { $this->response_body = gzinflate(substr($this->response_body,10)); // // if($temp = gzdecode($this->response_body)) {// $this->response_body = $temp;// } else {// $this->response_body = $this->mygzdecode($this->response_body);// } //$this->response_body = $this->mygzdecode($this->response_body); // $this->response_body = gzdecode($this->response_body); } } /** * @desc 自定义解压函数 */ function mygzdecode($data, &$filename = '', &$error = '', $maxlength = null) { $len = strlen($data); if ($len < 18 || strcmp(substr($data, 0, 2), "\x1f\x8b")) { $error = "Not in GZIP format."; return null; // Not GZIP format (See RFC 1952) } $method = ord(substr($data, 2, 1)); // Compression method $flags = ord(substr($data, 3, 1)); // Flags if ($flags & 31 != $flags) { $error = "Reserved bits not allowed."; return null; } // NOTE: $mtime may be negative (PHP integer limitations) $mtime = unpack("V", substr($data, 4, 4)); $mtime = $mtime[1]; $xfl = substr($data, 8, 1); $os = substr($data, 8, 1); $headerlen = 10; $extralen = 0; $extra = ""; if ($flags & 4) { // 2-byte length prefixed EXTRA data in header if ($len - $headerlen - 2 < 8) { return false; // invalid } $extralen = unpack("v", substr($data, 8, 2)); $extralen = $extralen[1]; if ($len - $headerlen - 2 - $extralen < 8) { return false; // invalid } $extra = substr($data, 10, $extralen); $headerlen += 2 + $extralen; } $filenamelen = 0; $filename = ""; if ($flags & 8) { // C-style string if ($len - $headerlen - 1 < 8) { return false; // invalid } $filenamelen = strpos(substr($data, $headerlen), chr(0)); if ($filenamelen === false || $len - $headerlen - $filenamelen - 1 < 8) { return false; // invalid } $filename = substr($data, $headerlen, $filenamelen); $headerlen += $filenamelen + 1; } $commentlen = 0; $comment = ""; if ($flags & 16) { // C-style string COMMENT data in header if ($len - $headerlen - 1 < 8) { return false; // invalid } $commentlen = strpos(substr($data, $headerlen), chr(0)); if ($commentlen === false || $len - $headerlen - $commentlen - 1 < 8) { return false; // Invalid header format } $comment = substr($data, $headerlen, $commentlen); $headerlen += $commentlen + 1; } $headercrc = ""; if ($flags & 2) { // 2-bytes (lowest order) of CRC32 on header present if ($len - $headerlen - 2 < 8) { return false; // invalid } $calccrc = crc32(substr($data, 0, $headerlen)) & 0xffff; $headercrc = unpack("v", substr($data, $headerlen, 2)); $headercrc = $headercrc[1]; if ($headercrc != $calccrc) { $error = "Header checksum failed."; return false; // Bad header CRC } $headerlen += 2; } // GZIP FOOTER $datacrc = unpack("V", substr($data, -8, 4)); $datacrc = sprintf('%u', $datacrc[1] & 0xFFFFFFFF); $isize = unpack("V", substr($data, -4)); $isize = $isize[1]; // decompression: $bodylen = $len - $headerlen - 8; if ($bodylen < 1) { // IMPLEMENTATION BUG! return null; } $body = substr($data, $headerlen, $bodylen); $data = ""; if ($bodylen > 0) { switch ($method) { case 8: // Currently the only supported compression method: $data = gzinflate($body, $maxlength); break; default: $error = "Unknown compression method."; return false; } } // zero-byte body content is allowed // Verifiy CRC32 $crc = sprintf("%u", crc32($data)); $crcOK = $crc == $datacrc; $lenOK = $isize == strlen($data); if (!$lenOK || !$crcOK) { $error = ( $lenOK ? '' : 'Length check FAILED. ') . ( $crcOK ? '' : 'Checksum FAILED.'); return false; } return $data; }
php 已经提供了 gzdecode 函数
如果你的 php 版本实在很低，没有 gzdecode 函数
那么 php 代码级的 gzdecode 函数是
function gzdecode($data) { $len = strlen($data); if ($len < 18 || strcmp(substr($data,0,2),"\x1f\x8b")) { return $data; // Not GZIP format (See RFC 1952) } $method = ord(substr($data,2,1)); // Compression method $flags = ord(substr($data,3,1)); // Flags if ($flags & 31 != $flags) { // Reserved bits are set -- NOT ALLOWED by RFC 1952 return data; } // NOTE: $mtime may be negative (PHP integer limitations) $mtime = unpack("V", substr($data,4,4)); $mtime = $mtime[1]; $xfl = substr($data,8,1); $os = substr($data,8,1); $headerlen = 10; $extralen = 0; $extra = ""; if ($flags & 4) { // 2-byte length prefixed EXTRA data in header if ($len - $headerlen - 2 < 8) { return false; // Invalid format } $extralen = unpack("v",substr($data,8,2)); $extralen = $extralen[1]; if ($len - $headerlen - 2 - $extralen < 8) { return false; // Invalid format } $extra = substr($data,10,$extralen); $headerlen += 2 + $extralen; } $filenamelen = 0; $filename = ""; if ($flags & 8) { // C-style string file NAME data in header if ($len - $headerlen - 1 < 8) { return false; // Invalid format } $filenamelen = strpos(substr($data,8+$extralen),chr(0)); if ($filenamelen === false || $len - $headerlen - $filenamelen - 1 < 8) { return false; // Invalid format } $filename = substr($data,$headerlen,$filenamelen); $headerlen += $filenamelen + 1; } $commentlen = 0; $comment = ""; if ($flags & 16) { // C-style string COMMENT data in header if ($len - $headerlen - 1 < 8) { return false; // Invalid format } $commentlen = strpos(substr($data,8+$extralen+$filenamelen),chr(0)); if ($commentlen === false || $len - $headerlen - $commentlen - 1 < 8) { return false; // Invalid header format } $comment = substr($data,$headerlen,$commentlen); $headerlen += $commentlen + 1; } $headercrc = ""; if ($flags & 1) { // 2-bytes (lowest order) of CRC32 on header present if ($len - $headerlen - 2 < 8) { return false; // Invalid format } $calccrc = crc32(substr($data,0,$headerlen)) & 0xffff; $headercrc = unpack("v", substr($data,$headerlen,2)); $headercrc = $headercrc[1]; if ($headercrc != $calccrc) { return false; // Bad header CRC } $headerlen += 2; } // GZIP FOOTER - These be negative due to PHP's limitations $datacrc = unpack("V",substr($data,-8,4)); $datacrc = $datacrc[1]; $isize = unpack("V",substr($data,-4)); $isize = $isize[1]; // Perform the decompression: $bodylen = $len-$headerlen-8; if ($bodylen < 1) { // This should never happen - IMPLEMENTATION BUG! return null; } $body = substr($data,$headerlen,$bodylen); $data = ""; if ($bodylen > 0) { switch ($method) { case 8: // Currently the only supported compression method: $data = gzinflate($body); break; default: // Unknown compression method return false; } } else { // I'm not sure if zero-byte body content is allowed. // Allow it for now... Do nothing... } // Verifiy decompressed size and CRC32: // NOTE: This may fail with large data sizes depending on how // PHP's integer limitations affect strlen() since $isize // may be negative for large sizes. if ($isize != strlen($data) || crc32($data) != $datacrc) { // Bad format! Length or CRC doesn't match! return false; } return $data; } 自己对比一下，看看是否是你抄写错了
既然函数会在 传入长度 和 crc32 校验失败时返回假，那么你就应该判断一下再进行下一步工作
php 已经提供了 gzdecode 函数
如果你的 php 版本实在很低，没有 gzdecode 函数
那么 php 代码级的 gzdecode 函数是
function gzdecode($data) { $len = strlen($data); if ($len < 18 || strcmp(substr($data,0,2),"\x1f\x8b")) { return $data; // Not GZIP format (See RFC 1952) } $method = ord(substr($data,2,1)); // Compression method $flags = ord(substr($data,3,1)); // Flags if ($flags & 31 != $flags) { // Reserved bits are set -- NOT ALLOWED by RFC 1952 return data; } // NOTE: $mtime may be negative (PHP integer limitations) $mtime = unpack("V", substr($data,4,4)); $mtime = $mtime[1]; $xfl = substr($data,8,1); $os = substr($data,8,1); $headerlen = 10; $extralen = 0; $extra = ""; if ($flags & 4) { // 2-byte length prefixed EXTRA data in header if ($len - $headerlen - 2 < 8) { return false; // Invalid format } $extralen = unpack("v",substr($data,8,2)); $extralen = $extralen[1]; if ($len - $headerlen - 2 - $extralen < 8) { return false; // Invalid format } $extra = substr($data,10,$extralen); $headerlen += 2 + $extralen; } $filenamelen = 0; $filename = ""; if ($flags & 8) { // C-style string file NAME data in header if ($len - $headerlen - 1 < 8) { return false; // Invalid format } $filenamelen = strpos(substr($data,8+$extralen),chr(0)); if ($filenamelen === false || $len - $headerlen - $filenamelen - 1 < 8) { return false; // Invalid format } $filename = substr($data,$headerlen,$filenamelen); $headerlen += $filenamelen + 1; } $commentlen = 0; $comment = ""; if ($flags & 16) { // C-style string COMMENT data in header if ($len - $headerlen - 1 < 8) { return false; // Invalid format } $commentlen = strpos(substr($data,8+$extralen+$filenamelen),chr(0)); if ($commentlen === false || $len - $headerlen - $commentlen - 1 < 8) { return false; // Invalid header format } $comment = substr($data,$headerlen,$commentlen); $headerlen += $commentlen + 1; } $headercrc = ""; if ($flags & 1) { // 2-bytes (lowest order) of CRC32 on header present if ($len - $headerlen - 2 < 8) { return false; // Invalid format } $calccrc = crc32(substr($data,0,$headerlen)) & 0xffff; $headercrc = unpack("v", substr($data,$headerlen,2)); $headercrc = $headercrc[1]; if ($headercrc != $calccrc) { return false; // Bad header CRC } $headerlen += 2; } // GZIP FOOTER - These be negative due to PHP's limitations $datacrc = unpack("V",substr($data,-8,4)); $datacrc = $datacrc[1]; $isize = unpack("V",substr($data,-4)); $isize = $isize[1]; // Perform the decompression: $bodylen = $len-$headerlen-8; if ($bodylen < 1) { // This should never happen - IMPLEMENTATION BUG! return null; } $body = substr($data,$headerlen,$bodylen); $data = ""; if ($bodylen > 0) { switch ($method) { case 8: // Currently the only supported compression method: $data = gzinflate($body); break; default: // Unknown compression method return false; } } else { // I'm not sure if zero-byte body content is allowed. // Allow it for now... Do nothing... } // Verifiy decompressed size and CRC32: // NOTE: This may fail with large data sizes depending on how // PHP's integer limitations affect strlen() since $isize // may be negative for large sizes. if ($isize != strlen($data) || crc32($data) != $datacrc) { // Bad format! Length or CRC doesn't match! return false; } return $data; }
自己对比一下，看看是否是你抄写错了
既然函数会在 传入长度 和 crc32 校验失败时返回假，那么你就应该判断一下再进行下一步工作
在网络上传输的数据，出现错误是不可避免的，但概率不高
重读一下，通常就可以了
主要是你要有容错策略
自己对比一下，看看是否是你抄写错了
既然函数会在 传入长度 和 crc32 校验失败时返回假，那么你就应该判断一下再进行下一步工作
在网络上传输的数据，出现错误是不可避免的，但概率不高
重读一下，通常就可以了
主要是你要有容错策略
在网络上传输的数据，出现错误是不可避免的，但概率不高
重读一下，通常就可以了
主要是你要有容错策略
传输过程出问题，导致部分数据没有了，而解压失败。
把需要解压的文件加入解压列表，每隔5秒-10秒判断解压文件是否变化，如无变化，则解压，解压失败做标记，继续下一个解压。
传输过程出问题，导致部分数据没有了，而解压失败。
Why Windows cannot access the specified device path or file
What does terminal equipment mean?
Usage of fscanf function
Permanently free oa system
css
How to open vcf file in windows
oracle clear table data
What should I do if English letters appear when I turn on the computer and the computer cannot be turned on?
![[Web front-end] Node.js quick start](https://img.php.cn/upload/course/000/000/067/662b5d34ba7c0227.png)
