采集遇到一个怪异有关问题
采集遇到一个怪异问题
目标地址 www.zhibo8.com
本地环境:windows xp + xmapp 1.7.7
file_get_contents 或者 curl 都没问题,首页面完全获取;
放到国外的服务器上,linux, Apache 2.0
file_get_contents 为空
curl 会和浏览器访问打开的不一致,内容少好多,但是
而且我用curl发送模拟浏览器的全部header也是一样不完整。
请大伙帮分析一下,可能存在的原因及如何解决,谢谢!
------解决方案--------------------
估计是linux环境的问题吧
我之前抓www.fblife.com也是windows下curl测试通过,但同样代码linux下就得不到完整的
这个网站我在Ubuntu9.10下CURL测试通过,可以抓取完整,不知道是不是你linux配置问题?
------解决方案--------------------
暂时木有。。。很纠结
说来奇怪,程序放到网站目录下,从浏览器访问程序文件抓取的信息很完整,linux,windows都可以,但是,linux后台抓取就是不行,怎么做都抓不全,我估计可能是linux配置的问题。楼主可以试一下wget命令
------解决方案--------------------
服务器的网络.访问国内的不行.这个是我的猜测.在国内主机上多半没问题
------解决方案--------------------
看看编码问题吧
$string = iconv('gb2312', 'utf-8//ignore', $string);
------解决方案--------------------
到国外的服务器上
file_get_contents 为空
可能未开放权限,也可能超时
curl 会和浏览器访问打开的不一致,内容少好多
那是一个纯中文网站,在国外看不到或看不全是很正常的。没有多少人能看懂中文
当然也与超时和curl支持库有关
------解决方案--------------------
这就是一种优越性
------解决方案--------------------

Hot AI Tools

Undresser.AI Undress
AI-powered app for creating realistic nude photos

AI Clothes Remover
Online AI tool for removing clothes from photos.

Undress AI Tool
Undress images for free

Clothoff.io
AI clothes remover

Video Face Swap
Swap faces in any video effortlessly with our completely free AI face swap tool!

Hot Article

Hot Tools

Notepad++7.3.1
Easy-to-use and free code editor

SublimeText3 Chinese version
Chinese version, very easy to use

Zend Studio 13.0.1
Powerful PHP integrated development environment

Dreamweaver CS6
Visual web development tools

SublimeText3 Mac version
God-level code editing software (SublimeText3)

Hot Topics



VS Code can run on Windows 8, but the experience may not be great. First make sure the system has been updated to the latest patch, then download the VS Code installation package that matches the system architecture and install it as prompted. After installation, be aware that some extensions may be incompatible with Windows 8 and need to look for alternative extensions or use newer Windows systems in a virtual machine. Install the necessary extensions to check whether they work properly. Although VS Code is feasible on Windows 8, it is recommended to upgrade to a newer Windows system for a better development experience and security.

VS Code system requirements: Operating system: Windows 10 and above, macOS 10.12 and above, Linux distribution processor: minimum 1.6 GHz, recommended 2.0 GHz and above memory: minimum 512 MB, recommended 4 GB and above storage space: minimum 250 MB, recommended 1 GB and above other requirements: stable network connection, Xorg/Wayland (Linux)

The reasons for the installation of VS Code extensions may be: network instability, insufficient permissions, system compatibility issues, VS Code version is too old, antivirus software or firewall interference. By checking network connections, permissions, log files, updating VS Code, disabling security software, and restarting VS Code or computers, you can gradually troubleshoot and resolve issues.

VS Code is the full name Visual Studio Code, which is a free and open source cross-platform code editor and development environment developed by Microsoft. It supports a wide range of programming languages and provides syntax highlighting, code automatic completion, code snippets and smart prompts to improve development efficiency. Through a rich extension ecosystem, users can add extensions to specific needs and languages, such as debuggers, code formatting tools, and Git integrations. VS Code also includes an intuitive debugger that helps quickly find and resolve bugs in your code.

VS Code is available on Mac. It has powerful extensions, Git integration, terminal and debugger, and also offers a wealth of setup options. However, for particularly large projects or highly professional development, VS Code may have performance or functional limitations.

VS Code To switch Chinese mode: Open the settings interface (Windows/Linux: Ctrl, macOS: Cmd,) Search for "Editor: Language" settings Select "Chinese" in the drop-down menu Save settings and restart VS Code

The most common "cannot run Python" problem stems from the misconfiguration of the Python interpreter path. Solutions include: confirming Python installation, configuring VS Code, and using a virtual environment. In addition, there are efficient debugging techniques and best practices such as breakpoint debugging, variable monitoring, log output, and code formatting, such as isolating dependencies using virtual environments, tracking code execution using breakpoints, and tracking variable changes in real time using monitoring expressions, etc., which can greatly improve development efficiency.

The main uses of Linux include: 1. Server operating system, 2. Embedded system, 3. Desktop operating system, 4. Development and testing environment. Linux excels in these areas, providing stability, security and efficient development tools.
