What is HTTP Referer
In short, HTTP Referer is part of the header. When the browser sends a request to the web server, it usually brings the Referer to tell the server where I am from. The page is linked to, so that the server can obtain some information for processing. For example, if I link to a friend from my homepage, his server can count from the HTTP Referer how many users click on the link on my homepage to visit his website every day.
Referer should actually be the English word Referrer, but too many people misspelled it, so the people who wrote the standard just made the mistake.
My problem
I just changed the feed reader to Gregarius, but it is not like the liferea I used before. When I visit the Sina blog, the pictures cannot be displayed, and it prompts "This picture is limited to Sina blog users" "Exchange and communication", I know, this is caused by HTTP Referer.
Due to the particularity of my Internet client configuration, I first suspected that it was a problem with Squid, but it was ruled out through experiments. However, I also discovered a privacy leak issue involving the collaborative use of Squid, Tor, and Privoxy, which will be left for future research.
Can Gregarius handle this problem?
The answer is no, because Gregarius is only responsible for outputting html code, and access to images is requested by the client browser from the server.
However, installing a firefox extension may solve the problem. I did not find the "Send Referrer" recommended in the article, but I found another available one: "RefControl", which can control the use of different Referrers according to different websites visited.
But I don’t like using Firefox extensions to solve the problem because I think it is too inefficient, so I use a better way - Privoxy.
Privoxy is awesome
Add two lines in Privoxy's default.action:
{+hide-referrer{forge}}
.album.sina.com.cn
This way Gregarius is in Sina Will the pictures on the blog come out? +hide-referrer is a filter of Privoxy. It sets the processing method of HTTP Referer during access. The following forge means using the access address as the Referer. It can also be changed to block, which means canceling the Referer, or directly using the Referer that needs to be used. The URL is written here.
Using Privoxy is much easier than using Firefox, so switch quickly.
From https to http
I also found that when accessing a non-encrypted http page from a link on an https page, the HTTP Referer cannot be checked on the http page. For example, when I click on my The w3c xhtml verification icon under the https page (the URL is http://validator.w3.org/check?uri=referer) has never been able to complete the verification, prompting:
No Referer header found!
It turns out that, It is defined in the rfc document of the http protocol:
Clients SHOULD NOT include a Referer header field in a (non-secure)
HTTP request if the referring page was transferred with a secure
protocol.
This is for security reasons. When accessing a non-encrypted page, if the source is an encrypted page, the client will not send a Referer. IE has always implemented this, and Firefox browser is no exception. But this does not affect access from encrypted pages to encrypted pages.
The settings for Referer in Firefox
are all there, there are two key values:
network.http.sendRefererHeader (default=2) Set the sending method of Referer, 0 means not sending at all, 1 means only It is sent when the link is clicked, not when the image in the page is accessed, and 2 is always sent. See Privacy Tip #3: Block Referer Headers in Firefox
network.http.sendSecureXSiteReferrer (default=true) Set whether to send Referer when accessing from one encrypted page to another encrypted page. True means sending, false means not sending.
Use Referer to prevent picture hotlinking
Although Referer is not reliable, it is enough to prevent picture hotlinking. After all, not everyone will modify the configuration of the client. . The implementation is generally through the apache configuration file. First, set the address that is allowed to be accessed and mark it:
# Only access from don.com is allowed. The image may be placed on the page of the don.com website
SetEnvIfNoCase Referer "^http://www.don.com/" local_ref
# Access directly through the address
SetEnvIf Referer "^$" local_ref
Then specify that only marked access is allowed:
or
The variable HTTP-REFERER has become increasingly unreliable, and can be forged.
Here’s how to fake it:
PHP (provided curl is installed):
javascript
xmlHttp.setRequestHeader("Referer", "http://URL");// Haha~fake~
JS does not support ^_^
The principle is that sock constructs the http header to send data. Other languages such as perl can also be used.
Currently the simplest way to defend against forged referers is to use a verification code (Session).
There are now some commercial companies that can prevent hotlinking software, such as UUDOG, linkgate, VirtualWall, etc., all of which have developed dlls for use on IIS.
Some use cookie verification and thread control, and some can randomly generate file names and then perform URL rewriting. Some methods can indeed achieve good results.
However, the truth is as high as the devil, and there are ways to crack these trivial tricks after all.
This is generally the case, but the server is not easy to forge and can only create a small amount of data. If it can be forged by accessing the web page, then real forgery can be achieved and natural IP distribution can be achieved.