Home Technology peripherals It Industry Reddit was revealed to have blocked multiple search engines and AI crawlers, and officials said it had nothing to do with Google's cooperation

Reddit was revealed to have blocked multiple search engines and AI crawlers, and officials said it had nothing to do with Google's cooperation

Jul 25, 2024 pm 04:43 PM
reddit search engine AI crawler

News from this site on July 25. In February this year, Google announced a new agreement with Reddit. Reddit data will be used to train Google’s AI model and highlight Reddit results in Google search results. Bloomberg reports the deal is worth about $60 million. Since then, Reddit has also seen a significant increase in impressions on Google searches. The latest news is that Reddit has banned other search engines from retrieving its website content. Foreign media 404 Media first reported this behavior and pointed out that Reddit has updated its robots.txt file (this site’s note: the agreement file between the website and the crawler) to prevent all robots from crawling any content on the site. This site inquired about the robots.txt file of the Reddit website and learned that the platform stated that "Reddit believes in an open Internet, but does not believe in the abuse of public content." Its "Disallow" option is set to "/", which means that retrieval of content in the root directory of the website is prohibited. All files.

Reddit 被曝屏蔽多个搜索引擎和 AI 爬虫,官方称与谷歌合作无关

The document was first updated in June of this year, with Reddit explaining that the change was due to "a significant increase in commercial entities scraping Reddit content and misuse of the content."

Foreign media 9To5Google said that search engines such as Bing, DuckDuckGo, Mojeek and Qwant have all been affected. When searching for content using "site:reddit.com", either the latest content or complete website results are not displayed. .

Reddit 被曝屏蔽多个搜索引擎和 AI 爬虫,官方称与谷歌合作无关

However, in the latest test of this site, Bing and DuckDuckGo have re-displayed the search content of "site:reddit.com".

Reddit 被曝屏蔽多个搜索引擎和 AI 爬虫,官方称与谷歌合作无关

Reddit spokesperson Tim Rathschmidt said in a statement to The Verge:

This is completely unrelated to our recent partnership with Google .

We have been in discussions with multiple search engines, but have been unable to Come to terms with everyone because some people are unable or unwilling to make enforceable commitments regarding their use of Reddit content, including their use of AI training.”

The above is the detailed content of Reddit was revealed to have blocked multiple search engines and AI crawlers, and officials said it had nothing to do with Google's cooperation. For more information, please follow other related articles on the PHP Chinese website!

Statement of this Website
The content of this article is voluntarily contributed by netizens, and the copyright belongs to the original author. This site does not assume corresponding legal responsibility. If you find any content suspected of plagiarism or infringement, please contact admin@php.cn

Hot AI Tools

Undresser.AI Undress

Undresser.AI Undress

AI-powered app for creating realistic nude photos

AI Clothes Remover

AI Clothes Remover

Online AI tool for removing clothes from photos.

Undress AI Tool

Undress AI Tool

Undress images for free

Clothoff.io

Clothoff.io

AI clothes remover

AI Hentai Generator

AI Hentai Generator

Generate AI Hentai for free.

Hot Article

R.E.P.O. Energy Crystals Explained and What They Do (Yellow Crystal)
1 months ago By 尊渡假赌尊渡假赌尊渡假赌
R.E.P.O. Best Graphic Settings
1 months ago By 尊渡假赌尊渡假赌尊渡假赌
R.E.P.O. How to Fix Audio if You Can't Hear Anyone
1 months ago By 尊渡假赌尊渡假赌尊渡假赌
R.E.P.O. Chat Commands and How to Use Them
1 months ago By 尊渡假赌尊渡假赌尊渡假赌

Hot Tools

Notepad++7.3.1

Notepad++7.3.1

Easy-to-use and free code editor

SublimeText3 Chinese version

SublimeText3 Chinese version

Chinese version, very easy to use

Zend Studio 13.0.1

Zend Studio 13.0.1

Powerful PHP integrated development environment

Dreamweaver CS6

Dreamweaver CS6

Visual web development tools

SublimeText3 Mac version

SublimeText3 Mac version

God-level code editing software (SublimeText3)

Users encounter rare glitches: Samsung Watch smartwatches suddenly experience white screen issues Users encounter rare glitches: Samsung Watch smartwatches suddenly experience white screen issues Apr 03, 2024 am 08:13 AM

You may have encountered the problem of green lines appearing on the screen of your smartphone. Even if you have never seen it, you must have seen related pictures on the Internet. So, have you ever encountered a situation where the smart watch screen turns white? On April 2, CNMO learned from foreign media that a Reddit user shared a picture on the social platform, showing the screen of the Samsung Watch series smart watches turning white. The user wrote: "I was charging when I left, and when I came back, it was like this. I tried to restart, but the screen was still like this during the restart process." Samsung Watch smart watch screen turned white. The Reddit user did not specify the smart watch. Specific model. However, judging from the picture, it should be Samsung Watch5. Previously, another Reddit user also reported

How to change search engines on iPhone and iPad How to change search engines on iPhone and iPad Apr 25, 2023 am 08:28 AM

It's easy to change the search engine in Safari, Google Chrome, or other browsers on your iPhone or iPad. This tutorial will show you how to do it on four different web browsers available on iPhone and iPad. How to Change the Safari Search Engine on iPhone or iPad Safari is the default web browser on iOS and iPadOS, but you might not like the search engine. Fortunately, you can use these steps to change it: On your iPhone or iPad, launch Settings from the Home screen. Swipe down and tap Safari from the list. In the next menu,

How to Fix Reddit is Banned Funny Error in Reddit How to Fix Reddit is Banned Funny Error in Reddit Oct 26, 2023 pm 11:21 PM

What is "RedditisFunErrorForbidden"? Reddit is an interesting application with error-free messages that are commonly found on websites and social forums. This error pops up when a user tries to access certain areas or perform specific actions on the platform without the required permissions. The platform is a large online forum with many smaller forums and different ways for users to interact. It has certain rules and restrictions to keep things safe and orderly. What is the reason for "Reddit is banned for funny bug"? Understanding the problem will help users identify common causes. Here are some common causes of errors. Private subreddits: If the subreddit you are trying to access is set to private, you will need to

Baidu cloud disk search engine entrance Baidu cloud disk search engine entrance Feb 27, 2024 pm 01:00 PM

Baidu Cloud is a software that allows users to store many files. So what is the entrance to Baidu Cloud Disk search engine? Users can enter the URL https://pan.baidu.com to enter Baidu Cloud Disk. This sharing of the latest entrance to Baidu Cloud Disk search engine will give you a detailed introduction. The following is a detailed introduction. Take a look. . Baidu cloud disk search engine entrance 1. Qianfan search website: https://pan.qianfan.app Supports network disk: aggregate search, Alibaba, Baidu, Quark, Lanzuo, Tianyi, Xunlei network disk viewing method: login required, follow the company Advantages of obtaining the activation code: The network disk is comprehensive, there are many resources, and the interface is simple. 2. Maolipansou website: alipansou.c

Java development: How to implement search engine and full-text retrieval functions Java development: How to implement search engine and full-text retrieval functions Sep 21, 2023 pm 01:10 PM

Java development: How to implement search engine and full-text retrieval functions, specific code examples are required Search engines and full-text retrieval are important functions in the modern Internet era. Not only do they help users find what they want quickly, they also provide a better user experience for websites and apps. This article will introduce how to use Java to develop search engines and full-text retrieval functions, and provide some specific code examples. Full-text search using Lucene library Lucene is an open source full-text search engine library, developed by ApacheSo

Q&A platform Reddit launches post translation function, supports 8 languages Q&A platform Reddit launches post translation function, supports 8 languages Sep 17, 2023 am 11:21 AM

According to news from this website on September 9, the Q&A platform Reddit officially announced the launch of a translation function, which can translate iOS and Android posts into other languages, providing eight different languages ​​to choose from: English, Spanish, German, French, and Portuguese. , Italian, Dutch and Swedish. Translating from French to English The site noted that users can click the "Translate" button below their Reddit username and the post will be translated into the language selected by the user's settings. Reddit has also begun trying to translate comments on iOS and Android. "Soon, you will have a multilingual conversation experience on Reddit." Reddit reminded users to update the help center, merge the moderator help center and Redditor help center, and provide User provided

PHP search engine performance optimization: Algolia's magic trick PHP search engine performance optimization: Algolia's magic trick Jul 23, 2023 pm 04:21 PM

PHP Search Engine Performance Optimization: Algolia’s Magical Way With the development of the Internet and the increasing user requirements for search experience, search engine performance optimization has become crucial. In the world of PHP development, Algolia is a powerful and easy-to-integrate search engine service. This article will introduce the magical uses of Algolia and how to optimize the performance of PHP search engines through Algolia. Algolia introduction Algolia is a search engine service provider based on SaaS model.

In the field of artificial intelligence search, Google and Microsoft compete In the field of artificial intelligence search, Google and Microsoft compete Apr 08, 2023 am 11:31 AM

Since its launch late last year, ChatGPT has been seen as a major threat to traditional ways of searching for information. Because it is diverse, you can answer people's questions, write essays or poems, or even write program code. The ability of conversational AI to provide coherent answers is considered a threat to Google's search engine, which for decades has been the benchmark platform for people to search for information on the Internet. OpenAI’s ChatGPT can tailor answers to specific questions asked by users, which can save time browsing websites. A report published by The New York Times in December revealed that ChatGPT’s overnight success forced Google to call it “Code Red” and begin addressing the threat posed by artificial intelligence chatbots to its search engine business. according to

See all articles