ChatGPT's 'nemesis” is here! Is the author of the article human or AI? It took a few seconds to detect it. The Chinese guy got it done during the New Year's Day holiday. The server was already overcrowded.

You are talking about ChatGPT, the things it writes are so realistic that you can’t even distinguish them?

It doesn't matter!

Now, its “nemesis” is here—

A Chinese guy has created such a website to identify whether text is written by AI or human beings.

You only need to paste the corresponding content and the results will be analyzed within a few seconds.

It: "I knew it was AI."

As soon as such a magical tool came out, it can be said that it quickly captured everyone's hearts.

#No, it will directly crush the server (fortunately, there is a spare one).

Now, those "naughty kids" who use ChatGPT to do their homework will also suffer?

Is it a human or ChatGPT? GPTZero: Look at my

This website is called GPTZero.

It mainly relies on "perplexity", that is, the "perplexity" of the text, as an indicator to determine who wrote the given content.

Friends in the NLP field know that this indicator is used to evaluate the quality of a language model.

Here, whenever you feed GPTZero a piece of test content, it will calculate respectively:

1. Total text confusion

The higher this value, the The more likely it was made by human hands.

#2. Average confusion of all sentences

The longer the sentence, the lower this value is usually.

3. The confusion level of each sentence

is presented in the form of a bar chart. You can hover the mouse over each box to see what the corresponding sentence is. (There are only two pieces here, because the test content I entered at this time is only two sentences).

The author also explained the reason why such a bar chart is drawn:

According to some of the latest research: some of the human writings The sentence may have a low degree of confusion (as mentioned earlier, human confusion is relatively high), but as you continue to write, the degree of confusion is bound to peak.

In contrast, with machine-generated text, the perplexity is evenly distributed and always low.

In addition, GPTZero will also pick out the sentence with the highest degree of confusion (that is, the sentence that is most human-like):

Those are the rules, let’s take a practical test.

Test 1: Success

First come to the latest English news (Chinese is not currently supported):

Paste it Enter the test box (please note that each sentence must have at least 5 words, it is recommended that each paragraph have 10 sentences, the results will be more accurate).

Soon, GPTZero calculated that the total text perplexity of this text was 27, and the average sentence perplexity was 171.2. The perplexity graph of each sentence looked like this:

and the sentence with the highest confusion have a value of 476.

Looking at this result, you may have guessed the answer to GPTZero:

is written by humans.

bingo.

Test 2: Failed

Let’s take a look at ChatGPT.

Paste the content into the test box, and the total text confusion of this paragraph is 31, the average sentence confusion is 76.67, and the confusion graph of each sentence is long This way:

and the sentence with the highest confusion has a value of 99.

It seems that each confusion value is quite different from the one above. You should be able to guess that it was written by AI.

Unfortunately, GPTZero cannot give the answer, and I hope we can try to add a little more text.

Obviously, this paragraph has enough words, but not enough sentences to allow GPTZero to see through ChatGPT’s “disguise” at a glance.

Test 3: Success

Let’s try ChatGPT again.

This time the content is long enough and there are enough sentences.

Sure enough, there is no problem. GPTZero answered directly:

It seems that he has indeed figured out some tricks:

That is, while there are many words, The number of sentences must also be large, at least 5 sentences, so that GPTZero can recognize through the distribution of histograms, and the final accuracy can be improved.

Another thing to mention is that when the challenge fails, in addition to the above-mentioned failure to recognize, there are also direct recognition errors -

For example, if the AI ​​writes Judging the news written by adults:

Judging the news written by humans as AI:

In this case, according to the tips mentioned above, adding more content may be able to bring the results back.

(Note that it is possible. As for the news above, we pasted all the content in, but it said no, and asked for more.)

Princeton Chinese Elementary School Produced by Brother

The person who created this website is a 22-year-old Chinese brother Edward Tian from Toronto, Canada.

I am currently a senior student majoring in CS at Princeton in the United States, with minors in cognitive science and journalism. I have a strong interest in software engineering, machine learning, etc.

He is now an intern at Microsoft, has written technical articles for the BBC, Bellingcat and other companies, and has also joined the founding team of the map tool Representable.

During his studies, he came to Tsinghua University to participate in a four-week urban research seminar.

According to him, this application was completed in a coffee shop during the New Year holiday.

The reason why I want to develop this application is simply because there is too much hype about ChatGPT, and humans should know which articles are written by AI.

Currently still in the barebones stage, improvements to the model and analysis are expected in the coming weeks.

In addition, he also revealed that he is testing a data set of news articles written by students and hopes to eventually publish a paper.

More ChatGPT Killers

In fact, this guy is not the only one who dislikes ChatGPT. There are other human organizations that have developed AI text detectors, AKA ChatGPT killers.

The general idea is similar, that is, "use magic to defeat magic" and use things written by AI to train new AI.

A while ago, OpenAI teamed up with Harvard and other universities to jointly create a detector: GPT-2 Output Detector.

The authors first released a "GPT-2 generated content" and WebText (specially taken from Reddit, a foreign post bar) data set to allow AI to understand "AI language" The difference between ” and “human speech”.

Subsequently, this data set was used to fine-tune the RoBERTa model, and the AI ​​detector was obtained. Human speech is always recognized as True, and AI-generated content is always recognized as Fake.

(RoBERTa is an improved version of BERT. The original BERT used a 13GB data set, but RoBERTa used a 160GB data set containing 63 million English news items.)

Another one The representative player in the early years was also mentioned again because of this new development.

It is the GLTR model developed by the MIT-IBM Watson AI Laboratory and the Harvard NLP Laboratory.

It mainly performs statistical analysis and visualization of text, and is used to detect the same or similar model originally used to generate text. Currently, it mainly supports two models: GPT-2 and BERT.

Since the output is the ranking of all words known to the model, each word of the text is color-coded according to the ranking. The top 10 are green, the top 100 are yellow, the top 1000 are red, and purple is not so good. possible words.

If a piece of text has too much yellow-green color, then this text is mainly generated by AI.

This time GPTZero appeared again, and many netizens exclaimed: Incredible work!

But some people say that text detectors are simply a failed arms race, and their actual results are not good. It cannot stop the development of ChatGPT.

#At the same time, some netizens discussed the necessity of "whether the article needs to be clearly indicated to be written by AI or humans."

A songwriter believes that this is necessary, just like when reading a magazine, it will mark "advertisement", this should be a simple requirement.

But some netizens immediately expressed their opposition. Why set up obstacles? This is like using PS and then adding Adobe watermark, which does not bring any benefit to the product.

What do you think about this issue? ​

Trial link:

https://etedward-gptzero-main-zqgfwb.streamlit.app/

Reference link:

[1 ]https://brackets.substack.com/about

[2]https://www.reddit.com/r/programming/comments/102hxlg/gptzero_an_app_to_efficiently_tell_if_an_essay_is/

[3] https://twitter.com/Marc__Watkins/status/1601746409203863553

[4]http://gltr.io/

The above is the detailed content of ChatGPT's 'nemesis” is here! Is the author of the article human or AI? It took a few seconds to detect it. The Chinese guy got it done during the New Year's Day holiday. The server was already overcrowded.. For more information, please follow other related articles on the PHP Chinese website!