How to Extract Data from Websites Using Java and Jsoup?

Extracting Website Data with Java

To retrieve information from a website into your Java program, employ an HTML parser such as Jsoup. Jsoup's jQuery-like CSS selectors and iterable class representing a node list (Elements) make it a preferred choice.

Process:

1. Establish a Connection: Use the Jsoup.connect(url) method to establish a connection to the desired webpage.
2. Retrieve the Page: Execute the get() method on the connection to obtain the page's HTML document.
3. Parse the Document: Utilize Jsoup's select() method with appropriate CSS selectors to extract the data you need.

Example (retrieving question and answerers from Stack Overflow):

<code class="java">import org.jsoup.Jsoup;
import org.jsoup.nodes.Document;
import org.jsoup.nodes.Element;
import org.jsoup.select.Elements;

public class WebsiteDataRetrieval {

    public static void main(String[] args) throws Exception {
        String url = "https://stackoverflow.com/questions/2835505";
        Document document = Jsoup.connect(url).get();

        // Extract the question
        String question = document.select("#question .post-text").text();
        System.out.println("Question: " + question);

        // Extract the answerers
        Elements answerers = document.select("#answers .user-details a");
        for (Element answerer : answerers) {
            System.out.println("Answerer: " + answerer.text());
        }
    }
}</code>

This example outputs the question and the list of answerers for the specified Stack Overflow question.

Troubleshooting the NoSuchMethodError:

The error you're encountering with Jsoup relates to an issue in the version you're using. Ensure that you're working with a recent version of Jsoup, as it has resolved this issue. Additionally, make sure you have the Apache Commons libraries in your classpath.

The above is the detailed content of How to Extract Data from Websites Using Java and Jsoup?. For more information, please follow other related articles on the PHP Chinese website!