What are the methods to remove duplicate objects in java8

1. Remove duplicate Strings in List

public List<String> removeStringListDupli(List<String> stringList) {
    Set<String> set = new LinkedHashSet<>();
    set.addAll(stringList);

    stringList.clear();

    stringList.addAll(set);
    return stringList;
}

or use Java8 writing method:

List<String> unique = list.stream().distinct().collect(Collectors.toList());

2. Remove duplicate objects in List

For example, there is a Person class now :

public class Person {
    private Long id;

    private String name;

    public Person(Long id, String name) {
        this.id = id;
        this.name = name;
    }

    public Long getId() {
        return id;
    }

    public void setId(Long id) {
        this.id = id;
    }

    public String getName() {
        return name;
    }

    public void setName(String name) {
        this.name = name;
    }

    @Override
    public String toString() {
        return "Person{" +
                "id=" + id +
                ", name='" + name + '\'' +
                '}';
    }
}

Rewrite the equals() method and hashCode() method of the Person object:

 @Override
    public boolean equals(Object o) {
        if (this == o) return true;
        if (o == null || getClass() != o.getClass()) return false;

        Person person = (Person) o;

        if (!id.equals(person.id)) return false;
        return name.equals(person.name);

    }

    @Override
    public int hashCode() {
        int result = id.hashCode();
        result = 31 * result + name.hashCode();
        return result;
    }

The following object deduplication code:

 Person p1 = new Person(1l, "jack");
        Person p2 = new Person(3l, "jack chou");
        Person p3 = new Person(2l, "tom");
        Person p4 = new Person(4l, "hanson");
        Person p5 = new Person(5l, "胶布虫");

        List<Person> persons = Arrays.asList(p1, p2, p3, p4, p5, p5, p1, p2, p2);

        List<Person> personList = new ArrayList<>();
        // 去重
        persons.stream().forEach(
                p -> {
                    if (!personList.contains(p)) {
                        personList.add(p);
                    }
                }
        );
        System.out.println(personList);

contains() method of List The underlying implementation uses the equals method of the object for comparison. In fact, just override equals(), but if you override equals, it is best to also rewrite the hashCode.

3. Deduplication based on the attributes of the object

Next, we need to deduplicate based on the id of the Person object. How to do it?

Write a method:

  public static List<Person> removeDupliById(List<Person> persons) {
        Set<Person> personSet = new TreeSet<>((o1, o2) -> o1.getId().compareTo(o2.getId()));
        personSet.addAll(persons);

        return new ArrayList<>(personSet);
    }

Compare the object properties through the Comparator. If they are the same, 0 will be returned to achieve the purpose of filtering.

Let’s take a look at the cool Java8 writing method:

import static java.util.Comparator.comparingLong;
import static java.util.stream.Collectors.collectingAndThen;
import static java.util.stream.Collectors.toCollection;

// 根据id去重
     List<Person> unique = persons.stream().collect(
                collectingAndThen(
                        toCollection(() -> new TreeSet<>(comparingLong(Person::getId))), ArrayList::new)
        );

There is another way of writing:

  public static <T> Predicate<T> distinctByKey(Function<? super T, Object> keyExtractor) {
        Map<Object, Boolean> map = new ConcurrentHashMap<>();
        return t -> map.putIfAbsent(keyExtractor.apply(t), Boolean.TRUE) == null;
    }

// remove duplicate
        persons.stream().filter(distinctByKey(p -> p.getId())).forEach(p -> System.out.println(p));

The above is the detailed content of What are the methods to remove duplicate objects in java8. For more information, please follow other related articles on the PHP Chinese website!