JavaScript array deduplication method-JS Tutorial-php.cn

We all know that different programming languages have array deduplication. In this article, we will talk about JavaScript array deduplication, hoping to help everyone.

Double-layer loop

Perhaps the first thing we think of is to use indexOf to loop through the judgment, but before this method, let us first look at the most original method:

var array = [1, 1, &#39;1&#39;, &#39;1&#39;];function unique(array) {    // res用来存储结果
    var res = [];    for (var i = 0, arrayLen = array.length; i < arrayLen; i++) {        for (var j = 0, resLen = res.length; j < resLen; j++ ) {            if (array[i] === res[j]) {                break;
            }
        }        // 如果array[i]是唯一的，那么执行完循环，j等于resLen
        if (j === resLen) {
            res.push(array[i])
        }
    }    return res;
}console.log(unique(array)); // [1, "1"]

Copy after login

In this method, we use loop nesting, the outermost loop array, and the inner loop res. If the value of array[i] is equal to the value of res[j], jump out of the loop. If they are not equal, the element is The only thing is that at this time the value of j will be equal to the length of res. Based on this feature, we will judge and add the value to res.

It seems very simple. The reason why I want to talk about this method is because——————It has good compatibility!

indexOf

We can use indexOf to simplify the inner loop:

var array = [1, 1, &#39;1&#39;];function unique(array) {    var res = [];    for (var i = 0, len = array.length; i < len; i++) {        var current = array[i];        if (res.indexOf(current) === -1) {
            res.push(current)
        }
    }    return res;
}console.log(unique(array));

Copy after login

Deduplication after sorting

Imagine that we first use sort to deduplicate the array After the method is sorted, the same values will be arranged together, and then we can only judge whether the current element is the same as the previous element. If they are the same, it means duplication. If they are not the same, they will be added to res. Let us write a demo:

var array = [1, 1, &#39;1&#39;];function unique(array) {    var res = [];    var sortedArray = array.concat().sort();    var seen;    for (var i = 0, len = sortedArray.length; i < len; i++) {        // 如果是第一个元素或者相邻的元素不相同
        if (!i || seen !== sortedArray[i]) {
            res.push(sortedArray[i])
        }
        seen = sortedArray[i];
    }    return res;
}console.log(unique(array));

Copy after login

If we are deduplicating a sorted array, this method is definitely more efficient than using indexOf.

unique API

After knowing these two methods, we can try to write a tool function named unique. We determine whether the incoming array is duplicated based on a parameter isSorted. If it is true, we will judge whether the adjacent elements are the same. If it is false, we will use indexOf to judge

var array1 = [1, 2, &#39;1&#39;, 2, 1];var array2 = [1, 1, &#39;1&#39;, 2, 2];// 第一版function unique(array, isSorted) {    var res = [];    var seen = [];    for (var i = 0, len = array.length; i < len; i++) {        var value = array[i];        if (isSorted) {            if (!i || seen !== value) {
                res.push(value)
            }
            seen = value;
        }        else if (res.indexOf(value) === -1) {
            res.push(value);
        }        
    }    return res;
}console.log(unique(array1)); // [1, 2, "1"]console.log(unique(array2, true)); // [1, "1", 2]

Copy after login

Optimization

Although unqique can already try the deduplication function, in order to make this The API is more powerful, let's consider a requirement:

New requirement: The upper and lower cases of letters are considered consistent, such as 'a' and 'A', just keep one!

Although we can process all the data in the array first, such as converting all letters to lowercase, and then pass it into the unique function, is there a way to save the loop of processing the array and just do it directly? What about doing it in a deduplication cycle? Let us complete this requirement:

var array3 = [1, 1, &#39;a&#39;, &#39;A&#39;, 2, 2];// 第二版// iteratee 英文释义：迭代 重复function unique(array, isSorted, iteratee) {    var res = [];    var seen = [];    for (var i = 0, len = array.length; i < len; i++) {        var value = array[i];        var computed = iteratee ? iteratee(value, i, array) : value;        if (isSorted) {            if (!i || seen !== value) {
                res.push(value)
            }
            seen = value;
        }        else if (iteratee) {            if (seen.indexOf(computed) === -1) {
                seen.push(computed);
                res.push(value);
            }
        }        else if (res.indexOf(value) === -1) {
            res.push(value);
        }        
    }    return res;
}console.log(unique(array3, false, function(item){    return typeof item == &#39;string&#39; ? item.toLowerCase() : item
})); // [1, "a", 2]

Copy after login

In this and the last version of the implementation, the function passes three parameters:

array: indicates the array to remove duplicates, required

isSorted: Indicates whether the array passed in by the function has been sorted. If true, a faster method will be used to remove duplicates

iteratee: Pass in a function that can sort each The elements are recalculated, and then deduplicated based on the processing results.

So far, we have written a unique function following the idea of underscore. You can view Github for details.

filter

ES5 provides the filter method, which we can use to simplify the outer loop:

For example, use the indexOf method:

var array = [1, 2, 1, 1, &#39;1&#39;];function unique(array) {    var res = array.filter(function(item, index, array){        return array.indexOf(item) === index;
    })    return res;
}console.log(unique(array));

Copy after login

Sort and remove duplicates Method:

var array = [1, 2, 1, 1, &#39;1&#39;];function unique(array) {    return array.concat().sort().filter(function(item, index, array){        return !index || item !== array[index - 1]
    })
}console.log(unique(array));

Copy after login

Object key-value pair

There are many ways to remove duplicates. Although we have written an unqiue API following underscore, let us look at other methods to expand our horizons:

This method uses an empty Object object. We store the value of the array as the key value of the Object, such as Object[value1] = true. When judging another value, if Object[value2] exists If , it means that the value is repeated. The sample code is as follows:

var array = [1, 2, 1, 1, &#39;1&#39;];function unique(array) {    var obj = {};    return array.filter(function(item, index, array){        return obj.hasOwnProperty(item) ? false : (obj[item] = true)
    })
}console.log(unique(array)); // [1, 2]

Copy after login

We can find that there is a problem because 1 and '1' are different, but this method will judge it to be the same value. This is because the key value of the object can only is a string, so we can use typeof item + item to form a string as the key value to avoid this problem:

var array = [1, 2, 1, 1, &#39;1&#39;];function unique(array) {    var obj = {};    return array.filter(function(item, index, array)
{        return obj.hasOwnProperty(typeof item + item) ? false : (obj[typeof item + item] = true)
    })
}console.log(unique(array)); // [1, 2, "1"]

Copy after login

ES6

With the arrival of ES6, there are new ways to remove duplicates For example, we can use Set and Map data structures. Taking Set as an example, ES6 provides a new data structure Set. It is similar to an array, but the values of the members are unique and there are no duplicate values.

Does it feel like you are preparing to lose weight? Let’s write a version:

var array = [1, 2, 1, 1, &#39;1&#39;];function unique(array) {   return Array.from(new Set(array));
}console.log(unique(array)); // [1, 2, "1"]

Copy after login

It can even be simplified:

function unique(array) {    return [...new Set(array)];
}

Copy after login

It can also be simplified:

var unique = (a) => [...new Set(a)]

Copy after login

In addition, if you use Map:

function unique (arr) {    const seen = new Map()    return arr.filter((a) => !seen.has(a) && seen.set(a, 1))
}

Copy after login

The evolution of JavaScript

We can see that the deduplication method has gone from the original 14 lines of code to ES6’s 1 line of code. This actually shows that the JavaScript language is constantly improving. I believe that in the future The development will become more and more efficient.

Special type comparison

The method of deduplication ends here. However, the types of elements to be deduplicated may be diverse, except for the simple 1 and '1' in the example. , in fact, there are also null, undefined, NaN, objects, etc. So for these elements, what is the deduplication result of the previous methods?

Before that, let us first look at a few examples:

var str1 = &#39;1&#39;;var str2 = new String(&#39;1&#39;);console.log(str1 == str2);
 // trueconsole.log(str1 === str2); // falseconsole.log(null == null);
  // trueconsole.log(null === null); // trueconsole.log(undefined == undefined);
   // trueconsole.log(undefined === undefined); // trueconsole.log(NaN == NaN);
    // falseconsole.log(NaN === NaN); // falseconsole.log(/a/ == /a/); 
    // falseconsole.log(/a/ === /a/); // falseconsole.log({} == {});
     // falseconsole.log({} === {});
      // false

Copy after login

So, for such an array

var array = [1, 1, &#39;1&#39;, &#39;1&#39;, null, null, undefined, undefined, new String(&#39;1&#39;), new String(&#39;1&#39;), /a/, /a/, NaN, NaN];

Copy after login

What are the results of the above methods to remove duplicates? What kind of thing?

I specially compiled a list, we focus on the deduplication of objects and NaN:

Method Result Description

for loop [1, "1", null, undefined, String, String, /a/, /a/, NaN, NaN] Objects and NaN are not duplicated

indexOf [1, "1", null, undefined, String, String, /a/, /a/, NaN, NaN] 对象和 NaN 不去重

sort [/a/, /a/, "1", 1, String, 1, String, NaN, NaN, null, undefined] 对象和 NaN 不去重数字 1 也不去重

filter + indexOf [1, "1", null, undefined, String, String, /a/, /a/] 对象不去重 NaN 会被忽略掉

filter + sort [/a/, /a/, "1", 1, String, 1, String, NaN, NaN, null, undefined] 对象和 NaN 不去重数字 1 不去重

优化后的键值对方法 [1, "1", null, undefined, String, /a/, NaN] 全部去重

Set [1, "1", null, undefined, String, String, /a/, /a/, NaN] 对象不去重 NaN 去重

想了解为什么会出现以上的结果，看两个 demo 便能明白：

// demo1var arr = [1, 2, NaN];
arr.indexOf(NaN); // -1

indexOf 底层还是使用 === 进行判断，因为 NaN ==== NaN的结果为 false，所以使用 indexOf 查找不到 NaN 元素

// demo2function unique(array) {   return Array.from(new Set(array));
}console.log(unique([NaN, NaN])) // [NaN]

Copy after login

Set 认为尽管 NaN === NaN 为 false，但是这两个元素是重复的。

写在最后

虽然去重的结果有所不同，但更重要的是让我们知道在合适的场景要选择合适的方法。

以上内容就是各种不同的JavaScript数组去重方法，如果大家觉得有用的话赶紧收藏起来吧。