JavaScript study notes: Array deduplication_javascript skills

Recommended reading: JavaScript study notes: Adding, deleting, modifying and checking arrays

JavaScript Study Notes Array Sum Method

JavaScript Study Notes Array Random Sorting

In interviews, interviewers often ask questions about implementing array deduplication in JavaScript. Recently, I happened to be learning about JavaScript arrays and took this opportunity to sort out some methods for deduplicating arrays in JavaScript.

The following array deduplication methods were collected and organized by myself. If there are any errors, please point out the errors in the text.

Double loop deduplication

This method uses two for loops for traversal. The whole idea is:

Construct an empty array to store the deduplicated array

The outer for loop traverses the original array, and each time it takes out an element from the array and compares it with the result array
If the elements taken out from the original array are the same as the elements in the result array, break out of the loop; otherwise, store them in the result array

The code is as follows:

Array.prototype.unique1 = function () {
// 构建一个新数组，存放结果
var newArray = [this[0]];
// for循环，每次从原数组中取出一个元素
// 用取出的元素循环与结果数组对比
for (var i = 1; i < this.length; i++) {
var repeat = false;
for (var j=0; j < newArray.length; j++) {
// 原数组取出的元素与结果数组元素相同
if(this[i] == newArray[j]) {
repeat = true;
break;
}
}
if(!repeat) {
// 如果结果数组中没有该元素，则存放到结果数组中
newArray.push(this[i]);
}
}
return newArray;
}

Suppose we have an array like this:

var arr = [1,2,3,4,'a','b',1,3,4,56,32,34,2,'b','c',5,'1',`2`];
arr.unique1(); // [1, 2, 3, 4, "a", "b", 56, 32, 34, "c", 5]

It is said that this method is time-consuming and performance-intensive. Just do a simple test (the test method is poorly written):

function test () {
var arr = [];
for (var i = 0; i < 1000000; i++) {
arr.push(Math.round(Math.random(i) * 10000));
}
doTest(arr, 1);
}
function doTest(arr, n) {
var tStart = (new Date()).getTime();
var re = arr.unique1();
var tEnd = (new Date()).getTime();
console.log('双重循环去重方法使用时间是:' + (tEnd - tStart) + 'ms');
return re;
}
test();

Run the above code in the Chrome controller and test the time taken for double loop deduplication: 11031ms.

The above method can be simulated using the forEach() method and indexOf() method:

function unique1() {
var newArray = [];
this.forEach(function (index) {
if (newArray.indexOf(index) == -1) {
newArray.push(index);
}
});
return newArray;
}

Call via unique1.apply(arr) or unique1.call(arr). However, this method is much more efficient. The same test code above takes 5423ms, which is almost half as fast.

Sort traversal to remove duplicates

First use the sort() method to sort the original array. After sorting, traverse the array and check whether the i-th element in the array is the same as the last element in the result array. If different, the elements are placed into the resulting array.

Array.prototype.unique2 = function () {
// 原数组先排序
this.sort();
// 构建一个新数组存放结果
var newArray = [];
for (var i = 1; i < this.length; i++) {
// 检查原数中的第i个元素与结果中的最后一个元素是否相同
// 因为排序了，所以重复元素会在相邻位置
if(this[i] !== newArray[newArray.length - 1]) {
// 如果不同，将元素放到结果数组中
newArray.push(this[i]);
}
}
return newArray;
}

For example:

var arr = [1,2,3,4,'a','b',1,3,4,56,32,34,2,'b','c',5,'1','2'];
arr.unique2(); // ["1", 1, 2, "2", 3, 32, 34, 4, 5, 56, "a", "b", "c"]

This method has two characteristics:

The array after deduplication will be sorted, mainly because the original numbers were sorted before deduplication

In the array after deduplication, numeric characters that are the same as numbers cannot be distinguished, such as '1' and 1

Using the same method, the test time is: 1232ms.

Object key-value pair method

The implementation idea of ​​this deduplication method is:

Create a JavaScript object and new array

Use a for loop to traverse the original array, taking out one element each time and comparing it with the key of the JavaScript object

If it is not included, push the value of the element stored in the object into the result array, and set the value of the attribute name stored in the object object to 1

The code is as follows:

Array.prototype.unique3 = function () {
// 构建一个新数组存放结果
var newArray = [];
// 创建一个空对象
var object = {};
// for循环时，每次取出一个元素与对象进行对比
// 如果这个元素不重复，则将它存放到结果数中
// 同时把这个元素的内容作为对象的一个属性，并赋值为1,
// 存入到第2步建立的对象中
for (var i = 0; i < this.length; i++){
// 检测在object对象中是否包含遍历到的元素的值
if(!object[typeof(this[i]) + this[i]]) {
// 如果不包含，将存入对象的元素的值推入到结果数组中
newArray.push(this[i]);
// 如果不包含，存入object对象中该属性名的值设置为1
object[typeof(this[i]) + this[i]] = 1;
}
}
return newArray;
}

Run the previous example:

var arr = [1,2,3,4,'a','b',1,3,4,56,32,34,2,'b','c',5,'1','2'];
arr.unique3(); // [1, 2, 3, 4, "a", "b", 56, 32, 34, "c", 5, "1", "2"]

Similarly, different keys may be mistaken for the same; for example: a[1], a["1"] . Time taken by this method: 621ms. This method takes the shortest time, but takes up more memory.

In addition to the above methods, there are several other methods as follows:

// 方法四
Array.prototype.unique4 = function () {
// 构建一个新数组存放结果
var newArray = [];
// 遍历整个数组
for (var i = 0; i < this.length; i++) {
// 遍历是否有重复的值
for (j = i + 1; j < this.length; j++) {
// 如果有相同元素，自增i变量，跳出i的循环
if(this[i] === this[j]) {
j = ++i;
}
}
// 如果没有相同元素，将元素推入到结果数组中
newArray.push(this[i]);
}
return newArray;
}

Chrome test results

var arr = [1,2,3,4,'a','b',1,3,4,56,32,34,2,'b','c',5,'1','2'];
arr.unique4(); // ["a", 1, 3, 4, 56, 32, 34, 2, "b", "c", 5, "1", "2"]

Similarly, 1 and '1' are indistinguishable.

// 方法五
Array.prototype.unique5 = function () {
// 构建一个新数组存放结果
var newArray = [];
// 遍历整个数组
for (var i = 0; i < this.length; i++) {
// 如果当前数组的第i值保存到临时数组，那么跳过
var index = this[i];
// 如果数组项不在结果数组中，将这个值推入结果数组中
if (newArray.indexOf(index) === -1) {
newArray.push(index);
}
}
return newArray;
}

Chrome test results:

var arr = [1,2,3,4,'a','b',1,3,4,56,32,34,2,'b','c',5,'1','2'];
arr.unique6(); // [1, 2, 3, 4, "a", "b", 56, 32, 34, "c", 5, "1", "2"]

Similarly, 1 and '1' are indistinguishable. Time spent: 14361ms.

// 方法六
Array.prototype.unique6 = function () {
return this.reduce(function (newArray, index) {
if(newArray.indexOf(index) < 0) {
newArray.push(index);
}
return newArray;
},[]);
}

The test results are as follows:

var arr = [1,2,3,4,'a','b',1,3,4,56,32,34,2,'b','c',5,'1','2'];
arr.unique6(); // [1, 2, 3, 4, "a", "b", 56, 32, 34, "c", 5, "1", "2"]

Time taken: 16490ms.

// 方法七
Array.prototype.unique7 = function(){
var newArray;
newArray = this.filter(function (ele,i,arr) {
return arr.indexOf(ele) === i;
});
return newArray;
}

Test results:

var arr = [1,2,3,4,'a','b',1,3,4,56,32,34,2,'b','c',5,'1','2'];
arr.unique6(); // [1, 2, 3, 4, "a", "b", 56, 32, 34, "c", 5, "1", "2"]

Time taken: 13201ms.

Although there are many methods, in comparison, the following method is a better solution:

Array.prototype.unique3 = function () {
// 构建一个新数组存放结果
var newArray = [];
// 创建一个空对象
var object = {};
// for循环时，每次取出一个元素与对象进行对比
// 如果这个元素不重复，则将它存放到结果数中
// 同时把这个元素的内容作为对象的一个属性，并赋值为1,
// 存入到第2步建立的对象中
for (var i = 0; i < this.length; i++){
// 检测在object对象中是否包含遍历到的元素的值
if(!object[typeof(this[i]) + this[i]]) {
// 如果不包含，将存入对象的元素的值推入到结果数组中
newArray.push(this[i]);
// 如果不包含，存入object对象中该属性名的值设置为1
object[typeof(this[i]) + this[i]] = 1;
}
}
return newArray;
}

But there are simpler and more optimized solutions for deduplication in ES6, such as:

// ES6
function unique (arr) {
const seen = new Map()
return arr.filter((a) => !seen.has(a) && seen.set(a, 1))
}
// or
function unique (arr) {
return Array.from(new Set(arr))
}

The above is the JavaScript learning notes introduced by the editor to remove duplicates from arrays. I hope it will be helpful to you!