Python has built-in map() and reduce() functions.
If you have read Google's famous paper "MapReduce: Simplified Data Processing on Large Clusters", you can roughly understand the concept of map/reduce.
Let’s look at the map first. The map() function receives two parameters, one is a function and the other is an Iterable. map applies the passed function to each element of the sequence in turn and returns the result as a new Iterator.
For example, for example, we have a function f(x)=x2, and we want to apply this function to a list [1, 2, 3, 4, 5, 6, 7, 8, 9]. You can use map() to implement it as follows:
Now, we use Python code to implement it:
>>> def f(x):... return x * x ...>>> r = map(f, [1, 2, 3, 4, 5, 6, 7, 8, 9])>>> list(r) [1, 4, 9, 16, 25, 36, 49, 64, 81]
map() Pass in The first parameter is f, which is the function object itself. Since the result r is a Iterator and Iterator is a lazy sequence, we use the list() function to let it calculate the entire sequence. and returns a list.
You may think that you don’t need the map() function. You can also calculate the result by writing a loop:
L = []for n in [1, 2, 3, 4, 5, 6, 7, 8, 9]: L.append(f(n)) print(L)
It is indeed possible, but, from the above loop Can you understand the code at a glance "apply f(x) to each element of the list and generate a new list as a result"?
So, map(), as a high-order function, actually abstracts the operation rules. Therefore, we can not only calculate simple f(x)=x2, but also calculate any Complex functions, for example, convert all numbers in this list to strings:
>>> list(map(str, [1, 2, 3, 4, 5, 6, 7, 8, 9])) ['1', '2', '3', '4', '5', '6', '7', '8', '9']
The above is the detailed content of What is MapReduce? Learn how MapReduce works in three minutes.. For more information, please follow other related articles on the PHP Chinese website!