Scaling Big Data Mining Infrastructure at Twitter

I’m almost always enjoying the lessons learned-style presentations from Twitter’s people. The slides below, by Jimmy Lin and Dmitriy Ryaboy, have been used at HadoopSummit. Besides the technical and practical details, there are two thing

I’m almost always enjoying the lessons learned-style presentations from Twitter’s people. The slides below, by Jimmy Lin and Dmitriy Ryaboy, have been used at HadoopSummit. Besides the technical and practical details, there are two things that I really like:

DJ Patil: “It’s impossible to overstress this: 80% of the work in any data project is in cleaning the data”

and then the reality check:

1. Your boss says something vague
2. You think very hard on how to move the needle
3. Where’s the data?
4. What’s in this dataset?
5. What’s all the f#$#$ crap in the data?
6. Clean the data
7. Run some off-the-shelf data mining algorithm
8. …
9. Productionize, act on the insight
10. Rinse, repeat

Enjoy!

Scaling Big Data Mining Infrastructure Twitter Experience

Original title and link: Scaling Big Data Mining Infrastructure at Twitter (NoSQL database?myNoSQL)

原文地址：Scaling Big Data Mining Infrastructure at Twitter, 感谢原作者分享。