Data vs Models in Machine Learning

Recently I came across this talk by Peter Norvig titled "The unreasonable effectiveness of data". There are some interesting insights on using machine learning for large scale problems.

1. All models are wrong but some are useful.
On the internet since we are trying to solve problems in domains where humans are involved, any model which we can come up with would be wrong. So instead of developing complex models we could use a simple model with large amounts of data. For example we could develop a spell checker using bayesian probability and a large corpus instead of trying to create linguistic models.

2. For problems which involve humans like game theoretic problmes, non-parametric methods like density estimation using Kmeans or Kernels would be much more useful when used with large dataset. Using parametric methods in such scenarios will hide complexity and will make the models brittle.

3. When using a large dataset with simple models a lot of attention would have to be given to the source of data and how the data is obtained. (For eg Spams or Memes could skew data)