No-Free-Lunch theorem

ml

what

• states that there is no universal learner
• in order to success, every learner has to
  • specify to some task
  • and use some prior knowledge about that task

application

• the prior knowledge can be modelled by restricting the output hypothesis to be a member of a hypothesis class $H$
  • when choosing $H$, we face the Bias-Complexity tradeoff between
    • a larger/more complex $H$ is more likely to have a small approximation error
    • or a more restricted class $H$ that would have a small estimation error

references

• understanding-machine-learning-theory-algorithms.pdf