Learning as Function Induction
Given a set of tuples (the training set)
- (x11, x12, …, x1n, y1)
- (x21, x22, …, x2n, y2)
- …
Produce a function f(x1, …, xn) = y
- a compact function
- that works on unobserved examples, not just the training set
Without some restrictions on f’s functional form, this will be impossible
- numeric functions: linear, polynomial
- symbolic functions: disjunctive normal form
Inevitable tradeoff between the complexity of the function learned and the computational complexity of learning it.