Have you ever had brainfreeze when trying to fit what you were doing into a framework that wasn’t suitable? Maybe it was a particular format for some lesson objectives, or a form you had to fill in which didn’t provide space for what you wanted to say. Or even an automated telephone menu that didn’t seem to take all your needs and wishes into account. I recently had such an experience, and it got me thinking…
I recently read a book called “Algorithms to Live By”, where I came across the concept of overfitting for the first time. Or at least it was the first time I had read about it as an actual thing with a name, rather than just having a general sense of it and a vague unarticulated unease…
Overfitting is a concept often associated with statistical models and machine learning, but which can be extended to many other areas of life. Here is a technical definition:
Overfitting is a modeling error which occurs when a function is too closely fit to a limited set of data points.
The potential implications of this type of behaviour are vast. For example, understanding overfitting makes it clear that there really is such a thing as
- too much practice
- too much planning
- too much detail
- too much effort
- too much deliberation
In same ways this is like the law of diminishing returns. But it’s more than that – beyond a certain point, further investment is not just a waste of time, energy and resources, but is often actually detrimental to achieving the goals your “fitting” was designed for.
Thinking more about this issue, I remembered a book by Edward de Bono about lateral thinking (a term which he invented himself). In it there is an exercise which relates to overfitting. De Bono makes the point that we are culturally indoctrinated to expect every move in a a process to create a valid result – that is we have to justify every step we take. This can be disastrous for creativity and innovation.
Justify Everything You Do Every Step of the Way
Take this example:
You are given the first two pieces. The instruction is “arrange them into a shape which is easy to describe.” The two pieces are usually arranged to give a square, as shown. Another piece is added with the same instruction. Keep going. Once you are given the final piece, although you been correct at each stage, you are unable to proceed further. The new piece cannot be fitted into the existing pattern
Try this exorcise for yourself using cut-out card if you are curious about this subject…
Don’t Justify Every Step
Click below to reveal an alternative approach.
If one started off with the square then one would have to go back and rearrange the pieces at some stage to give a parallelogram before one could proceed. Thus even though one had been correct at each stage one would still have to restructure the situation before being able to proceed. Yet the square seemed so much more “justified” somehow!
Note: the more observant of you might have noticed that this exercise doesn’t perfectly illustrate the point about justifying steps, as the rules ask that the shape at each stage be “easy to describe”. I remembered the exercise and tried to fit it into the subject of the blog post as I think it is very interesting in its own right. De Bono illustrates the point about justifying every step better elsewhere, but I can’t recall where for now.
The Big Takeaway
Overfitting destroys space. Creativity needs space. Like a 15-puzzle with no empty slot – lack of space restricts movement, and thus creativity.