Machine Learning and Meta-Meta-Data
I read a recent post on slashdot talking about machine learning and some new ideas coming out of MIT and it got me thinking. Since I’m a programmer, I’ve always been fascinated by how well my children learn by experience, training, categorization, and classification of all of their available input to produce (mostly) sensible output. I have two wonderful children and watching them grow is the ultimate first-hand experience for a programmer to see the baby bootstrap.
Here’s a simple example: One night my daughter woke up crying. I asked her what the problem was and she couldn’t really give me an answer. Just some babbling and more crying. Since it was 2 in the morning and my programmer brain was just nodding off, I figured that A) she had some kind of dream and her brain didn’t really know what to do with it and B) she couldn’t focus on what I was asking her because of A. All it took to fix the problem was a simple processor interrupt. I just carried her into the bathroom, sat her on the toilet and said “go potty”. Since that was something we had practiced many times, it was very easy for her to focus on that instead of the problem at hand, and the crying instantly stopped. (Parents take note, beatings and yelling only get you so far…) As a programmer, I found that little trick worked perfectly well.
So back to the learning thing… The brain doesn’t come pre-wired. It doesn’t come with a blueprint, and it sure doesn’t come with a users-manual. So how in the world does it come up with a superb sensory storage system that doesn’t cook itself given all of the input we experience on a daily basis? A common buzzword in this synergy-infected business world is “thinking outside the box”. Typically this means taking a step back and re-evaluating the problem. In the database world, we had data, we stepped back and came up with metadata to fit the data into. It worked, and worked surprisingly well. All of the techniques that I’ve seen from MIT and Caltech and every other AI research group out there are focused on how to categorize the data. They start with some data, then they find these wonderfully sophisticated algorithms to pour through hundreds of thousands of samples and produce some very good output, but the framework is very rigid.
Brain cells die. New brain cells are created. The brain re-wires itself constantly. You just can’t cram that rigid square-peg-algorithm into a dynamic round-hole-environment. It won’t work. We need to take another step backwards. Call it a meta-meta-data if you’d like. Call it dynamic metadata. I don’t care what you call it, just call it something.
Instead of toiling over algorithms and data structures, come up with a simple genetic sequence of virtual parallel computer instructions. The Cell processor from IBM is a good starting point. Each SPE has a fixed instruction set. Use that. Now write some code that randomly scrambles and reassembles generations of genetic code and run it through fitness trials (i.e. real life situations), but don’t train the model based on example input and yes/no output – train the model on example input and “classification methodologies”. We train our children how to organize by color, size, shape, feel, etc. Train the genetic model that a picture of a car can be a “red car”, a “big car”, a “race car”, whatever. Let the model decide how to store that data. Let the model decide where to store that data.
Then give the model a toddler to learn with. They might just be able to teach each other something.