Notional Slurry Logo

I’d hammer in the morning

September 23, 2005 at 6:50 am · Filed under Uncategorized

Why don’t more people know about (and use) genetic programming, especially for symbolic regression? GP is an approach that can be useful in all sorts of domains, for problems ranging from exploratory data analysis to design automation. SR can be a subtly informative complement to statistical modeling projects, or it can be used as a monstrously powerful open-ended exploratory machine learning engine. It rocks.

So. Do you know anything about it? [Cheating has been discouraged by eliminating outbound links from this post.]

This has become a problem for me. In seven conversations in two weeks with colleagues about work, including bosses and peers, I’ve mentioned or advised or absolutely insisted they consider GP/SR. In one case my opposite knew about GP but hadn’t considered it because he only knew about pole-balancing and stuff; in four cases they thought I was talking about genetic algorithms for parameter optimization (not that there’s anything wrong with that, but… no); in two cases I suppose they thought “symbolic regression” meant something ickily statisticky, and didn’t want to go down that road, so they played like it was some fancy newfangled numerical regression technique fad-of-the-day. Then, yesterday, in a room full of people using fast but utterly opaque SVMs to do machine learning, where the goal is to understand the system, they had thought about neither Bayesian networks nor GP/SR, both of which could tell them important things about how the system works. And in this latter case they hadn’t ever heard of SR.

I suppose now I have to do something about it.

Sigh. More in a while.

son1 said,

September 23, 2005 @ 10:21 am

Just out of curiosity, do you really think that any reasonably complex Bayesian network, whose parameter-set or structure is learned from the data, is likely to be any less opaque than the results of a well-designed SVM?

Never-mind the fact that the graph representation of a bayesian network often invites people to consider the edges which are drawn (when in fact, it’s what isn’t drawn that’s important, graphically) and, at the same time, to consider the edges singly (when in fact, it’s a “set of edges” which form the incoming arcs to a node that are the irreducible elements in a BN), which is totally wrong.

Bleh. (what I’m saying is, it’s not about your machine-learning tool-of-choice, it’s about the auxiliary tools you have available to summarize and visualize the results of whatever method you’ve chosen.)

But I’m intrigued… what is GP/SR? and how is it different from a genetic algorithm?

aleks said,

September 28, 2005 @ 7:13 am

Just BTW: One can convert a SVM into the form of a generalized additive model and then show it graphically: http://portal.acm.org/citation.cfm?id=1081870.1081886 The dimensionality of the visualization depends on the kernel.

Bill said,

September 28, 2005 @ 10:32 am

This is true… but you can’t really make strong statements about the role of, say, qualitative relationship between mass and time variables,as you can with symbolic regression approaches.

In many cases, the customer–the person who wants and pays to see the analysis done–may not realize initially that they don’t just want a classifier or a model, but also want one that provides insight to drive further first-principles modeling and experimentation. That’s the shortcoming of most NN-like and PCA-like approaches: the intermediate calculations are some function of almost all the inputs, and as a result it’s nigh impossible to tease out meaningful insight.

Anyway… I’m working on it…..

Alexandre Grings said,

September 28, 2005 @ 4:04 pm

What the alternatives to do Symbolic Regression?
Make a web search for the term. You will see that there is no definition for the term outside GP world.
My guess is that the most of people don’t even know that such a problem can be solved by a machine.
Take a look in http://www.it.lut.fi/mat/EcmiNL/ecmi35/node70.html

dsquared said,

October 1, 2005 @ 11:12 am

It’s what you do for when mixtures of logistic functions aren’t flexible enough, isn’t it? You tend to get laughed out of the shop in econometrics when you get to this level of complexity, but that’s mainly because we’re usually looking for patterns that probably don’t exist.

RSS feed for comments on this post · TrackBack URI

Leave a Comment