the beauty of statistics…

… lies in its ability to provide us with frameworks and structures with which to see the world as it may be.

 

I am constantly barraged by questions on ‘goodness-of-fit’ measures.  People question models based on their adjusted r-squared’s, their AIC, their-D-W indicators and everything else that you can think about.  There are discussions that sometimes lead to frustrations from either the analysts’ part, the clients’, or the strategists’.

Everyone wants to be sure that the model is accurate.

That when 100 dollars are invested, indeed, there will be a return of 50%.

That when 200 dollars are invested, indeed, there will be 65% returns.

The thing is, every single model – regardless of how they were derived – are based on part observations.

And as far as I know, no one has seen the future.

 

There’s only one thing in my mind whenever I approach modeling projects (not the kind which involve photographers, human models, etc. – although I wouldn’t mind that kind of modeling project, too):  interpretation.

Can I explain the model?

Can the variables explain the movements?

Does the explanation hold?

Could it be the same in the future?

How can the model be wrong?

Are there alternative interpretations?

  

 

Here’s a wonderful quote from Andreas Buja:

 

A lot of people think that analytics – done by statisticians and econometricians – are all about creating models with ‘accuracies approaching 100%’.

I think analytics is all about interpretation that empowers others (and me) to more deeply understand, appreciate, and explore an idea.

 

After all, analytics people are still humans – and they have not seen the future.

Comments

[Neil Charles]

Nice post. I always think it's fun explaining to clients that if they want to explain the past (and work out ROI) they're going to need one model. For accurate forecasting they'll need a different one...

Response is almost always 'but surely you only need one model if it's right?'

[Philip Tiongson]

Thanks very much for the comment. Indeed, fun! Although sometimes, it could painfully be one great fun!

What's worse really is when people rely too much on "tools that automate the model selection process". To which I typically say, yeah: saves me perhaps some time by consolidating computer power - BUT it does not replace the thinking process which ultimately I will have to do any way - software or no software.

Interpretation is still best left to us - and yes, just because we got the past explained doesn't mean we've got the future locked on. :D