Thursday, January 25, 2007

Are we lacking innovation?

Is the Canadian marketing analytics community lacking in innovation?

I had an interesting conversation with an executive from the US yesterday; he was bemoaning the fact that he didn't see anything interesting happening in Canadian data mining community. He commented that it was all the "same old, same old"....lots of logistic regression response and propensity models and k-means clustering. He claimed that the US was miles ahead and doing much more interesting things. This certainly got me thinking.; I am not sure I agree with him... but I suspect that we are not as innovative as we could be.


Are we really pushing ourselves to try new things or are we happy with the status quo?? I know we are exploring more new ideas than we used to be; new tools have made coding less onerous and advanced statistics knowledge a little less critical. But are we given the freedom to explore truly new things?

Is this the fault of clients and executitves not giving data miners freedom to test new things? Or is the fault of data miners not pushing boundaries?

What do you think?

4 comments:

Anonymous said...

Well, I'd rather focus on value creation rather than mining advancement for its own sake, as that's what I'm paid to do by my employer.

If advancement can indeed bring extra value and benefits, we'd be all over it. Are we doing enough of it? Just from the scale of things, our bigger neighbour will have more buzz. They are also probably better funded and equipped to do so.

I think there is some truth in that statement. But I would beg to differ that we are that far behind. Perhaps being Canadian we just don't brag as much.

Daymond Ling, CIBC

Anonymous said...
This comment has been removed by a blog administrator.
Anonymous said...

Comparing ourselves to the US in the world of data mining and analytics is a tricky subject because there are many facets to think about. In some areas the gap is entirely fictional and in some areas there is probably something there to look at.

Certainly in terms of software creation, and features being added to the software, the US market will get the majority the press, and of course they host many more conferences where "new" software and features get unveiled for the first time. One can see how the perception would be that in technical terms, new and interesting stuff is happening in the US. Many of the major analytics tools have been and continue to be developed in the US (i.e. SAS, etc) so the optics will be that the US is where all of the "new stuff" is happening. (Ironically many of these companies employ Canadian and other nationalities as programmers but that's another story).

In terms of implementation and execution of CRM tools I think there is where it's a mixed bag, varying by industry. Traditionally the US had a strong experience with retail markets, credit cards markets, automotive markets and those markets were far larger than anything comparable in Canada so one could argue that more experimentation was and is possible. Also there is a healthy competition in those markets and more overall money to be spent on CRM and marketing programs, period. Where there is both money and competition, usually there is innovation. On the other hand Canada does have some interesting advantages in certain categories. Certainly one could argue that the Canadian banking industry, being relatively strong and stable, has done just as good if not better CRM work relative to the US market. The same could be said for the investment and brokerage community although it's on a smaller scale. In reality, any thorough objective comparison would have to go through industry by industry in great detail to make any sort of conclusion, and the results would certainly vary by industry.

The academic world is a whole other ball game which I think is more based on opinion than anything else. Can someone seriously suggest that the US math professors are better than the Canadian or visa-versa....I'm not even sure how to measure that. Certainly the US has big name universities so even a minor improvement and slightly different iteration of a formula, while by itself not that exciting, could easily be hyped-up more based on name recognition of the university than anything else. If a Harvard business professor and a Brock university business professor came out with the exact same thing at the exact same time what do you think would happen...(no offense intended to any Brock grads out there)

All of it is a moot point of course, if the "innovations" cannot be used. Slight tweaks to a regression formula will not mean anything unless the user can take advantage of that extra lift. A good example of this is conjoint analysis. In the practical world, the decision to do a conjoint, the design of it, and the execution of it is far more important than the software package used or the specific iteration of background formula used. I have used several different software packages to do conjoint analysis, all with different "features"... and never had a complaint with any of them and never really used 90% of the so called extra features. So if those same packages come out with even more extra features it really isn't going to matter in any real business sense.

Okay, there's my two cents worth!

Anonymous said...

I attended a data mining conference last summer in Philadelphia (KD Nuggets). Although these conferences tend to be more academic in nature, there was one whole day devoted to practitioners. I actually spoke at this conference and it was comforting to note that amongst our community, we all still have the same challenges and issues. In fact, the senior VP at Capital One for credit risk decision sciences spoke at this conference. Being a leading edge user of data mining techniques, they still use logistic regression and his key challenges were not about more mathematical research. Rather, it was about doing more with less, hiring the right people, and more automation of the data processes used in data mining.



Richard Boire, Boire Filler Group