Information Week: Credibility Of Analysts
February 7 2006
An interesting subject that has been discussed
here in the past. Information Week talks to the "big guns"
of the analyst world: IDC, Gartner, Forrester. There is a smattering
of discussion about boutique analyst firms, but there are so many that
it would be tough ground to cover.
It's an interesting article, but I think it is difficult to apply a single
lens to all analyst firms. Burton Group, for example, doesn't do
vendor-sponsored work, as Peter
O'Kelly notes on his blog.
Other analysts exclusively do so, so-called "white paper for
hire" analysts. This class of analyst doesn't get much attention
in the Information Week article, yet I think it's more relevant for readers
to be aware of.
Predicting tech trends is notoriously difficult, and no research firm is perfect in its advice. But their motives must be beyond reproach. Gartner VP Larry Perlstein tackled the pay-for-play question on the company's blog. "Aside from the occasional comment on a blog or backroom chatter, we have not had any vendors complain to the office of the ombudsman directly since the office opened in September 2004," Perlstein wrote. "So my message is, if anyone believes that they know of a legitimate issue, please step forward regardless of your client status."The article also doesn't really tackle this subject -- track record. The article quotes a reader saying that some analysts erroenously (and egotistically) see their role as "market makers" rather than trend watchers -- I agree completely. It would be very interesting for those analysts that have been around a long time to self-publish a scorecard, and one that is somehow indepedently auditable. I know that, for example, I still think about Tom Austin/Gartner's "Great Train Wreck" report on the messaging market in 1996 -- which was remarkably prescient for the time. Can't say that I have seen the same from certain others.
Link: Information Week: Credibility Of Analysts >
Post a Comment
- 2
Chris Bordeleau http://chris.bordeleau.net | 2/7/2006 6:19:42 PM
My company has Gartner access and my boss is a regular reader of their reports. If we can find a report that agrees with a project we are looking at it strengthens our possibility of getting time & funding.
From the reports I have read you can find a report to justify any decision you want to whether it be Exchange vs. Domino, Websphere vs BEA vs Oracle (big at my site), Large servers using LPAR's vs lots of small server, Domino vs Sharepoint...
So I think the analysts are in business to keep people buying their reports... they do this by creating reports that many times agree with the buyer of the reports position. They create enough reports to make sure the have all their basis covered.
Often the rank and file workers at my site have pondered re-directing Gartner inside of our site to a page saying how underpaid and over worked we are... Could work
- 3
Nathan T. Freeman | 2/8/2006 2:02:03 AM
"So I think the analysts are in business to keep people buying their reports."
To the extent that this is true, they are shills. Never a good thing.
- 4
Alan Bell http://www.dominux.co.uk | 2/8/2006 5:11:22 AM
One thing I really don't understand is how a survey of opinions of decision makers can in any way influence a technology decision. It just seems irrational to me. Decisions should be based on what is the best fit for the requirements, not on the direction the rest of the flock is taking.
- 5
Paul Pentony | 2/8/2006 5:18:58 AM
More insidious than straight out vendor sponsored work is the consultancy side line. I have certainly heard allegations from one major vendor that the only way to get coverage from a particular major analyst was to hire them for consultancy and market research work.
This is similar to accountancy firms using audit as a loss leader to get consultancy work , and shock jocks giving good publicity to companies (and governments) which pay them significant fees as PR consultants.
- 6
Axel | 2/8/2006 5:51:34 AM
I believe to be in a world with faster release cycles, more competition on platform level, faster transmission from research to real world app, etc.
All this creates more complexity and so it is getting more difficult for the analysts to provide guidance for strategic IT decisions, even if they want to and are not bought by some platform vendor or write to feed existing customer expectation.
- 7
Jack Dausman http://www.Leadershipbynumbers.com | 2/8/2006 11:11:16 AM
Ed, it is very interesting to read what the analysts, themselves, say about their job and others. I was surprised to read David Ferris grousing that at Lotusphere, of the 20 analysts, only "three or four had some level of competence." ( { Link } ) I'd really like to know who made his list.
James Governor of RedMonk cites Richard Holway, "During my 40 years in this business I’ve been pretty good at spotting trends. However, I’ve been pretty p**s poor at getting the timing right." ( { Link } )
What a great quote.
- 8
Danny Lawrence | 2/8/2006 1:40:47 PM
Cringely posts how he did on last years predictions { Link } along with his predictions for this year. Not exactly the same thing, but it is an interesting read.
IMHO analysts and their firms have one "important" purpose: to offer numbers, where the companies can't due to SEC-regulations, etc. That's the only thing they are good for, at least from a press context. IDC, Gartner, give us the numbers we need for the mor economical oriented publications that need yearly/monthly/... figures, perferrably broken down to product and country.
I can't recall that any analyst was really right with his outlook into the next 3/6/12/18 months.
And regarding credibility: If you as a customer/writer/... dont't check out where that report is coming from. Well ...
But that are just my 2 Euro-Cent.