A week ago, the Microsoft Exchange team's unified communications blog published a chest-beating missive on their perceptions of competition in the messaging market.  I addressed some of their points a week ago.  One claim, however, took some additional examination.

Cost savings is the number one driver for migrations.  Notes systems simply cost more to run and manage.  A Ferris Research survey of 917 companies found that Notes systems can cost twice as much as Exchange 2007.
As vendors, we love to find third-party sources that validate our perspective on the market -- why our product is great, why we are better than the competition, etc.  However, in eleven years of competing with Microsoft, I've seen the truth stretched in countless ways, and the latest round are, simply, outright lies.

Today, I'd like to address the Exchange team's citation of Ferris Research.  The survey results in question was published in January, 2008, from data gathered as far back as 2006.  At IBM, we wouldn't even be allowed to cite this report, because it's over a year old; Ferris points out that none of the respondents were even running Notes 8.  We certainly wouldn't keep going back to the report after its methodology had been invalidated.  We would realize that an opt-in survey conducted over three years by an industry analyst off their subscriber mailing list isn't necessarily reflective of the market at large -- it's like a BMW dealer publishing a survey of the types of cars driven by their customers and representing it as a market share report.  We would accept criticisms that an opt-in survey does not equal a rigorous market share analysis.  We would recognize the folly of on the one hand clinging to a report that claims our competitor has only a small share of the market, while simultaneously declaring that that competitor has millions of customers that our partners should be targeting for migration.

But let's get back to the Exchange team's assertion about cost.

Ferris's survey asked customers only about the price paid for software.  They asked: "In US$, what do you expect to spend on email software licenses, email software maintenance contracts, email software updates, and so on as of January 2006, January 2007, and January 2008? (Please do not include professional services fees; we are only interested in software product costs. Please leave this empty if you cannot answer or find it hard to estimate.)"

On page 58 of the report that Microsoft linked to, Ferris Research admits that only 8% of their 917 respondents answered the question about price paid.  On page 60, the truth comes out -- the data on "price paid" for Lotus Notes cited in this report comes from only six organizations.  So Microsoft and all of its corporate ethics are standing up behind a Ferris Research report that asserts facts about the price paid -- which Microsoft uses the word "cost", feinting the reader to think they mean the total cost of ownership -- based on six responses.

Even that would be fine if the facts supported the Microsoft assertion.  But they don't!  On page 59 of the report, the median "price paid" figure from the Notes customers who responded (all six of them) was US$100/mailbox/year.  On page 58 of the report, the median "price paid" figure from the Exchange customers who responded -- 76 responses -- was US$130/user/year.  The Microsoft Exchange blog asserts, to reiterate:
a Ferris Research survey ... found that Notes systems can cost twice as much as Exchange 2007
Yet the median price paid for Microsoft was actually 30% higher.  The only way to get to "can cost twice as much" is looking at one specific segment of Microsoft Exchange respondents (all 12 of them) whose median figure was US$50/user/year.  So based on 18 total responses out of 917, Microsoft asserts that Ferris Research supports a statement that Lotus Notes "can cost twice as much", and attempt to get the reader to believe the assertion is about the cost "to run and manage".  And their assertion doesn't carry the report's "Caveat: Replies are confusing" that applies to this section.

At Lotusphere, I met with David Ferris as part of the executive / analyst meetings gauntlet.  David and I have known each other for many years, and IBM has been a client of his off and on during that time.  I think David, and the team of messaging industry experts who consult for him, know this industry well.  Their qualitative analysis is useful and reliable.  However, this single report has done more to undermine Ferris Research's credibility than the goodness of all their other work combined.  I've told David this, as have other IBMers.  Yet David stands behind the report -- and Microsoft's interpretation of it -- as you can see in his comments on his site, and continued to do so when contacted by my IBM colleagues last week.

I think Microsoft's bold-faced deceptive interpretation of this report -- which will never be fact-checked by those reading -- is the worst kind of unethical competitive marketing.  But Ferris Research's complicit involvement in this deception is even more troubling for the industry.  When a supposedly-independent analysis allows their work to be distorted, and chooses not to respond to criticisms about the report's methodology nor its interpretation, the industry is left to wonder -- how confident can we be in other work of this nature?  Meanwhile, a company that publishes "standards of business conduct" has senior managers publishing wildly distorted interpretations on the company's own website (key phrase: "Our advertising, sales, and promotional literature seeks to be truthful, accurate, and free from false claims."), while the company's COO rallies the troops to "eradicate" Notes and lies about its "eradication" from whole countries (Hey Kevin Turner -- prove it).  

For me personally, I'm sure glad I turned down the offer to interview for Julia White's job at Microsoft four years ago.  I don't think I could look at myself in the mirror in the morning.

Post a Comment