Ed Brill

Post a Comment

1. 1  Daniel Lieber http://www.iiui.com | 4/13/2007 2:31:08 PM

   Too bad they didn't actually follow the law ( { Link } )! Lotus Notes is the only fully integrated system with the availability of true records management built directly into it (we know this as we wrote the certified application -- see the link at top of this comment if you want to know more). What is also fascinating is that by statute the President is directly responsible for thses records. No consequences are directly referenced in the laws that I could find, so any possibilities I will leave to your imagination, other legal scholars, and possibly Congress. There will be many more discussions on records management coming soon!

1. 2  Nathan T. Freeman http://nathan.lotus911.com | 4/13/2007 2:40:28 PM

   AHAHAHAHAHAHAHAHAHA

   That's awesome. I want to know the consulting firm and the toolkit used for the migration.

1. 3  Craig Wiseman http:\\www.wiseman.la/cpw | 4/13/2007 2:42:56 PM

   5 million emails more or less than 18 minutes?

1. 4  Luis Guirigay http://lguiriga.blogspot.com | 4/13/2007 3:15:10 PM

   I don't want to be the company that was in charge of the migration process.

   I have seen other situations like this one, but, I think we all agree, this is going to be a good selling point if IBM wants to stop the customers from moving to another mail platforms.

1. 5  Chris Whisonant http://cwhisonant.blogspot.com | 4/13/2007 3:21:23 PM

   WOW!!! That puts an entirely new spin on it... :)

1. 6  the nutjob conspiracist http://andy.the-broyles.com | 4/13/2007 3:21:39 PM

   So maybe there is some method to the madness of the migration decision afterall? I can see the scene now...

   GWB: Karl, we really need to have these troublesome emails magically disappear, what do you propose?

   KR: Hmmm....I know Mr President, we can arbitrarily decide to migrate the WH to MS Exchange.

   GWB: What will that get us?

   KR: Well, all the corporate guys have do for no apparently sound business reason so we have precidence and deniability AND the tech guys will back us up that it is commonplace for the migration process to 'lose' vast amounts of historically significant email AND Bill Gates (& Bono too) will love us as well for the sales and reference account story. Heck, we might even get a campaign contribution from them if we do...flawless plan, if I do say so myself, Mr. President.

   GWB: Make it so Karl.

1. 7  Ed Maloney  | 4/13/2007 3:39:39 PM

   I didn't realize that the white house had migrated from Notes. I used to use GWB as proof that Notes is easy to use. Maybe not easy enough...

1. 8  Ed Brill http://www.edbrill.com | 4/13/2007 3:43:02 PM

   Ed, the decision to migrate is usually a political one. :-)

1. 9  Flemming Riis  | 4/13/2007 3:50:36 PM

   Even if they didnt migrate everything for some odd reason.

   what they needed from domino they should still have the backups somewhere shouldnt they.

   { Link }

   Could be a good item on the next budget bill.

1. 10  Charles Robinson http://cubert-codepoet.blogspot.com | 4/13/2007 3:56:50 PM

   I suppose they weren't using mail journaling, which I think was added in R6.

1. 11  Richard Schwartz http://www.rhs.com/poweroftheschwartz | 4/13/2007 5:27:45 PM

   Ed, unless I'm missing something, that quote is no longer in the linked article.

   @9, Flemming: thanks for the plug for our product (I'm the architect for the Notes/Domino solution on that hardware), but I do want to stress that it's an archiving solution, not a backup solution. Subtle difference, but important. And also: I can say with assurance that the White House is not and was not a customer for our archiving product for Lotus Notes. It's possible that they are using it for Exchange, but I haven't heard about it, and I think I probably would have since the architect for that product sits one office away from me.

1. 12  Ed Brill http://www.edbrill.com | 4/13/2007 5:29:23 PM

   True, it's gone. But it's still in the minutes of the White House press conference:

   { Link }

1. 13  Chris Whisonant http://cwhisonant.blogspot.com | 4/13/2007 9:17:21 PM

   Remember, we're not talking about White House emails. We're talking about Campaign and Republican Party emails.

   { Link }

1. 14  Craig Wiseman http://www.wiseman.la/cpw | 4/13/2007 9:24:48 PM

   @13 - In general that appears to be true, but we don't actually know that for sure.

1. 15  Craig Wiseman http://www.wiseman.la/cpw | 4/13/2007 9:26:41 PM

   @13 - Actually, the 5 million lost emails are White House emails, from the Exchange downgrade early in the 1st W Administration. I think you're referring to the other email situation. Seems hard to keep up with all the issues going on now...

1. 16  Rob McDonagh http://www.CaptainOblivious.com | 4/13/2007 9:46:47 PM

   @15 Right, there are two separate sets of emails involved.

   First, many White House officials have been using unofficial email accounts hosted by the RNC to get around the record maintenance and transparency requirements that are supposed to apply to official communications between members of the government. The logic they use to justify the use of those accounts is that they weren't talking about governing, they were talking about campaigning. Whether that's actually true or not, Patrick Fitzgerald's investigation into the leaking of Valerie Plame's identity led him to those email accounts, and he instructed the RNC to save all those messages. The RNC now says that there is a certain amount of Karl Rove's messages that are missing, but that they didn't delete them, and in fact they have since then removed Rove's ability to delete his own emails (after Fitzgerald's instructions arrived).

   Now we're also finding out that the official White House email system lost some unimaginable number of messages, presumably while they were migrating from Notes to Exchange. Which, of course, is both appalling and absolutely hysterical. Especially given the industry we work in. I must second Nathan (@2), with a slight modification: BWA-HA-HA-HA-HA-HA-HA-HA!

1. 17  Craig Wiseman http://www.wiseman.la/cpw | 4/13/2007 10:22:43 PM

   @Ed - "I'm not writing about politics...."

   Technically correct, but it'd be reallll hard to believe that you didn't know your post might, just might, spawn some political responses.

   Which makes this post just blend perfectly with all hair-splitting garbage coming from Washington (and from both sides of the aisle)...

   8-)

1. 18  Richard Schwartz http://www.rhs.com/poweroftheschwartz | 4/14/2007 1:09:52 AM

   @12 Ed, This is funny... I found the minutes on the RNC site { Link } . So it seems that email isn't the only thing they use duplicate systems for ;-)

1. 19  Chris Whisonant http://cwhisonant.blogspot.com | 4/14/2007 7:19:24 AM

   Apparently, at least on the RNC e-mails, Rove thought the emails were being archived:

   { Link }

1. 20  cb  | 4/14/2007 9:31:41 AM

   Interesting that the Reuter's article no longer contains that paragraph. You can still see it here though: { Link }

1. 21  Flemming Riis  | 4/14/2007 4:33:31 PM

   -but I do want to stress that it's an archiving solution, not a backup solution. Subtle difference, but important

   Yeah once in never out :)

1. 22  Rob McDonagh http://www.CaptainOblivious.com | 4/14/2007 10:23:37 PM

   Interesting take on this from someone who claims to have worked on the White House's email migration: { Link }

   The ultimate point is the the migration tools used would only lose messages if the admin doing the migration chose to lose messages (which matches my own email migration experience), and also the timing of the loss is odd because the White House's migration off Notes started in 1999. It wouldn't surprise anyone here to learn that Notes applications were still around after several years, but we all know that it truly isn't that difficult to switch out the email itself. Email is a commodity, after all, and businesses "switch" from one messaging platform to another without this sort of multi-million message loss all the time. So something seems a bit odd about that aspect of the report.

1. 23  Nathan T. Freeman http://nathan.lotus911.com | 4/15/2007 7:16:09 AM

   So, Rob, THAT'S the company and HIS is the tool for the migration?

   Now we just need to know who the heck he works for. :-)

   Of course, it's pretty obvious on the face of it that either a) the messages were lost deliberately or b) the White House employed staggeringly incompetent email admins. I can believe either one. It's not exactly a shocker in the world of politics, now is it?

1. 24  Ed Brill http://www.edbrill.com | 4/15/2007 8:34:32 AM

   Rob, very interesting link.

   I sure don't remember EOP starting the migration in 1999...I thought it was coincident with regime change. But I could be wrong.

   Also it puzzles me that an Australian (see profile) would have been that closely involved in the EOP migration. Based on his use of first person and his location in Seattle, my guess is he works for MS directly.

1. 25  Keith Brooks http://www.keithbrooks.com | 4/15/2007 8:39:40 AM

   And here I was wondering if there was a market for finding lost emails, I guess I have an answer.

   Of course what they meant was they did a rip and replace.

   But funny how the backups don't exist isn't it?

   Bummer the SEC can't nail them for it like they nail every corp.

1. 26  Daniel Lieber http://www.iiui.com | 4/15/2007 10:28:13 AM

   @25; The National Archives is the recipient of the White House records when they are ascended (aka promoted or transferred) from the White House itself at the directive of the President. The President is directly responsible, explicitly by law, for all White House records being retained in accordance with the Presidential Records Act. Violations of law by a current President are prosecuted exclusively in Congress. Time will tell whether this will result in E-MailGate (aka A bad "Outlook" possibly related to the "Exchange" of systems) or is simply dismissed as a matter of course. In reality, official records are inadvertently destroyed or "lost" on an all-too-frequent basis; the vast majority are of these losses are due to force majeure causes such as hurricanes, floods, fires, etc.

   Also, the SEC does not go after public corporations unless there is a specific reason relating to an investment vehicle; the primary SEC rule affecting e-mail is for investment traders, generally requiring the retention of all relevant communications for 3+ years.

1. 27  Charles Robinson http://cubert-codepoet.blogspot.com | 4/15/2007 10:39:52 AM

   @23 - Nathan, I think he means it's "his" in the same way Damien Katz can take credit (and rightfully so) for the Notes 6 formula engine.

1. 28  Craig Wiseman http://www.wiseman.la/cpw | 4/15/2007 1:43:14 PM

   @23 - or both. One can elect to use shoddy migration contractors and then be "shocked, just shocked AND surprised" at the outcome.

   I'm pretty sure I watched too much x files late at night when I was younger.

1. 29  David Bell  | 4/15/2007 9:28:10 PM

   @17 - not just both sides of the aisle, but both sides of the country.

1. 30  Paul Robichaux http://www.robichaux.net/blog | 4/16/2007 8:57:57 AM

   There's nothing new under the sun. Check out this NYT story { Link } from May 2000, in which the White House counsel is grilled about missing messages from 1998.

1. 31  Nathan T. Freeman http://nathan.lotus911.com | 4/16/2007 11:37:42 AM

   @30 - Except in the 1996-1998 timeframe, it was pretty normal to have lousy mail infrastructure deployments. Email came a long way even in the 5 years between '98 and '03.

   That being said, it's not like saying "a politician lied!" should be a stretch for anyone.

1. 32  Dave Madison  | 4/16/2007 4:49:39 PM

   @30. Paul, would that have been when the WH was a Notes shop? :-)

   Ed, where does Perino say they are running an Exchange server? I can't seem to find that reference anywhere. She says they switched from Notes (that's the client, right?), to Outlook. Nowhere in the press conference does she mention the server back end. Now we can speculate all we want, but for all we know they could just have easily been using DAMO. :-).

   Can someone provide public information on the fact they are running an Exchange server? (Not, "I know someone who knows someone", or "I'm not allowed to say officially, but....") Afterall, isn't want of the goals of DAMO to ensure the Domino servers stay around? Right now all we have to go on is Perino's public statement that they switched from "Notes" to "Outlook".

1. 33  Jim Bernardo  | 4/16/2007 7:48:01 PM

   Uh huh...lost the emails...and the dog ate my homework...

1. 34  Nathan T. Freeman http://nathan.lotus911.com | 4/16/2007 8:07:59 PM

   @32 - HUH!? Yeah, I'm sure the White House migrated the client, but lost 5 million emails while on the Domino platform. 'Cause, y'know, clustered replica servers are prone to massive mail loss.

   Hell, even if they STAYED on Exchange from one client to another, they're unlikely to have lost that kind of mail volume in anyway unintentional.

   Are you a lawyer by trade? Or are you simply a Microsoft employee, who's time would be better spent integrating Exchange with SQL Server?

1. 35  Richard Schwartz http://www.rhs.com/poweroftheschwartz | 4/16/2007 8:22:23 PM

   @31 Nathan: A long way... that's for sure. Prior to 2002, Domino didn't support journaling. Before that, loss of messages could only be avoided by denying delete rights or through the use of a third party journaling product.

1. 36  The Unofficial Poster Child For Lotus Notes and Domino http://www.bobbaehr.com | 4/17/2007 4:43:33 AM

   Greetings:

   Looks like the proper 'strategery' to 'securitize' the messaging infrastructure was lost with the migration.

   On the bright side, hopefully one of the emails lost was Dick Cheney's list of his favorite hunting places.

   :)

1. 37  Jim Bernardo  | 4/17/2007 8:24:47 AM

   New headline: White House May Find Countless E-Mails { Link } Imagine that... :-)

1. 38  Mike Brown  | 4/17/2007 10:01:47 AM

   Stop Press:

   Missing emails found, buried alongside weapons of mass destruction.

   Cheers,

   - Mike

1. 39  Bill McCuistion  | 4/17/2007 3:49:03 PM

   I'm confused. Mail migrations don't "move" messages from one platform to another... they copy and transform them.

   A migration project would not "destroy" data, unless that was a (perhaps hidden) objective of the migration.

   It may be correct that not all data is migrated, but it would still be available in the source system, if normal backups (and perhaps archives too) were maintained.

   Sounds like smoke cover-up.

   ----------

   Using the WH and RNC accounts seems to be a legal requirement, and I can see how it would be easy to have email contacts duplicated between the two, and have an email thread start properly in one system, then morph into topics that would properly be on the other system.

   I assume that Congress and the DNC have the same issue.

1. 40  Dave Madison  | 4/18/2007 5:01:55 PM

   @34 Nope, not a Microsoft employee. Not sure the how the ad hominem comment regarding Exchange and SQL has any relevance whatsoever to my question. What was the point of that? Nevertheless, I see the question still remains unanswered. What evidence do we have, based on what is publically known, that they are running an Exchange server? or even a Domino server? And,if they are or were running a Domino server, what public evidence do we have that they were running them in a clustered configuration?

   I agree with your overall point that regardless of what email server they are using, losing that much email is more likely due to either incompetence or a deliberate act rather than any suspected fault of the server itself. It just seems Ed was implying this may not have happened had they not moved to Outlook. And there seems to have been some unspoken implication that there was an Exchange server involved. My point is, we don't know based on the public information we have. But, it sure makes a great possible MS bashing story.

1. 41  Ed Brill http://www.edbrill.com | 4/18/2007 5:10:12 PM

   Come on Dave, you're being a happy troll and honestly, I'm done engaging in tail-chasing from your hypothetical queries.

   I can't publicly on my own weblog say anything about the EOP, whether I have first or second-hand information. Do you have a reason to doubt there is an Exchange server involved in this Outlook deployment? Something you know from when you -were- at Microsoft?