Several bloggers have picked this up during Lotusphere (thanks Mick, Paul, Chris, Bruce etc.)... the Domino development team has posted a Domino Attachment and Object Service Estimator:
The DAOS Estimator (daosest) is a tool for planning the roll-out of DAOS on the Domino 8.5 server. The tool iterates through all the requested databases, scanning for documents with attachments. It keeps a list of all the attachments so it can estimate savings based on duplicate attachments found both within a database and across all databases.I am starting to hear crazy good numbers from real-world estimates and deployments of DAOS -- 50-70% disk space savings overall (which cascades into reduced I/O, faster backup, etc.). Somewhere the past few days I saw DAOS described as "the very definition of a killer feature" -- worth the upgrade alone. Couldn't say it better myself.
Link: IBM.com: Download the Domino Attachment and Object Service Estimator tool >
Post a Comment
- 2
Andy M | 1/25/2009 6:57:50 PM
It reliably crashes the Mac 8.5 client!
- 3
Stephen Coombs http://www.frontier-group.co.uk | 1/26/2009 3:53:37 AM
I have just finished running the tool and it’s returned some incredible figures:-
Summary:
Total DB's analyzed: 207
Total DB's skipped due to errors: 0
Total Size of NSF's Examined: 58.6GB
Total Attachments found: 127641
Total Duplicate Attachments found: 64737
Total DAOS Eligible Attachments: 127641
Estimated Size of DAOSified NSF's: 10.6 GB
Estimate Size of DAOS dir: 26.3 GB
Total Disk Savings: 24.1 GB
Just need to sort out transaction logging to take advantage of the space savings!
- 4
Dave Harris http://www.wavysworld.com | 1/26/2009 6:09:50 AM
@Robert, do you know if there's a PMR for that? I have a client that's mostly System i, and they're on 8.0.1 at the moment (so even 8.02 with document compression is going to save them heaps), but I reeally want to get them up to 8.5 since disk space is one of their biggest concerns.
- 5
Robert Laing | 1/26/2009 7:57:25 AM
@Dave, I don't know if there's a PMR. I also have a couple System i customers who would jump on 8.5 if they could realize significant disk savings.
- 6
John Head http://www.johndavidhead.com | 1/26/2009 9:14:28 AM
@5 Its the DAOS Estimator that is not working on the System i for you, not DAOS itself, correct? I know we did testing of DAOS in the System i and it works great
- 7
Robert Laing | 1/26/2009 9:48:01 AM
@6, yes - the DAOS Estimator does not work on the System i. I have not tried DAOS yet.
- 8
Dave Harris http://www.wavysworld.com | 1/26/2009 10:21:58 AM
@5, 6, 7, Yeah I realised it's the estimator not working on system i - but I need to give them proof that DAOS is going to give them lots of disk space happies, for which I need the estimator.
BTW, Ed, and everyone, let me wish you all a very healthy, happy and prosperous Year of the Ox.
- 9
paul http://no web | 1/26/2009 11:28:43 AM
I am afraid of DAOS. I went to the labs at lotusphere and I found out that in fact DAOS would consume more disk after initial installation.
This is why: You install DAOS and you instantly get 30% savings... sounds good! but after DAOS is installed and running, the pruning deferred date is set to 30. Why? because the guys at the labs told me that for technical reasons these days must (prune days) must be greater than the period of my backup utility (that is 30). So... during 30 days I will have plenty of .nlo files that will be there and will be pruned only after 30 days. My users receive many spams and them remove many many emails per day... all this space will not be removed from DAOS! only after pruning.
So this estimator is good to estimage the instant savings but it doesn't tell you the number of unreferenced .nlo files that will be left there!
- 10
Kerr | 1/26/2009 1:16:12 PM
@Paul, clearly every environment will have it's own unique set of criteria that influence whether any feature should be enabled. Can you put some more numbers on this? When I look at some sample numbers the only why that I can end up with DAOS using more space for mail on a 30day retention policy is if you receive over 100 spam messages per user per day.
Let's flesh that out. If you are saving 30% on mail file sizes and you have an average mail file size of 100MB then you will save an average of 30MB. So you are going to have to receive 1MB of spam per user per day to use up more space than DAOS consolidates. A quick google gives me figure of 95% of all spam is less than 10k. So lets be generous and say that all your spam has 10k attachments. That gives 100 spam messages a day per user before DAOS is not pulling its weight.
This is being conservative. If you've got 500MB mail boxes and say for every spam mail that is over 10K you have 9 that are below the DAOS threshold (or no attachments) you could easily see spam numbers of 1000 per user before DAOS used more space.
If that's the case, I'd agree that you have other problems to solve before thinking about DAOS.
- 11
John Smart http://www.greyduck.com | 1/26/2009 1:56:24 PM
@9, @10: DAOS only applies to file attachments, so you'd have to receive 30Mb of spam file attachments.
Also, Paul, contact me about spam filtering at john.smart@greyduck.com I rarely receive any spam at all. (Case in point: I'm confident enough to put my email address here, where email harvesters can get it.)
- 12
Kerr | 1/26/2009 3:18:10 PM
@11, Yeah, I know, I was using a worst case scenario where all spam has attachments. Pretty unlikely, but it proves the point.
- 13
Dan Rosato | 1/26/2009 7:15:08 PM
Version 1.1 of the DAOS Estimator is now available for download. It addresses the MIME-CD hang as well as the System i issues. Sorry for any inconvenience.
{ Link }
- 14
Robert Laing | 1/26/2009 9:48:49 PM
Downloaded the latest daosest and it now works great on the System i. Space savings in our environment will be about 33%
- 15
paul http://no web | 1/27/2009 4:45:19 AM
@10: Spams are not the only problem. Uses deletes emails they usually receive. At lotusphere 2009 a statistics from IBM says that users receive 200 msgs per day and they remove 150.
@11: A spam email that has images or a big body uses attachments internally. This is because the images are stored in $file and if the size of the text of a smap is huge then it is stored in a $file field too. If you don't receive spams in your mailbox this is probably because your spam program is working just fine! but DAOS happens before your spam filter. It means that if you receive a SPAM email containing an attachment, even if your spam filter removes it from the mail.box, DAOS already stored the attachment.
- 16
Kerr | 1/27/2009 5:27:00 AM
@15, Well, it's certainly possible that you have a highly unusual environment, where DAOS would not be the most efficient route to go. Remember this effects only mail with attachments and even then only attachments over 4K in size (by default, you can tune it). So of all the mail your users receive they delete 1M per day in unwanted attachments? And these attachments are deleted by everyone? If one person decides to not delete the attachment then DAOS has marginal overhead and if 2 people do then DAOS is saving space.
On your second point, you should certainly be identifying and blocking spam before it hits any DAOS enabled mail.box. If you don't have a solution to this
Also remember that you can try this out in your environment and see how the numbers actually work out. Enable DAOS on one server and not on another and see what kind of real world results you get. Try enabling it for some mail boxes and not for others. Try only using it for document libraries. There are lots of ways you can tune this to save space and at the end of the day, if you're the 0.001% of environments that won't see any benefit from DAOS, don't use it, but at least recognise that you're a special case.
- 17
Scott Marchione http://scottmarchione.blogspot.com | 1/27/2009 12:28:01 PM
maybe I'm just not cut out for IT work, but I can't seem to get the thing to analyze anything other than my default directory. I tried daosest data\mail, daosest \mail , daoest mail and after I type the password for the ID file, nothing happens..... what the heck am I doing wrong?
- 18
Mike Burford | 1/28/2009 6:55:45 PM
@17 - the following worked for me: At the Domino console type: load daosest mail -o c:\file\location
I've run the Estimator on a (8.0.2) mail server with 1,384 mail files and it reports a saving of 43.4% saving in disk space - 145.9 GB. The results for the archived mail files (1,457 of them) was 40.6% - 112.6 GB. It was enough to encourage the customer to start planning to upgrade to 8.5.
- 19
Jack Milunsky http://www.agilebuddy.com | 1/29/2009 9:21:06 PM
Interesting to hear that the Notes team is doing Agile development.
Jack
blog.agilebuddy.com
www.agilebuddy.com
- 20
Shrinivas Sahadev | 2/4/2009 3:16:17 AM
Hi have problem while running it on AIX 64
Could not load program daosest:
Dependent module libnotes_r.a could not be loaded.
Could not load module libnotes_r.a.
System error: No such file or directory
How do i overcome that
- 21
AirWolf | 6/18/2010 6:37:39 AM
I have the same problem on AIX 5.3L.
Domino ver: 7.0.3 FP1. Any Idea?
"0509-150 Dependent module libnotes_r.a could not be loaded.
0509-022 Cannot load module libnotes_r.a.
0509-026 System error: A file or directory in the path name does not exist."
Sounds like a great tool. Unfortunately it's not working on the system i yet. I spoke to a couple developers at Lotusphere this past week and they said it was packaged incorrectly for system i. They said they'd notify me when it was working but nothing yet.