Bill's Blog

1 - I just wonder: isn't this almost the same feature as the famous Single Copy Object Store a.k.a. Shared Mail which exists in Domino for a long time yet?

That feature that caused so much problems everyone recommended NOT to use it?

2 - In the very early days of Iris when the team was adding file attachment support, there was debate over whether to store files in the NSF or externally. They went with the in-the-NSF model because it made management of NSF files easier.

3 - @1 - According to Russ Holden, there is ZERO common code between Shared Mail and DAOS. It's not an extension, update or improvement. It is a complete refactoring with a totally different approach.

The biggest fundamental difference is that each unique attachment is stored as an encrypted discrete file on the OS, so backups become dramatically more efficient. And that's where the big win comes in.

If you think about the process of backing up your Domino server, the original attachment storage model meant that you were not only repeating the attachment in the NSF of each recipient, but you were backing up a complete version of all those attachments at every backup cycle, because there was no difference between adding a new attachment and changing an existing one. The whole NSF was different, and that was the file processing unit.

With DAOS, because the attachment is outside the NSF, and is a discrete file, unless that single attachment reference has changed, there's no need to back it up.

So not only are you not storing the attachment 10 times on the server for 10 users, but you're not storing it 10 times 100 days of backing up the same content in the NSF file.

Think about that. If you sent a file attachment to 50 people in Domino 7, and backed up their mail for 200 days, you'd have 10000 copies of that same series of bits in your backup system. With DAOS, you can have ONE.

That's the big motivation here. Drive space for an actual server is cheap. Archival space, on the other hand, can get expensive real quick.

4 - Bob - I can imagine in those early v1 days, the idea of attaching a few k or a few hundred kilobytes of file wasnt going to stretch the database capability *lots*. And I can also imagine given the Notes server profile - as a guerilla departmental server ran outside the computer department - this was absolutely the right thing to do.

These days however, users tend to use their mailfile as a file system. The largest single, running, inuse mailbox I have *seen* - 240 GIGABYTES. Crazy. And since this is an optional thing - will be used buy those folks who are comfortable with this idea and can manage the separate file directory, etc.. (At least, we can all say, they *should* be able to manage, etc..)

This will do amazing things to database sizes (pruning out duplicate files) as well as (as Nathan points out) really reducing backup cycles.

Now - I'll say something controversial here. Disk is *expensive*. Not the disk itself (though that is still not cheap), but the management of the RAID/SAN, the offsite backup, and the business continuity of having such a lot of information - some of it business critical - in one place.

And the more we can help our customers with this issue, the better.

(I feel sorry for one site I know of - who have just taken delivery of their fifth PETABYTE of SAN. Scary..)

--* Bill

5 - @4, Bill, can you say for how many users that 5PB is for?

6 - We (with other administrators) have been analyzing DAOS but we're not yet convinced. First people talk about backup that it will consume less storage when backupping, but they don't talk about restoring. Suppose I have a generic backup utility (not a Domino dedicated backup utility) and I need to restore one NSF that was backupped 6 months ago... in this case I know I can restore the NSF but which files from the DAOS folder should I restore? I cannot restore the complete DAOS folder!!!

As you can see, if you use DAOS then you cannot use a generic backup utility.

Also DAOS is unsecure. The stored attachments are encoded using HUFFMAN or LZ1... they are public known algorithms. It's easy to crack them. You can say that you if you're local then you can also go to the NSF, but NSF has some kind of local access protection and it's not as easy as searching in the DAOS folder directly.
In the DAOS folder I can use the standard Windows search, and I can get valuable information. Encrypting the file system is not an option since it will reduce the overall server performance.

Also I must use transaction logging... that means getting another RAID (IBM recommends you to get a RAID when using TLogs).

So DAOS will cause more work for me (deciding which DBs should have DAOS, mantaining the file system of DAOS, getting crazy when restoring a single NSF, putting more security access to the server, getting an extra RAID, mantaining Transaction logging, ...).

On the top of that this is just to save disk space... not even bandwidth! And I personally doubt if disk space will be saved: The only fact that I must enable transaction logging makes me think that the savings I obtain from DAOS will be lost using Transaction logging.

For me, DAOS will be the same than shared mail and I remember I wasted too much time with shared mail and I don't want to pass through the same experience.

7 - Austin - I think your being unnecessarily hard on this stuff.

Firstly, on ANY server, if you lose physical security of the device or the backup tapes - you've lost security. I dont care if its a Domino server, SQL server, Oracle server, DB2, whatever.

DAOS, being on the server, is as secure as any other service. Yes its lightly encrypted. Its meant to be *fast*.

There's a timeout period on the DAOS store purge. Set that to two months or so - your median 'christ, this had better come back in a hurry' period. So you dont have to worry about restoring the DAOS filestore to the same point as the database to restore stuff. Remember, its a write once, read-many store, so it wont do any redmondian-stupid things like reuse references.

No. Its not shared mail. Yes, it works. No. Seriously. And on medium to large servers, properly set up transaction logging will save you a bucketload of time and performance (especially on startup). Just remember to point translog at a separate disk/raid/LUN/san partition from the data. (Paul Mooney's favourite mantra on this)

Give it a whirl, you'd be surprised.

--* Bil

8 - Austin - Just for the record, Huffman and LZ1 are not encryption algorithms, they are Loss Less Compression algorithms.

Among security experts (I'm not, but I consider myself rather knowledgeable on this subject) publicly known algorithms are considered much more secure than non public. Why? because the public algorithms have been analyzed by a large number of cryptologists; any flaws have been found (or will be).

Symmetric algorithms such as AES or blowfish and Asymmetric algorithms like RSA and Diffie Hellman are fully specified to the public; however they are considered the most secure algorithms.

The key to a good encryption is a good (publicly known) algorithm and a good key (or set of keys).

I haven't been able to find any specifics on the algorithms used for encrypting the DAOS files; but I'd expect Lotus to use algorithms similar in strength as the algorithms used for encrypting data/databases in Notes. Bill, do you have any info on that?

Regards
Mads

9 - Mads - sorry - I dont, I do know they intercept the file from the transaction log mechanism for speed, etc.. But not what kind of encryption is used. I'm guessing this will become more and more visible - here's hoping we get another beta soon.


Cheers,

---* Bill

10 - I can see that DAOS will store a single copy of an attachment sent to multiple people on the same email, but what happens if the same attachment is then attached to a completely separate email a week or so later and emailed to another 10 people? Will the attachment be recognised as the same attachment as the previous attachment and still only stored once or will it be treated as a separate attachment and stored again in the DAOS store?

11 - While following the blog here another option to DAOS came into my mind: a (unfortunately not cost-free) tool called Domino Storage Optimizer provided by www.proclients.de. Using this tool you don't encounter any performance issues as it is linked with a EMC Centera that does all the storage organisation and retrieval. The Centera is a black box to Lotus Notes and just returns an attachment when handed over a ticket (called CLIP). So security and encryption isn't an issue. In addition to that you can optimize all your databases on all (!, not only one) your servers by letting the local replicas in tact. So, even the restoring of an nsf isn't an issue anymore. Also, if you run clusters you can "eliminate" 100% of the optimised attachment data.
The only advantage DAOS has is that it comes without any extra costs.

12 - One thing I would like more information on is how it handles replication, particularly cluster replication. Can you have a cluster-mate configured to a different DAOS, so that you can have a real-time, off-site replica of not only the mail message, but also the attachment? Because if not....I'll take the disk bloat and peace of mind, thank you!