Backup storage

SATA based. This meant for storing data which we primarily need in case systems die etc.

Online storage

SAS based. This is storage intended to be actively used, not just written to for backup purposes.

Misc notes

NetApp fiber shelf is 14 drives. NetApp SAS shelf is 24 drives. 2 TB SATA drives only supported in SAS shelf.

3 disks are needed for root as we need RAID-DP, as per NetApp's own recommendation (RAID-DP allows upgrade of drive firmware if needed without disrupting the running system).

SAS and SATA disks need to be separated, both in different RAID sets / aggregates and in separate disk shelves. We assume RAID-DP for everything, so we have min. 2 parity disks per RAID set. We assume 2 hot-spares to allow for higher resiliency and for non-disruptive upgrade of disk firmware.

WIP Mail to NetApp

Hello,

We are trying to set up our new New York Internet (NYI) site to have a full mirror of all important data and all the main systems which run the FreeBSD project. The plan is also to have enough storage etc. to be able to run as the primary site for the FreeBSD project in case the current primary site at Yahoo! fails.

Initially I should say I'm far from an expert on NetApp products - I'm just an enthusiastic user :-), so I very much value input on best configuration etc. As don't know what the relative prices are between different NetApp components etc. my main guideline for which components to suggest has been to take the "smallest" possible system, which is still future proof as possible. To better be able to find out the right solution (hardware and software) I also tried to describe in part what we plan to use the system for.

I do not know if there is a difference in the software between the different systems (I don't think so), but if there is we would prefer systems which are based FreeBSD. This is certainly not a requirement, but if possible we would like to say that the entire setup runs entirely on FreeBSD :-).

After trying to estimate storage requirements for the new FreeBSD.org NYI site we found out that we "need" something along the lines of this (taking future use into account):

10 TB "slow" storage (IE, SATA) for backup / replication target. 6 TB fast storage (IE, SAS) for storage "online" data, such as FreeBSD FTP site.

The slow storage will, among other things, be used as a replication of the NetApp FAS 3050 (with 4TB of storage) we have located at Yahoo!. Some of the fast storage will be connected to a public facing FTP server, which of course should not have access to the backup data, so we will probably be looking at using vfiler to separate data access.

For replication from our Yahoo! filer to the new NYI filer I have been mainly looking at using SnapMirror, but perhaps SnapVault would be better - I'm not really sure. We don't need to have live replication from Yahoo! to NYI, but of course the smaller the delay the better. We also have a number of systems with local storage (IE, important data is not on a filer etc) which needs to be backed up to NYI. I was mainly planning on using a simple rsync based system where a server at NYI NFS mounts the new filer, and we then snapshot this volume. Perhaps Open Systems SnapVault might be better, but among other things I do not know if the client software runs on FreeBSD.

I have been trying to look at different configurations which would accommodate this, and be as flexible as possible for future expansion. To find out how much available space we would get for each drive I used the "Available disk capacity by disk size" from "Data ONTAP 8.0 7-Mode Storage Management Guide" and subtracted 10% extra for ONTAP reserve based on the same documentation.

This could e.g. by done by 20 600GB SAS drives (3 root, 2 hot-spare, 2 RAID-DP parity, 13 data). For the SATA part this could be handled by 12 2TB drives (2 hot-spare, 2 RAID-DP parity, 8 data).

I have mainly been looking mainly at FAS 2040 and FAS 3140 due to their support for ONTAP 8 and for "larger" deduplication volumes. In the end I think the one which fits best is FAS 3140 as it is more flexible with regard to future expansion.

So, the main configuration I came up with was the following:

• FAS 3140
• SAS adapter - quad port
• 4-Port 1GbE adapter (for NFS access etc)
• 1 DS4243 with 24 600GB SAS drives
• 1 DS4243 with 14 2TB SATA drives

For SAS drives I filled the shelf as it was mostly full, and for SATA I added 2 extra drives as they give us some extra margin to run on, and I don't think it makes much difference with regard to total cost.

For the system we would also like the following licenses:

• NFS
• CIFS
• HTTP
• iSCSI
• ASIS
• SnapMirror
• SnapRestore
• MultiStore

-- Simon L. Nielsen FreeBSD.org admins team

Current system use

Does not include dumpster. All sizes in Giga Bytes.