So my FreeNAS server is full.<\/p>\n\n\n\n
I wanted to build it again in a rack mount case for inclusion in my new A\/V rack in the new cinema room, and bring all my ‘home compute & network’ together in a single place – but I couldn’t afford to take it offline. <\/p>\n\n\n\n
Since it’s been a while since I built it, I had a look on eBay for all the same parts (since it’s just a FreeNAS file server, it doesn’t need a whole bunch of power) and I was really quite happy with how solid the original build has been. I therefore purchased all the same parts again (except for a different case)<\/p>\n\n\n\n
SuperMicro X9-SCL-O motherboard
Xeon E3-1260v3 Processor
32Gb ECC RAM
8 x 6Tb Seagate NAS drives (increasing from 6 x 3Tb drives)
2 x 120Gb SSD
1 x 800w PSU
1 x Rosewill 4U rack mount case<\/p>\n\n\n\n
Apart from the drives, I managed to get the whole lot for about $350 which was a steal compared to my original build.<\/p>\n\n\n\n
Throw on the latest FreeNAS 11 and it’s ticking along nicely.<\/p>\n\n\n\n
However, I now have about 12Tb of data to move from the old server to the new one, without bringing the old one offline.<\/p>\n\n\n\n
I’ve always been a fan of rsync… I know there are faster ways of transferring data, but rsync is solid, handles incremental updates, and is very very mature. But it ain’t fast.<\/p>\n\n\n\n
After having a dig around, I finally found a magic incantation which upped my transfer rate from about 150Mbps to about 850Mbps!!!<\/p>\n\n\n\n
rsync -aHAXv -e \"ssh -T -o Compression=no -x\" user@<source>:<source_dir> <dest_dir><\/code><\/pre>\n\n\n\nThere’s a fantastic discussion here :<\/p>\n\n\n\n
https:\/\/gist.github.com\/KartikTalwar\/4393116<\/code><\/pre>\n\n\n\n[UPDATE] — so, using this to migrate servers is missing some special sauce. Using the command line as-is was resulting in some odd copy failures, mainly related to the mismatch of user\/group permissions between the source and target machines.<\/p>\n\n\n\n
We need to tell rsync NOT to transfer permissions (and I added a parameter which cleans up the target folder too) – this seems to work well, and saturates my network connection while running \ud83d\ude42<\/p>\n\n\n\n
rsync -aHAXv -A --no-perms --stats --delete -e \"ssh -T -o Compression=no -x\" remote:\/Music\/Compressed\/ \/Media\/Music\/MP3 <\/code><\/code><\/code><\/p>\n\n\n\n<\/p>\n","protected":false},"excerpt":{"rendered":"
So my FreeNAS server is full. I wanted to build it again in a rack mount case for inclusion in my new A\/V rack in the new cinema room, and bring all my ‘home compute & network’ together in a single place – but I couldn’t afford to take it offline. Since it’s been a […]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[],"class_list":["post-73","post","type-post","status-publish","format-standard","hentry","category-uncategorized"],"_links":{"self":[{"href":"http:\/\/www.hexland.com\/blog\/index.php?rest_route=\/wp\/v2\/posts\/73","targetHints":{"allow":["GET"]}}],"collection":[{"href":"http:\/\/www.hexland.com\/blog\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/www.hexland.com\/blog\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/www.hexland.com\/blog\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"http:\/\/www.hexland.com\/blog\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=73"}],"version-history":[{"count":2,"href":"http:\/\/www.hexland.com\/blog\/index.php?rest_route=\/wp\/v2\/posts\/73\/revisions"}],"predecessor-version":[{"id":76,"href":"http:\/\/www.hexland.com\/blog\/index.php?rest_route=\/wp\/v2\/posts\/73\/revisions\/76"}],"wp:attachment":[{"href":"http:\/\/www.hexland.com\/blog\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=73"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/www.hexland.com\/blog\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=73"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/www.hexland.com\/blog\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=73"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}