I’ve been exploring Thunderbolt quite a bit lately. As a follow up to my post about using Thunderbolt in a NAS, I recently bought a Gigabyte Maple Ridge Thunderbolt 4 add on card. I’ve captured a few thoughts, observations and pics here.
In this post I discuss how you can upgrade a NAS Server by adding Thunderbolt 3 for lightning fast connectivity at 20 or 40Gbps. This particular implementation is specific to an Apple Mac Mini M1 and a Linux NAS server on an older Supermicro X10SRL-F motherboard, but the principles should be the same on similar architectures and operating systems.
Updated for 2021 !!!
Update: I’ve recently added a Thunderbolt 3 PCIe card for easy 40 Gbps connectivity.
There are a lot resources online for building your own DIY DAS. A lot of them contain good information but none of them were comprehensive for the DAS I wanted to build so I compiled the information that was useful for my specific 2019 build here. The prices linked below are subject to market fluctuations and timing, so some of the components I found at a great price. Overall they should be roughly the same as what you can find. Your mileage may vary.
I’m limited to about 80MB/s on downloads on my VPC at Digital Ocean, but I run Nzbget for downloading large files from usenet. It doesn’t take long to download at all, but out of curiosity I wanted to see if I could parallelize this and download multiple files at the same. I use Sonarr for searching usenet for freely distributable training videos which then sends them to NZBget for downloading. Since Sonarr can send multiple files to nzbget which get queued up, I figured I can reduce the queue by downloading them at the same time.
Using Ansible and Terraform (devops automation tools), I can spin up VPC on demand, provision them, configure them as nzbget download nodes and then destroy the instances when complete.
I’m limited to about 40MB/s on downloads on my VPC at Digital Ocean, but I run Sabnzbd for downloading large files from usenet. It doesn’t take long to download at all, but out of curiosity I wanted to see if I could parallelize this and download multiple files at the same. I use Sonarr for searching usenet for freely distributable training videos which then sends them to SABnzbd for downloading. Since Sonarr can send multiple files to sabnzbd which get queued up, I figured I can reduce the queue by downloading them at the same time.
Using Ansible and Terraform (devops automation tools), I can spin up VPC on demand, provision them, configure them as sabnzbd download nodes and then destroy the instances when complete.
The instances all run the same sabnzbd config and the instances use haproxy for round-robin distribution. I will probably change this to Consul, but I just wanted something quick so I used a basic haproxy config.
O’Reilly’s Radar - Release 2.0 Magazine interviewed me about Dashwerks, Inc’s role in the proliferation of open-source technology and how we were able to develop a profitable business model using a hybrid open-source / proprietary business model. My interview is in the Winter 2007 issue.
 | In this issue of Release 2.0, we consider the state of the open source hardware products and business models that are emerging. It’s the future of manufacturing - and early signs of it are here now. |
Update July, 16, 2018:
O’Reilly is updating their website and they have released the eBook version of this for free.
From their site:
O’Reilly published Release 2.0 from February 2007 (when we acquired Esther Dyson’s Release 1.0 newsletter) through April 2009. In this print journal, we endeavored to provide “news from the future.” Looking back, we did a pretty good job. Themes included open source hardware, information visualization, web operations and performance, geo, and big data–all of which matter even more today. We’ve made PDFs of the entire archive freely available, so read, remember, and learn.
