U.2 NVME to Thunderbolt Enclosure
This is an enclosure I built to put 8x U.2 enterprise NVME drives on a Mac Studio.
Thunderbolt 4 gives 4 PCIe 3.0 lanes, which is about 4 GBytes/s of throughput, and most NVME drives support a 4x interface. There are several products which support NVME to thunderbolt, but most focus on m.2 consumer drives and don't support the 110mm long enterprise m.2. The OWC Thunderblade, is listed as up to 32 TB of NVME, but is internally is 8x M.2 drives. What's worse is these are connected via 1x buses internally, as there is no PCI switch. This limits speed to any one drive, so any advanced raid setups will be limited. The consumer SSD's lack Power Loss Protection as well.
The OWC Thunderblade is 5500 USD for the 32 GB model. This is retarded. They refuse to sell the enclosure without disks, and make it non-upgradeable; so it's perfect for the typical Mac user (Why would you want to create and empty file!). OWC also claims a 2622 MB/s of peek throughput. This soultion has real results to 3376 MB/s read and 3220 MB/s write after exhausting on disk cache. In this case the limit is the PCI interface being maxed out.
With this you can upgrade NVME drives and I've found used NVME disks to have over a decade of life still left on them when used in typical service on the desktop.
This is $64/TiB vs. $189/TiB for the Thunderblade, or 1/3 the cost with 30% better performance.
NVME on OSX
First not all NVME U.2 drives are compatible with OSX.
Github discussion about NVME issues.
List of known OSX supported NVME U.2 disks
Parts
The following is needed to build this.
| QTY | Part | Price | Extended | Source |
|---|---|---|---|---|
| 8 | Micron 9300 PRO 7.68TB | 375 | 3000 | ebay |
| 1 | CI Designs Ci2000 enclosure | 54 | 54 | ebay |
| 1 | HPE 8sff cage | 84 | 84 | ebay |
| 1 | SFX PSSU | 64 | 64 | https://www.amazon.com/dp/B0DCJT8W2G |
| 4 | ππ π -ππππ 74pin to 2*ππ π -ππππ | 28 | 112 | https://www.amazon.com/dp/B098JBS7QD |
| 1 | PCI Express reiser | 9 | 9 | https://www.amazon.com/dp/B08R38B818 |
| 1 | 80 mm fan | 26 | 26 | https://www.amazon.com/dp/B00VG9PJBO |
| 1 | Fan control | 13.5 | 13.5 | https://www.amazon.com/dp/B0D4YGKC6G |
| 1 | UT3G USB to PCI | 115 | 115 | Aliexpress |
| 1 | CEACENT CNS44PE16 PCI Switch | 130 | 130 | Aliexpress |
| Total | 3607.50 |
Performance
This was tested as a ZFS Stripe or pool of the disks using AmorphousDiskMark using a 64 GB file, to ensure we're not hitting the disk cache or ZFS ram cache.
Construction notes
There's a ton of cable in here, but it's the least bad way to do this. It's important to have most of the holes other than a small vent for the PCI cards and the disks so that air is sucked though the enclosure. In testing I've seen the disks idle at 55-60c which is not warm for die temp on these disks; they are happy up to 80c. I put the temp sensor on the top of the disk cage, and configure the ramp setting so that it's very quiet and I can just feel some air movement with my hand in front of the disks.
