Difference between revisions of "Storage Server"
Jump to navigation
Jump to search
| Line 34: | Line 34: | ||
https://docs.broadcom.com/doc/pub-005110 | https://docs.broadcom.com/doc/pub-005110 | ||
| + | |||
| + | = Updating Disks = | ||
| + | |||
| + | == Firmware == | ||
| + | |||
| + | Current firmware on the disks is E004, but disks have E002 on them | ||
| + | |||
| + | <pre> | ||
| + | for i in `seq 2 25` ; do SeaChest_Firmware --downloadFW ./EvansExosX16SAS-STD-512E-E004.LOD -d /dev/sg$i; done | ||
| + | |||
| + | #SeaChest_Firmware -s | ||
| + | ========================================================================================== | ||
| + | SeaChest_Firmware - Seagate drive utilities - NVMe Enabled | ||
| + | Copyright (c) 2014-2021 Seagate Technology LLC and/or its Affiliates, All Rights Reserved | ||
| + | SeaChest_Firmware Version: 3.0.0-2_2_1 X86_64 | ||
| + | Build Date: Apr 27 2021 | ||
| + | Today: Tue Oct 10 22:14:38 2023 User: root | ||
| + | ========================================================================================== | ||
| + | nvme_ioctl_id: Inappropriate ioctl for device | ||
| + | Vendor Handle Model Number Serial Number FwRev | ||
| + | LSI /dev/sg0 SAS3x28 0705 | ||
| + | LSI /dev/sg1 SAS3x28 0705 | ||
| + | SEAGATE /dev/sg10 ST16000NM002G ZL20AJ3P E004 | ||
| + | SEAGATE /dev/sg11 ST16000NM002G ZL231860 E004 | ||
| + | SEAGATE /dev/sg12 ST16000NM002G ZL21T6Q1 E004 | ||
| + | SEAGATE /dev/sg13 ST16000NM002G ZL231PV0 E004 | ||
| + | SEAGATE /dev/sg14 ST16000NM002G ZL21TDHH E004 | ||
| + | SEAGATE /dev/sg15 ST16000NM002G ZL21Y85R E004 | ||
| + | SEAGATE /dev/sg16 ST16000NM002G ZL21S9FQ E004 | ||
| + | SEAGATE /dev/sg17 ST16000NM002G ZL21T38M E004 | ||
| + | SEAGATE /dev/sg18 ST16000NM002G ZL22C3MJ E004 | ||
| + | SEAGATE /dev/sg19 ST16000NM002G ZL21S9J3 E004 | ||
| + | SEAGATE /dev/sg2 ST16000NM002G ZL21S9WK E004 | ||
| + | SEAGATE /dev/sg20 ST16000NM002G ZL21S9AW E004 | ||
| + | SEAGATE /dev/sg21 ST16000NM002G ZL21TGY1 E004 | ||
| + | SEAGATE /dev/sg22 ST16000NM002G ZL20CRL7 E004 | ||
| + | SEAGATE /dev/sg23 ST16000NM002G ZL21RP9E E004 | ||
| + | SEAGATE /dev/sg24 ST16000NM002G ZL21RNZW E004 | ||
| + | SEAGATE /dev/sg25 ST16000NM002G ZL21JYXF E004 | ||
| + | ATA /dev/sg26 Samsung SSD 870 EVO 1TB S75BNL0W812633P SVT03B6Q | ||
| + | ATA /dev/sg27 Samsung SSD 870 EVO 1TB S75BNS0W642820L SVT03B6Q | ||
| + | SEAGATE /dev/sg3 ST16000NM002G ZL21V48W E004 | ||
| + | SEAGATE /dev/sg4 ST16000NM002G ZL21T7XK E004 | ||
| + | SEAGATE /dev/sg5 ST16000NM002G ZL21T8HS E004 | ||
| + | SEAGATE /dev/sg6 ST16000NM002G ZL21SBMS E004 | ||
| + | SEAGATE /dev/sg7 ST16000NM002G ZL21SRYP E004 | ||
| + | SEAGATE /dev/sg8 ST16000NM002G ZL21LVPQ E004 | ||
| + | SEAGATE /dev/sg9 ST16000NM002G ZL21TB40 E004 | ||
| + | |||
| + | NVMe /dev/nvme0n1 Samsung SSD 990 PRO 4TB S7KGNJ0W912464T 0B2QJXG7 | ||
| + | |||
| + | </pre> | ||
Revision as of 18:15, 10 October 2023
My documents on building a storage server and proxmox host.
Parts
- X10DSC+ Motherboard
- AOM-S3108M-H8L RAID/HBA
- AOC-MTG-I4S quad SFP+ NIC
- 8x32 GB RAM (256g)
- 24x Seagate ST16000NM002G Exos X16 16TB 12Gb/s SAS
- 375 GB Optane SSD for l2arc/ZIL SSDPEL1K375GA01
- 3 4tb m2 SSD for Special Devices
Disk Layout
ZFS zraid2
Zpool 3 - 6 disk raidz2 1 - 5 disk raidz2 Special - 3 m2 flash Zil - 16 GiB - optane l2arc - 333 GiB - optane Note optane is shared between Zil and l2arc
Space on this should be 15 disks of 16 tb, with redundancy or 240tb. This will leave one disk open as a hot spare.
The sata disks will be linux boot disks in a standard linux raid (maybe zfs mirror?)
SAS controller
The built in controller supports HBA mode per supermicro.
https://docs.broadcom.com/doc/pub-005110
Updating Disks
Firmware
Current firmware on the disks is E004, but disks have E002 on them
for i in `seq 2 25` ; do SeaChest_Firmware --downloadFW ./EvansExosX16SAS-STD-512E-E004.LOD -d /dev/sg$i; done #SeaChest_Firmware -s ========================================================================================== SeaChest_Firmware - Seagate drive utilities - NVMe Enabled Copyright (c) 2014-2021 Seagate Technology LLC and/or its Affiliates, All Rights Reserved SeaChest_Firmware Version: 3.0.0-2_2_1 X86_64 Build Date: Apr 27 2021 Today: Tue Oct 10 22:14:38 2023 User: root ========================================================================================== nvme_ioctl_id: Inappropriate ioctl for device Vendor Handle Model Number Serial Number FwRev LSI /dev/sg0 SAS3x28 0705 LSI /dev/sg1 SAS3x28 0705 SEAGATE /dev/sg10 ST16000NM002G ZL20AJ3P E004 SEAGATE /dev/sg11 ST16000NM002G ZL231860 E004 SEAGATE /dev/sg12 ST16000NM002G ZL21T6Q1 E004 SEAGATE /dev/sg13 ST16000NM002G ZL231PV0 E004 SEAGATE /dev/sg14 ST16000NM002G ZL21TDHH E004 SEAGATE /dev/sg15 ST16000NM002G ZL21Y85R E004 SEAGATE /dev/sg16 ST16000NM002G ZL21S9FQ E004 SEAGATE /dev/sg17 ST16000NM002G ZL21T38M E004 SEAGATE /dev/sg18 ST16000NM002G ZL22C3MJ E004 SEAGATE /dev/sg19 ST16000NM002G ZL21S9J3 E004 SEAGATE /dev/sg2 ST16000NM002G ZL21S9WK E004 SEAGATE /dev/sg20 ST16000NM002G ZL21S9AW E004 SEAGATE /dev/sg21 ST16000NM002G ZL21TGY1 E004 SEAGATE /dev/sg22 ST16000NM002G ZL20CRL7 E004 SEAGATE /dev/sg23 ST16000NM002G ZL21RP9E E004 SEAGATE /dev/sg24 ST16000NM002G ZL21RNZW E004 SEAGATE /dev/sg25 ST16000NM002G ZL21JYXF E004 ATA /dev/sg26 Samsung SSD 870 EVO 1TB S75BNL0W812633P SVT03B6Q ATA /dev/sg27 Samsung SSD 870 EVO 1TB S75BNS0W642820L SVT03B6Q SEAGATE /dev/sg3 ST16000NM002G ZL21V48W E004 SEAGATE /dev/sg4 ST16000NM002G ZL21T7XK E004 SEAGATE /dev/sg5 ST16000NM002G ZL21T8HS E004 SEAGATE /dev/sg6 ST16000NM002G ZL21SBMS E004 SEAGATE /dev/sg7 ST16000NM002G ZL21SRYP E004 SEAGATE /dev/sg8 ST16000NM002G ZL21LVPQ E004 SEAGATE /dev/sg9 ST16000NM002G ZL21TB40 E004 NVMe /dev/nvme0n1 Samsung SSD 990 PRO 4TB S7KGNJ0W912464T 0B2QJXG7
