ZFS - Revision history

Bryan at 19:11, 8 September 2024

2024-09-08T19:11:12Z

Bryan: /* Optimization */

2023-06-05T17:55:05Z

Optimization

Bryan: /* Disk Notes */

2023-06-05T16:09:46Z

Disk Notes

Show changes

Bryan: /* Disk Notes */

2023-06-04T16:06:17Z

Disk Notes

Show changes

Bryan: /* Disk Notes */

2023-06-04T16:01:50Z

Disk Notes

Bryan at 13:49, 3 June 2023

2023-06-03T13:49:37Z

Bryan: /* Disk Notes */

2023-06-01T23:35:22Z

Disk Notes

Bryan: /* Disk Notes */

2023-06-01T21:33:18Z

Disk Notes

Bryan at 19:26, 1 June 2023

2023-06-01T19:26:36Z

Bryan at 00:14, 9 May 2023

2023-05-09T00:14:22Z

← Older revision		Revision as of 19:11, 8 September 2024
Line 1:		Line 1:
		+	[[Category:Keekles Infrastructure]]
		+
	Notes on ZFS		Notes on ZFS

@@ Line 72: / Line 72: @@
   zdb -Lbbb  PoolName
-  zpool create -f -o ashift=12 -O casesensitivity=insensitive -O normalization=formD -O compression=lz4 -O atime=off -O recordsize=256k ZfsMediaPool \
+  zpool create -f -o ashift=13 -O normalization=formD \
-  raidz2 /var/run/disk/by-path/PCI0@0-SAT0@17-PRT5@5-PMP@0-@0:0 /var/run/disk/by-path/PCI0@0-SAT0@17-PRT4@4-PMP@0-@0:0 \
+ -O compression=lz4 -O atime=off -O recordsize=256k -O special_small_blocks=128k BigPool \
- /var/run/disk/by-path/PCI0@0-RP21@1B,4-PXSX@0-PRT31@1f-PMP@0-@0:0 /var/run/disk/by-path/PCI0@0-SAT0@17-PRT3@3-PMP@0-@0:0 \
+  raidz2 /dev/sdk /dev/sdu /dev/sdo /dev/sdq /dev/sds /dev/sdb /dev/sdc /dev/sdd \
-  /var/run/disk/by-path/PCI0@0-SAT0@17-PRT2@2-PMP@0-@0:0 /var/run/disk/by-path/PCI0@0-SAT0@17-PRT1@1-PMP@0-@0:0 \
+  raidz2 /dev/sdl /dev/sdm /dev/sdn /dev/sde /dev/sdp /dev/sdf /dev/sdg /dev/sdi \
- /var/run/disk/by-path/PCI0@0-SAT0@17-PRT0@0-PMP@0-@0:0 /var/run/disk/by-path/PCI0@0-RP21@1B,4-PXSX@0-PRT2@2-PMP@0-@0:0 \
+  special mirror /dev/sdt /dev/sdw /dev/sdv \
-  /var/run/disk/by-path/PCI0@0-RP21@1B,4-PXSX@0-PRT3@3-PMP@0-@0:0 /var/run/disk/by-path/PCI0@0-RP21@1B,4-PXSX@0-PRT28@1c-PMP@0-@0:0 \
+  spare /dev/sdh /dev/sdx /dev/sdr /dev/sdj
   zpool add ZfsMediaPool log /dev/disk5s3

@@ Line 98: / Line 98: @@
 This is strange the size is different.  I ran the info command on these
-SeaChest_Basics_x86_64-redhat-linux  -d /dev/sg11 -i
+ SeaChest_Basics_x86_64-redhat-linux  -d /dev/sg11 -i
-SeaChest_Basics_x86_64-redhat-linux  -d /dev/sg15 -i
+ SeaChest_Basics_x86_64-redhat-linux  -d /dev/sg15 -i
-−
 This shows the following different options
   Drive Capacity (TB/TiB): 12.00/10.91
@@ Line 219: / Line 219: @@
   Progress will be updated every  5 minutes
           Percent Complete: 0.00%

← Older revision		Revision as of 13:49, 3 June 2023
Line 33:		Line 33:
	</pre>		</pre>

		+	= Things to do =
		+
		+	# Set Write Cache
		+	# set MRIE = 4
		+	# ll format
		+	# record address and hours.
		+
		+	One liner to do this
		+	for i in `seq 2 25` ; do SG="/dev/sg$i" ; echo $SG ;sdparm --set=WCE --save $SG ; sdparm --get=WCE $SG; sdparm --set=MRIE=4 --save $SG; sdparm --get=MRIE $SG; done
	==Optimization==		==Optimization==

← Older revision		Revision as of 23:35, 1 June 2023
Line 130:		Line 130:

	Essentially this is a hold over from the 520 byte sectors and 8 bytes being used for a checksum/parity. This isn't needed in ZFS. It wastes 79.872 GiBytes of space from the disk too!		Essentially this is a hold over from the 520 byte sectors and 8 bytes being used for a checksum/parity. This isn't needed in ZFS. It wastes 79.872 GiBytes of space from the disk too!
		+
		+	Note: I'm not sure this is due to the waste of space on this disk. It could be something the OEM does different for them as they were original 520b disks.
		+
		+	I tried
		+	# SeaChest_Basics_x86_64-redhat-linux -d /dev/sg11 --restoreMaxLBA
		+	And it didn't increase the LBA.
		+
		+	So lets look at the supported formats for this:
		+
		+	# ./SeaChest_Format_x86_64-redhat-linux -d /dev/sg11 --showSupportedFormats
		+	==========================================================================================
		+	SeaChest_Format - Seagate drive utilities - NVMe Enabled
		+	Copyright (c) 2014-2021 Seagate Technology LLC and/or its Affiliates, All Rights Reserved
		+	SeaChest_Format Version: 2.3.1-2_2_3 X86_64
		+	Build Date: Jun 17 2021
		+	Today: Thu Jun 1 23:15:34 2023 User: root
		+	==========================================================================================
		+
		+	/dev/sg12 - ST12000NM0027 - ZJV1HW2T0000C8496S1T - SCSI
		+
		+	Supported Logical Block Sizes and Protection Types:
		+	---------------------------------------------------
		+	* - current device format
		+	PI Key:
		+	Y - protection type supported at specified block size
		+	N - protection type not supported at specified block size
		+	? - unable to determine support for protection type at specified block size
		+	Relative performance key:
		+	N/A - relative performance not available.
		+	Best
		+	Better
		+	Good
		+	Degraded
		+	--------------------------------------------------------------------------------
		+	Logical Block Size PI-0 PI-1 PI-2 PI-3 Relative Performance Metadata Size
		+	--------------------------------------------------------------------------------
		+	512 Y ? ? N N/A N/A
		+	520 Y ? ? N N/A N/A
		+	528 Y ? ? N N/A N/A
		+	* 4096 Y ? ? N N/A N/A
		+	4112 Y ? ? N N/A N/A
		+	4160 Y ? ? N N/A N/A
		+	--------------------------------------------------------------------------------
		+	NOTE: Device is not capable of showing all sizes it supports. Only common
		+	sizes are listed. Please consult the product manual for all supported
		+	combinations.
		+	NOTE: This device supports protection information (PI) (a.k.a. End to End protection).
		+	Type 0 - No protection beyond transport protocol
		+	Type 1 - Logical Block Guard and Logical Block Reference Tag
		+	Type 2 - Logical Block Guard and Logical Block Reference Tag (except first block)
		+	32byte read/write CDBs allowed
		+	Not all forms of PI are supported on all sector sizes unless otherwise indicated
		+	in the device product manual.
		+	NOTE: This device supports Fast Format. Fast format is not instantaneous and is used for
		+	switching between 5xx and 4xxx sector sizes. A fast format may take a few minutes or longer
		+	but may take longer depending on the size of the drive. Fast format support does not necessarily
		+	mean switching sector sizes AND changing PI at the same time is supported. In most cases, a
		+	switch of PI type will require a full device format.
		+	Fast format mode 1 is typically used to switch from 512 to 4096 block sizes with the current
		+	PI scheme.
		+
		+	Well that was a bust. But from reading the last thing, it looks like we'll have to try a format of the disk (again)
		+
		+	# ./SeaChest_Format_x86_64-redhat-linux --protectionType 0 --formatUnit 4096 --confirm this-will-erase-data --poll -d /dev/sg11
		+	==========================================================================================
		+	SeaChest_Format - Seagate drive utilities - NVMe Enabled
		+	Copyright (c) 2014-2021 Seagate Technology LLC and/or its Affiliates, All Rights Reserved
		+	SeaChest_Format Version: 2.3.1-2_2_3 X86_64
		+	Build Date: Jun 17 2021
		+	Today: Thu Jun 1 23:34:40 2023 User: root
		+	==========================================================================================
		+
		+	/dev/sg11 - ST12000NM0027 - ZJV2GTFT0000C9069US9 - SCSI
		+	Format Unit
		+	Performing SCSI drive format.
		+	Depending on the format request, this could take minutes to hours or days.
		+	Do not remove power or attempt other access as interrupting it may make
		+	the drive unusable or require performing this command again!!
		+	Progress will be updated every 5 minutes
		+	Percent Complete: 0.00%
		+

	== Informational Exceptions ==		== Informational Exceptions ==

← Older revision		Revision as of 21:33, 1 June 2023
Line 130:		Line 130:

	Essentially this is a hold over from the 520 byte sectors and 8 bytes being used for a checksum/parity. This isn't needed in ZFS. It wastes 79.872 GiBytes of space from the disk too!		Essentially this is a hold over from the 520 byte sectors and 8 bytes being used for a checksum/parity. This isn't needed in ZFS. It wastes 79.872 GiBytes of space from the disk too!
		+
		+	== Informational Exceptions ==
		+
		+	'''MRIE (Method Of Reporting Informational Exceptions) field'''
		+
		+	This defines how the disk handles errors on the SAS level.
		+
		+	[https://www.seagate.com/files/staticfiles/support/docs/manual/Interface%20manuals/100293068j.pdf Page 417, table 391, of this document] shows the following explanations:
		+
		+	{\| class="wikitable"
		+	\|-
		+	! MRIE<br />
		+	! style="font-weight:bold;" \| Description
		+	\|-
		+	\| 0
		+	\| No reporting of informational exception condition: The device server shall not report information exception conditions.
		+	\|-
		+	\| 1
		+	\| Asynchronous event reporting: Obsolete
		+	\|-
		+	\| 2
		+	\| Generate unit attention: <br />The device server shall report informational exception conditions by establishing a unit attention condition (see SAM-5) for the initiator port associated with every I_T nexus, with the additional sense code set to indicate the cause of the informational exception condition.<br /><br />As defined in SAM-5, the command that has the CHECK CONDITION status with the sense key set to UNIT ATTENTION is not processed before the informational exception condition is reported
		+	\|-
		+	\| 3
		+	\| Conditionally generate recovered error: The device server shall report informational exception conditions, if the reporting of recovered errors is allowed, by returning a CHECK CONDITION status. If the TEST bit is set to zero, the status may be returned after the informational exception condition occurs on any command for which GOOD status or INTERMEDIATE status would have been returned. If the TEST bit is set to one, the status shall be returned on the next command received on any I_T nexus that is normally capable of returning an informational exception condition when the test bit is set to zero. The sense key shall be set to RECOVERED ERROR and the additional sense code shall indicate the cause of the informational exception condition.<br /><br />The command that returns the CHECK CONDITION for the informational exception shall complete without error before any informational exception condition may be reported.
		+	\|-
		+	\| 4
		+	\| Unconditionally generate recovered error: The device server shall report informational exception conditions, regardless of whether the reporting of recovered errors is allowed, by returning a CHECK CONDITION status. If the TEST bit is set to zero, thestatus may be returned after the informational exception condition occurs on any command for which GOOD status or INTERMEDIATE status would have been returned. If the TEST bit is set to one, the status shall be returned on the next command received on any I_T nexus that is normally capable of returning an informational exception condition when the TEST bit is set to zero. The sense key shall be set to RECOVERED ERROR and the additional sense code shall indicate the cause of the informational exception condition.<br /><br />The command that returns the CHECK CONDITION for the informational exception shall complete without error before any informational exception condition may be reported.
		+	\|-
		+	\| 5
		+	\| Generate no sense: The device server shall report informational exception conditions by returning a CHECK CONDITION status. If the TEST bit is set to zero, the status may be returned after the informational exception condition occurs on any command for which GOOD status or INTERMEDIATE status would have been returned. If the TEST bit is set to one, the status shall be returned on the next command received on any I_T nexus that is normally capable of returning an informational exception condition when the TEST bit is set to zero. The sense key shall be set to NO SENSE and the additional sense code shall indicate the cause of the informational exception condition.<br /><br />The command that returns the CHECK CONDITION for the informational exception shall complete without error before any informational exception condition may be reported.
		+	\|-
		+	\| 6
		+	\| Only report informational exception condition on request: The device server shall preserve the informational exception(s) information. To find out about information exception conditions the application client polls the device server by issuing a REQUEST SENSE command. In the REQUEST SENSE parameter data that contains the sense data, the sense key shall be set to NO SENSE and the additional sense code shall indicate the cause of the informational exception condition.
		+	\|-
		+	\| 7-B
		+	\| Reserved
		+	\|-
		+	\| C-F
		+	\| Vendor specific
		+	\|-
		+	\|
		+	\|
		+	\|}
		+
		+	I can find no suggested setting of this for Linux or ZFS use. From my reading I will choose to set this to 4.
		+
		+	There a two ways to set this.
		+
		+	=== SeaChest_SMART ===
		+
		+	This is the command to set it via SeaChest
		+
		+	# ./SeaChest_SMART_x86_64-redhat-linux -d /dev/sg11 --setMRIE 4
		+	==========================================================================================
		+	SeaChest_SMART - Seagate drive utilities - NVMe Enabled
		+	Copyright (c) 2014-2021 Seagate Technology LLC and/or its Affiliates, All Rights Reserved
		+	SeaChest_SMART Version: 2.0.1-2_2_3 X86_64
		+	Build Date: Jun 17 2021
		+	Today: Thu Jun 1 21:29:34 2023 User: root
		+	==========================================================================================
		+
		+	/dev/sg11 - ST12000NM0027 - ZJV2GTFT0000C9069US9 - SCSI
		+	Successfully set MRIE mode to 4
		+
		+	# ./SeaChest_SMART_x86_64-redhat-linux -d /dev/sg11 -i
		+	==========================================================================================
		+	SeaChest_SMART - Seagate drive utilities - NVMe Enabled
		+	Copyright (c) 2014-2021 Seagate Technology LLC and/or its Affiliates, All Rights Reserved
		+	SeaChest_SMART Version: 2.0.1-2_2_3 X86_64
		+	Build Date: Jun 17 2021
		+	Today: Thu Jun 1 21:29:37 2023 User: root
		+	==========================================================================================
		+
		+	/dev/sg11 - ST12000NM0027 - ZJV2GTFT0000C9069US9 - SCSI
		+	Vendor ID: SEAGATE
		+	Model Number: ST12000NM0027
		+	Serial Number: ZJV2GTFT
		+	PCBA Serial Number: 0000C9069US9
		+	Firmware Revision: E004
		+	World Wide Name: 5000C500A6F0CC8B
		+	Copyright: Copyright (c) 2020 Seagate All rights reserved
		+	Drive Capacity (TB/TiB): 11.92/10.84
		+	Temperature Data:
		+	Current Temperature (C): 36
		+	Highest Temperature (C): Not Reported
		+	Lowest Temperature (C): Not Reported
		+	Power On Time: 2 years 100 days 15 hours 26 minutes
		+	Power On Hours: 19935.43
		+	MaxLBA: 2909274111
		+	Native MaxLBA: Not Reported
		+	Logical Sector Size (B): 4096
		+	Physical Sector Size (B): 4096
		+	Sector Alignment: 0
		+	Rotation Rate (RPM): 7200
		+	Form Factor: 3.5"
		+	Last DST information:
		+	DST has never been run
		+	Long Drive Self Test Time: 19 hours 1 minute
		+	Interface speed:
		+	Port 0 (Current Port)
		+	Max Speed (GB/s): 12.0
		+	Negotiated Speed (Gb/s): 12.0
		+	Port 1
		+	Max Speed (GB/s): 12.0
		+	Negotiated Speed (Gb/s): Not Reported
		+	Annualized Workload Rate (TB/yr): 12.36
		+	Total Bytes Read (TB): 23.82
		+	Total Bytes Written (TB): 4.30
		+	Encryption Support: Not Supported
		+	Cache Size (MiB): Not Reported
		+	Read Look-Ahead: Enabled
		+	Non-Volatile Cache: Enabled
		+	Write Cache: Enabled
		+	SMART Status: Good
		+	ATA Security Information: Not Supported
		+	Firmware Download Support: Full, Segmented, Deferred
		+	Number of Logical Units: 1
		+	Specifications Supported:
		+	SPC-4
		+	SAM-5
		+	SAS-3
		+	SPL-3
		+	SPC-4
		+	SBC-3
		+	Features Supported:
		+	Protection Type 1
		+	Protection Type 2 [Enabled]
		+	Application Client Logging
		+	Self Test
		+	Automatic Write Reassignment [Enabled]
		+	Automatic Read Reassignment [Enabled]
		+	EPC [Enabled]
		+	'''Informational Exceptions [Mode 4]'''
		+	Translate Address
		+	Rebuild Assist
		+	Seagate Remanufacture
		+	Seagate In Drive Diagnostics (IDD)
		+	Format Unit
		+	Fast Format
		+	Sanitize
		+	Adapter Information:
		+	Vendor ID: 117Ch
		+	Product ID: 8072h
		+	Revision: 0006h
		+
		+	=== sg utils ===
		+
		+	# sdparm /dev/sg11 --set=MRIE=4
		+	/dev/sg11: SEAGATE ST12000NM0027 E004
		+
		+	# sdparm /dev/sg11 --get=MRIE
		+	/dev/sg11: SEAGATE ST12000NM0027 E004
		+	MRIE 4 [cha: y, def: 0, sav: 4]

@@ Line 34: / Line 34: @@
 ==Optimization==
 All disks should be updated
@@ Line 76: / Line 77: @@
 https://github.com/openzfs/zfs/discussions/12769

← Older revision		Revision as of 00:14, 9 May 2023
Line 8:		Line 8:

	This should give about 174.56 TiB of space.		This should give about 174.56 TiB of space.
		+
		+
		+	<pre>
		+	Block Size Histogram
		+
		+	block psize lsize asize
		+	size Count Size Cum. Count Size Cum. Count Size Cum.
		+	512: 350K 175M 175M 350K 175M 175M 0 0 0
		+	1K: 348K 413M 589M 348K 413M 589M 0 0 0
		+	2K: 273K 722M 1.28G 273K 722M 1.28G 0 0 0
		+	4K: 669K 2.65G 3.93G 221K 1.17G 2.45G 0 0 0
		+	8K: 925K 8.50G 12.4G 176K 1.91G 4.36G 1.23M 14.7G 14.7G
		+	16K: 620M 9.69T 9.70T 621M 9.70T 9.70T 621M 14.6T 14.6T
		+	32K: 1.39M 62.8G 9.76T 82.2K 3.57G 9.70T 410K 19.0G 14.6T
		+	64K: 548K 47.3G 9.81T 47.2K 4.06G 9.71T 1.58M 153G 14.7T
		+	128K: 825K 150G 9.95T 1014K 128G 9.83T 699K 133G 14.9T
		+	256K: 66.3M 16.6T 26.5T 68.4M 17.1T 26.9T 66.6M 20.3T 35.1T
		+	512K: 0 0 26.5T 0 0 26.9T 0 0 35.1T
		+	1M: 0 0 26.5T 0 0 26.9T 0 0 35.1T
		+	2M: 0 0 26.5T 0 0 26.9T 0 0 35.1T
		+	4M: 0 0 26.5T 0 0 26.9T 0 0 35.1T
		+	8M: 0 0 26.5T 0 0 26.9T 0 0 35.1T
		+	16M: 0 0 26.5T 0 0 26.9T 0 0 35.1T
		+	</pre>

	==Optimization==		==Optimization==