EXADATA: How to enable Flash Cache WriteBack on a running system

In a recent tuning activity it was necessary to change the Exadata Smart Flash Cache from “WriteThrough” to “WriteBack“. Because the system was used in a 24/7 environment we had to implement the change in a Rolling Upgrade Fashion.

Here below are described the different steps.

 

From one DB node using dcli check the currest status of the storage cells:

[root@efudbadm02 ~]# dcli -g ~/cell_group -l root cellcli -e "list cell attributes flashcachemode"
efuceladm01: WriteThrough
efuceladm02: WriteThrough
efuceladm03: WriteThrough
efuceladm04: WriteThrough
efuceladm05: WriteThrough
efuceladm06: WriteThrough
efuceladm07: WriteThrough
efuceladm08: WriteThrough
efuceladm09: WriteThrough
efuceladm10: WriteThrough
efuceladm11: WriteThrough

From one DB node using dcli check that the properties asmdeactivationoutcome and asmmodestatus of all griddisks are respectively “Yes” and “ONLINE” before continuing with the change.

[root@efudbadm02 ~]# dcli -g cell_group -l root cellcli -e list griddisk attributes asmdeactivationoutcome, asmmodestatus
efuceladm01: Yes ONLINE
efuceladm01: Yes ONLINE
efuceladm01: Yes ONLINE
efuceladm01: Yes ONLINE
efuceladm01: Yes ONLINE
efuceladm01: Yes ONLINE
efuceladm01: Yes ONLINE
efuceladm01: Yes ONLINE
efuceladm01: Yes ONLINE
efuceladm01: Yes ONLINE
efuceladm01: Yes ONLINE
efuceladm01: Yes ONLINE
efuceladm01: Yes ONLINE
efuceladm01: Yes ONLINE
efuceladm01: Yes ONLINE
efuceladm01: Yes ONLINE
efuceladm02: Yes ONLINE
efuceladm02: Yes ONLINE
efuceladm02: Yes ONLINE
efuceladm02: Yes ONLINE
efuceladm02: Yes ONLINE
efuceladm02: Yes ONLINE
efuceladm02: Yes ONLINE
efuceladm02: Yes ONLINE
efuceladm02: Yes ONLINE
efuceladm02: Yes ONLINE
efuceladm02: Yes ONLINE
efuceladm02: Yes ONLINE
efuceladm02: Yes ONLINE
efuceladm02: Yes ONLINE
efuceladm02: Yes ONLINE
efuceladm02: Yes ONLINE
...
..
.

From one DB node using dcli check that all flashcache modules are in “normal” state and no flash disk is in degraded or critical state.

[root@efudbadm02 ~]# dcli -g cell_group -l root cellcli -e list flashcache detail
efuceladm01: name: efuceladm01_FLASHCACHE
efuceladm01: cellDisk: FD_00_efuceladm01,FD_07_efuceladm01,FD_06_efuceladm01,FD_03_efuceladm01,FD_05_efuceladm01,FD_01_efuceladm01,FD_02_efuceladm01,FD_04_efuceladm01
efuceladm01: creationTime: 2013-06-18T15:21:13+02:00
efuceladm01: degradedCelldisks:
efuceladm01: effectiveCacheSize: 744.125G
efuceladm01: id: 35b61001-438f-4d66-8ce9-40704f758d3f
efuceladm01: size: 744.125G
efuceladm01: status: normal
efuceladm02: name: efuceladm02_FLASHCACHE
efuceladm02: cellDisk: FD_06_efuceladm02,FD_05_efuceladm02,FD_00_efuceladm02,FD_02_efuceladm02,FD_01_efuceladm02,FD_07_efuceladm02,FD_03_efuceladm02,FD_04_efuceladm02
efuceladm02: creationTime: 2013-06-18T15:21:12+02:00
efuceladm02: degradedCelldisks:
efuceladm02: effectiveCacheSize: 744.125G
efuceladm02: id: 2f7eedd6-cda2-496e-98ec-417b94fb8ee7
efuceladm02: size: 744.125G
efuceladm02: status: normal
efuceladm03: name: efuceladm03_FLASHCACHE
efuceladm03: cellDisk: FD_00_efuceladm03,FD_04_efuceladm03,FD_01_efuceladm03,FD_02_efuceladm03,FD_03_efuceladm03,FD_06_efuceladm03,FD_05_efuceladm03,FD_07_efuceladm03
efuceladm03: creationTime: 2013-06-18T15:21:10+02:00
efuceladm03: degradedCelldisks:
efuceladm03: effectiveCacheSize: 744.125G
efuceladm03: id: c271cdb8-dc70-4009-ba97-dfc4c26b00ef
efuceladm03: size: 744.125G
efuceladm03: status: normal
...
..
.

Logon on the first Cell Storage and using CellCli interface perform the following procedure to enable the WriteBack Flash Cache in a rolling upgrade fashion.

 

Drop the existing flash cache

CellCLI> drop flashcache
Flash cache efuceladm01_FLASHCACHE successfully dropped

Inactivate the griddisk on the cell

CellCLI> alter griddisk all inactive
GridDisk DATA_CD_00_efuceladm01 successfully altered
GridDisk DATA_CD_01_efuceladm01 successfully altered
GridDisk DATA_CD_02_efuceladm01 successfully altered
GridDisk DATA_CD_03_efuceladm01 successfully altered
GridDisk DATA_CD_04_efuceladm01 successfully altered
GridDisk DATA_CD_05_efuceladm01 successfully altered
GridDisk DBFS_DG_CD_02_efuceladm01 successfully altered
GridDisk DBFS_DG_CD_03_efuceladm01 successfully altered
GridDisk DBFS_DG_CD_04_efuceladm01 successfully altered
GridDisk DBFS_DG_CD_05_efuceladm01 successfully altered
GridDisk RECO_CD_00_efuceladm01 successfully altered
GridDisk RECO_CD_01_efuceladm01 successfully altered
GridDisk RECO_CD_02_efuceladm01 successfully altered
GridDisk RECO_CD_03_efuceladm01 successfully altered
GridDisk RECO_CD_04_efuceladm01 successfully altered
GridDisk RECO_CD_05_efuceladm01 successfully altered

Shut down cellsrv service

CellCLI> alter cell shutdown services cellsrv

Stopping CELLSRV services...
The SHUTDOWN of CELLSRV services was successful.

Enable the Smart Flash Cache WriteBack

CellCLI> alter cell flashCacheMode=writeback
Cell efuceladm01 successfully altered

Restart the cellsrv service

CellCLI> alter cell startup services cellsrv

Starting CELLSRV services...
The STARTUP of CELLSRV services was successful.

Reactivate the griddisk on the cell

CellCLI> alter griddisk all active
GridDisk DATA_CD_00_efuceladm01 successfully altered
GridDisk DATA_CD_01_efuceladm01 successfully altered
GridDisk DATA_CD_02_efuceladm01 successfully altered
GridDisk DATA_CD_03_efuceladm01 successfully altered
GridDisk DATA_CD_04_efuceladm01 successfully altered
GridDisk DATA_CD_05_efuceladm01 successfully altered
GridDisk DBFS_DG_CD_02_efuceladm01 successfully altered
GridDisk DBFS_DG_CD_03_efuceladm01 successfully altered
GridDisk DBFS_DG_CD_04_efuceladm01 successfully altered
GridDisk DBFS_DG_CD_05_efuceladm01 successfully altered
GridDisk RECO_CD_00_efuceladm01 successfully altered
GridDisk RECO_CD_01_efuceladm01 successfully altered
GridDisk RECO_CD_02_efuceladm01 successfully altered
GridDisk RECO_CD_03_efuceladm01 successfully altered
GridDisk RECO_CD_04_efuceladm01 successfully altered
GridDisk RECO_CD_05_efuceladm01 successfully altered

Recreate the flash cache

CellCLI> create flashcache all
Flash cache efuceladm01_FLASHCACHE successfully created

 


Verify that the Smart Flash Cache WriteBackWriteBack option is enabled

[root@efuceladm01 ~]# cellcli -e list cell detail | grep flashCacheMode
 flashCacheMode: writeback

Before applying the change to the next Exadata Storage Server  wait that all griddisk are synronized and online.

[root@efuceladm01 ~]# cellcli -e list griddisk attributes name,asmmodestatus,asmdeactivationoutcome
 DATA_CD_00_efuceladm01 SYNCING Yes
 DATA_CD_01_efuceladm01 SYNCING Yes
 DATA_CD_02_efuceladm01 SYNCING Yes
 DATA_CD_03_efuceladm01 SYNCING Yes
 DATA_CD_04_efuceladm01 SYNCING Yes
 DATA_CD_05_efuceladm01 SYNCING Yes
 DBFS_DG_CD_02_efuceladm01 ONLINE Yes
 DBFS_DG_CD_03_efuceladm01 ONLINE Yes
 DBFS_DG_CD_04_efuceladm01 ONLINE Yes
 DBFS_DG_CD_05_efuceladm01 ONLINE Yes
 RECO_CD_00_efuceladm01 OFFLINE Yes
 RECO_CD_01_efuceladm01 OFFLINE Yes
 RECO_CD_02_efuceladm01 OFFLINE Yes
 RECO_CD_03_efuceladm01 OFFLINE Yes
 RECO_CD_04_efuceladm01 OFFLINE Yes
 RECO_CD_05_efuceladm01 OFFLINE Yes

Once the asmmodestatus is ONLINE on all griddisks it is safe to move to the next Storage Server.


 

At the end of the procedure all Storage Servers are configured with Smart Flash Cache WriteBach option:

[root@efudbadm02 ~]# dcli -g ~/cell_group -l root cellcli -e "list cell attributes flashcachemode"
efuceladm01: writeback
efuceladm02: writeback
efuceladm03: writeback
efuceladm04: writeback
efuceladm05: writeback
efuceladm06: writeback
efuceladm07: writeback
efuceladm08: writeback
efuceladm09: writeback
efuceladm10: writeback
efuceladm11: writeback


		
Advertisements