The \u201cnormal\u201d Exadata storage cell has 12 disks, the Extreme Flash version uses 8 disks per storage. If you have doubt about how many disks you have per storage cell, you can connect in each one and check the number of celldisks you have. And before continuing, be aware of Exadata disk division:<\/p>\n
To do this change we execute three major steps: ASM, Exadata Storage, and ASM again.<\/p>\n <\/p>\n Inside ASM we can use this query to collect some information about the diskgroups:<\/p>\n With that, I have three important information: number the disks (60)<\/strong>, redundancy type (NORMAL)<\/strong>, total actual size (RAW value – 15790080)<\/strong>. To discover the size for each disk in ASM (here I do manually and not check in v$asm_disk just to show you the steps and to be more didact) you can divide the raw space\/#disks:<\/p>\n So, each disk has 257GB of space size (in raw). Since the actual free space is 1100GB (1.07TB) and we want to add more 2TB we need to increase the value for each disk.<\/p>\n The formula is simple: NewValue(inGB)*#OfDisksPerCell*#NumberofCells<\/strong>. Here I choose 330GB per disk, so, the new size for diskgroup will be:<\/p>\n But this value is not correct because does not consider the mirror type, so, we need to divide this value. If it NORMAL, divide by 2, if HIGH, divide by 3. To compare, the old and new expected value:<\/p>\n So, the new total space for diskgroup will be around 9.6 TB (9900 GB). And we will add (as free space) around 2.1 TB. Probably you need to execute these formulas more than one time to find the desired size per disk.<\/p>\n I start to calculate by disk (and after discovering the final diskgroup size) instead of starting with diskgroup size (and dividing to discover the disk size) because doing this way, the size for disk will be always correct and align with storage cell grid disk. Remember that grid disks are aligned in 16MB and, if you start to choose one arbitrary value to the max size for the ASM diskgroup, you can reach a value per grid disk that is not 16MB aligned. As an example, if I start choosing 20TB for diskgroup, the size per disk will be (20*1024)\/60 = 341.33GB and this is not aligned with 16MB.<\/p>\n For 16 Mb explanation, you can check in the Exadata docs<\/a>: <\/p>\n Find the closest 16 MB boundary for the new grid disk size. If you do not perform this check, then the cell will round down the grid disk size to the nearest 16 MB boundary automatically, and you could end up with a mismatch in size between the Oracle ASM disks and the grid disks.<\/em><\/p>\n In the Exadata side, first, check some info about the actual state for grid disks. Here I connect in one cell (if you want you can use dcli to call every\/all cells) and check some info for the grid disk:<\/p>\n Here you can see that I checked:<\/p>\n This part was just to check and show you how to verify some info, with time, you don\u2019t need to check this in every maintenance (because you will be familiar with the environment). Be careful that, if you have different grid disk space division per storage cells, you need to check if you have available space in all your storage celldisks.<\/p>\n To expand the grid disks you have two options, enter in each cell and expand manually one by one, or create one script and call by dcli (the option that I choose). So, create one script that executes the ALTER GRIDDISK<\/strong> command for the new desired size. Just remember to be careful and choose the correct grid disks (here is for VM 03, that means DATAC3):<\/p>\n Above I cropped the output to reduce the size or post, but you can check the raw output here<\/a>. After the change you can check the info for the grid disk:<\/p>\n After you change the grid disks in the storage side, you can go back to ASM and extend the diskgroup:<\/p>\n And you can check that the size was already added (look that values hit what we calculated before):<\/p>\n And you can check the v$asm_operation to check the rebalance progress:<\/p>\n As you can see, the steps to do that are simple and not complex, you just need to take care about some details of your environment: Number of the disks per cell, number the cells and the VM where you want to add the space is critical to do the correct change.<\/strong> Remember to align with 16MB the size of your grid disk, when you are adding it is not a big deal, but if you want to shrink this can break your ASM diskgroup.<\/p>\n Check that the only change effectively is the size of the grid disk, all the others occur automatically because of the grid disk. ASM diskgroup will increase to the max value that it is available and space is available just after the command.<\/p>\n The steps above are more detailed that you will do in daily maintenance, but help you to understand most of the datils for this kind of change.<\/p>\n <\/p>\n Reference:<\/strong><\/p>\n How to Resize Grid Disks in Exadata (Doc ID 2176737.1) – https:\/\/support.oracle.com\/epmos\/faces\/DocContentDisplay?id=2176737.1<\/a><\/p>\n![\"\"](\"http:\/\/www.fernandosimon.com\/blog\/wp-content\/uploads\/2014\/11\/Divisao-Discos-Exa.png\")
<\/a><\/p>\nFor ASM<\/h2>\n
SQL> col name format a12 head 'Disk Group';\r\nSQL> col total_mb format 999999999 head 'Total GB|Raw';\r\nSQL> col free_mb format 999999999 head 'Free GB|Raw';\r\nSQL> col avail_mb format 999999999 head 'Total GB|Usable';\r\nSQL> col usable_mb format 999999999 head 'Free GB|Usable';\r\nSQL> col usable_mb format 999999999 head 'Free GB|Usable';\r\nSQL> col cdisks format 99999 head 'Cell|Disksl';\r\nSQL>\r\nSQL> select a.name,a.total_mb,a.free_mb,a.type,\r\n 2 decode(a.type,'NORMAL',a.total_mb\/2\/1024,'HIGH',a.total_mb\/3\/1024) avail_mb,\r\n 3 decode(a.type,'NORMAL',a.free_mb\/2\/1024,'HIGH',a.free_mb\/3\/1024) usable_mb,\r\n 4 count(b.path) cdisks\r\n 5 from v$asm_diskgroup a, v$asm_disk b\r\n 6 where a.group_number=b.group_number\r\n 7 group by a.name,a.total_mb,a.free_mb,a.type,\r\n 8 decode(a.type,'NORMAL',a.total_mb\/2\/1024,'HIGH',a.total_mb\/3\/1024) ,\r\n 9 decode(a.type,'NORMAL',a.free_mb\/2\/1024,'HIGH',a.free_mb\/3\/1024)\r\n 10 order by 2,1\r\n 11 \/\r\n\r\n Total GB Free GB Total GB Free GB Cell\r\nDisk Group Raw Raw TYPE Usable Usable Disksl\r\n------------ ---------- ---------- ------ ---------- ---------- ------\r\nRECOC3 4239360 2465540 NORMAL 2070 1204 60\r\nDATAC3 15790080 2253048 NORMAL 7710 1100 60\r\n\r\nSQL>\r\nSQL> select dg.name, d.failgroup, d.state, d.header_status, d.mount_status, d.mode_status, count(1) num_disks\r\n 2 from v$asm_disk d, v$asm_diskgroup dg\r\n 3 where d.group_number = dg.group_number\r\n 4 and dg.name IN ('DATAC3')\r\n 5 group by dg.name, d.failgroup, d.state, d.header_status, d.mount_status, d.mode_status\r\n 6 order by 1,2,3;\r\n\r\nNAME FAILGROUP STATE HEADER_STATU MOUNT_S MODE_ST NUM_DISKS\r\n------ ------------------------------ -------- ------------ ------- ------- ----------\r\nDATAC3 EXACELADM01 NORMAL MEMBER CACHED ONLINE 12\r\nDATAC3 EXACELADM02 NORMAL MEMBER CACHED ONLINE 12\r\nDATAC3 EXACELADM03 NORMAL MEMBER CACHED ONLINE 12\r\nDATAC3 EXACELADM04 NORMAL MEMBER CACHED ONLINE 12\r\nDATAC3 EXACELADM05 NORMAL MEMBER CACHED ONLINE 12\r\n\r\nSQL><\/pre>\nSQL> SELECT (15790080\/1024)\/60 as gbDISK FROM dual;\r\n\r\n GBDISK\r\n----------\r\n 257\r\n\r\nSQL>\r\n<\/pre>\n
SQL> SELECT (330*12*5) AS newsizeGB FROM dual;\r\n\r\n NEWSIZEGB\r\n----------\r\n 19800\r\n\r\nSQL><\/pre>\n
SQL> SELECT (257*12*5)\/2 as actualsizeGBUsable, (330*12*5)\/2 AS newsizeGBUSable FROM dual;\r\n\r\nACTUALSIZEGBUSABLE NEWSIZEGBUSABLE\r\n------------------ ---------------\r\n 7710 9900\r\n\r\nSQL><\/pre>\n
For Storage Cell<\/h2>\n
[root@exaceladm01 ~]# cellcli\r\nCellCLI: Release 18.1.6.0.0 - Production on Fri Jun 21 16:57:49 CEST 2019\r\n\r\nCopyright (c) 2007, 2016, Oracle and\/or its affiliates. All rights reserved.\r\n\r\nCellCLI> list celldisk\r\n CD_00_exaceladm01 normal\r\n CD_01_exaceladm01 normal\r\n CD_02_exaceladm01 normal\r\n CD_03_exaceladm01 normal\r\n CD_04_exaceladm01 normal\r\n CD_05_exaceladm01 normal\r\n CD_06_exaceladm01 normal\r\n CD_07_exaceladm01 normal\r\n CD_08_exaceladm01 normal\r\n CD_09_exaceladm01 normal\r\n CD_10_exaceladm01 normal\r\n CD_11_exaceladm01 normal\r\n FD_00_exaceladm01 normal\r\n FD_01_exaceladm01 normal\r\n FD_02_exaceladm01 normal\r\n FD_03_exaceladm01 normal\r\n\r\nCellCLI> list celldisk CD_02_exaceladm01 detail\r\n name: CD_02_exaceladm01\r\n comment:\r\n creationTime: 2016-11-29T10:23:35+01:00\r\n deviceName: \/dev\/sdc\r\n devicePartition: \/dev\/sdc\r\n diskType: HardDisk\r\n errorCount: 0\r\n freeSpace: 2.6688079833984375T\r\n id: d57b31bb-6043-4cea-b992-ef8075f42e77\r\n physicalDisk: PUT81V\r\n size: 7.152252197265625T\r\n status: normal\r\n\r\nCellCLI>\r\n\r\nCellCLI> list griddisk where name like 'DATAC3.*';\r\n DATAC3_CD_00_exaceladm01 active\r\n DATAC3_CD_01_exaceladm01 active\r\n DATAC3_CD_02_exaceladm01 active\r\n DATAC3_CD_03_exaceladm01 active\r\n DATAC3_CD_04_exaceladm01 active\r\n DATAC3_CD_05_exaceladm01 active\r\n DATAC3_CD_06_exaceladm01 active\r\n DATAC3_CD_07_exaceladm01 active\r\n DATAC3_CD_08_exaceladm01 active\r\n DATAC3_CD_09_exaceladm01 active\r\n DATAC3_CD_10_exaceladm01 active\r\n DATAC3_CD_11_exaceladm01 active\r\n\r\nCellCLI>\r\nCellCLI> list griddisk where name = 'DATAC3_CD_04_exaceladm01' detail;\r\n name: DATAC3_CD_04_exaceladm01\r\n asmDiskGroupName: DATAC3\r\n asmDiskName: DATAC3_CD_04_EXACELADM01\r\n asmFailGroupName: EXACELADM01\r\n availableTo:\r\n cachedBy: FD_00_exaceladm01\r\n cachingPolicy: default\r\n cellDisk: CD_04_exaceladm01\r\n comment: \"Cluster exa-cl3 diskgroup DATAC3\"\r\n creationTime: 2017-01-20T17:23:21+01:00\r\n diskType: HardDisk\r\n errorCount: 0\r\n id: 2cb2aecb-cfa1-4282-b90d-3a08ed079778\r\n size: 257G\r\n status: active\r\n\r\nCellCLI><\/pre>\n
\n
[DOM0 - root@exadbadm01 tmp]$ vi Change_Disk_Size_Of_DATAC3_Cluster_To_330G.sh\r\n[DOM0 - root@exadbadm01 tmp]$\r\n[DOM0 - root@exadbadm01 tmp]$\r\n[DOM0 - root@exadbadm01 tmp]$ cat Change_Disk_Size_Of_DATAC3_Cluster_To_330G.sh\r\ndcli -l root -c exaceladm01 cellcli -e ALTER GRIDDISK DATAC3_CD_00_EXACELADM01 size=330G;\r\ndcli -l root -c exaceladm02 cellcli -e ALTER GRIDDISK DATAC3_CD_00_EXACELADM02 size=330G;\r\n\u2026\r\ndcli -l root -c exaceladm02 cellcli -e ALTER GRIDDISK DATAC3_CD_11_EXACELADM02 size=330G;\r\ndcli -l root -c exaceladm03 cellcli -e ALTER GRIDDISK DATAC3_CD_11_EXACELADM03 size=330G;\r\ndcli -l root -c exaceladm04 cellcli -e ALTER GRIDDISK DATAC3_CD_11_EXACELADM04 size=330G;\r\ndcli -l root -c exaceladm05 cellcli -e ALTER GRIDDISK DATAC3_CD_11_EXACELADM05 size=330G;\r\n[DOM0 - root@exadbadm01 tmp]$\r\n[DOM0 - root@exadbadm01 tmp]$ chmod +x Change_Disk_Size_Of_DATAC3_Cluster_To_330G.sh\r\n[DOM0 - root@exadbadm01 tmp]$\r\n[DOM0 - root@exadbadm01 tmp]$ .\/Change_Disk_Size_Of_DATAC3_Cluster_To_330G.sh\r\nexaceladm01: GridDisk DATAC3_CD_00_exaceladm01 successfully altered\r\nexaceladm02: GridDisk DATAC3_CD_00_exaceladm02 successfully altered\r\nexaceladm03: GridDisk DATAC3_CD_00_exaceladm03 successfully altered\r\n\u2026\r\n\u2026\r\nexaceladm01: GridDisk DATAC3_CD_11_exaceladm01 successfully altered\r\nexaceladm02: GridDisk DATAC3_CD_11_exaceladm02 successfully altered\r\nexaceladm03: GridDisk DATAC3_CD_11_exaceladm03 successfully altered\r\nexaceladm04: GridDisk DATAC3_CD_11_exaceladm04 successfully altered\r\nexaceladm05: GridDisk DATAC3_CD_11_exaceladm05 successfully altered\r\n[DOM0 - root@exadbadm01 tmp]$\r\n[DOM0 - root@exadbadm01 tmp]$<\/pre>\n
[root@exaceladm01 ~]# cellcli\r\nCellCLI: Release 18.1.6.0.0 - Production on Fri Jun 21 17:24:03 CEST 2019\r\n\r\nCopyright (c) 2007, 2016, Oracle and\/or its affiliates. All rights reserved.\r\n\r\nCellCLI> list griddisk where name = 'DATAC3_CD_04_exaceladm01' detail;\r\n name: DATAC3_CD_04_exaceladm01\r\n asmDiskGroupName: DATAC3\r\n asmDiskName: DATAC3_CD_04_EXACELADM01\r\n asmFailGroupName: EXACELADM01\r\n availableTo:\r\n cachedBy: FD_00_exaceladm01\r\n cachingPolicy: default\r\n cellDisk: CD_04_exaceladm01\r\n comment: \"Cluster exa-cl3 diskgroup DATAC3\"\r\n creationTime: 2017-01-20T17:23:21+01:00\r\n diskType: HardDisk\r\n errorCount: 0\r\n id: 2cb2aecb-cfa1-4282-b90d-3a08ed079778\r\n size: 330G\r\n status: active\r\n\r\nCellCLI> list celldisk CD_02_exaceladm01 detail\r\n name: CD_02_exaceladm01\r\n comment:\r\n creationTime: 2016-11-29T10:23:35+01:00\r\n deviceName: \/dev\/sdc\r\n devicePartition: \/dev\/sdc\r\n diskType: HardDisk\r\n errorCount: 0\r\n freeSpace: 2.5975189208984375T\r\n id: d57b31bb-6043-4cea-b992-ef8075f42e77\r\n physicalDisk: PUT81V\r\n size: 7.152252197265625T\r\n status: normal\r\n\r\nCellCLI> exit\r\nquitting\r\n\r\n[root@exaceladm01 ~]#\r\n<\/pre>\n
For ASM – Part #2<\/h2>\n
SQL> ALTER DISKGROUP DATAC3 RESIZE ALL;\r\n\r\nDiskgroup altered.\r\n\r\nSQL><\/pre>\n
SQL> select a.name,a.total_mb,a.free_mb,a.type,\r\n 2 decode(a.type,'NORMAL',a.total_mb\/2\/1024,'HIGH',a.total_mb\/3\/1024) avail_mb,\r\n 3 decode(a.type,'NORMAL',a.free_mb\/2\/1024,'HIGH',a.free_mb\/3\/1024) usable_mb,\r\n 4 count(b.path) cdisks\r\n 5 from v$asm_diskgroup a, v$asm_disk b\r\n 6 where a.group_number=b.group_number\r\n 7 group by a.name,a.total_mb,a.free_mb,a.type,\r\n 8 decode(a.type,'NORMAL',a.total_mb\/2\/1024,'HIGH',a.total_mb\/3\/1024) ,\r\n 9 decode(a.type,'NORMAL',a.free_mb\/2\/1024,'HIGH',a.free_mb\/3\/1024)\r\n 10 order by 2,1\r\n 11 \/\r\n\r\n Total GB Free GB Total GB Free GB Cell\r\nDisk Group Raw Raw TYPE Usable Usable Disksl\r\n------------ ---------- ---------- ------ ---------- ---------- ------\r\nRECOC3 4239360 2465540 NORMAL 2070 1204 60\r\nDATAC3 20275200 6738128 NORMAL 9900 3290 60\r\n\r\nSQL><\/pre>\n
SQL> select operation, EST_MINUTES, EST_RATE, EST_WORK, sofar from v$asm_operation;\r\n\r\nOPERA EST_MINUTES EST_RATE EST_WORK SOFAR\r\n----- ----------- ---------- ---------- ----------\r\nREBAL 0 0 0 0\r\nREBAL 0 46158 18490 5761\r\nREBAL 0 0 0 0\r\nREBAL 0 0 0 0\r\nREBAL 0 0 0 0\r\n\r\nSQL><\/pre>\n
Conclusion<\/h2>\n