The process will be very similar and can be done in rolling and non-rolling mode. In the first, the services continue to run and you don\u2019t need to shutdown databases, but will take more time because the patchmgr applies server by server. At the second, you need to shutdown the entire GI and the patch is applied in parallel and will be faster.<\/p>\n
<\/p>\n
Exadata Version for ZDLRA<\/h2>\n
The Exadata version that can be used for ZDLRA needs to be marked as supported by the ZDLRA team and needs to be compatible with the version that you are running. This compatibility matrix is found at note Zero Data Loss Recovery Appliance Supported Versions (Doc ID 1927416.1)<\/a>.<\/p>\n I posted how to upgrade ZDLRA here<\/a>, and the first thing to download the correct version is discovering which version of ZDLRA you are running. To do that, the racli version is used:<\/p>\n With that, check what version is compatible at the matrix:<\/p>\n So, we can see that version 19.2.1.1.1.202001 is compatible with several Exadata version. Here, in this example, I will use the version 19.3.2.0.0.<\/p>\n Now, we start to patch whatever the Engineering System you are using. Exadata, ZDLRA, SuperCluster. The process follows almost the same to all.<\/p>\n I recommend doing some prechecks before start the patch itself. The first it checks the actual Exadata version that you are running at database servers, storage servers, and switches. Doing this you can check if you need to apply intermediate patches (usually for switches this can occurs) because you are in an old version.<\/p>\n After that, I recommend the restart of ILOM from database and storage (if you do for the switch, the entire switch will restart). This is needed to avoid fsck of ILOM filesystem due longtime online (in old machines like V2, X2, the FSCK can take a lot of time and trigger the timeout and rollback of patchmgr), and to show if some hardware error was not triggered yet:<\/p>\n As you can see, after the ILOM restart, there is no HW error (service leds are off).<\/p>\n To download the Exadata patch, the correct place is the note Zero Data Loss Recovery Appliance Supported Versions (Doc ID 1927416.1)<\/a>. And we can match the ZDLRA version that we are using with the Exadata:<\/p>\n If the link does not work (it\u2019s common) the patch can be downloaded at note 888828.1<\/a> from Exadata. Be careful to download the same patch numbers.<\/p>\n I recommend downloading the patches do \/radump folder. If you want, you can create one specific folder for that:<\/p>\n And after download, we can unzip the patches from Infiniband\/RCoE, Storage, and database patch manager. The patch from database server it is not needed to be unzipped:<\/p>\n Specific for Exadata, the right place it is download from the note 888828.1<\/a>. There, you can choose the desired version that you want. <\/p>\n After that, you can put in one specific folder and unzip (as shown for ZDLRA above) the files.<\/p>\n This step is needed just for ZDLRA to shutdown it (the \u201csoftware\u201d and the database). Just login with the sqlplus and execute dbms_ra.shutdown as rasys user:<\/p>\n Pre-Patch<\/strong><\/p>\n I recommend using the screen<\/em> to avoid that your session being killed by a firewall or network issues during the patch. With this, a session that simulates a console connection is created and you can reconnect it if needed. You can download from https:\/\/linux.oracle.com\/<\/a>.<\/p>\n Another detail is unmounting all NSF that you are using (and remove\/comment from fstab too to avoid mounting during the boot). If not, you will receive a warning about that. This is important for Exadata because you can have long mount time during the startup and this can force the rollback because the time to do the patch reaches the limit. If you don\u2019t want to unmount (because you are patching in rolling mode), there is one flag that can be passed at patchmgr to avoid mount during the reboot:<\/p>\n It is needed to create files with the hostname for the switches, dbnodes, and storage servers.<\/p>\n This step depends on the chosen mode that you are patching. If it is in rolling mode, you can skip. If no, continue with the shutdown.<\/p>\n Since the patch will be in non-rolling mode, it is needed to shutdown all the services running in the machine. To avoid any kind of service running, I shutdown the cluster services and CRS in all nodes:<\/p>\n Stopping cells services<\/strong><\/p>\n This part of the process is not a requirement but can be done only in non-rolling executions. The idea is to stop all services running at storage to avoid IB port errors. And doing this, the \u201calerthistory\u201d from cell doesn\u2019t report error:<\/p>\n After that, we can start the patch for Infiniband switches.<\/p>\n ibswitch_precheck<\/strong><\/p>\n This step checks if it is possible to apply the patch over ibswitch. Some versions have requirements that need to be matched, and if not, we need to fix (one example is hosts file that is wrong or space for FS).<\/p>\n To call the process we enter in the folder that corresponds to an unzipped patch from switches (patch 30469711 in this scenario):<\/p>\n If the status is a success, we can continue to the next step.<\/p>\n upgrade<\/strong><\/p>\n After the precheck, we can call the upgrade. The call is the same, just changing the parameter to upgrade<\/em>:<\/p>\n As you can see, the patch uses the file \u201c\/root\/ib_group\u201d that contains the switches hostname and the patchmgr patched one by one. This mode to patch (one by one) is needed because of the architecture of the IB network. One switch is the \u201cmaster\u201d of the network.<\/p>\n The patch process is simple, copy the new version (through scp) to the switch and call the upgrade script. After that, reboot to load the new version.<\/p>\n Startup cells services<\/strong><\/p>\n After the success of the patch, we can start the cell services:<\/p>\n In the end, you can see that there is no error for cells.<\/p>\n To patch the Storage Cell the process is similar, we use the patchmgr that it is inside of patch that was downloaded. The patch_19.3.2.0.0.191119<\/em> that we unzipped before.<\/p>\n cleanup<\/strong><\/p>\n I always like to start the patch with the cleanup option for patchmgr. This connects in every cell and removes all possible old files (from the previous patch). It is a good start to avoid errors:<\/p>\n patch_check_prereq<\/strong><\/p>\n The next is check the prereqs to do the patch (this test space and if the current version is compatible to do the direct upgrade):<\/p>\n patch<\/strong><\/p>\n After successfully prereqs check, we can do the patch. For rolling mode, you need to add the parameter “rolling” when you call the patchmgr. Doing that, it will patch one by one and just continue with the other cell if the resync from disks at ASM it is finished. And you need to be aware of the EX54 error<\/a>. If you hit this error, the best is to apply in rolling mode.<\/p>\n The process to patch it is interesting because the patch itself copy the image in a different MD device (the one that it is not running), and change the boot to mount this MD. So, you do not upgrade the current system but just change the running image.<\/p>\n[root@zerosita01 ~]# racli version\r\nRecovery Appliance Version:\r\n exadata image: 19.2.3.0.0.190621\r\n rarpm version: ra_automation-19.2.1.1.1.202001-RELEASE.x86_64\r\n rdbms version: RDBMS_19.3.0.0.190416DBRU_LINUX.X64_RELEASE\r\n transaction :\r\n zdlra version: ZDLRA_19.2.1.1.1.202001_LINUX.X64_RELEASE\r\n[root@zerosita01 ~]#<\/pre>\n
![\"\"](\"https:\/\/www.fernandosimon.com\/blog\/wp-content\/uploads\/2020\/04\/ZDLRA-PATCH-02-ORG.png\")
<\/a><\/p>\nPrechecks<\/h2>\n
[root@zerosita01 radump]# dcli -l root -g \/root\/cell_group \"imageinfo\" |grep \"Active image version\"\r\nzerocell01: Active image version: 19.2.3.0.0.190621\r\nzerocell02: Active image version: 19.2.3.0.0.190621\r\nzerocell03: Active image version: 19.2.3.0.0.190621\r\nzerocell04: Active image version: 19.2.3.0.0.190621\r\nzerocell05: Active image version: 19.2.3.0.0.190621\r\nzerocell06: Active image version: 19.2.3.0.0.190621\r\n[root@zerosita01 radump]#\r\n[root@zerosita01 radump]#\r\n[root@zerosita01 radump]#\r\n[root@zerosita01 radump]# dcli -l root -g \/root\/db_group \"imageinfo\" |grep \"Image version\"\r\nzerosita01: Image version: 19.2.3.0.0.190621\r\nzerosita02: Image version: 19.2.3.0.0.190621\r\n[root@zerosita01 radump]#\r\n[root@zerosita01 radump]# dcli -l root -g \/root\/ib_group \"version\" |grep version\r\nzdls-iba01: SUN DCS 36p version: 2.2.12-2\r\nzdls-iba01: BIOS version: NUP1R918\r\nzdls-ibb01: SUN DCS 36p version: 2.2.12-2\r\nzdls-ibb01: BIOS version: NUP1R918\r\n[root@zerosita01 radump]# dcli -l root -g \/root\/ib_group \"version\" |grep \"36p\"\r\nzdls-iba01: SUN DCS 36p version: 2.2.12-2\r\nzdls-ibb01: SUN DCS 36p version: 2.2.12-2\r\n[root@zerosita01 radump]#<\/pre>\n
[root@zerosita01 radump]# dcli -g \/root\/cell_group -l root 'ipmitool mc reset cold'\r\nzerocell01: Sent cold reset command to MC\r\nzerocell02: Sent cold reset command to MC\r\nzerocell03: Sent cold reset command to MC\r\nzerocell04: Sent cold reset command to MC\r\nzerocell05: Sent cold reset command to MC\r\nzerocell06: Sent cold reset command to MC\r\n[root@zerosita01 radump]#\r\n[root@zerosita01 radump]#\r\n[root@zerosita01 radump]# dcli -g \/root\/db_group -l root 'ipmitool mc reset cold'\r\nzerosita01: Sent cold reset command to MC\r\nzerosita02: Sent cold reset command to MC\r\n[root@zerosita01 radump]#\r\n[root@zerosita01 radump]#\r\n######################################################\r\nAfter some time\r\n######################################################\r\n[root@zerosita01 radump]# dcli -l root -g \/root\/cell_group \"ipmitool sunoem led get\" |grep \": SERVICE\"\r\nzerocell01: SERVICE | OFF\r\nzerocell02: SERVICE | OFF\r\nzerocell03: SERVICE | OFF\r\nzerocell04: SERVICE | OFF\r\nzerocell05: SERVICE | OFF\r\nzerocell06: SERVICE | OFF\r\n[root@zerosita01 radump]#\r\n[root@zerosita01 radump]#\r\n[root@zerosita01 radump]#\r\n[root@zerosita01 radump]# dcli -l root -g \/root\/db_group \"ipmitool sunoem led get\" |grep \": SERVICE\"\r\nzerosita01: SERVICE | OFF\r\nzerosita02: SERVICE | OFF\r\n[root@zerosita01 radump]#\r\n<\/pre>\n
Downloading the patch<\/h2>\n
ZDLRA<\/h3>\n
![\"\"](\"https:\/\/www.fernandosimon.com\/blog\/wp-content\/uploads\/2020\/04\/Exadata-ZDLRA-Patch.png\")
<\/a><\/p>\n[root@zerosita01 radump]# mkdir exa_stack\r\n[root@zerosita01 radump]# cd exa_stack\/\r\n[root@zerosita01 exa_stack]#\r\n[root@zerosita01 exa_stack]# cp \/zfs\/ZDLRA_PATCHING\/19.2.1.1.1\/19.2.1.1.1-201910-30614042\/EXA-STACK\/patchmgr\/p21634633_193100_Linux-x86-64.zip .\/\r\n[root@zerosita01 exa_stack]# cp \/zfs\/ZDLRA_PATCHING\/19.2.1.1.1\/19.2.1.1.1-201910-30614042\/EXA-STACK\/dbnode\/p30439212_193000_Linux-x86-64.zip .\/\r\n[root@zerosita01 exa_stack]# cp \/zfs\/ZDLRA_PATCHING\/19.2.1.1.1\/19.2.1.1.1-201910-30614042\/EXA-STACK\/switch\/p30469711_193000_Linux-x86-64.zip .\/\r\n[root@zerosita01 exa_stack]# cp \/zfs\/ZDLRA_PATCHING\/19.2.1.1.1\/19.2.1.1.1-201910-30614042\/EXA-STACK\/storage\/p30540336_193000_Linux-x86-64.zip .\/\r\n[root@zerosita01 exa_stack]#<\/pre>\n
[root@zerosita01 exa_stack]# unzip p21634633_193100_Linux-x86-64.zip\r\nArchive: p21634633_193100_Linux-x86-64.zip\r\n creating: dbserver_patch_19.191113\/\r\n...\r\n...\r\n inflating: dbserver_patch_19.191113\/patchmgr_functions\r\n inflating: dbserver_patch_19.191113\/imageLogger\r\n inflating: dbserver_patch_19.191113\/ExaXMLNode.pm\r\n[root@zerosita01 exa_stack]#\r\n[root@zerosita01 exa_stack]#\r\n[root@zerosita01 exa_stack]# unzip p30439212_193000_Linux-x86-64.zip\r\nArchive: p30439212_193000_Linux-x86-64.zip\r\n inflating: exadata_ol7_base_repo_19.3.2.0.0.191119.iso\r\n inflating: README.html\r\n[root@zerosita01 exa_stack]#\r\n[root@zerosita01 exa_stack]#\r\n[root@zerosita01 exa_stack]# unzip p30469711_193000_Linux-x86-64.zip\r\nArchive: p30469711_193000_Linux-x86-64.zip\r\n creating: patch_switch_19.3.2.0.0.191119\/\r\n inflating: patch_switch_19.3.2.0.0.191119\/nxos.7.0.3.I7.6.bin\r\n...\r\n...\r\n inflating: patch_switch_19.3.2.0.0.191119\/patchmgr_functions\r\n[root@zerosita01 exa_stack]#\r\n[root@zerosita01 exa_stack]#\r\n[root@zerosita01 exa_stack]#\r\n[root@zerosita01 exa_stack]# unzip p30540336_193000_Linux-x86-64.zip\r\nArchive: p30540336_193000_Linux-x86-64.zip\r\n creating: patch_19.3.2.0.0.191119\/\r\n inflating: patch_19.3.2.0.0.191119\/dostep.sh\r\n...\r\n...\r\n inflating: patch_19.3.2.0.0.191119\/19.3.2.0.0.191119.iso\r\n creating: patch_19.3.2.0.0.191119\/linux.db.rpms\/\r\n inflating: patch_19.3.2.0.0.191119\/README.html\r\n[root@zerosita01 exa_stack]#<\/pre>\n
Exadata<\/h3>\n
Shutdown ZDLRA<\/h2>\n
root@zerosita01 exa_stack]# su - oracle\r\nLast login: Wed Mar 4 14:07:44 CET 2020 from 99.234.123.138 on ssh\r\n[oracle@zerosita01 ~]$\r\n[oracle@zerosita01 ~]$ sqlplus rasys\/xxxxxx\r\n\r\nSQL*Plus: Release 19.0.0.0.0 - Production on Wed Mar 4 14:10:07 2020\r\nVersion 19.3.0.0.0\r\n\r\nCopyright (c) 1982, 2019, Oracle. All rights reserved.\r\n\r\nLast Successful login time: Wed Mar 04 2020 14:00:02 +01:00\r\n\r\nConnected to:\r\nOracle Database 19c Enterprise Edition Release 19.0.0.0.0 - Production\r\nVersion 19.3.0.0.0\r\n\r\nSQL> SELECT state FROM ra_server;\r\n\r\nSTATE\r\n-------------\r\nON\r\n\r\nSQL> exec dbms_ra.shutdown;\r\n\r\nPL\/SQL procedure successfully completed.\r\n\r\nSQL> SELECT state FROM ra_server;\r\n\r\nSTATE\r\n-------------\r\nOFF\r\n\r\nSQL> exit\r\nDisconnected from Oracle Database 19c Enterprise Edition Release 19.0.0.0.0 - Production\r\nVersion 19.3.0.0.0\r\n[oracle@zerosita01 ~]$\r\n[oracle@zerosita01 ~]$ srvctl stop database -d zdlras\r\n[oracle@zerosita01 ~]$<\/pre>\n
Patching<\/h2>\n
[root@zerosita01 exa_stack]# rpm -Uvh \/zfs\/ZDLRA_PATCHING\/19.2.1.1.1\/screen-4.1.0-0.25.20120314git3c2946.el7.x86_64.rpm\r\nPreparing... ################################# [100%]\r\nUpdating \/ installing...\r\n 1:screen-4.1.0-0.25.20120314git3c29################################# [100%]\r\n[root@zerosita01 exa_stack]#\r\n[root@zerosita01 exa_stack]# screen -L -RR Patch-Exadata-Stack\r\n[root@zerosita01 exa_stack]# pwd\r\n\/radump\/exa_stack\r\n[root@zerosita01 exa_stack]#<\/pre>\n
[root@zerosita01 exa_stack]# umount \/zfs\r\n[root@zerosita01 exa_stack]#\r\n\r\n SUMMARY OF WARNINGS AND ERRORS FOR zerosita01:\r\n\r\n zerosita01: WARNING: Active network mounts found on this DB node.\r\n zerosita01: The following known issues will be checked for but require manual follow-up:\r\n zerosita01: (*) - Yum rolling update requires fix for 11768055 when Grid Infrastructure is below 11.2.0.2 BP12\r\n<\/pre>\n
1 – Shutdown<\/h3>\n
[root@zerosita01 exa_stack]# \/u01\/app\/19.0.0.0\/grid\/bin\/crsctl stop cluster -all\r\nCRS-2673: Attempting to stop 'ora.crsd' on 'zerosita01'\r\nCRS-2673: Attempting to stop 'ora.crsd' on 'zerosita02'\r\n...\r\n...\r\nCRS-2673: Attempting to stop 'ora.diskmon' on 'zerosita02'\r\nCRS-2677: Stop of 'ora.diskmon' on 'zerosita02' succeeded\r\n[root@zerosita01 exa_stack]#\r\n[root@zerosita01 exa_stack]#\r\n[root@zerosita01 exa_stack]# \/u01\/app\/19.0.0.0\/grid\/bin\/crsctl stop crs\r\nCRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'zerosita01'\r\nCRS-2673: Attempting to stop 'ora.drivers.acfs' on 'zerosita01'\r\n...\r\n...\r\nCRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'zerosita01' has completed\r\nCRS-4133: Oracle High Availability Services has been stopped.\r\n[root@zerosita01 exa_stack]#\r\n[root@zerosita01 exa_stack]#\r\n[root@zerosita01 exa_stack]# ssh -q zerosita02\r\nLast login: Wed Mar 4 14:15:09 CET 2020 from 99.234.123.138 on ssh\r\nLast login: Wed Mar 4 14:15:57 2020 from zerosita01.zero.flisk.net\r\n[root@zerosita02 ~]#\r\n[root@zerosita02 ~]# \/u01\/app\/19.0.0.0\/grid\/bin\/crsctl stop crs\r\nCRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'zerosita02'\r\nCRS-2673: Attempting to stop 'ora.mdnsd' on 'zerosita02'\r\n...\r\n...\r\nCRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'zerosita02' has completed\r\nCRS-4133: Oracle High Availability Services has been stopped.\r\n[root@zerosita02 ~]#\r\n[root@zerosita02 ~]#\r\n[root@zerosita02 ~]# logout\r\n[root@zerosita01 exa_stack]#\r\n<\/pre>\n
2 \u2013 Patch Switches<\/h3>\n
[root@zerosita01 patch_switch_19.3.2.0.0.191119]# dcli -l root -g \/root\/cell_group \"cellcli -e list cell\"\r\nzerocell01: zerocell01 online\r\nzerocell02: zerocell02 online\r\nzerocell03: zerocell03 online\r\nzerocell04: zerocell04 online\r\nzerocell05: zerocell05 online\r\nzerocell06: zerocell06 online\r\n[root@zerosita01 patch_switch_19.3.2.0.0.191119]#\r\n[root@zerosita01 patch_switch_19.3.2.0.0.191119]#\r\n[root@zerosita01 patch_switch_19.3.2.0.0.191119]# dcli -l root -g \/root\/cell_group \"cellcli -e alter cell shutdown services all\"\r\nzerocell01:\r\nzerocell01: Stopping the RS, CELLSRV, and MS services...\r\nzerocell01: The SHUTDOWN of services was successful.\r\nzerocell02:\r\nzerocell02: Stopping the RS, CELLSRV, and MS services...\r\nzerocell02: The SHUTDOWN of services was successful.\r\nzerocell03:\r\nzerocell03: Stopping the RS, CELLSRV, and MS services...\r\nzerocell03: The SHUTDOWN of services was successful.\r\nzerocell04:\r\nzerocell04: Stopping the RS, CELLSRV, and MS services...\r\nzerocell04: The SHUTDOWN of services was successful.\r\nzerocell05:\r\nzerocell05: Stopping the RS, CELLSRV, and MS services...\r\nzerocell05: The SHUTDOWN of services was successful.\r\nzerocell06:\r\nzerocell06: Stopping the RS, CELLSRV, and MS services...\r\nzerocell06: The SHUTDOWN of services was successful.\r\n[root@zerosita01 patch_switch_19.3.2.0.0.191119]#<\/pre>\n
[root@zerosita01 exa_stack]# cd patch_switch_19.3.2.0.0.191119\r\n[root@zerosita01 patch_switch_19.3.2.0.0.191119]#\r\n[root@zerosita01 patch_switch_19.3.2.0.0.191119]#\r\n[root@zerosita01 patch_switch_19.3.2.0.0.191119]# .\/patchmgr -ibswitches \/root\/ib_group -upgrade -ibswitch_precheck\r\n\r\n2020-03-04 14:19:21 +0100 :Working: Verify SSH equivalence for the root user to node(s)\r\n2020-03-04 14:19:26 +0100 :SUCCESS: Verify SSH equivalence for the root user to node(s)\r\n2020-03-04 14:19:29 +0100 1 of 1 :Working: Initiate pre-upgrade validation check on InfiniBand switch(es).\r\n ----- InfiniBand switch update process started 2020-03-04 14:19:29 +0100 -----\r\n[NOTE ] Log file at \/radump\/exa_stack\/patch_switch_19.3.2.0.0.191119\/upgradeIBSwitch.log\r\n\r\n[INFO ] List of InfiniBand switches for upgrade: ( zdls-iba01 zdls-ibb01 )\r\n[SUCCESS ] Verifying Network connectivity to zdls-iba01\r\n[SUCCESS ] Verifying Network connectivity to zdls-ibb01\r\n[SUCCESS ] Validating verify-topology output\r\n[INFO ] Master Subnet Manager is set to \"zdls-iba01\" in all Switches\r\n\r\n[INFO ] ---------- Starting with InfiniBand Switch zdls-iba01\r\n[WARNING ] Infiniband switch meets minimal version requirements, but downgrade is only available to 2.2.13-2 with the current package.\r\n To downgrade to other versions:\r\n - Manually download the InfiniBand switch firmware package to the patch directory\r\n - Set export variable \"EXADATA_IMAGE_IBSWITCH_DOWNGRADE_VERSION\" to the appropriate version\r\n - Run patchmgr command to initiate downgrade.\r\n[SUCCESS ] Verify SSH access to the patchmgr host zerosita01.zero.flisk.net from the InfiniBand Switch zdls-iba01.\r\n[INFO ] Starting pre-update validation on zdls-iba01\r\n[SUCCESS ] Verifying that \/tmp has 150M in zdls-iba01, found 492M\r\n[SUCCESS ] Verifying that \/ has 20M in zdls-iba01, found 26M\r\n[SUCCESS ] NTP daemon is running on zdls-iba01.\r\n[INFO ] Manually validate the following entries Date:(YYYY-MM-DD) 2020-03-04 Time:(HH:MM:SS) 14:26:40\r\n[INFO ] Validating the current firmware on the InfiniBand Switch\r\n[SUCCESS ] Firmware verification on InfiniBand switch zdls-iba01\r\n[SUCCESS ] Verifying that the patchmgr host zerosita01.zero.flisk.net is recognized on the InfiniBand Switch zdls-iba01 through getHostByName\r\n[SUCCESS ] Execute plugin check for Patch Check Prereq on zdls-iba01\r\n[INFO ] Finished pre-update validation on zdls-iba01\r\n[SUCCESS ] Pre-update validation on zdls-iba01\r\n[SUCCESS ] Prereq check on zdls-iba01\r\n\r\n[INFO ] ---------- Starting with InfiniBand Switch zdls-ibb01\r\n[WARNING ] Infiniband switch meets minimal version requirements, but downgrade is only available to 2.2.13-2 with the current package.\r\n To downgrade to other versions:\r\n - Manually download the InfiniBand switch firmware package to the patch directory\r\n - Set export variable \"EXADATA_IMAGE_IBSWITCH_DOWNGRADE_VERSION\" to the appropriate version\r\n - Run patchmgr command to initiate downgrade.\r\n[SUCCESS ] Verify SSH access to the patchmgr host zerosita01.zero.flisk.net from the InfiniBand Switch zdls-ibb01.\r\n[INFO ] Starting pre-update validation on zdls-ibb01\r\n[SUCCESS ] Verifying that \/tmp has 150M in zdls-ibb01, found 492M\r\n[SUCCESS ] Verifying that \/ has 20M in zdls-ibb01, found 26M\r\n[SUCCESS ] NTP daemon is running on zdls-ibb01.\r\n[INFO ] Manually validate the following entries Date:(YYYY-MM-DD) 2020-03-04 Time:(HH:MM:SS) 14:27:13\r\n[INFO ] Validating the current firmware on the InfiniBand Switch\r\n[SUCCESS ] Firmware verification on InfiniBand switch zdls-ibb01\r\n[SUCCESS ] Verifying that the patchmgr host zerosita01.zero.flisk.net is recognized on the InfiniBand Switch zdls-ibb01 through getHostByName\r\n[SUCCESS ] Execute plugin check for Patch Check Prereq on zdls-ibb01\r\n[INFO ] Finished pre-update validation on zdls-ibb01\r\n[SUCCESS ] Pre-update validation on zdls-ibb01\r\n[SUCCESS ] Prereq check on zdls-ibb01\r\n[SUCCESS ] Overall status\r\n\r\n ----- InfiniBand switch update process ended 2020-03-04 14:20:54 +0100 -----\r\n2020-03-04 14:20:54 +0100 1 of 1 :SUCCESS: Initiate pre-upgrade validation check on InfiniBand switch(es).\r\n2020-03-04 14:20:54 +0100 :SUCCESS: Completed run of command: .\/patchmgr -ibswitches \/root\/ib_group -upgrade -ibswitch_precheck\r\n2020-03-04 14:20:54 +0100 :INFO : upgrade attempted on nodes in file \/root\/ib_group: [zdls-iba01 zdls-ibb01]\r\n2020-03-04 14:20:54 +0100 :INFO : For details, check the following files in \/radump\/exa_stack\/patch_switch_19.3.2.0.0.191119:\r\n2020-03-04 14:20:54 +0100 :INFO : - upgradeIBSwitch.log\r\n2020-03-04 14:20:54 +0100 :INFO : - upgradeIBSwitch.trc\r\n2020-03-04 14:20:54 +0100 :INFO : - patchmgr.stdout\r\n2020-03-04 14:20:54 +0100 :INFO : - patchmgr.stderr\r\n2020-03-04 14:20:54 +0100 :INFO : - patchmgr.log\r\n2020-03-04 14:20:54 +0100 :INFO : - patchmgr.trc\r\n2020-03-04 14:20:54 +0100 :INFO : Exit status:0\r\n2020-03-04 14:20:55 +0100 :INFO : Exiting.\r\n\r\n[root@zerosita01 patch_switch_19.3.2.0.0.191119]#\r\n<\/pre>\n
[root@zerosita01 patch_switch_19.3.2.0.0.191119]# .\/patchmgr -ibswitches \/root\/ib_group -upgrade\r\n\r\n2020-03-04 14:30:46 +0100 :Working: Verify SSH equivalence for the root user to node(s)\r\n2020-03-04 14:30:51 +0100 :SUCCESS: Verify SSH equivalence for the root user to node(s)\r\n2020-03-04 14:30:54 +0100 1 of 1 :Working: Initiate upgrade of InfiniBand switches to 2.2.14-1. Expect up to 40 minutes for each switch\r\n\r\n ----- InfiniBand switch update process started 2020-03-04 14:30:54 +0100 -----\r\n[NOTE ] Log file at \/radump\/exa_stack\/patch_switch_19.3.2.0.0.191119\/upgradeIBSwitch.log\r\n\r\n[INFO ] List of InfiniBand switches for upgrade: ( zdls-iba01 zdls-ibb01 )\r\n[SUCCESS ] Verifying Network connectivity to zdls-iba01\r\n[SUCCESS ] Verifying Network connectivity to zdls-ibb01\r\n[SUCCESS ] Validating verify-topology output\r\n[INFO ] Proceeding with upgrade of InfiniBand switches to version 2.2.14_1\r\n[INFO ] Master Subnet Manager is set to \"zdls-iba01\" in all Switches\r\n\r\n[INFO ] ---------- Starting with InfiniBand Switch zdls-iba01\r\n[WARNING ] Infiniband switch meets minimal version requirements, but downgrade is only available to 2.2.13-2 with the current package.\r\n To downgrade to other versions:\r\n - Manually download the InfiniBand switch firmware package to the patch directory\r\n - Set export variable \"EXADATA_IMAGE_IBSWITCH_DOWNGRADE_VERSION\" to the appropriate version\r\n - Run patchmgr command to initiate downgrade.\r\n[SUCCESS ] Verify SSH access to the patchmgr host zerosita01.zero.flisk.net from the InfiniBand Switch zdls-iba01.\r\n[INFO ] Starting pre-update validation on zdls-iba01\r\n[SUCCESS ] Verifying that \/tmp has 150M in zdls-iba01, found 492M\r\n[SUCCESS ] Verifying that \/ has 20M in zdls-iba01, found 26M\r\n[SUCCESS ] Service opensmd is running on InfiniBand Switch zdls-iba01\r\n[SUCCESS ] NTP daemon is running on zdls-iba01.\r\n[INFO ] Manually validate the following entries Date:(YYYY-MM-DD) 2020-03-04 Time:(HH:MM:SS) 14:38:06\r\n[INFO ] Validating the current firmware on the InfiniBand Switch\r\n[SUCCESS ] Firmware verification on InfiniBand switch zdls-iba01\r\n[SUCCESS ] Verifying that the patchmgr host zerosita01.zero.flisk.net is recognized on the InfiniBand Switch zdls-iba01 through getHostByName\r\n[SUCCESS ] Execute plugin check for Patch Check Prereq on zdls-iba01\r\n[INFO ] Finished pre-update validation on zdls-iba01\r\n[SUCCESS ] Pre-update validation on zdls-iba01\r\n[INFO ] Package will be downloaded at firmware update time via scp\r\n[SUCCESS ] Execute plugin check for Patching on zdls-iba01\r\n[INFO ] Starting upgrade on zdls-iba01 to 2.2.14_1. Please give upto 15 mins for the process to complete. DO NOT INTERRUPT or HIT CTRL+C during the upgrade\r\n[INFO ] Rebooting zdls-iba01 to complete the firmware update. Wait for 15 minutes before continuing. DO NOT MANUALLY REBOOT THE INFINIBAND SWITCH\r\n[SUCCESS ] Load firmware 2.2.14_1 onto zdls-iba01\r\n[SUCCESS ] Verify that \/conf\/configvalid is set to 1 on zdls-iba01\r\n[INFO ] Set SMPriority to 5 on zdls-iba01\r\n[INFO ] Starting post-update validation on zdls-iba01\r\n[SUCCESS ] Service opensmd is running on InfiniBand Switch zdls-iba01\r\n[SUCCESS ] NTP daemon is running on zdls-iba01.\r\n[INFO ] Manually validate the following entries Date:(YYYY-MM-DD) 2020-03-04 Time:(HH:MM:SS) 14:54:47\r\n[INFO ] \/conf\/configvalid is 1\r\n[INFO ] Validating the current firmware on the InfiniBand Switch\r\n[SUCCESS ] Firmware verification on InfiniBand switch zdls-iba01\r\n[SUCCESS ] Execute plugin check for Post Patch on zdls-iba01\r\n[INFO ] Finished post-update validation on zdls-iba01\r\n[SUCCESS ] Post-update validation on zdls-iba01\r\n[SUCCESS ] Update InfiniBand switch zdls-iba01 to 2.2.14_1\r\n\r\n[INFO ] ---------- Starting with InfiniBand Switch zdls-ibb01\r\n[WARNING ] Infiniband switch meets minimal version requirements, but downgrade is only available to 2.2.13-2 with the current package.\r\n To downgrade to other versions:\r\n - Manually download the InfiniBand switch firmware package to the patch directory\r\n - Set export variable \"EXADATA_IMAGE_IBSWITCH_DOWNGRADE_VERSION\" to the appropriate version\r\n - Run patchmgr command to initiate downgrade.\r\n[SUCCESS ] Verify SSH access to the patchmgr host zerosita01.zero.flisk.net from the InfiniBand Switch zdls-ibb01.\r\n[INFO ] Starting pre-update validation on zdls-ibb01\r\n[SUCCESS ] Verifying that \/tmp has 150M in zdls-ibb01, found 492M\r\n[SUCCESS ] Verifying that \/ has 20M in zdls-ibb01, found 26M\r\n[SUCCESS ] Service opensmd is running on InfiniBand Switch zdls-ibb01\r\n[SUCCESS ] NTP daemon is running on zdls-ibb01.\r\n[INFO ] Manually validate the following entries Date:(YYYY-MM-DD) 2020-03-04 Time:(HH:MM:SS) 14:55:24\r\n[INFO ] Validating the current firmware on the InfiniBand Switch\r\n[SUCCESS ] Firmware verification on InfiniBand switch zdls-ibb01\r\n[SUCCESS ] Verifying that the patchmgr host zerosita01.zero.flisk.net is recognized on the InfiniBand Switch zdls-ibb01 through getHostByName\r\n[SUCCESS ] Execute plugin check for Patch Check Prereq on zdls-ibb01\r\n[INFO ] Finished pre-update validation on zdls-ibb01\r\n[SUCCESS ] Pre-update validation on zdls-ibb01\r\n[INFO ] Package will be downloaded at firmware update time via scp\r\n[SUCCESS ] Execute plugin check for Patching on zdls-ibb01\r\n[INFO ] Starting upgrade on zdls-ibb01 to 2.2.14_1. Please give upto 15 mins for the process to complete. DO NOT INTERRUPT or HIT CTRL+C during the upgrade\r\n[INFO ] Rebooting zdls-ibb01 to complete the firmware update. Wait for 15 minutes before continuing. DO NOT MANUALLY REBOOT THE INFINIBAND SWITCH\r\n[SUCCESS ] Load firmware 2.2.14_1 onto zdls-ibb01\r\n[SUCCESS ] Verify that \/conf\/configvalid is set to 1 on zdls-ibb01\r\n[INFO ] Set SMPriority to 5 on zdls-ibb01\r\n[INFO ] Starting post-update validation on zdls-ibb01\r\n[SUCCESS ] Service opensmd is running on InfiniBand Switch zdls-ibb01\r\n[SUCCESS ] NTP daemon is running on zdls-ibb01.\r\n[INFO ] Manually validate the following entries Date:(YYYY-MM-DD) 2020-03-04 Time:(HH:MM:SS) 15:12:05\r\n[INFO ] \/conf\/configvalid is 1\r\n[INFO ] Validating the current firmware on the InfiniBand Switch\r\n[SUCCESS ] Firmware verification on InfiniBand switch zdls-ibb01\r\n[SUCCESS ] Execute plugin check for Post Patch on zdls-ibb01\r\n[INFO ] Finished post-update validation on zdls-ibb01\r\n[SUCCESS ] Post-update validation on zdls-ibb01\r\n[SUCCESS ] Update InfiniBand switch zdls-ibb01 to 2.2.14_1\r\n[INFO ] InfiniBand Switches ( zdls-iba01 zdls-ibb01 ) updated to 2.2.14_1\r\n[SUCCESS ] Overall status\r\n\r\n ----- InfiniBand switch update process ended 2020-03-04 15:05:48 +0100 -----\r\n2020-03-04 15:05:48 +0100 1 of 1 :SUCCESS: Upgrade InfiniBand switch(es) to 2.2.14-1.\r\n2020-03-04 15:05:48 +0100 :SUCCESS: Completed run of command: .\/patchmgr -ibswitches \/root\/ib_group -upgrade\r\n2020-03-04 15:05:48 +0100 :INFO : upgrade attempted on nodes in file \/root\/ib_group: [zdls-iba01 zdls-ibb01]\r\n2020-03-04 15:05:48 +0100 :INFO : For details, check the following files in \/radump\/exa_stack\/patch_switch_19.3.2.0.0.191119:\r\n2020-03-04 15:05:48 +0100 :INFO : - upgradeIBSwitch.log\r\n2020-03-04 15:05:48 +0100 :INFO : - upgradeIBSwitch.trc\r\n2020-03-04 15:05:48 +0100 :INFO : - patchmgr.stdout\r\n2020-03-04 15:05:48 +0100 :INFO : - patchmgr.stderr\r\n2020-03-04 15:05:48 +0100 :INFO : - patchmgr.log\r\n2020-03-04 15:05:48 +0100 :INFO : - patchmgr.trc\r\n2020-03-04 15:05:48 +0100 :INFO : Exit status:0\r\n2020-03-04 15:05:48 +0100 :INFO : Exiting.\r\n\r\nYou have new mail in \/var\/spool\/mail\/root\r\n[root@zerosita01 patch_switch_19.3.2.0.0.191119]# cd ..\r\n[root@zerosita01 exa_stack]# dcli -l root -g \/root\/ib_group \"version\" |grep version\r\nzdls-iba01: SUN DCS 36p version: 2.2.14-1\r\nzdls-iba01: BIOS version: NUP1R918\r\nzdls-ibb01: SUN DCS 36p version: 2.2.14-1\r\nzdls-ibb01: BIOS version: NUP1R918\r\n[root@zerosita01 exa_stack]#<\/pre>\n
[root@zerosita01 exa_stack]# dcli -l root -g \/root\/cell_group \"cellcli -e alter cell startup services all\"\r\nzerocell01:\r\nzerocell01: Starting the RS, CELLSRV, and MS services...\r\nzerocell01: Getting the state of RS services... running\r\nzerocell01: Starting CELLSRV services...\r\nzerocell01: The STARTUP of CELLSRV services was successful.\r\nzerocell01: Starting MS services...\r\nzerocell01: The STARTUP of MS services was successful.\r\nzerocell02:\r\nzerocell02: Starting the RS, CELLSRV, and MS services...\r\nzerocell02: Getting the state of RS services... running\r\nzerocell02: Starting CELLSRV services...\r\nzerocell02: The STARTUP of CELLSRV services was successful.\r\nzerocell02: Starting MS services...\r\nzerocell02: The STARTUP of MS services was successful.\r\nzerocell03:\r\nzerocell03: Starting the RS, CELLSRV, and MS services...\r\nzerocell03: Getting the state of RS services... running\r\nzerocell03: Starting CELLSRV services...\r\nzerocell03: The STARTUP of CELLSRV services was successful.\r\nzerocell03: Starting MS services...\r\nzerocell03: The STARTUP of MS services was successful.\r\nzerocell04:\r\nzerocell04: Starting the RS, CELLSRV, and MS services...\r\nzerocell04: Getting the state of RS services... running\r\nzerocell04: Starting CELLSRV services...\r\nzerocell04: The STARTUP of CELLSRV services was successful.\r\nzerocell04: Starting MS services...\r\nzerocell04: The STARTUP of MS services was successful.\r\nzerocell05:\r\nzerocell05: Starting the RS, CELLSRV, and MS services...\r\nzerocell05: Getting the state of RS services... running\r\nzerocell05: Starting CELLSRV services...\r\nzerocell05: The STARTUP of CELLSRV services was successful.\r\nzerocell05: Starting MS services...\r\nzerocell05: The STARTUP of MS services was successful.\r\nzerocell06:\r\nzerocell06: Starting the RS, CELLSRV, and MS services...\r\nzerocell06: Getting the state of RS services... running\r\nzerocell06: Starting CELLSRV services...\r\nzerocell06: The STARTUP of CELLSRV services was successful.\r\nzerocell06: Starting MS services...\r\nzerocell06: The STARTUP of MS services was successful.\r\n[root@zerosita01 exa_stack]#\r\n[root@zerosita01 exa_stack]# dcli -l root -g \/root\/cell_group \"cellcli -e list cell\"\r\nzerocell01: zerocell01 online\r\nzerocell02: zerocell02 online\r\nzerocell03: zerocell03 online\r\nzerocell04: zerocell04 online\r\nzerocell05: zerocell05 online\r\nzerocell06: zerocell06 online\r\n[root@zerosita01 exa_stack]# \r\n[root@zerosita01 exa_stack]# dcli -l root -g \/root\/cell_group \"cellcli -e list alerthistory\"\r\n[root@zerosita01 exa_stack]#<\/pre>\n
3 – Patch Storage Cells<\/h3>\n
[root@zerosita01 exa_stack]# ls -l\r\ntotal 7577888\r\ndrwxrwxr-x 3 root root 4096 Nov 13 16:11 dbserver_patch_19.191113\r\n-rw-rw-r-- 1 root root 1505257472 Nov 19 19:23 exadata_ol7_base_repo_19.3.2.0.0.191119.iso\r\n-rw-r--r-- 1 root root 438739135 Mar 4 13:40 p21634633_193100_Linux-x86-64.zip\r\n-rw-r--r-- 1 root root 1460815642 Mar 4 13:46 p30439212_193000_Linux-x86-64.zip\r\n-rw-r--r-- 1 root root 2872128400 Mar 4 13:47 p30469711_193000_Linux-x86-64.zip\r\n-rw-r--r-- 1 root root 1474890484 Mar 4 13:48 p30540336_193000_Linux-x86-64.zip\r\ndrwxrwxr-x 5 root root 4096 Nov 19 19:29 patch_19.3.2.0.0.191119\r\ndrwxrwxr-x 6 root root 4096 Mar 4 15:05 patch_switch_19.3.2.0.0.191119\r\n-rw-rw-r-- 1 root root 264228 Nov 19 19:29 README.html\r\n-rw-r--r-- 1 root root 31912 Mar 4 15:05 screenlog.0\r\n[root@zerosita01 exa_stack]#<\/pre>\n
[root@zerosita01 exa_stack]# cd patch_19.3.2.0.0.191119\r\n[root@zerosita01 patch_19.3.2.0.0.191119]#\r\n[root@zerosita01 patch_19.3.2.0.0.191119]# .\/patchmgr -cells \/root\/cell_group -cleanup\r\n\r\n2020-03-04 16:08:06 +0100 :Working: Cleanup\r\n2020-03-04 16:08:08 +0100 :SUCCESS: Cleanup\r\n2020-03-04 16:08:09 +0100 :SUCCESS: Completed run of command: .\/patchmgr -cells \/root\/cell_group -cleanup\r\n2020-03-04 16:08:09 +0100 :INFO : Cleanup attempted on nodes in file \/root\/cell_group: [zerocell01 zerocell02 zerocell03 zerocell04 zerocell05 zerocell06]\r\n2020-03-04 16:08:09 +0100 :INFO : Current image version on cell(s) is:\r\n2020-03-04 16:08:09 +0100 :INFO : zerocell01: 19.2.3.0.0.190621\r\n2020-03-04 16:08:09 +0100 :INFO : zerocell02: 19.2.3.0.0.190621\r\n2020-03-04 16:08:09 +0100 :INFO : zerocell03: 19.2.3.0.0.190621\r\n2020-03-04 16:08:09 +0100 :INFO : zerocell04: 19.2.3.0.0.190621\r\n2020-03-04 16:08:09 +0100 :INFO : zerocell05: 19.2.3.0.0.190621\r\n2020-03-04 16:08:09 +0100 :INFO : zerocell06: 19.2.3.0.0.190621\r\n2020-03-04 16:08:09 +0100 :INFO : For details, check the following files in \/radump\/exa_stack\/patch_19.3.2.0.0.191119:\r\n2020-03-04 16:08:09 +0100 :INFO : - <cell_name>.log\r\n2020-03-04 16:08:09 +0100 :INFO : - patchmgr.stdout\r\n2020-03-04 16:08:09 +0100 :INFO : - patchmgr.stderr\r\n2020-03-04 16:08:09 +0100 :INFO : - patchmgr.log\r\n2020-03-04 16:08:09 +0100 :INFO : - patchmgr.trc\r\n2020-03-04 16:08:09 +0100 :INFO : Exit status:0\r\n2020-03-04 16:08:09 +0100 :INFO : Exiting.\r\n\r\n[root@zerosita01 patch_19.3.2.0.0.191119]#<\/pre>\n
[root@zerosita01 patch_19.3.2.0.0.191119]# .\/patchmgr -cells \/root\/cell_group -patch_check_prereq\r\n\r\n2020-03-04 16:09:22 +0100 :Working: Check cells have ssh equivalence for root user. Up to 10 seconds per cell ...\r\n2020-03-04 16:09:24 +0100 :SUCCESS: Check cells have ssh equivalence for root user.\r\n2020-03-04 16:09:30 +0100 :Working: Initialize files. Up to 1 minute ...\r\n2020-03-04 16:09:32 +0100 :Working: Setup work directory\r\n2020-03-04 16:09:34 +0100 :SUCCESS: Setup work directory\r\n2020-03-04 16:09:37 +0100 :SUCCESS: Initialize files.\r\n2020-03-04 16:09:37 +0100 :Working: Copy, extract prerequisite check archive to cells. If required start md11 mismatched partner size correction. Up to 40 minutes ...\r\n2020-03-04 16:09:52 +0100 :INFO : Wait correction of degraded md11 due to md partner size mismatch. Up to 30 minutes.\r\n2020-03-04 16:09:53 +0100 :SUCCESS: Copy, extract prerequisite check archive to cells. If required start md11 mismatched partner size correction.\r\n2020-03-04 16:09:53 +0100 :Working: Check space and state of cell services. Up to 20 minutes ...\r\n2020-03-04 16:11:10 +0100 :SUCCESS: Check space and state of cell services.\r\n2020-03-04 16:11:10 +0100 :Working: Check prerequisites on all cells. Up to 2 minutes ...\r\n2020-03-04 16:11:13 +0100 :SUCCESS: Check prerequisites on all cells.\r\n2020-03-04 16:11:13 +0100 :Working: Execute plugin check for Patch Check Prereq ...\r\n2020-03-04 16:11:14 +0100 :INFO : Patchmgr plugin start: Prereq check for exposure to bug 22909764 v1.0.\r\n2020-03-04 16:11:14 +0100 :INFO : Details in logfile \/radump\/exa_stack\/patch_19.3.2.0.0.191119\/patchmgr.stdout.\r\n2020-03-04 16:11:14 +0100 :INFO : Patchmgr plugin start: Prereq check for exposure to bug 17854520 v1.3.\r\n2020-03-04 16:11:14 +0100 :INFO : Details in logfile \/radump\/exa_stack\/patch_19.3.2.0.0.191119\/patchmgr.stdout.\r\n2020-03-04 16:11:14 +0100 :SUCCESS: No exposure to bug 17854520 with non-rolling patching\r\n2020-03-04 16:11:14 +0100 :INFO : Patchmgr plugin start: Prereq check for exposure to bug 22468216 v1.0.\r\n2020-03-04 16:11:14 +0100 :INFO : Details in logfile \/radump\/exa_stack\/patch_19.3.2.0.0.191119\/patchmgr.stdout.\r\n2020-03-04 16:11:14 +0100 :SUCCESS: Patchmgr plugin complete: Prereq check passed for the bug 22468216\r\n2020-03-04 16:11:14 +0100 :INFO : Patchmgr plugin start: Prereq check for exposure to bug 24625612 v1.0.\r\n2020-03-04 16:11:14 +0100 :INFO : Details in logfile \/radump\/exa_stack\/patch_19.3.2.0.0.191119\/patchmgr.stdout.\r\n2020-03-04 16:11:14 +0100 :SUCCESS: Patchmgr plugin complete: Prereq check passed for the bug 24625612\r\n2020-03-04 16:11:14 +0100 :SUCCESS: No exposure to bug 22896791 with non-rolling patching\r\n2020-03-04 16:11:14 +0100 :INFO : Patchmgr plugin start: Prereq check for exposure to bug 22651315 v1.0.\r\n2020-03-04 16:11:14 +0100 :INFO : Details in logfile \/radump\/exa_stack\/patch_19.3.2.0.0.191119\/patchmgr.stdout.\r\n2020-03-04 16:11:16 +0100 :SUCCESS: Patchmgr plugin complete: Prereq check passed for the bug 22651315\r\n2020-03-04 16:11:17 +0100 :SUCCESS: Execute plugin check for Patch Check Prereq.\r\n2020-03-04 16:11:17 +0100 :Working: Check ASM deactivation outcome. Up to 1 minute ...\r\n2020-03-04 16:11:28 +0100 :SUCCESS: Check ASM deactivation outcome.\r\n2020-03-04 16:11:28 +0100 :Working: check if MS SOFTWAREUPDATE is scheduled. up to 5 minutes...\r\n2020-03-04 16:11:29 +0100 :NO ACTION NEEDED: No cells found with SOFTWAREUPDATE scheduled by MS\r\n2020-03-04 16:11:30 +0100 :SUCCESS: check if MS SOFTWAREUPDATE is scheduled\r\n2020-03-04 16:11:32 +0100 :SUCCESS: Completed run of command: .\/patchmgr -cells \/root\/cell_group -patch_check_prereq\r\n2020-03-04 16:11:32 +0100 :INFO : patch_prereq attempted on nodes in file \/root\/cell_group: [zerocell01 zerocell02 zerocell03 zerocell04 zerocell05 zerocell06]\r\n2020-03-04 16:11:32 +0100 :INFO : Current image version on cell(s) is:\r\n2020-03-04 16:11:32 +0100 :INFO : zerocell01: 19.2.3.0.0.190621\r\n2020-03-04 16:11:32 +0100 :INFO : zerocell02: 19.2.3.0.0.190621\r\n2020-03-04 16:11:32 +0100 :INFO : zerocell03: 19.2.3.0.0.190621\r\n2020-03-04 16:11:32 +0100 :INFO : zerocell04: 19.2.3.0.0.190621\r\n2020-03-04 16:11:32 +0100 :INFO : zerocell05: 19.2.3.0.0.190621\r\n2020-03-04 16:11:32 +0100 :INFO : zerocell06: 19.2.3.0.0.190621\r\n2020-03-04 16:11:32 +0100 :INFO : For details, check the following files in \/radump\/exa_stack\/patch_19.3.2.0.0.191119:\r\n2020-03-04 16:11:32 +0100 :INFO : - <cell_name>.log\r\n2020-03-04 16:11:32 +0100 :INFO : - patchmgr.stdout\r\n2020-03-04 16:11:32 +0100 :INFO : - patchmgr.stderr\r\n2020-03-04 16:11:32 +0100 :INFO : - patchmgr.log\r\n2020-03-04 16:11:32 +0100 :INFO : - patchmgr.trc\r\n2020-03-04 16:11:32 +0100 :INFO : Exit status:0\r\n2020-03-04 16:11:32 +0100 :INFO : Exiting.\r\n\r\n[root@zerosita01 patch_19.3.2.0.0.191119]#<\/pre>\n
[root@zerosita01 patch_19.3.2.0.0.191119]# .\/patchmgr -cells \/root\/cell_group -patch\r\n********************************************************************************\r\nNOTE Cells will reboot during the patch or rollback process.\r\nNOTE For non-rolling patch or rollback, ensure all ASM instances using\r\nNOTE the cells are shut down for the duration of the patch or rollback.\r\nNOTE For rolling patch or rollback, ensure all ASM instances using\r\nNOTE the cells are up for the duration of the patch or rollback.\r\n\r\nWARNING Do not interrupt the patchmgr session.\r\nWARNING Do not alter state of ASM instances during patch or rollback.\r\nWARNING Do not resize the screen. It may disturb the screen layout.\r\nWARNING Do not reboot cells or alter cell services during patch or rollback.\r\nWARNING Do not open log files in editor in write mode or try to alter them.\r\n\r\nNOTE All time estimates are approximate.\r\n********************************************************************************\r\n\r\n\r\n2020-03-04 16:11:57 +0100 :Working: Check cells have ssh equivalence for root user. Up to 10 seconds per cell ...\r\n2020-03-04 16:11:59 +0100 :SUCCESS: Check cells have ssh equivalence for root user.\r\n2020-03-04 16:12:05 +0100 :Working: Initialize files. Up to 1 minute ...\r\n2020-03-04 16:12:07 +0100 :Working: Setup work directory\r\n2020-03-04 16:12:41 +0100 :SUCCESS: Setup work directory\r\n2020-03-04 16:12:44 +0100 :SUCCESS: Initialize files.\r\n2020-03-04 16:12:44 +0100 :Working: Copy, extract prerequisite check archive to cells. If required start md11 mismatched partner size correction. Up to 40 minutes ...\r\n2020-03-04 16:12:58 +0100 :INFO : Wait correction of degraded md11 due to md partner size mismatch. Up to 30 minutes.\r\n2020-03-04 16:13:00 +0100 :SUCCESS: Copy, extract prerequisite check archive to cells. If required start md11 mismatched partner size correction.\r\n2020-03-04 16:13:00 +0100 :Working: Check space and state of cell services. Up to 20 minutes ...\r\n2020-03-04 16:14:15 +0100 :SUCCESS: Check space and state of cell services.\r\n2020-03-04 16:14:15 +0100 :Working: Check prerequisites on all cells. Up to 2 minutes ...\r\n2020-03-04 16:14:18 +0100 :SUCCESS: Check prerequisites on all cells.\r\n2020-03-04 16:14:18 +0100 :Working: Copy the patch to all cells. Up to 3 minutes ...\r\n2020-03-04 16:15:36 +0100 :SUCCESS: Copy the patch to all cells.\r\n2020-03-04 16:15:38 +0100 :Working: Execute plugin check for Patch Check Prereq ...\r\n2020-03-04 16:15:38 +0100 :INFO : Patchmgr plugin start: Prereq check for exposure to bug 22909764 v1.0.\r\n2020-03-04 16:15:38 +0100 :INFO : Details in logfile \/radump\/exa_stack\/patch_19.3.2.0.0.191119\/patchmgr.stdout.\r\n2020-03-04 16:15:38 +0100 :INFO : Patchmgr plugin start: Prereq check for exposure to bug 17854520 v1.3.\r\n2020-03-04 16:15:38 +0100 :INFO : Details in logfile \/radump\/exa_stack\/patch_19.3.2.0.0.191119\/patchmgr.stdout.\r\n2020-03-04 16:15:38 +0100 :SUCCESS: No exposure to bug 17854520 with non-rolling patching\r\n2020-03-04 16:15:38 +0100 :INFO : Patchmgr plugin start: Prereq check for exposure to bug 22468216 v1.0.\r\n2020-03-04 16:15:38 +0100 :INFO : Details in logfile \/radump\/exa_stack\/patch_19.3.2.0.0.191119\/patchmgr.stdout.\r\n2020-03-04 16:15:39 +0100 :SUCCESS: Patchmgr plugin complete: Prereq check passed for the bug 22468216\r\n2020-03-04 16:15:39 +0100 :INFO : Patchmgr plugin start: Prereq check for exposure to bug 24625612 v1.0.\r\n2020-03-04 16:15:39 +0100 :INFO : Details in logfile \/radump\/exa_stack\/patch_19.3.2.0.0.191119\/patchmgr.stdout.\r\n2020-03-04 16:15:39 +0100 :SUCCESS: Patchmgr plugin complete: Prereq check passed for the bug 24625612\r\n2020-03-04 16:15:39 +0100 :SUCCESS: No exposure to bug 22896791 with non-rolling patching\r\n2020-03-04 16:15:39 +0100 :INFO : Patchmgr plugin start: Prereq check for exposure to bug 22651315 v1.0.\r\n2020-03-04 16:15:39 +0100 :INFO : Details in logfile \/radump\/exa_stack\/patch_19.3.2.0.0.191119\/patchmgr.stdout.\r\n2020-03-04 16:15:41 +0100 :SUCCESS: Patchmgr plugin complete: Prereq check passed for the bug 22651315\r\n2020-03-04 16:15:42 +0100 :SUCCESS: Execute plugin check for Patch Check Prereq.\r\n2020-03-04 16:15:42 +0100 :Working: check if MS SOFTWAREUPDATE is scheduled. up to 5 minutes...\r\n2020-03-04 16:15:43 +0100 :NO ACTION NEEDED: No cells found with SOFTWAREUPDATE scheduled by MS\r\n2020-03-04 16:15:44 +0100 :SUCCESS: check if MS SOFTWAREUPDATE is scheduled\r\n2020-03-04 16:15:52 +0100 1 of 5 :Working: Initiate patch on cells. Cells will remain up. Up to 5 minutes ...\r\n2020-03-04 16:15:58 +0100 1 of 5 :SUCCESS: Initiate patch on cells.\r\n2020-03-04 16:15:58 +0100 2 of 5 :Working: Waiting to finish pre-reboot patch actions. Cells will remain up. Up to 45 minutes ...\r\n2020-03-04 16:16:58 +0100 :INFO : Wait for patch pre-reboot procedures\r\n2020-03-04 16:18:46 +0100 2 of 5 :SUCCESS: Waiting to finish pre-reboot patch actions.\r\n2020-03-04 16:18:47 +0100 :Working: Execute plugin check for Patching ...\r\n2020-03-04 16:18:47 +0100 :SUCCESS: Execute plugin check for Patching.\r\n2020-03-04 16:18:47 +0100 3 of 5 :Working: Finalize patch on cells. Cells will reboot. Up to 5 minutes ...\r\n2020-03-04 16:18:59 +0100 3 of 5 :SUCCESS: Finalize patch on cells.\r\n2020-03-04 16:19:16 +0100 4 of 5 :Working: Wait for cells to reboot and come online. Up to 120 minutes ...\r\n2020-03-04 16:20:16 +0100 :INFO : Wait for patch finalization and reboot\r\n2020-03-04 16:51:25 +0100 4 of 5 :SUCCESS: Wait for cells to reboot and come online.\r\n2020-03-04 16:51:26 +0100 5 of 5 :Working: Check the state of patch on cells. Up to 5 minutes ...\r\n2020-03-04 16:52:40 +0100 5 of 5 :SUCCESS: Check the state of patch on cells.\r\n2020-03-04 16:52:40 +0100 :Working: Execute plugin check for Pre Disk Activation ...\r\n2020-03-04 16:52:41 +0100 :SUCCESS: Execute plugin check for Pre Disk Activation.\r\n2020-03-04 16:52:41 +0100 :Working: Activate grid disks...\r\n2020-03-04 16:52:42 +0100 :INFO : Wait for checking and activating grid disks\r\n2020-03-04 16:54:27 +0100 :SUCCESS: Activate grid disks.\r\n2020-03-04 16:54:31 +0100 :Working: Execute plugin check for Post Patch ...\r\n2020-03-04 16:54:32 +0100 :SUCCESS: Execute plugin check for Post Patch.\r\n2020-03-04 16:54:33 +0100 :Working: Cleanup\r\n2020-03-04 16:55:08 +0100 :SUCCESS: Cleanup\r\n2020-03-04 16:55:10 +0100 :SUCCESS: Completed run of command: .\/patchmgr -cells \/root\/cell_group -patch\r\n2020-03-04 16:55:10 +0100 :INFO : patch attempted on nodes in file \/root\/cell_group: [zerocell01 zerocell02 zerocell03 zerocell04 zerocell05 zerocell06]\r\n2020-03-04 16:55:10 +0100 :INFO : Current image version on cell(s) is:\r\n2020-03-04 16:55:10 +0100 :INFO : zerocell01: 19.3.2.0.0.191119\r\n2020-03-04 16:55:10 +0100 :INFO : zerocell02: 19.3.2.0.0.191119\r\n2020-03-04 16:55:10 +0100 :INFO : zerocell03: 19.3.2.0.0.191119\r\n2020-03-04 16:55:10 +0100 :INFO : zerocell04: 19.3.2.0.0.191119\r\n2020-03-04 16:55:10 +0100 :INFO : zerocell05: 19.3.2.0.0.191119\r\n2020-03-04 16:55:10 +0100 :INFO : zerocell06: 19.3.2.0.0.191119\r\n2020-03-04 16:55:10 +0100 :INFO : For details, check the following files in \/radump\/exa_stack\/patch_19.3.2.0.0.191119:\r\n2020-03-04 16:55:10 +0100 :INFO : - <cell_name>.log\r\n2020-03-04 16:55:10 +0100 :INFO : - patchmgr.stdout\r\n2020-03-04 16:55:10 +0100 :INFO : - patchmgr.stderr\r\n2020-03-04 16:55:10 +0100 :INFO : - patchmgr.log\r\n2020-03-04 16:55:10 +0100 :INFO : - patchmgr.trc\r\n2020-03-04 16:55:10 +0100 :INFO : Exit status:0\r\n2020-03-04 16:55:10 +0100 :INFO : Exiting.\r\n\r\n[root@zerosita01 patch_19.3.2.0.0.191119]#<\/pre>\n
![\"\"](\"https:\/\/www.fernandosimon.com\/blog\/wp-content\/uploads\/2020\/04\/Exadata-ZDLRA-DB-Patch.png\")
<\/a><\/p>\n