The process of patch ZDLRA is not complicated, but it is important to be aware of some details. The most important is from where you are until where you want to go. This is crucial because it will define what commands you will need to execute.
If you read the previous post about the process, you can notice that I was running the ZDLRA 12.2 version, and forwarded to 19.2 version. In that case, I needed to use the upgrade path since I was changing the major release and the racli commands had the “upgrade” parameter.
In this post I will show how to do a simple update (or patch apply) for ZDLRA, this means that I will remain inside the same major release for recovery appliance library. Some steps and checks are the same.
Whatever you need to do (patch or upgrade), the startup point it is the note 1927416.1 that cover the supported versions for ZDLRA. There it is possible to find all the supported versions for the recovery appliance library as well as the Exadata versions. Please, not upgrade the Exadata stack with a version that is not listed on this page.
Where we are
The first step is always to discover what version for ZDLRA and Exadata Stack you are running. The simple way to do that is using the command “racli version”:
[root@zeroinsg01 ~]# racli version
Recovery Appliance Version:
exadata image: 19.2.3.0.0.190621
rarpm version: ra_automation-19.2.1.1.1.202001-31014797.x86_64
rdbms version: RDBMS_19.3.0.0.190416DBRU_LINUX.X64_RELEASE
transaction : kadjei_bug-31014797
zdlra version: ZDLRA_19.2.1.1.1.202001_LINUX.X64_RELEASE
[root@zeroinsg01 ~]#
The way to read this is that: rpm version is the running version, zdlra version is the base release (202001). The major release is 19.2.1.1.1. So, it is possible to discover that the currently running version for ZDLRA is 19.2.1.1.1.202001-31014797 and the Exadata image is 19.2.3.0.0.190621.
Where we want to go
As I wrote before, the startup point it is the note 1927416.1. There we can check the available versions for ZDLRA, and in this example, I will patch to version 19.2.1.1.1-202003-31304538 (released 18-May-2020).
So, I will jump from 19.2.1.1.1.202001-31014797 to 19.2.1.1.1-202003-31304538, but I will remain inside of the same major release of ZDLRA, the 19.2.1.1.1. I will jump from base release (like PSU for OH) from 202001 (released in January) to 202003 (released in March).
Exadata Supported Versions
Again, checking the note 1927416.1 that covers the supported versions for ZDLRA is possible to check the compatibility matrix for Exadata System Software (database and storage servers) and discover if it is needed to upgrade/patch the Exadata stack.
Look above that the current version for Exadata is supported. So, this means that I can do just the patch apply for ZDLRA. If you want to update the Exadata Stack you can check my post on how to do that, but the process is very, very, similar for Exadata Patch apply.
The only detail is that usually, we update the ZDLRA library first, and after the Exadata Stack. This occurs, (because as you can see in the image above), that the current version that I am running (19.2.1.1.1.202001-31014797) is not compatible with the Exadata 19.3.7.0.0. So, first, update ZDLRA, and after Exadata stack. In this example, I will just update the ZDLRA library.
Patch Process
Download
Since we will continue in the same major release of ZDLRA (19.2.1.1.1), this means that the OH and GI will continue at the same release and will not suffer any kind of version upgrade (like 12 to 19). So, we don’t need to download additional patches or binaries. In my previous post, I already explained this.
As you know, the ZDLRA library contains the updates for the OH and GI, so the compatible PSU’s will be applied and they are inside of downloaded patched:
Where put the files
As I wrote in my previous post, all the files for ZDLRA need to be stored at /radump at the database server node 01. It is fixed in the procedure and it is a requirement.
And as best practices, I recommend that before copy the new files, remove all older patches from /radump in both nodes. And this include files that are inside the ZDLRA patch (like ra_init_param_check.pl, load_init_param.sh, load_init_param.pl, dbmsrsadm.sql, dbmsrsadmpreq.sql, prvtrsadm.sql, ra_preinstall.pl).
01 – Precheck
The first step is to do a simple precheck of the ZDLRA stack. The simple and fast way is executing the “racli status appliance”:
[root@zeroinsg01 ~]# racli status appliance zeroinsg01 crs state: [ONLINE] zeroinsg01 db state: [ONLINE] zeroinsg01 ra_server state: [ONLINE] zeroinsg02 crs state: [ONLINE] zeroinsg02 ra_server state: [ONLINE] zeroinsg02 db state: [ONLINE] [root@zeroinsg01 ~]#
And the “racli run check –all“:
[root@zeroinsg01 ~]# racli run check --all Wed Jun 3 10:07:21 2020: Start: racli run check --all Created log file zeroinsg01:/opt/oracle.RecoveryAppliance/log/racli_run_check_20200603.1007.log Wed Jun 3 10:07:26 2020: CHECK: RA Services - PASS Wed Jun 3 10:07:37 2020: CHECK: Exadata Image Version - PASS Wed Jun 3 10:07:38 2020: CHECK: Active Incidents - PASS Wed Jun 3 10:07:46 2020: CHECK: Init Parameters - PASS Wed Jun 3 10:07:47 2020: CHECK: Invalid Objects - PASS Wed Jun 3 10:07:48 2020: CHECK: Export Backup - PASS Wed Jun 3 10:07:48 2020: CHECK: ZDLRA Rasys Wallet - PASS Wed Jun 3 10:07:51 2020: CHECK: Compute Node AlertHistory Wed Jun 3 10:07:51 2020: HOST: [zeroinsg02] - PASS Wed Jun 3 10:07:51 2020: HOST: [zeroinsg01] - PASS Wed Jun 3 10:07:59 2020: CHECK: Storage Cell AlertHistory Wed Jun 3 10:07:59 2020: HOST: [zerocadm04] - PASS Wed Jun 3 10:07:59 2020: HOST: [zerocadm02] - PASS Wed Jun 3 10:07:59 2020: HOST: [zerocadm06] - PASS Wed Jun 3 10:07:59 2020: HOST: [zerocadm05] - PASS Wed Jun 3 10:07:59 2020: HOST: [zerocadm01] - PASS Wed Jun 3 10:07:59 2020: HOST: [zerocadm03] - PASS Wed Jun 3 10:07:59 2020: CHECK: Oracle User Password Expires Wed Jun 3 10:07:59 2020: HOST: [zeroinsg01] - PASS Wed Jun 3 10:07:59 2020: HOST: [zeroinsg02] - PASS Wed Jun 3 10:08:00 2020: CHECK: ZDLRA Version Wed Jun 3 10:08:00 2020: HOST: [zeroinsg02] - PASS Wed Jun 3 10:08:00 2020: HOST: [zeroinsg01] - PASS Wed Jun 3 10:08:00 2020: End: racli run check --all [root@zeroinsg01 ~]#
With that, every error can be checked, reported, and fixed before continuing the process.
02 – Unzip Library
The second step is, as oracle user, unzip the ZDLRA patch. This is done inside of /radump folder at node 01:
[oracle@zeroinsg01 radump]$ unzip p31304538_192111_Linux-x86-64.zip Archive: p31304538_192111_Linux-x86-64.zip inflating: set_env.sh extracting: p6880880_180000_Linux-x86-64.zip inflating: dbmsrsadmpreq.sql inflating: dbmsrsadm.sql inflating: prvtrsadm.sql inflating: create_raoratab.pl inflating: create_raoratab.sh inflating: get_versions.pl inflating: get_versions.sh inflating: ra_init_param_check.pl inflating: ra_init_param_check.sh inflating: ra_precheck.pl inflating: ra_precheck.sh inflating: ra_preinstall.pl inflating: ra_automation-19.2.1.1.1.202003-31304538.x86_64.rpm inflating: run_set_env.sh extracting: p29908639_193000DBRU_Linux-x86-64.zip extracting: p29232533_190000_Linux-x86-64.zip extracting: p29726449_190000_Linux-x86-64.zip extracting: p29391849_194000DBRU_Linux-x86-64.zip extracting: p30177140_194000DBRU_Linux-x86-64.zip extracting: p30540969_190000_Linux-x86-64.zip extracting: p30312546_190000_Linux-x86-64.zip extracting: p30458593_190000_Linux-x86-64.zip extracting: p30363621_190000_Linux-x86-64.zip extracting: p28538439_190000_Linux-x86-64.zip extracting: p30094929_190000_Linux-x86-64.zip extracting: p29708769_190000_Linux-x86-64.zip inflating: README.txt [oracle@zeroinsg01 radump]$
As you can see the patch p31304538_192111_Linux-x86-64.zip contains inside the PSU/One-offs (p* files) patches for OH and GI.
03 – ra_preinstall.pl
The third is step is to execute the ra_preinstall.pl, as root user, to start the process. This step removes the old ZDLRA rpm library from the system and installs the new one (in both nodes). Besides that, execute some checks as well.
As explained in my previous post, if you have some open incidents you can clean it (the process is described there), but also is not recommended to restart this ZDLRA after this moment because the ZDLRA binaries were changed (rpm was changed) but the database itself was not upgraded. Of course, that library supposes to be/have interoperability, but if it is possible to avoid the problem, it is desired.
[root@zeroinsg01 ~]# cd /radump/
[root@zeroinsg01 radump]#
[root@zeroinsg01 radump]# which perl
/opt/oracle.RecoveryAppliance/bin/perl
[root@zeroinsg01 radump]#
[root@zeroinsg01 radump]# perl ra_preinstall.pl
Start: Running ra_preinstall.pl on zeroinsg01.
NOTE:
Current deployed RPM [ra_automation-19.2.1.1.1.202001-31014797.x86_64.rpm] not found!
If you continue without an old RPM, rollback will not be possible.
Refer to the README.txt included in this ZDLRA Patch for more details.
Do you want to continue? (y|n): y
Note:
The ra_preinstall.pl manages the ra_automation RPM,
and provides a --rollback option.
The RPM is updated during ra preinstall. Rollback is feasible if the old rpm is found.
You do not need to update RPM separately.
Refer to the README.txt included in this ZDLRA Patch for more details.
Do you want to continue (y|n): y
Deployed RPM: ZDLRA_19.2.1.1.1.202001_LINUX.X64_RELEASE
Installed RPM: ZDLRA_19.2.1.1.1.202001_LINUX.X64_RELEASE
RPM matches.
Fuse group already exists. Skipping.
Start Update sshd_config
End Update sshd_config
End: Running ra_preinstall.pl on zeroinsg01.
Copying ra_preinstall.pl, create_raoratab.* and new RPM to remote node zeroinsg02. Password may be required to connect.
ra_preinstall.pl 100% 42KB 62.5MB/s 00:00
create_raoratab.pl 100% 3251 13.7MB/s 00:00
create_raoratab.sh 100% 991 5.5MB/s 00:00
run_set_env.sh 100% 1000 4.4MB/s 00:00
set_env.sh 100% 2420 13.5MB/s 00:00
ra_automation-19.2.1.1.1.202003-31304538.x86_64.rpm 100% 1903MB 227.2MB/s 00:08
Created log /opt/oracle.RecoveryAppliance/log/create_raoratab.log
Start: Running ra_preinstall.pl on zeroinsg02.
Fuse group already exists. Skipping.
Start Update sshd_config
End Update sshd_config
End: Running ra_preinstall.pl on zeroinsg02.
Start Restart sshd
End Restart sshd
Start: Check Init Parameters
Created log file /opt/oracle.RecoveryAppliance/log/ra_init_param_check.log
All init parameters have been validated.
End: Check Init Parameters
Start: Remove current RPM and install new RPM on zeroinsg01.
End: Remove current RPM and install new RPM on zeroinsg01.
Start: Remove current RPM and install new RPM on zeroinsg02.
End: Remove current RPM and install new RPM on zeroinsg02.
Created log /opt/oracle.RecoveryAppliance/log/raprecheck.log
Start: Run Pre check for Patch
Wed Jun 3 10:10:34 2020: Start: Check ZDLRA Services
Wed Jun 3 10:10:38 2020: End: Check ZDLRA Services
Wed Jun 3 10:10:38 2020: Start: Check ASM rebalance
Wed Jun 3 10:10:39 2020: End: Check ASM rebalance
Wed Jun 3 10:10:39 2020: Start: Check Cluster
Wed Jun 3 10:11:48 2020: End: Check Cluster
Wed Jun 3 10:11:53 2020: Start: Check Open Incidents
Wed Jun 3 10:11:54 2020: End: Check Open Incidents
Wed Jun 3 10:11:54 2020: Start: Check Invalid Objects
Wed Jun 3 10:11:58 2020: End: Check Invalid Objects
Wed Jun 3 10:11:58 2020: Start: Check Init Parameters
Wed Jun 3 10:11:59 2020: End: Check Init Parameters
Wed Jun 3 10:11:59 2020: Start: Check compute node oracle access
Wed Jun 3 10:11:59 2020: End: Check compute node oracle access
Wed Jun 3 10:11:59 2020: Start: Check TFA/AHF status
Wed Jun 3 10:12:03 2020: End: Check TFA/AHF status
End: Run Pre check for Patch
Start Restart sshd
!!!NOTE!!!
Exit and log back into [zeroinsg01] prior to continuing
!!!NOTE!!!
End Restart sshd
[root@zeroinsg01 radump]#
Last login: Wed Jun 3 10:12:30 2020 from
[root@zeroinsg01 ~]#
I always check where Perl is coming, be sure that it is from the ZDLRA folder. The warning about rpm file not found means that the current rpm file was not found at /radump (probably cleaned in some moment), but since I have it in local NFS, I can just copy it if needed.
04 – patch appliance –step=1
After the rpm is changed in both nodes, we can call the racli to do continue with the patch. The first step is just a precheck phase. This is executed as root:
[root@zeroinsg01 ~]# /opt/oracle.RecoveryAppliance/bin/racli patch appliance --step=1 Created log /opt/oracle.RecoveryAppliance/log/racli_patch_appliance.log Wed Jun 3 10:13:36 2020: Start: Patch Recovery Appliance - Step [PreCheck] Wed Jun 3 10:13:36 2020: Start: Check ZDLRA Services Wed Jun 3 10:13:40 2020: End: Check ZDLRA Services Wed Jun 3 10:13:40 2020: Start: Check ASM rebalance Wed Jun 3 10:13:41 2020: End: Check ASM rebalance Wed Jun 3 10:13:41 2020: Start: Check Cluster Wed Jun 3 10:14:57 2020: End: Check Cluster Wed Jun 3 10:15:00 2020: Start: Check Open Incidents Wed Jun 3 10:15:01 2020: End: Check Open Incidents Wed Jun 3 10:15:01 2020: Start: Check Invalid Objects Wed Jun 3 10:15:05 2020: End: Check Invalid Objects Wed Jun 3 10:15:05 2020: Start: Check Init Parameters Wed Jun 3 10:15:07 2020: End: Check Init Parameters Wed Jun 3 10:15:07 2020: Start: Check compute node oracle access Wed Jun 3 10:15:07 2020: End: Check compute node oracle access Wed Jun 3 10:15:07 2020: Start: Check TFA/AHF status Wed Jun 3 10:15:10 2020: End: Check TFA/AHF status Wed Jun 3 10:15:13 2020: Start: Check RA Patch Levels Wed Jun 3 10:15:13 2020: End: Check RA Patch Levels Wed Jun 3 10:15:13 2020: Start: Check Patches Wed Jun 3 10:15:25 2020: End: Check Patches Wed Jun 3 10:15:25 2020: End: Patch Recovery Appliance - Step [PreCheck] [root@zeroinsg01 ~]#
This check if some errors exist inside ASM or ZDLRA database metadata.
As you can above, the command racli now uses the parameter “patch appliance”. In my previous posts, I used “upgrade appliance”. The patch, as explained, is used when we are inside of the same major release.
05 – patch appliance –step=2
The next step is the patch itself, the racli called as root will apply all patches needed.
[root@zeroinsg01 ~]# /opt/oracle.RecoveryAppliance/bin/racli patch appliance --step=2 Created log /opt/oracle.RecoveryAppliance/log/racli_patch_appliance.log Wed Jun 3 10:17:46 2020: Start: Patch Recovery Appliance - Step [Patch] Wed Jun 3 10:17:46 2020: Skip: Patch Recovery Appliance - Step [PreCheck] [Already Run] Wed Jun 3 10:17:48 2020: Start: Pre Patch Release Steps Wed Jun 3 10:17:48 2020: End: Pre Patch Release Steps Wed Jun 3 10:17:48 2020: Start: Unpack Patches Wed Jun 3 10:19:24 2020: End: Unpack Patches Wed Jun 3 10:19:24 2020: Start: Rasys configuration Wed Jun 3 10:19:27 2020: End: Rasys configuration Wed Jun 3 10:19:27 2020: Start: Stop ZDLRA Services Wed Jun 3 10:23:26 2020: End: Stop ZDLRA Services Wed Jun 3 10:23:26 2020: Start: Disable ZDLRA Exports Wed Jun 3 10:23:26 2020: End: Disable ZDLRA Exports Wed Jun 3 10:23:26 2020: Start: Check Active Programs Wed Jun 3 10:23:57 2020: Start: Stop Clusterware Wed Jun 3 10:24:34 2020: End: Stop Clusterware Wed Jun 3 10:24:34 2020: Start: Checking active programs Wed Jun 3 10:24:46 2020: End: Checking active programs Wed Jun 3 10:24:46 2020: Start: Start Clusterware Wed Jun 3 10:30:27 2020: End: Start Clusterware Wed Jun 3 10:30:27 2020: End: Check Active Programs Wed Jun 3 10:30:27 2020: Start: Patch OPatch Wed Jun 3 10:30:29 2020: End: Patch OPatch Wed Jun 3 10:30:29 2020: Start: CRS start Wed Jun 3 10:32:56 2020: End: CRS start Wed Jun 3 10:32:56 2020: Start: Installed Patch Check Wed Jun 3 10:35:40 2020: End: Installed Patch Check Wed Jun 3 10:35:40 2020: Start: Apply Bundle Patch Wed Jun 3 10:36:52 2020: Skipped: Bundle Patch is Current. Wed Jun 3 10:36:52 2020: End: Apply Bundle Patch Wed Jun 3 10:36:53 2020: End: Apply Bundle Patch Wed Jun 3 10:36:53 2020: Start: Apply GI Patches Wed Jun 3 10:54:56 2020: End: Apply GI Patches Wed Jun 3 10:54:56 2020: Start: Install ZDLRA Patch Wed Jun 3 11:00:16 2020: End: Install ZDLRA Patch Wed Jun 3 11:00:16 2020: Start: Apply Datapatch Wed Jun 3 11:00:37 2020: End: Apply Datapatch Wed Jun 3 11:00:37 2020: Start: Rasys Update Wed Jun 3 11:02:37 2020: End: Rasys Update Wed Jun 3 11:02:37 2020: Start: ZDLRA OS Updates Wed Jun 3 11:02:38 2020: End: ZDLRA OS Updates Wed Jun 3 11:02:38 2020: Start: Exachk Update Wed Jun 3 11:02:40 2020: End: Exachk Update Wed Jun 3 11:02:40 2020: Start: AHF Update Wed Jun 3 11:09:24 2020: End: AHF Update Wed Jun 3 11:09:28 2020: Start: Secure Backup Update Wed Jun 3 11:09:28 2020: Start: Tape Update Wed Jun 3 11:09:58 2020: End: Tape Update Wed Jun 3 11:09:58 2020: End: Secure Backup Update Wed Jun 3 11:09:58 2020: Start: Post Patch Service Enable Wed Jun 3 11:10:01 2020: Start: Stop ZDLRA Stack Wed Jun 3 11:13:36 2020: End: Stop ZDLRA Stack Wed Jun 3 11:13:36 2020: Start: Enable ZDLRA Wed Jun 3 11:13:37 2020: End: Enable ZDLRA Wed Jun 3 11:18:08 2020: Start: Enable RA Backup Wed Jun 3 11:18:08 2020: End: Enable RA Backup Wed Jun 3 11:18:08 2020: End: Post Patch Service Enable Wed Jun 3 11:18:08 2020: Start: Reset RA user password Wed Jun 3 11:18:09 2020: End: Reset RA user password Wed Jun 3 11:18:09 2020: Start: Check for EM Agents Wed Jun 3 11:19:00 2020: End: Check for EM Agents Wed Jun 3 11:19:00 2020: Start: Check Init Parameters Wed Jun 3 11:19:00 2020: End: Check Init Parameters Wed Jun 3 11:19:00 2020: Start: Setup OS Wed Jun 3 11:19:35 2020: End: Setup OS Wed Jun 3 11:19:35 2020: Start: Post Patch Clean Up Wed Jun 3 11:34:14 2020: End: Post Patch Clean Up Wed Jun 3 11:34:14 2020: End: Patch Recovery Appliance - Step [Patch] [root@zeroinsg01 ~]#
The patch will apply everything that is needed: OH Patches, GI Patches, OSB, and ZDLRA stack. In this process, the ZDLRA will be “offline” and not receive ingested backups. I recommend not even try to send backups (or archivelogs) to ZDLRA to avoid concurrencies inside rman catalog (since the patch process touches it too).
You can see below some parts of the patch apply process:
Wed Jun 3 10:17:48 2020: Got /u01/app/oracle/product/19.0.0.0/dbhome_1 patch status Wed Jun 3 10:17:48 2020: Checking for 29517242. Wed Jun 3 10:17:48 2020: Found 29517242. Wed Jun 3 10:17:48 2020: Start: Pre Patch Release Steps Wed Jun 3 10:17:48 2020: End: RunLevel 1001 Wed Jun 3 10:17:48 2020: Start: RunLevel 1002 Wed Jun 3 10:17:48 2020: End: Pre Patch Release Steps Wed Jun 3 10:17:48 2020: Start: Unpack Patches Wed Jun 3 10:17:48 2020: Start: Unpack Patches Wed Jun 3 10:17:48 2020: Checking for patch [29726449] Wed Jun 3 10:17:48 2020: Found patch [29726449]. Checking 29726449. Wed Jun 3 10:17:48 2020: Patch [29726449] md5sum matches. Unpacking 29726449. Wed Jun 3 10:17:48 2020: Switching to UID: 1001, GID: 1002 Wed Jun 3 10:17:48 2020: Set Command '/usr/bin/unzip -nuq /radump//p29726449_190000_Linux-x86-64.zip -d //radump//patch//general' timeout to 900. Wed Jun 3 10:17:48 2020: Unpacking 29726449 complete. OK Wed Jun 3 10:17:48 2020: Checking for patch [29232533] Wed Jun 3 10:17:48 2020: Found patch [29232533]. Checking 29232533. Wed Jun 3 10:17:48 2020: Patch [29232533] md5sum matches. Unpacking 29232533. … … Wed Jun 3 10:24:00 2020: spawn /usr/bin/ssh -o ConnectTimeout=20 -o LogLevel=error -l root zeroinsg02 /u01/app/19.0.0.0/grid/bin/crsctl stop crs -f … … Wed Jun 3 10:33:16 2020: Found 28279612. Wed Jun 3 10:33:16 2020: Removing 28279612 Wed Jun 3 10:33:16 2020: Set Command '/usr/bin/expect ' timeout to 305. Wed Jun 3 10:33:43 2020: spawn /usr/bin/ssh -o ConnectTimeout=20 -o LogLevel=error -l oracle zeroinsg02 /u01/app/oracle/product/19.0.0.0/dbhome_1/OPatch/opatch rollback -id 28279612 -local_node -no_relink -oh /u01/app/oracle/product/19.0.0.0/dbhome_1 -silent Oracle Interim Patch Installer version 12.2.0.1.17 ... ... Wed Jun 3 10:36:52 2020: spawn /usr/bin/ssh -o ConnectTimeout=20 -o LogLevel=error -l root zeroinsg01 /u01/app/19.0.0.0/grid/OPatch/opatchauto apply //radump//patch//gi_opatchauto/29708769 -oh /u01/app/19.0.0.0/grid OPatchauto session is initiated at Wed Jun 3 10:36:21 2020 … … =========================================================== install-zdlra-patch starting =========================================================== Processing in install mode. Source root is /opt/oracle.RecoveryAppliance/zdlra Destination root is /u01/app/oracle/product/19.0.0.0/dbhome_1/rdbms Backout root is /u01/app/oracle/product/19.0.0.0/dbhome_1/rdbms/install/zdlra/backout_0001 Destination appears to be an installed shiphome. library = libzdlraserver19.a Performing object copies for libserver19.a. … … Processing install/zdlra files. Copying /opt/oracle.RecoveryAppliance/zdlra/install/zdlra/zdlra-software-id to /u01/app/oracle/product/19.0.0.0/dbhome_1/rdbms/install/zdlra/zdlra-software-id … … Wed Jun 3 11:00:39 2020: Start: Update Sys Package. Wed Jun 3 11:00:39 2020: Switching to UID: 1001, GID: 1002 Wed Jun 3 11:00:39 2020: Set Command '/u01/app/oracle/product/19.0.0.0/dbhome_1/bin/sqlplus -s / AS SYSDBA <<EOF … … Granting VPD privileges to the owner of the base catalog schema RASYS ======================================== VPD SETUP STATUS: VPD privileges granted successfully! Connect to RMAN base catalog and perform UPGRADE CATALOG. … … Wed Jun 3 11:00:51 2020: Set Command '/u01/app/oracle/product/19.0.0.0/dbhome_1/bin/rman catalog /@install.local cmdfile=/opt/oracle.RecoveryAppliance/install/upgrade.rman' timeout to 0. … … Wed Jun 3 11:18:08 2020: Start: Enable RA Backup Wed Jun 3 11:18:08 2020: Create Symlink succeeded on zeroinsg02. Wed Jun 3 11:18:08 2020: Create Symlink succeeded on zeroinsg01. Wed Jun 3 11:18:08 2020: ra_export.pl successfully enabled. … … Wed Jun 3 11:34:14 2020: End: Post Patch Clean Up Wed Jun 3 11:34:14 2020: Deleted restart flag /opt/oracle.RecoveryAppliance/data/racli_patch_appliance.restart Wed Jun 3 11:34:14 2020: End: Patch Recovery Appliance - Step [Patch]
As you can see, everything is done by racli, since GI patches, OH metadata, and ZDLRA patches.
06 – Post Patch
As a post patch, we can check again the appliance and verify if everything is running correctly:
[root@zeroinsg01 ~]# racli version
Recovery Appliance Version:
exadata image: 19.2.3.0.0.190621
rarpm version: ra_automation-19.2.1.1.1.202003-31304538.x86_64
rdbms version: RDBMS_19.3.0.0.190416DBRU_LINUX.X64_RELEASE
transaction : kadjei_bug-31304538
zdlra version: ZDLRA_19.2.1.1.1.202003_LINUX.X64_RELEASE
[root@zeroinsg01 ~]# racli status appliance
zeroinsg01 db state: [ONLINE]
zeroinsg01 crs state: [ONLINE]
zeroinsg01 ra_server state: [ONLINE]
zeroinsg02 ra_server state: [ONLINE]
zeroinsg02 db state: [ONLINE]
zeroinsg02 crs state: [ONLINE]
[root@zeroinsg01 ~]#
[root@zeroinsg01 ~]#
[root@zeroinsg01 ~]# racli run check --all
Wed Jun 3 12:16:18 2020: Start: racli run check --all
Created log file zeroinsg01:/opt/oracle.RecoveryAppliance/log/racli_run_check_20200603.1216.log
Wed Jun 3 12:16:21 2020: CHECK: RA Services - PASS
Wed Jun 3 12:16:31 2020: CHECK: Exadata Image Version - PASS
Wed Jun 3 12:16:32 2020: CHECK: Active Incidents - PASS
Wed Jun 3 12:16:40 2020: CHECK: Init Parameters - PASS
Wed Jun 3 12:16:41 2020: CHECK: Invalid Objects - PASS
Wed Jun 3 12:16:41 2020: CHECK: Export Backup - PASS
Wed Jun 3 12:16:42 2020: CHECK: ZDLRA Rasys Wallet - PASS
Wed Jun 3 12:16:46 2020: CHECK: Compute Node AlertHistory
Wed Jun 3 12:16:46 2020: HOST: [zeroinsg02] - PASS
Wed Jun 3 12:16:46 2020: HOST: [zeroinsg01] - PASS
Wed Jun 3 12:16:53 2020: CHECK: Storage Cell AlertHistory
Wed Jun 3 12:16:53 2020: HOST: [zerocadm02] - PASS
Wed Jun 3 12:16:53 2020: HOST: [zerocadm05] - PASS
Wed Jun 3 12:16:53 2020: HOST: [zerocadm01] - PASS
Wed Jun 3 12:16:53 2020: HOST: [zerocadm04] - PASS
Wed Jun 3 12:16:53 2020: HOST: [zerocadm06] - PASS
Wed Jun 3 12:16:53 2020: HOST: [zerocadm03] - PASS
Wed Jun 3 12:16:53 2020: CHECK: Oracle User Password Expires
Wed Jun 3 12:16:53 2020: HOST: [zeroinsg01] - PASS
Wed Jun 3 12:16:53 2020: HOST: [zeroinsg02] - PASS
Wed Jun 3 12:16:54 2020: CHECK: ZDLRA Version
Wed Jun 3 12:16:54 2020: HOST: [zeroinsg01] - PASS
Wed Jun 3 12:16:54 2020: HOST: [zeroinsg02] - PASS
Wed Jun 3 12:16:54 2020: End: racli run check --all
[root@zeroinsg01 ~]#
If you faced whatever the error, open one SR against Oracle at MOS to receive the correct feedback and fix. Never tries to execute some commands to “fix” because you can corrupt the database and lost data, backups, and your job.
Upgrade and Replication
Since ZDLRA can be used in replicated mode, where the upstream send to downstream, it is important that whoever receives the backup can do that. This means that it is recommended to update always the downstream first. This allows that when receiving the backups from upstream, the internal ZDLRA database and the library can handle it.
Conclusion
The process to update is not complicated, but some details need to be checked before start the process. The patch process is different than the upgrade that I described at my previous post since the major release continued the same. It can be simpler, but the details are very, very, similar.
References
Some references about how to do the patch and handle the issues
- Zero Data Loss Recovery Appliance Supported Versions (Doc ID 1927416.1)
- ZDLRA Release and Patching Policy (Doc ID 2410137.1)
- ZDLRA Upgrade and Patching Troubleshooting Guide (Doc ID 2639262.1)
- Zero Data Loss Recovery Appliance Upgrade and Patching (Doc ID 2028931.1)
- ZDLRA Detailed Troubleshooting Methodology (Doc ID 2408256.1)
Disclaimer: “The postings on this site are my own and don’t necessarily represent my actual employer positions, strategies or opinions. The information here was edited to be useful for general purpose, specific data and identifications were removed to allow reach the generic audience and to be useful for the community. Post protected by copyright.”
Pingback: ZDLRA patching | Amardeep Sidhu