ZDLRA, Protecting Databases with Replication

The replication for ZDLRA works differently than normal DataGuard, but you can reach almost the same level of multiple site protection with that. The replication for ZDLRA is not complicated but can be divided into several steps. Basically, to protect a database (since you have everything configure) is done linking the database with the protection policy that is replicated.

In my previous posts, I already wrote about all the steps to reach this configuration. Starting with an explanation about replication, how to configure the replication network between ZDLRA’s, how to configure the replication server,  and how to create the replication config (that links everything is done before).

But most of the time we don’t need to pass through all of these steps. Usually, the ZDLRA is deployed with the replication network already configured, or you already deploy two ZDLRA’s that will operate replicated. This part I consider the “physical” part of the configuration because evolves network and details that we usually don’t touch after configured. The “logical” part comes after and evolves all the definitions about what policies will be replicated, which databases will be part of each policy, and so on. This “logical” configuration I explained in this previous post.

But it is important to know how it’s working to understand all the details. And if you need, you can check my posts about the replication for ZDLRA.

In this post, I will show more details on how the ZDLRA replication impact over the backup for your database, and show the protection occurs to reduce your RPO to a minimum.

Replication, Policies, and Databases

A little recap. The most important to understand about the replication is that is done based on protection policies. ZDLRA does not replicate database per database, it replicates policies. So, every database that is linked with the policy will be replicated.

You can have more than one protection policy that is replicated between the same ZDLRA’s, and this link directly with the MAA Architecture where you have more than one SLA that needs to be sustained (like recovery window requirements as an example). And the caveat it is that, maybe, you need to double part of your policies because it is possible that you don’t want to replicate all databases for that policy. So, you will create another policy and drop all databases to be replicated there. But remember that when you do that, you need to check again your clone to tapes because they can be linked with policies (and not with backups tags of another configuration). And if this occurs, you will need to reorganize your clone to the tape schedule.

It is important to also understand that replication is done as soon as possible. So, after every backup that is ingested at upstream ZDLRA (and after it is validated), it is replicated ASAP. I already explained this in previous posts, but the full backup it is not replicated, just the incremental and the archivelogs are. Think that your datafile has more than 20TB, you will wait until the virtual full is generated and replicate after that? If you do this you lose protection because, in case of an entire datacenter failure, your backup is not replicate in a secure environment.  And since the virtual full backup will be created (at downstream) it will be equal to that one that was created at upstream.

Base Config

The base config is the same as I create in my previous post. I have the ORCL19 database that is linked with the protection policy ZDLRAUP_BRONZE trough a replication server called ZDLRAS2_REP.

Replicated Protection

To use the replication we just need to do a backup of the database. So, below you can see a simple backup for datafile 1. This backup is the first backup that I am doing for the database at this ZDLRA, so it is a level 0 backup.

First, look that I connected at upstream ZDLRA (ZDLRAS1 in this scenario).

[oracle@orcloel7 ~]$ rman target=/ catalog=vpcsrc/vpcsrc@zdlras1-scan:1521/zdlras1

Recovery Manager: Release 19.0.0.0.0 - Production on Thu Jul 9 22:54:55 2020
Version 19.3.0.0.0

Copyright (c) 1982, 2019, Oracle and/or its affiliates.  All rights reserved.

connected to target database: ORCL19 (DBID=323177095)
connected to recovery catalog database

RMAN>

And executed the first backup of datafile 1:

RMAN> BACKUP INCREMENTAL LEVEL 0 DEVICE TYPE SBT FILESPERSET 1 DATAFILE 1 TAG 'BKP-DB-INCR0-REP';

Starting backup at 09/07/2020 23:31:06
released channel: ORA_DISK_1
allocated channel: ORA_SBT_TAPE_1
channel ORA_SBT_TAPE_1: SID=37 device type=SBT_TAPE
channel ORA_SBT_TAPE_1: RA Library (ZDLRAS1) SID=AA098857038E149DE053010310AC801E
channel ORA_SBT_TAPE_1: starting incremental level 0 datafile backup set
channel ORA_SBT_TAPE_1: specifying datafile(s) in backup set
input datafile file number=00001 name=/u01/app/oracle/oradata/ORCL19/system01.dbf
channel ORA_SBT_TAPE_1: starting piece 1 at 09/07/2020 23:31:09
channel ORA_SBT_TAPE_1: finished piece 1 at 09/07/2020 23:33:24
piece handle=ORCL19_3fv4tjdt_1_1 tag=BKP-DB-INCR0-REP comment=API Version 2.0,MMS Version 12.2.0.2
channel ORA_SBT_TAPE_1: backup set complete, elapsed time: 00:02:15
Finished backup at 09/07/2020 23:33:24

Starting Control File and SPFILE Autobackup at 09/07/2020 23:33:24
piece handle=c-323177095-20200709-02 comment=API Version 2.0,MMS Version 12.2.0.2
Finished Control File and SPFILE Autobackup at 09/07/2020 23:33:41

RMAN>

And as you can see here, it was replicated:

RMAN> list backup of datafile 1 tag = 'BKP-DB-INCR0-REP';


List of Backup Sets
===================


BS Key  Type LV Size
------- ---- -- ----------
25797   Incr 0  325.02M
  List of Datafiles in backup set 25797
  File LV Type Ckp SCN    Ckp Time            Abs Fuz SCN Sparse Name
  ---- -- ---- ---------- ------------------- ----------- ------ ----
  1    0  Incr 2414591    09/07/2020 23:31:09              NO    /u01/app/oracle/oradata/ORCL19/system01.dbf

  Backup Set Copy #1 of backup set 25797
  Device Type Elapsed Time Completion Time     Compressed Tag
  ----------- ------------ ------------------- ---------- ---
  SBT_TAPE    00:04:57     09/07/2020 23:36:11 YES        BKP-DB-INCR0-REP

    List of Backup Pieces for backup set 25797 Copy #1
    BP Key  Pc# Status      Media                   Piece Name
    ------- --- ----------- ----------------------- ----------
    25798   1   AVAILABLE                           VB$_1891149551_25782I

  Backup Set Copy #2 of backup set 25797
  Device Type Elapsed Time Completion Time     Compressed Tag
  ----------- ------------ ------------------- ---------- ---
  SBT_TAPE    00:04:57     09/07/2020 23:36:06 YES        BKP-DB-INCR0-REP

    List of Backup Pieces for backup set 25797 Copy #2
    BP Key  Pc# Status      Media                   Piece Name
    ------- --- ----------- ----------------------- ----------
    25811   1   AVAILABLE   ZDLRAS2_REP             VB$_2127575003_12535I

RMAN>

Some details here:

  • The replication occurred at the backupset level.
  • The backupset has two backuppiesces.
  • The copy without “Media” information is local (at upstream), the “Media” for replicate will show the same name as the server.
  • The TAG is the same for every copy.
  • Completion time will be different for each ZDLRA.
  • All the integration at the ZDLRA catalog was made automatically by ZDLRA. Like registration of additional copies.

Inside of ZDLRA, the replication occurs based in “tasks” for each backupset and you can follow at rasys.ra_task table:

SQL> select TASK_ID, TASK_TYPE, STATE, WAITING_ON, DB_KEY, DB_UNIQUE_NAME, CREATION_TIME, ERROR_COUNT, INTERRUPT_COUNT, BP_KEY,BS_KEY,DF_KEY,VB_KEY from rasys.ra_task where archived = 'N' order by 5,2,7,10,11,12,13;

   TASK_ID TASK_TYPE                      STATE                     WAITING_ON     DB_KEY DB_UNIQUE_NAME                 CREATION_TIME                       ERROR_COUNT INTERRUPT_COUNT     BP_KEY     BS_KEY     DF_KEY     VB_KEY
---------- ------------------------------ ------------------------- ---------- ---------- ------------------------------ ----------------------------------- ----------- --------------- ---------- ---------- ---------- ----------
     42384 BACKUP_SBT                     RUNNING                                   25140 ORCL19                         09-JUL-20 11.33.16.448006 PM +02:00           0               0                 25667
     42386 BACKUP_SBT                     RUNNING                                   25140 ORCL19                         09-JUL-20 11.33.16.999633 PM +02:00           0               0                 25732
     42390 BACKUP_SBT                     RUNNING                                   25140 ORCL19                         09-JUL-20 11.33.35.585173 PM +02:00           0               0                 25761
     42389 INDEX_BACKUP                   RUNNING                                   25140 ORCL19                         09-JUL-20 11.33.17.548351 PM +02:00           0               0      25733

SQL>

And doing a subsequent level 1 of datafile 1:

RMAN>  backup incremental level 1 datafile 1 tag = 'BKP-DB-INCR1-REP';

Starting backup at 09/07/2020 23:52:10
allocated channel: ORA_SBT_TAPE_1
channel ORA_SBT_TAPE_1: SID=37 device type=SBT_TAPE
channel ORA_SBT_TAPE_1: RA Library (ZDLRAS1) SID=AA0988570390149DE053010310AC801E
channel ORA_SBT_TAPE_1: starting incremental level 1 datafile backup set
channel ORA_SBT_TAPE_1: specifying datafile(s) in backup set
input datafile file number=00001 name=/u01/app/oracle/oradata/ORCL19/system01.dbf
channel ORA_SBT_TAPE_1: starting piece 1 at 09/07/2020 23:52:11
channel ORA_SBT_TAPE_1: finished piece 1 at 09/07/2020 23:52:14
piece handle=ORCL19_3lv4tklb_1_1 tag=BKP-DB-INCR1-REP comment=API Version 2.0,MMS Version 12.2.0.2
channel ORA_SBT_TAPE_1: backup set complete, elapsed time: 00:00:03
Finished backup at 09/07/2020 23:52:14

Starting Control File and SPFILE Autobackup at 09/07/2020 23:52:14
piece handle=c-323177095-20200709-05 comment=API Version 2.0,MMS Version 12.2.0.2
Finished Control File and SPFILE Autobackup at 09/07/2020 23:52:22

RMAN>

I will have both replicated. The level 1, and the virtual full backup:

RMAN> list backup tag = 'BKP-DB-INCR1-REP';


List of Backup Sets
===================


BS Key  Type LV Size
------- ---- -- ----------
26088   Incr 1  48.00K
  List of Datafiles in backup set 26088
  File LV Type Ckp SCN    Ckp Time            Abs Fuz SCN Sparse Name
  ---- -- ---- ---------- ------------------- ----------- ------ ----
  1    1  Incr 2417536    09/07/2020 23:52:11              NO    /u01/app/oracle/oradata/ORCL19/system01.dbf

  Backup Set Copy #1 of backup set 26088
  Device Type Elapsed Time Completion Time     Compressed Tag
  ----------- ------------ ------------------- ---------- ---
  SBT_TAPE    00:06:06     09/07/2020 23:52:12 YES        BKP-DB-INCR1-REP

    List of Backup Pieces for backup set 26088 Copy #1
    BP Key  Pc# Status      Media                   Piece Name
    ------- --- ----------- ----------------------- ----------
    26089   1   AVAILABLE                           VB$_1891149551_26087I

  Backup Set Copy #3 of backup set 26088
  Device Type Elapsed Time Completion Time     Compressed Tag
  ----------- ------------ ------------------- ---------- ---
  SBT_TAPE    00:06:06     09/07/2020 23:58:17 YES        BKP-DB-INCR1-REP

    List of Backup Pieces for backup set 26088 Copy #3
    BP Key  Pc# Status      Media                   Piece Name
    ------- --- ----------- ----------------------- ----------
    26136   1   AVAILABLE   ZDLRAS2_REP             VB$_2127575003_12565I

BS Key  Type LV Size
------- ---- -- ----------
26092   Incr 0  323.90M
  List of Datafiles in backup set 26092
  File LV Type Ckp SCN    Ckp Time            Abs Fuz SCN Sparse Name
  ---- -- ---- ---------- ------------------- ----------- ------ ----
  1    0  Incr 2417536    09/07/2020 23:52:11              NO    /u01/app/oracle/oradata/ORCL19/system01.dbf

  Backup Set Copy #1 of backup set 26092
  Device Type Elapsed Time Completion Time     Compressed Tag
  ----------- ------------ ------------------- ---------- ---
  SBT_TAPE    00:06:06     09/07/2020 23:52:12 YES        BKP-DB-INCR1-REP

    List of Backup Pieces for backup set 26092 Copy #1
    BP Key  Pc# Status      Media                   Piece Name
    ------- --- ----------- ----------------------- ----------
    26093   1   AVAILABLE                           VB$_1891149551_26087_1

  Backup Set Copy #2 of backup set 26092
  Device Type Elapsed Time Completion Time     Compressed Tag
  ----------- ------------ ------------------- ---------- ---
  SBT_TAPE    00:06:06     09/07/2020 23:58:17 YES        BKP-DB-INCR1-REP

    List of Backup Pieces for backup set 26092 Copy #2
    BP Key  Pc# Status      Media                   Piece Name
    ------- --- ----------- ----------------------- ----------
    26134   1   AVAILABLE   ZDLRAS2_REP             VB$_2127575003_12565_1

RMAN>

As you can see above, each backup was replicated. And the replication occurred for the incremental level 1, and for the virtual full backup. Above, look that the tags are the same. About how the virtual full backups works I explained in a dedicated post.

The virtual full backup was generated at both ZDLRA’s (upstream and downstream) based on the ingested incremental level 1. It was ingested at upstream, sent do downstream and both generated the virtual full. And since they have the same SCN, they are considered equal.

Archivelogs

The same protection is done for backups of archivelogs:

RMAN> BACKUP DEVICE TYPE SBT ARCHIVELOG ALL TAG 'BKP-ARCH';

Starting backup at 10/07/2020 00:04:59
current log archived
using channel ORA_SBT_TAPE_1
channel ORA_SBT_TAPE_1: starting archived log backup set
channel ORA_SBT_TAPE_1: specifying archived log(s) in backup set
input archived log thread=1 sequence=28 RECID=57 STAMP=1029524127
input archived log thread=1 sequence=29 RECID=59 STAMP=1029524475
input archived log thread=1 sequence=30 RECID=61 STAMP=1029524606
input archived log thread=1 sequence=31 RECID=63 STAMP=1037543212
input archived log thread=1 sequence=32 RECID=65 STAMP=1045347143
input archived log thread=1 sequence=33 RECID=66 STAMP=1045353901
channel ORA_SBT_TAPE_1: starting piece 1 at 10/07/2020 00:05:05
channel ORA_SBT_TAPE_1: finished piece 1 at 10/07/2020 00:05:40
piece handle=ORCL19_3nv4tldh_1_1 tag=BKP-ARCH comment=API Version 2.0,MMS Version 12.2.0.2
channel ORA_SBT_TAPE_1: backup set complete, elapsed time: 00:00:35
Finished backup at 10/07/2020 00:05:40

Starting Control File and SPFILE Autobackup at 10/07/2020 00:05:40
piece handle=c-323177095-20200710-00 comment=API Version 2.0,MMS Version 12.2.0.2
Finished Control File and SPFILE Autobackup at 10/07/2020 00:05:49

RMAN> list backup tag = 'BKP-ARCH';


List of Backup Sets
===================


BS Key  Size
------- ----------
26199   142.50M

  List of Archived Logs in backup set 26199
  Thrd Seq     Low SCN    Low Time            Next SCN   Next Time
  ---- ------- ---------- ------------------- ---------- ---------
  1    28      2297422    10/01/2020 01:08:09 2298810    12/01/2020 18:55:26
  1    29      2298810    12/01/2020 18:55:26 2310760    12/01/2020 19:01:07
  1    30      2310760    12/01/2020 19:01:07 2311714    12/01/2020 19:03:26
  1    31      2311714    12/01/2020 19:03:26 2360020    12/04/2020 14:26:44
  1    32      2360020    12/04/2020 14:26:44 2401381    09/07/2020 22:12:12
  1    33      2401381    09/07/2020 22:12:12 2419539    10/07/2020 00:04:59

  Backup Set Copy #1 of backup set 26199
  Device Type Elapsed Time Completion Time     Compressed Tag
  ----------- ------------ ------------------- ---------- ---
  SBT_TAPE    01:58:52     10/07/2020 00:05:36 NO         BKP-ARCH

    List of Backup Pieces for backup set 26199 Copy #1
    BP Key  Pc# Status      Media                   Piece Name
    ------- --- ----------- ----------------------- ----------
    26200   1   AVAILABLE   Recovery Appliance (ZDLRAS1) ORCL19_3nv4tldh_1_1

  Backup Set Copy #2 of backup set 26199
  Device Type Elapsed Time Completion Time     Compressed Tag
  ----------- ------------ ------------------- ---------- ---
  SBT_TAPE    01:58:52     09/07/2020 22:06:13 NO         BKP-ARCH

    List of Backup Pieces for backup set 26199 Copy #2
    BP Key  Pc# Status      Media                   Piece Name
    ------- --- ----------- ----------------------- ----------
    26336   1   AVAILABLE   Recovery Appliance (ZDLRAS2) RA_SBT_ORCL19_323177095_25742_3nv4tldh_1_2_26199

RMAN>

Check above that backup was replicated too.

Real-time redo

The same occurs for the real-time redo. Since the backup of archivelog is generated automatically at ZDLRA (you can read more at my previous post), it is replicated too by:

RMAN> alter system archive log current;

Statement processed

RMAN> list backup of archivelog sequence 48;


List of Backup Sets
===================


BS Key  Size
------- ----------
28186   198.00K

  List of Archived Logs in backup set 28186
  Thrd Seq     Low SCN    Low Time            Next SCN   Next Time
  ---- ------- ---------- ------------------- ---------- ---------
  1    48      3148635    18/07/2020 01:20:24 3149484    18/07/2020 01:26:35

  Backup Set Copy #1 of backup set 28186
  Device Type Elapsed Time Completion Time     Compressed Tag
  ----------- ------------ ------------------- ---------- ---
  SBT_TAPE    01:59:28     18/07/2020 01:27:04 YES        TAG20200718T012704

    List of Backup Pieces for backup set 28186 Copy #1
    BP Key  Pc# Status      Media                   Piece Name
    ------- --- ----------- ----------------------- ----------
    28187   1   AVAILABLE                           $RSCN_2029050_RTIM_1021856938_THRD_1_SEQ_48_CTKEY_28039_BACKUP

  Backup Set Copy #2 of backup set 28186
  Device Type Elapsed Time Completion Time     Compressed Tag
  ----------- ------------ ------------------- ---------- ---
  SBT_TAPE    01:59:28     17/07/2020 23:27:36 YES        TAG20200718T012704

    List of Backup Pieces for backup set 28186 Copy #2
    BP Key  Pc# Status      Media                   Piece Name
    ------- --- ----------- ----------------------- ----------
    28190   1   AVAILABLE   Recovery Appliance (ZDLRAS2) RA_SBT_ORCL19_323177095_25742_m2v5it78_1_2_28186

RMAN>

And if you have a failure (simulated here with a kill), the partial archivelog was replicated too. And with the ZDLRA replication, if your database crash and you need to restore the database, you have multi-site protection even without dataguard:

[oracle@orcloel7 ~]$ ps -ef |grep smon
oracle    6087     1  0 00:59 ?        00:00:00 ora_smon_ORCL19
oracle    8519  1937  0 01:35 pts/0    00:00:00 grep --color=auto smon
[oracle@orcloel7 ~]$ ps -ef |grep pmon
oracle    6047     1  0 00:59 ?        00:00:00 ora_pmon_ORCL19
oracle    8524  1937  0 01:35 pts/0    00:00:00 grep --color=auto pmon
[oracle@orcloel7 ~]$ ps -ef |grep mmon
oracle    6104     1  0 00:59 ?        00:00:01 ora_mmon_ORCL19
oracle    8530  1937  0 01:35 pts/0    00:00:00 grep --color=auto mmon
[oracle@orcloel7 ~]$ kill -9 6087 6047 6104
[oracle@orcloel7 ~]$
[oracle@orcloel7 ~]$ rman target=/ catalog=vpcsrc/vpcsrc@zdlras1-scan:1521/zdlras1

Recovery Manager: Release 19.0.0.0.0 - Production on Sat Jul 18 01:36:35 2020
Version 19.3.0.0.0

Copyright (c) 1982, 2019, Oracle and/or its affiliates.  All rights reserved.

connected to target database (not started)
connected to recovery catalog database

RMAN> list db_unique_name of database orcl19;


List of Databases
DB Key  DB Name  DB ID            Database Role    Db_unique_name
------- ------- ----------------- ---------------  ------------------
25140   ORCL19   323177095        PRIMARY          ORCL19

RMAN> set dbid 323177095;

executing command: SET DBID
database name is "ORCL19" and DBID is 323177095

RMAN> list backup of archivelog sequence 49;


List of Backup Sets
===================


BS Key  Size
------- ----------
28209   1.31M

  List of Archived Logs in backup set 28209
  Thrd Seq     Low SCN    Low Time            Next SCN   Next Time
  ---- ------- ---------- ------------------- ---------- ---------
  1    49      3149484    18/07/2020 01:26:35 3150908    18/07/2020 01:35:40

  Backup Set Copy #1 of backup set 28209
  Device Type Elapsed Time Completion Time     Compressed Tag
  ----------- ------------ ------------------- ---------- ---
  SBT_TAPE    01:59:01     18/07/2020 01:35:56 YES        TAG20200718T013555

    List of Backup Pieces for backup set 28209 Copy #1
    BP Key  Pc# Status      Media                   Piece Name
    ------- --- ----------- ----------------------- ----------
    28210   1   AVAILABLE                           $RSCN_2029050_RTIM_1021856938_THRD_1_SEQ_49_CTKEY_28155_BACKUP

  Backup Set Copy #2 of backup set 28209
  Device Type Elapsed Time Completion Time     Compressed Tag
  ----------- ------------ ------------------- ---------- ---
  SBT_TAPE    01:59:01     17/07/2020 23:36:54 YES        TAG20200718T013555

    List of Backup Pieces for backup set 28209 Copy #2
    BP Key  Pc# Status      Media                   Piece Name
    ------- --- ----------- ----------------------- ----------
    28213   1   AVAILABLE   Recovery Appliance (ZDLRAS2) RA_SBT_ORCL19_323177095_25742_m4v5itnr_1_2_28209

RMAN>

More copies

Since every ZDLRA can have a different environment, you can one that has clone to tape enabled (in this scenario the downstream has a clone to tape enabled). And since the backup is cloned, it appears automatically at rman catalog:

RMAN> list backup of datafile 1 completed after "to_date('10/07/2020 01:15:00', 'DD/MM/RRRR HH24:MI:SS')";


List of Backup Sets
===================


BS Key  Type LV Size
------- ---- -- ----------
26752   Incr 0  325.02M
  List of Datafiles in backup set 26752
  File LV Type Ckp SCN    Ckp Time            Abs Fuz SCN Sparse Name
  ---- -- ---- ---------- ------------------- ----------- ------ ----
  1    0  Incr 2429273    10/07/2020 01:18:02              NO    /u01/app/oracle/oradata/ORCL19/system01.dbf

  Backup Set Copy #1 of backup set 26752
  Device Type Elapsed Time Completion Time     Compressed Tag
  ----------- ------------ ------------------- ---------- ---
  SBT_TAPE    00:05:24     10/07/2020 01:23:27 YES        TAG20200710T011802

    List of Backup Pieces for backup set 26752 Copy #1
    BP Key  Pc# Status      Media                   Piece Name
    ------- --- ----------- ----------------------- ----------
    26753   1   AVAILABLE                           VB$_1891149551_26619I

  Backup Set Copy #2 of backup set 26752
  Device Type Elapsed Time Completion Time     Compressed Tag
  ----------- ------------ ------------------- ---------- ---
  SBT_TAPE    00:05:24     10/07/2020 01:23:26 YES        TAG20200710T011802

    List of Backup Pieces for backup set 26752 Copy #2
    BP Key  Pc# Status      Media                   Piece Name
    ------- --- ----------- ----------------------- ----------
    26775   1   AVAILABLE   ZDLRAS2_REP             VB$_2127575003_13258I

RMAN>

###########################################################################
After the clone to tape occurs (at downstream in this example)
###########################################################################

RMAN> list backup of datafile 1 completed after "to_date('10/07/2020 01:15:00', 'DD/MM/RRRR HH24:MI:SS')";


List of Backup Sets
===================


BS Key  Type LV Size
------- ---- -- ----------
26752   Incr 0  325.02M
  List of Datafiles in backup set 26752
  File LV Type Ckp SCN    Ckp Time            Abs Fuz SCN Sparse Name
  ---- -- ---- ---------- ------------------- ----------- ------ ----
  1    0  Incr 2429273    10/07/2020 01:18:02              NO    /u01/app/oracle/oradata/ORCL19/system01.dbf

  Backup Set Copy #1 of backup set 26752
  Device Type Elapsed Time Completion Time     Compressed Tag
  ----------- ------------ ------------------- ---------- ---
  SBT_TAPE    09:11:37     10/07/2020 01:23:27 YES        TAG20200710T011802

    List of Backup Pieces for backup set 26752 Copy #1
    BP Key  Pc# Status      Media                   Piece Name
    ------- --- ----------- ----------------------- ----------
    26753   1   AVAILABLE                           VB$_1891149551_26619I

  Backup Set Copy #3 of backup set 26752
  Device Type Elapsed Time Completion Time     Compressed Tag
  ----------- ------------ ------------------- ---------- ---
  SBT_TAPE    09:11:37     10/07/2020 10:29:39 NO         TAG20200710T011802

    List of Backup Pieces for backup set 26752 Copy #3
    BP Key  Pc# Status      Media                   Piece Name
    ------- --- ----------- ----------------------- ----------
    26889   1   AVAILABLE   ZDLRAS2_REP             RA_SBT_ORCL19_323177095_8206_0sv4tpma_1_2_13259

  Backup Set Copy #2 of backup set 26752
  Device Type Elapsed Time Completion Time     Compressed Tag
  ----------- ------------ ------------------- ---------- ---
  SBT_TAPE    09:11:37     10/07/2020 01:23:26 YES        TAG20200710T011802

    List of Backup Pieces for backup set 26752 Copy #2
    BP Key  Pc# Status      Media                   Piece Name
    ------- --- ----------- ----------------------- ----------
    26775   1   AVAILABLE   ZDLRAS2_REP             VB$_2127575003_13258I

RMAN>

Check that now the backupset for the virtual full level 0 exists 3 copies: local at upstream, one at downstream ZDLRA, and another at the tape attached at downstream ZDLRA. About how the clone to tape works you can at my post about this topic. And if you have a clone to tape enabled at upstream, a new copy will appear as available for rman catalog automatically.

Restoring backups

ZDLRA will choose the best source to restore the backup. The preference is always to restore from the catalog that you are connected to. But remember that you can have different recovery windows for your protection policies (at upstream and downstream). So, it is possible that your backup was already deleted from upstream, but is still available at downstream and listed at upstream catalog as a copy.

The process to restore a backup is the same as a traditional ZDLRA without replication. You can connect at the upstream catalog (recommended), or directly at the downstream catalog. In this example, I will restore the datafile connecting at upstream ZDLRA.

Check in the example below that the exists just the copies #2 e #3 of the backupset and both are at downstream ZDLRA. Check also that I connected at upstream and from there I called the restore:

[oracle@orcloel7 ~]$ rman target=/ catalog=vpcsrc/vpcsrc@zdlras1-scan:1521/zdlras1

Recovery Manager: Release 19.0.0.0.0 - Production on Sat Jul 18 00:13:29 2020
Version 19.3.0.0.0

Copyright (c) 1982, 2019, Oracle and/or its affiliates.  All rights reserved.

connected to target database: ORCL19 (DBID=323177095)
connected to recovery catalog database

RMAN>

RMAN> list backup of datafile 1 completed between "to_date('10/07/2020 01:15:00', 'DD/MM/RRRR HH24:MI:SS')" and "to_date('13/07/2020 01:15:00', 'DD/MM/RRRR HH24:MI:SS')";


List of Backup Sets
===================


BS Key  Type LV Size
------- ---- -- ----------
26752   Incr 0  785.06M
  List of Datafiles in backup set 26752
  File LV Type Ckp SCN    Ckp Time            Abs Fuz SCN Sparse Name
  ---- -- ---- ---------- ------------------- ----------- ------ ----
  1    0  Incr 2429273    10/07/2020 01:18:02              NO    /u01/app/oracle/oradata/ORCL19/system01.dbf

  Backup Set Copy #3 of backup set 26752
  Device Type Elapsed Time Completion Time     Compressed Tag
  ----------- ------------ ------------------- ---------- ---
  SBT_TAPE    09:11:37     10/07/2020 10:29:39 NO         TAG20200710T011802

    List of Backup Pieces for backup set 26752 Copy #3
    BP Key  Pc# Status      Media                   Piece Name
    ------- --- ----------- ----------------------- ----------
    26889   1   AVAILABLE   ZDLRAS2_REP             RA_SBT_ORCL19_323177095_8206_0sv4tpma_1_2_13259

  Backup Set Copy #2 of backup set 26752
  Device Type Elapsed Time Completion Time     Compressed Tag
  ----------- ------------ ------------------- ---------- ---
  SBT_TAPE    09:11:37     10/07/2020 01:23:26 YES        TAG20200710T011802

    List of Backup Pieces for backup set 26752 Copy #2
    BP Key  Pc# Status      Media                   Piece Name
    ------- --- ----------- ----------------------- ----------
    26775   1   AVAILABLE   ZDLRAS2_REP             VB$_2127575003_13258I

RMAN> shutdown immediate;

database closed
database dismounted
Oracle instance shut down

RMAN> startup mount;

connected to target database (not started)
Oracle instance started
database mounted

Total System Global Area    4294963272 bytes

Fixed Size                     8904776 bytes
Variable Size               1694498816 bytes
Database Buffers            2583691264 bytes
Redo Buffers                   7868416 bytes

RMAN> restore datafile 1 from tag = 'TAG20200710T011802';

Starting restore at 18/07/2020 00:23:54
allocated channel: ORA_SBT_TAPE_1
channel ORA_SBT_TAPE_1: SID=44 device type=SBT_TAPE
channel ORA_SBT_TAPE_1: RA Library (ZDLRAS1) SID=AAAB9D0E05120DEEE053010310AC604B
allocated channel: ORA_DISK_1
channel ORA_DISK_1: SID=45 device type=DISK

channel ORA_SBT_TAPE_1: starting datafile backup set restore
channel ORA_SBT_TAPE_1: specifying datafile(s) to restore from backup set
channel ORA_SBT_TAPE_1: restoring datafile 00001 to /u01/app/oracle/oradata/ORCL19/system01.dbf
channel ORA_SBT_TAPE_1: reading from backup piece VB$_2127575003_13258I
channel ORA_SBT_TAPE_1: piece handle=VB$_2127575003_13258I tag=TAG20200710T011802
channel ORA_SBT_TAPE_1: restored backup piece 1
channel ORA_SBT_TAPE_1: restore complete, elapsed time: 00:01:54
Finished restore at 18/07/2020 00:26:06

RMAN>

As you can see above, the restore read the piece “channel ORA_SBT_TAPE_1: piece handle=VB$_2127575003_13258I tag=TAG20200710T011802”, and this came directly from downstream. And even connected at upstream, it (the upstream) connected downstream and send to you transparently the backups from downstream. Working as a proxy.

ZDLRA Native Replication

The ZDLRA native replication increases a lot of the options for database protection. Even without dataguard you can have multi-site replication for your backupsets and archivelogs. And everything is automatically managed by the ZDLRA library. The catalog is self-managed and the backupsets copies are updated without interaction.

The native replication is not just an additional copy of your backupsets. Is part of architecture design from your protection. Depending on the architecture (and needs), is possible to have downstream with different protection windows and sustain for more time your protection. Imagine that you don’t need to replicate all databases that you do backup at upstream, so, you will have more space (and consequently more time) to your downstream store backups.

The ZDLRA native replication is also interesting because allows us to have multi-site protection because can work together with real-time redo to reach near-zero RPO. Is near-zero because the replication between upstream and downstream is not in real-time (like DG or real-time redo). So, in case of a complete datacenter failure, you can have some gap at downstream (since the last archivelog received – but this can be mitigated/reduced if needed).

Understand native replication for ZDLRA is more than just copy backups sets. As you saw above, it can interconnect with a lot of previous details, like different recovery windows goals, real-time redo, and clone to tapes. Understand the details and how to use it to create good architecture is important.

In the next posts, I will dig more about how to create these kinds of designs to protect all your databases. Reaching the MAA Reference architecture from the simplest single instance, to multi RAC and multi-site database operating with DataGuard. Everything with zero RPO.

The ZDLRA replication work to provide multiple copies of the backupset. Check how replication works for backups and to restore it.  

 

Disclaimer: “The postings on this site are my own and don’t necessarily represent my actual employer positions, strategies or opinions. The information here was edited to be useful for general purpose, specific data and identifications were removed to allow reach the generic audience and to be useful for the community. Post protected by copyright.”

Leave a Reply

Your email address will not be published. Required fields are marked *