![\"\"](\"http:\/\/www.fernandosimon.com\/blog\/wp-content\/uploads\/2020\/04\/HAIP.jpg\")
<\/a><\/p>\n*Image removed from Oracle Presentation: Oracle Real Application Clusters 19c- Best Practices and Internals- EMEA Tour Oct 2019<\/em><\/p>\n This procedure can be used for a non-Exadata environment too. But before continue it is important to be aware of some details with ACFS. If you use ACFS, be careful since the HAIP IP is used in the ASM proxy and needs to be updated; and if ACFS is in place, this procedure is valid just for GI 12.2+.<\/p>\n In this scenario, I am removing HAIP from ZDLRA GI. The HAIP has configured automatically and we are using ACFS.<\/p>\n The focus is to disable HAIP and remove it from CRS. Another detail is that we need to remove the dependencies between the CRS resources (like ASM that need HAIP) too.<\/p>\n One important detail is that since we are removing the HAIP, we need to swap the HAIP IP that it is used by services (like ASM Proxy) for something that exists. So, you need to have high availability IP\u2019s to use. For Exadata, we use the interconnect IP\u2019s, but if you are using in another environment, be careful that your network has the needed requirements (throughput, failover, and others).<\/p>\n During the GI upgrade from 12.2 to 19c we need to run the runcluvfy and it will detect if HAIP is enabled:<\/p>\n The steps need to be executed in order to avoid errors during the procedure. Unfortunately, it is needed to have one maintenance window due to the unavailability of the services. The running databases need to be shutdown due to the CRS restart by an example.<\/p>\n Bellow, the steps are summarized and explained to be followed. Please be aware of the IP\u2019s involved (they will be different in your case).<\/strong><\/p>\n So, this first point is to shutdown all databases running in this cluster:<\/p>\n It is needed to check if HAIP is enabled. Another detail is check for ASM proxy, and ACFS too:<\/p>\n The next step is to check the new IP that will be used. Since ASM does not use HAIP, we can (at Exadata) pickup the same used IP:<\/p>\n In this case, the IP that will substitute HAIP IP is 192.168.10.1\/192.168.10.2. And, as you can see above, they are from the interconnect network.<\/p>\n The next step is to fix the IP that is used by the ASM Proxy instance. But default, it pickup the HAIP IP during the startup. The idea here is to force the same IP than the ASM instance for cluster_interconnect parameter.<\/p>\n Connect at APX instance, check the parameters (and see that are without values). After that we create one init file from memory to have the backup:<\/p>\n For ASM Proxy instance we need to set the IP for cluster_interconnect parameter with the same IP used by ASM instance.<\/strong><\/p>\n To do that, we need to set it in init for ASM Proxy. But since the ASM Proxy instance doesn\u2019t have one, we create one at $GI_HOME\/dbs with the connect name (look below the folder and instance name). Below I set just the cluster to interconnect parameter and dedicated to the instance running in this node.<\/p>\n After creating the init, we restart the ASM Proxy in this node.<\/p>\n Look that instance restarted correctly and it is up.<\/p>\n And we can check the parameter if it is OK with the IP:<\/p>\n If the instance not startup (or the IP\u2019s are wrong) you need to check the init file that you created in $GIHOME\/dbs folder.<\/p>\n Since we restarted the ASM Proxy instance, the ACFS mountpoints went down and need to be up again:<\/p>\n I started up using the CRS resource for that.<\/p>\n After it is Ok for node 1, we can do it for other nodes. Here, since I made for ZDLRA, I have just the second node to fix ASM Proxy. Be careful with the IP\u2019s used, they are specific for this node and remember to use the correct init file.<\/strong><\/p>\n Since the ASM since the CRS depends on HAIP we need to remove this dependency. Here we have a tricky part. It is needed to completely change the dependencies for ASM, setting it to CRS\/CSS directly.<\/strong><\/p>\n First, at first node we check the current dependency for START and STOP:<\/p>\n And now we change it. Look at the parameters values. They are completely different, and need to be with these specific values:<\/p>\n If you see closely the HAIP it is not listed there as a dependency, setting with the values, we completely remove HAIP dependency and inheritance.<\/strong><\/p>\n And we need to do the same for the other node:<\/p>\n The next step is to disable the HAIP resource from startup at CRS. We do this in both nodes.<\/p>\n Node 1:<\/p>\n Node 2:<\/p>\n But check that we do not stop yet the HAIP resource. It is needed to remain up at this moment.<\/strong><\/p>\n If we check with the init for CRS, it is still there in node 1:<\/p>\n And for the other node too:<\/p>\n After we configure the dependencies for ASM we can restart CRS to shutdown HAIP (and test the changes that we made). I prefer to execute, first, in just one node and if everything goes well, I do for the others.<\/p>\n So, first the STOP CLUSTER in the first node:<\/p>\n And if everything was successful, we can stop CRS. This is needed because HAIP is from init from CRS:<\/p>\n After that we can start again CRS in this node:<\/p>\n And after some time we can see if the HAIP not restarted (as expected):<\/p>\n As you can see above, the HAIP not started during the init of CRS. If the HAIP still up, please check the topic 5.<\/p>\n After that, we can do the same in the other nodes.<\/p>\n After doing the changes in both nodes we can see if everything is up and running:<\/p>\n As you can see, everything is up. ASM is online in both nodes, as the listeners for ASM too.<\/p>\n To finish, we need to restart all databases that were stopped at topic 1.<\/p>\n After you complete the GI upgrade for 19c, you can remove the ASM Proxy init file (from $GI_HOME\/dbs) that was created before. It is not needed anymore because the upgrade removes completely the HAIP from CRS because it is not in use.<\/p>\n As told before, HAIP is not supported by the Exadata environment. If you check the definition for HAIP and the hardware design for the Exadata\/Engineering System, it is completely redundant. The InfiniBand network and the dual-channel\/ports have the same effect (even better)<\/p>\n But unfortunately, HAIP can be up for several reasons (like ZDLRA) and because of the way that ASM Proxy starts and pick up the interconnect IP, HAIP can be selected. Remove HAIP it is a little tricky because of the ASM dependency, we need to set some specific parameters.<\/p>\n As you saw before, there are several steps. But you can use and adapt them to your environment. If it is Exadata\/Engineering System (like ZDLRA) they can be used almost as-is (just check the IPs). If it is not, read and adapt.<\/p>\n References:<\/p>\n <\/p>\n <\/p>\n Disclaimer<\/strong>: \u201cThe postings on this site are my own and don\u2019t necessarily represent my actual employer positions, strategies or opinions. The information here was edited to be useful for general purpose, specific data and identifications were removed to allow reach the generic audience and to be useful for the community. Post protected by copyright.\u201d<\/em><\/p>\n","protected":false},"excerpt":{"rendered":" HAIP (High Availability IP) is not supported for the Exadata environment but can occur (if you did not create the cluster using OEDA) that HAIP became in use. And this particularity true for ZDLRA. So, during the upgrade from the previous version (12.2) to a higher version, it is needed to remove HAIP. Usually, when […]<\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"ngg_post_thumbnail":0,"jetpack_post_was_ever_published":false,"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_publicize_message":"","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":true,"jetpack_social_options":{"image_generator_settings":{"template":"highway","default_image_id":0,"font":"","enabled":false},"version":2}},"categories":[29,77,6,56,5,51],"tags":[134,100,69,81,65,74],"class_list":["post-716","post","type-post","status-publish","format-standard","hentry","category-database","category-engineeredsystems","category-exadata","category-grid-infrastructure","category-oracle","category-zdlra","tag-cluster","tag-engineered-systems","tag-exadata","tag-grid","tag-oracle","tag-zdlra"],"yoast_head":"\nEnvironment<\/h2>\n
GI Upgrade<\/h3>\n
....\r\n....\r\nChecks did not pass for the following nodes:\r\n zeroserv02,zeroserv01\r\n\r\n\r\nFailures were encountered during execution of CVU verification request \"stage -pre crsinst\".\r\n\r\nVerifying Node Connectivity ...FAILED\r\nzeroserv02: PRVG-11068 : Highly Available IP (HAIP) is enabled on the nodes\r\n \"zeroserv01,zeroserv02\".\r\n\r\nzeroserv01: PRVG-11068 : Highly Available IP (HAIP) is enabled on the nodes\r\n \"zeroserv01,zeroserv02\".\r\n\r\nVerifying RPM Package Manager database ...INFORMATION\r\nPRVG-11250 : The check \"RPM Package Manager database\" was not performed because\r\nit needs 'root' user privileges.\r\n\r\n\r\nCVU operation performed: stage -pre crsinst\r\nDate: Dec 9, 2019 1:38:46 PM\r\nCVU home: \/u01\/app\/19.0.0.0\/grid\/\r\nUser: oracle\r\n[root@zeroserv01 ~]#<\/pre>\n
Remove HAIP<\/h2>\n
1 – Shutdown services<\/h3>\n
[oracle@zeroserv01 ~]$ srvctl stop database -d zdlras -o immediate\r\n[oracle@zeroserv01 ~]$<\/pre>\n
2- Check HAIP, ASM Proxy, and ACFS<\/h3>\n
[oracle@zeroserv01 ~]$ $ORACLE_HOME\/bin\/crsctl stat res ora.cluster_interconnect.haip -t -init\r\n--------------------------------------------------------------------------------\r\nName Target State Server State details\r\n--------------------------------------------------------------------------------\r\nCluster Resources\r\n--------------------------------------------------------------------------------\r\nora.cluster_interconnect.haip\r\n 1 ONLINE ONLINE zeroserv01 STABLE\r\n--------------------------------------------------------------------------------\r\n[oracle@zeroserv01 ~]$\r\n[oracle@zeroserv01 ~]$ $ORACLE_HOME\/bin\/crsctl stat res ora.proxy_advm -t\r\n--------------------------------------------------------------------------------\r\nName Target State Server State details\r\n--------------------------------------------------------------------------------\r\nLocal Resources\r\n--------------------------------------------------------------------------------\r\nora.proxy_advm\r\n ONLINE ONLINE zeroserv01 STABLE\r\n ONLINE ONLINE zeroserv02 STABLE\r\n--------------------------------------------------------------------------------\r\n[oracle@zeroserv01 ~]$\r\n[oracle@zeroserv01 ~]$ $ORACLE_HOME\/bin\/crsctl stat res -w \"TYPE = ora.acfs.type\" -t\r\n--------------------------------------------------------------------------------\r\nName Target State Server State details\r\n--------------------------------------------------------------------------------\r\nLocal Resources\r\n--------------------------------------------------------------------------------\r\nora.catalog.raadmin.acfs\r\n ONLINE ONLINE zeroserv01 mounted on \/raacfs\/r\r\n aadmin,STABLE\r\n ONLINE ONLINE zeroserv02 mounted on \/raacfs\/r\r\n aadmin,STABLE\r\nora.catalog.raosbadmin.acfs\r\n ONLINE ONLINE zeroserv01 mounted on \/osbcat,S\r\n TABLE\r\n ONLINE ONLINE zeroserv02 mounted on \/osbcat,S\r\n TABLE\r\n--------------------------------------------------------------------------------\r\n[oracle@zeroserv01 ~]$<\/pre>\n
3 – Check New IP<\/h3>\n
[oracle@zeroserv01 ~]$ export ORACLE_SID=+ASM1\r\n[oracle@zeroserv01 ~]$ export ORACLE_HOME=\/u01\/app\/12.2.0.1\/grid\r\n[oracle@zeroserv01 ~]$ export PATH=$ORACLE_HOME\/bin:$PATH\r\n[oracle@zeroserv01 ~]$\r\n[oracle@zeroserv01 ~]$ sqlplus \/ as sysasm\r\n\r\nSQL*Plus: Release 12.2.0.1.0 Production on Mon Dec 9 15:00:55 2019\r\n\r\nCopyright (c) 1982, 2016, Oracle. All rights reserved.\r\n\r\n\r\nConnected to:\r\nOracle Database 12c Enterprise Edition Release 12.2.0.1.0 - 64bit Production\r\n\r\nSQL> show parameter cluster_interconnects\r\n\r\nNAME TYPE VALUE\r\n------------------------------------ ----------- ------------------------------\r\ncluster_interconnects string 192.168.10.1:192.168.10.2\r\nSQL> exit\r\nDisconnected from Oracle Database 12c Enterprise Edition Release 12.2.0.1.0 - 64bit Production\r\n[oracle@zeroserv01 ~]$\r\n[oracle@zeroserv01 ~]$ $ORACLE_HOME\/bin\/oifcfg getif\r\nbondeth0 10.208.68.0 global public\r\nib0 192.168.8.0 global cluster_interconnect,asm\r\nib1 192.168.8.0 global cluster_interconnect,asm\r\n[oracle@zeroserv01 ~]$<\/pre>\n
4 – Fixing IP for ASM Proxy<\/h3>\n
Connecting at APX instance<\/h4>\n
[oracle@zeroserv01 ~]$ export ORACLE_HOME=\/u01\/app\/12.2.0.1\/grid\r\n[oracle@zeroserv01 ~]$ export PATH=$ORACLE_HOME\/bin:$PATH\r\n[oracle@zeroserv01 ~]$ export ORACLE_SID=+APX1\r\n[oracle@zeroserv01 ~]$\r\n[oracle@zeroserv01 ~]$ sqlplus \/ as sysasm\r\n\r\nSQL*Plus: Release 12.2.0.1.0 Production on Tue Dec 10 10:20:59 2019\r\n\r\nCopyright (c) 1982, 2016, Oracle. All rights reserved.\r\n\r\n\r\nConnected to:\r\nOracle Database 12c Enterprise Edition Release 12.2.0.1.0 - 64bit Production\r\n\r\nSQL> show parameter instance_name\r\n\r\nNAME TYPE VALUE\r\n------------------------------------ ----------- ------------------------------\r\ninstance_name string +APX1\r\nSQL> show parameter instance_type\r\n\r\nNAME TYPE VALUE\r\n------------------------------------ ----------- ------------------------------\r\ninstance_type string ASMPROXY\r\nSQL> show parameter cluster_interconnects\r\n\r\nNAME TYPE VALUE\r\n------------------------------------ ----------- ------------------------------\r\ncluster_interconnects string\r\nSQL> create pfile = '\/tmp\/pfileapx1' from memory;\r\n\r\nFile created.\r\n\r\nSQL> exit\r\nDisconnected from Oracle Database 12c Enterprise Edition Release 12.2.0.1.0 - 64bit Production\r\n[oracle@zeroserv01 ~]$<\/pre>\n
Changing the IP<\/h4>\n
[oracle@zeroserv01 ~]$ echo \"+APX1.cluster_interconnects='192.168.10.1:192.168.10.2'\" > $ORACLE_HOME\/dbs\/init+APX1.ora\r\n[oracle@zeroserv01 ~]$\r\n[oracle@zeroserv01 ~]$ cat $ORACLE_HOME\/dbs\/init+APX1.ora\r\n+APX1.cluster_interconnects='192.168.10.1:192.168.10.2'\r\n[oracle@zeroserv01 ~]$\r\n<\/pre>\n
Restart the ASM Proxy<\/h4>\n
[root@zeroserv01 ~]# export ORACLE_HOME=\/u01\/app\/12.2.0.1\/grid\r\n[root@zeroserv01 ~]# export PATH=$ORACLE_HOME\/bin:$PATH\r\n[root@zeroserv01 ~]#\r\n[root@zeroserv01 ~]# $ORACLE_HOME\/bin\/srvctl stop asm -proxy -node zeroserv01 -force\r\n[root@zeroserv01 ~]#\r\n[root@zeroserv01 ~]# $ORACLE_HOME\/bin\/srvctl start asm -proxy -node zeroserv01\r\n[root@zeroserv01 ~]#\r\n[root@zeroserv01 ~]# ps -ef |grep APX\r\noracle 267811 1 0 10:43 ? 00:00:00 apx_pmon_+APX1\r\noracle 267813 1 0 10:43 ? 00:00:00 apx_clmn_+APX1\r\noracle 267815 1 0 10:43 ? 00:00:00 apx_psp0_+APX1\r\noracle 267820 1 1 10:43 ? 00:00:00 apx_vktm_+APX1\r\noracle 267824 1 0 10:43 ? 00:00:00 apx_gen0_+APX1\r\noracle 267826 1 0 10:43 ? 00:00:00 apx_mman_+APX1\r\noracle 267830 1 0 10:43 ? 00:00:00 apx_gen1_+APX1\r\noracle 267834 1 0 10:43 ? 00:00:00 apx_diag_+APX1\r\noracle 267836 1 0 10:43 ? 00:00:00 apx_dskm_+APX1\r\noracle 267838 1 0 10:43 ? 00:00:00 apx_pman_+APX1\r\noracle 267840 1 0 10:43 ? 00:00:00 apx_dia0_+APX1\r\noracle 267842 1 0 10:43 ? 00:00:00 apx_lreg_+APX1\r\noracle 267845 1 0 10:43 ? 00:00:00 apx_pxmn_+APX1\r\noracle 267847 1 0 10:43 ? 00:00:00 apx_rbal_+APX1\r\noracle 267849 1 0 10:43 ? 00:00:00 apx_vbg0_+APX1\r\noracle 267851 1 0 10:43 ? 00:00:00 apx_vdbg_+APX1\r\noracle 267853 1 0 10:43 ? 00:00:00 apx_vubg_+APX1\r\nroot 267979 32720 0 10:43 pts\/0 00:00:00 grep --color=auto APX\r\n[root@zeroserv01 ~]#<\/pre>\n
[oracle@zeroserv01 ~]$ sqlplus \/ as sysasm\r\n\r\nSQL*Plus: Release 12.2.0.1.0 Production on Tue Dec 10 10:51:59 2019\r\n\r\nCopyright (c) 1982, 2016, Oracle. All rights reserved.\r\n\r\n\r\nConnected to:\r\nOracle Database 12c Enterprise Edition Release 12.2.0.1.0 - 64bit Production\r\n\r\nSQL> show parameter cluster_interconnects\r\n\r\nNAME TYPE VALUE\r\n------------------------------------ ----------- ------------------------------\r\ncluster_interconnects string 192.168.10.1:192.168.10.2\r\nSQL> show parameter instance_name\r\n\r\nNAME TYPE VALUE\r\n------------------------------------ ----------- ------------------------------\r\ninstance_name string +APX1\r\nSQL> exit\r\nDisconnected from Oracle Database 12c Enterprise Edition Release 12.2.0.1.0 - 64bit Production\r\n[oracle@zeroserv01 ~]$<\/pre>\n
Restart ACFS<\/h4>\n
[root@zeroserv01 ~]# $ORACLE_HOME\/bin\/crsctl start res -w \"TYPE = ora.acfs.type\" -n zeroserv01\r\nCRS-2672: Attempting to start 'ora.CATALOG.RAOSBADMIN.advm' on 'zeroserv01'\r\nCRS-2672: Attempting to start 'ora.CATALOG.RAADMIN.advm' on 'zeroserv01'\r\nCRS-2676: Start of 'ora.CATALOG.RAOSBADMIN.advm' on 'zeroserv01' succeeded\r\nCRS-2672: Attempting to start 'ora.catalog.raosbadmin.acfs' on 'zeroserv01'\r\nCRS-2676: Start of 'ora.CATALOG.RAADMIN.advm' on 'zeroserv01' succeeded\r\nCRS-2672: Attempting to start 'ora.catalog.raadmin.acfs' on 'zeroserv01'\r\nCRS-2676: Start of 'ora.catalog.raosbadmin.acfs' on 'zeroserv01' succeeded\r\nCRS-2676: Start of 'ora.catalog.raadmin.acfs' on 'zeroserv01' succeeded\r\n[root@zeroserv01 ~]#<\/pre>\n
Other Nodes<\/h4>\n
[root@zeroserv02 ~]# su - oracle\r\nLast login: Tue Dec 10 10:53:02 CET 2019\r\n[oracle@zeroserv02 ~]$ export ORACLE_SID=+ASM2\r\n[oracle@zeroserv02 ~]$ export ORACLE_HOME=\/u01\/app\/12.2.0.1\/grid\r\n[oracle@zeroserv02 ~]$ export PATH=$ORACLE_HOME\/bin:$PATH\r\n[oracle@zeroserv02 ~]$\r\n[oracle@zeroserv02 ~]$ sqlplus \/ as sysasm\r\n\r\nSQL*Plus: Release 12.2.0.1.0 Production on Tue Dec 10 10:56:24 2019\r\n\r\nCopyright (c) 1982, 2016, Oracle. All rights reserved.\r\n\r\n\r\nConnected to:\r\nOracle Database 12c Enterprise Edition Release 12.2.0.1.0 - 64bit Production\r\n\r\nSQL> show parameter cluster_interconnects\r\n\r\nNAME TYPE VALUE\r\n------------------------------------ ----------- ------------------------------\r\ncluster_interconnects string 192.168.10.3:192.168.10.4\r\nSQL> exit\r\nDisconnected from Oracle Database 12c Enterprise Edition Release 12.2.0.1.0 - 64bit Production\r\n[oracle@zeroserv02 ~]$\r\n[oracle@zeroserv02 ~]$\r\n[oracle@zeroserv02 ~]$\r\n[oracle@zeroserv02 ~]$ export ORACLE_HOME=\/u01\/app\/12.2.0.1\/grid\r\n[oracle@zeroserv02 ~]$ export PATH=$ORACLE_HOME\/bin:$PATH\r\n[oracle@zeroserv02 ~]$ export ORACLE_SID=+APX1\r\n[oracle@zeroserv02 ~]$ export ORACLE_SID=+APX2\r\n[oracle@zeroserv02 ~]$\r\n[oracle@zeroserv02 ~]$ sqlplus \/ as sysasm\r\n\r\nSQL*Plus: Release 12.2.0.1.0 Production on Tue Dec 10 10:57:15 2019\r\n\r\nCopyright (c) 1982, 2016, Oracle. All rights reserved.\r\n\r\n\r\nConnected to:\r\nOracle Database 12c Enterprise Edition Release 12.2.0.1.0 - 64bit Production\r\n\r\nSQL>\r\nSQL> show parameter instance_name\r\n\r\nNAME TYPE VALUE\r\n------------------------------------ ----------- ------------------------------\r\ninstance_name string +APX2\r\nSQL> show parameter cluster_interconnects\r\n\r\nNAME TYPE VALUE\r\n------------------------------------ ----------- ------------------------------\r\ncluster_interconnects string\r\nSQL> create pfile = '\/tmp\/pfileapx2' from memory;\r\n\r\nFile created.\r\n\r\nSQL> exit\r\nDisconnected from Oracle Database 12c Enterprise Edition Release 12.2.0.1.0 - 64bit Production\r\n[oracle@zeroserv02 ~]$\r\n[oracle@zeroserv02 ~]$\r\n[oracle@zeroserv02 ~]$\r\n[oracle@zeroserv02 ~]$\r\n[oracle@zeroserv02 ~]$ ls -l $ORACLE_HOME\/dbs\/init?APX*\r\nls: cannot access \/u01\/app\/12.2.0.1\/grid\/dbs\/init?APX*: No such file or directory\r\n[oracle@zeroserv02 ~]$\r\n[oracle@zeroserv02 ~]$\r\n[oracle@zeroserv02 ~]$ echo \"+APX2.cluster_interconnects='192.168.10.3:192.168.10.4'\" > $ORACLE_HOME\/dbs\/init+APX2.ora\r\n[oracle@zeroserv02 ~]$\r\n[oracle@zeroserv02 ~]$ cat $ORACLE_HOME\/dbs\/init+APX2.ora\r\n+APX2.cluster_interconnects='192.168.10.3:192.168.10.4'\r\n[oracle@zeroserv02 ~]$\r\n[oracle@zeroserv02 ~]$ exit \r\n[root@zeroserv02 ~]# export ORACLE_HOME=\/u01\/app\/12.2.0.1\/grid\r\n[root@zeroserv02 ~]# export PATH=$ORACLE_HOME\/bin:$PATH\r\n[root@zeroserv02 ~]#\r\n[root@zeroserv02 ~]#\r\n[root@zeroserv02 ~]# $ORACLE_HOME\/bin\/srvctl stop asm -proxy -node zeroserv02 -force\r\n[root@zeroserv02 ~]#\r\n[root@zeroserv02 ~]#\r\n[root@zeroserv02 ~]# $ORACLE_HOME\/bin\/srvctl start asm -proxy -node zeroserv02\r\n[root@zeroserv02 ~]#\r\n[root@zeroserv02 ~]#\r\n[root@zeroserv02 ~]#\r\n[root@zeroserv02 ~]# $ORACLE_HOME\/bin\/crsctl start res -w \"TYPE = ora.acfs.type\" -n zeroserv02\r\nCRS-2672: Attempting to start 'ora.CATALOG.RAOSBADMIN.advm' on 'zeroserv02'\r\nCRS-2672: Attempting to start 'ora.CATALOG.RAADMIN.advm' on 'zeroserv02'\r\nCRS-2676: Start of 'ora.CATALOG.RAOSBADMIN.advm' on 'zeroserv02' succeeded\r\nCRS-2672: Attempting to start 'ora.catalog.raosbadmin.acfs' on 'zeroserv02'\r\nCRS-2676: Start of 'ora.CATALOG.RAADMIN.advm' on 'zeroserv02' succeeded\r\nCRS-2672: Attempting to start 'ora.catalog.raadmin.acfs' on 'zeroserv02'\r\nCRS-2676: Start of 'ora.catalog.raosbadmin.acfs' on 'zeroserv02' succeeded\r\nCRS-2676: Start of 'ora.catalog.raadmin.acfs' on 'zeroserv02' succeeded\r\n[root@zeroserv02 ~]#\r\n[root@zeroserv02 ~]#\r\n[root@zeroserv02 ~]# su - oracle\r\nLast login: Tue Dec 10 10:54:10 CET 2019\r\n[oracle@zeroserv02 ~]$ export ORACLE_HOME=\/u01\/app\/12.2.0.1\/grid\r\n[oracle@zeroserv02 ~]$ export PATH=$ORACLE_HOME\/bin:$PATH\r\n[oracle@zeroserv02 ~]$ export ORACLE_SID=+APX1\r\n[oracle@zeroserv02 ~]$ export ORACLE_SID=+APX2\r\n\r\n[oracle@zeroserv02 ~]$ sqlplus \/ as sysasm\r\n\r\nSQL*Plus: Release 12.2.0.1.0 Production on Tue Dec 10 11:00:46 2019\r\n\r\nCopyright (c) 1982, 2016, Oracle. All rights reserved.\r\n\r\n\r\nConnected to:\r\nOracle Database 12c Enterprise Edition Release 12.2.0.1.0 - 64bit Production\r\n\r\nSQL> show parameter cluster_interconnects\r\n\r\nNAME TYPE VALUE\r\n------------------------------------ ----------- ------------------------------\r\ncluster_interconnects string 192.168.10.3:192.168.10.4\r\nSQL> show parameter instance_name\r\n\r\nNAME TYPE VALUE\r\n------------------------------------ ----------- ------------------------------\r\ninstance_name string +APX2\r\nSQL> exit\r\nDisconnected from Oracle Database 12c Enterprise Edition Release 12.2.0.1.0 - 64bit Production\r\nYou have new mail in \/var\/spool\/mail\/oracle\r\n[oracle@zeroserv02 ~]$<\/pre>\n
5 – ASM Dependency<\/h3>\n
[root@zeroserv01 ~]# \/u01\/app\/12.2.0.1\/grid\/bin\/crsctl stat res ora.asm -p |grep START_DEPENDENCIES=\r\nSTART_DEPENDENCIES=hard(ora.ASMNET1LSNR_ASM.lsnr,ora.ASMNET2LSNR_ASM.lsnr) weak(ora.LISTENER.lsnr) pullup(ora.ASMNET1LSNR_ASM.lsnr,ora.ASMNET2LSNR_ASM.lsnr) dispersion:active(site:type:ora.asm.type)\r\n[root@zeroserv01 ~]#\r\n[root@zeroserv01 ~]# \/u01\/app\/12.2.0.1\/grid\/bin\/crsctl stat res ora.asm -p |grep STOP_DEPENDENCIES=\r\nSTOP_DEPENDENCIES=hard(intermediate:ora.ASMNET1LSNR_ASM.lsnr,intermediate:ora.ASMNET2LSNR_ASM.lsnr)\r\n[root@zeroserv01 ~]#<\/pre>\n
[root@zeroserv01 ~]# \/u01\/app\/12.2.0.1\/grid\/bin\/crsctl modify res ora.asm -attr \"START_DEPENDENCIES='hard(ora.cssd,ora.ctssd) pullup(ora.cssd,ora.ctssd) weak(ora.drivers.acfs)',STOP_DEPENDENCIES='hard(intermediate:ora.cssd)'\" -init\r\n[root@zeroserv01 ~]#\r\n<\/pre>\n
[root@zeroserv02 ~]# \/u01\/app\/12.2.0.1\/grid\/bin\/crsctl modify res ora.asm -attr \"START_DEPENDENCIES='hard(ora.cssd,ora.ctssd) pullup(ora.cssd,ora.ctssd) weak(ora.drivers.acfs)',STOP_DEPENDENCIES='hard(intermediate:ora.cssd)'\" -init\r\n[root@zeroserv02 ~]#<\/pre>\n
6 – Disable HAIP Resource<\/h3>\n
[root@zeroserv01 ~]# \/u01\/app\/12.2.0.1\/grid\/bin\/crsctl modify res ora.cluster_interconnect.haip -attr \"ENABLED=0\" -init\r\n[root@zeroserv01 ~]#<\/pre>\n
[root@zeroserv02 ~]# \/u01\/app\/12.2.0.1\/grid\/bin\/crsctl modify res ora.cluster_interconnect.haip -attr \"ENABLED=0\" -init\r\n[root@zeroserv02 ~]#<\/pre>\n
[root@zeroserv01 ~]# \/u01\/app\/12.2.0.1\/grid\/bin\/crsctl stat res -t -init\r\n--------------------------------------------------------------------------------\r\nName Target State Server State details\r\n--------------------------------------------------------------------------------\r\nCluster Resources\r\n--------------------------------------------------------------------------------\r\nora.asm\r\n 1 ONLINE ONLINE zeroserv01 STABLE\r\nora.cluster_interconnect.haip\r\n 1 ONLINE ONLINE zeroserv01 STABLE\r\nora.crf\r\n 1 ONLINE ONLINE zeroserv01 STABLE\r\nora.crsd\r\n 1 ONLINE ONLINE zeroserv01 STABLE\r\nora.cssd\r\n 1 ONLINE ONLINE zeroserv01 STABLE\r\nora.cssdmonitor\r\n 1 ONLINE ONLINE zeroserv01 STABLE\r\nora.ctssd\r\n 1 ONLINE ONLINE zeroserv01 OBSERVER,STABLE\r\nora.diskmon\r\n 1 ONLINE ONLINE zeroserv01 STABLE\r\nora.drivers.acfs\r\n 1 ONLINE ONLINE zeroserv01 STABLE\r\nora.drivers.oka\r\n 1 OFFLINE OFFLINE STABLE\r\nora.evmd\r\n 1 ONLINE ONLINE zeroserv01 STABLE\r\nora.gipcd\r\n 1 ONLINE ONLINE zeroserv01 STABLE\r\nora.gpnpd\r\n 1 ONLINE ONLINE zeroserv01 STABLE\r\nora.mdnsd\r\n 1 ONLINE ONLINE zeroserv01 STABLE\r\nora.storage\r\n 1 ONLINE ONLINE zeroserv01 STABLE\r\n--------------------------------------------------------------------------------\r\n[root@zeroserv01 ~]#<\/pre>\n
[root@zeroserv02 ~]# \/u01\/app\/12.2.0.1\/grid\/bin\/crsctl stat res -t -init\r\n--------------------------------------------------------------------------------\r\nName Target State Server State details\r\n--------------------------------------------------------------------------------\r\nCluster Resources\r\n--------------------------------------------------------------------------------\r\nora.asm\r\n 1 ONLINE ONLINE zeroserv02 STABLE\r\nora.cluster_interconnect.haip\r\n 1 ONLINE ONLINE zeroserv02 STABLE\r\nora.crf\r\n 1 ONLINE ONLINE zeroserv02 STABLE\r\nora.crsd\r\n 1 ONLINE ONLINE zeroserv02 STABLE\r\nora.cssd\r\n 1 ONLINE ONLINE zeroserv02 STABLE\r\nora.cssdmonitor\r\n 1 ONLINE ONLINE zeroserv02 STABLE\r\nora.ctssd\r\n 1 ONLINE ONLINE zeroserv02 ACTIVE:0,STABLE\r\nora.diskmon\r\n 1 ONLINE ONLINE zeroserv02 STABLE\r\nora.drivers.acfs\r\n 1 ONLINE ONLINE zeroserv02 STABLE\r\nora.drivers.oka\r\n 1 OFFLINE OFFLINE STABLE\r\nora.evmd\r\n 1 ONLINE ONLINE zeroserv02 STABLE\r\nora.gipcd\r\n 1 ONLINE ONLINE zeroserv02 STABLE\r\nora.gpnpd\r\n 1 ONLINE ONLINE zeroserv02 STABLE\r\nora.mdnsd\r\n 1 ONLINE ONLINE zeroserv02 STABLE\r\nora.storage\r\n 1 ONLINE ONLINE zeroserv02 STABLE\r\n--------------------------------------------------------------------------------\r\n[root@zeroserv02 ~]#\r\n<\/pre>\n
7 \u2013 Restart CRS<\/h3>\n
Stop Cluster<\/h4>\n
[root@zeroserv01 ~]# \/u01\/app\/12.2.0.1\/grid\/bin\/crsctl stop cluster -f\r\nCRS-2673: Attempting to stop 'ora.crsd' on 'zeroserv01'\r\nCRS-2790: Starting shutdown of Cluster Ready Services-managed resources on server 'zeroserv01'\r\nCRS-2673: Attempting to stop 'ora.catalog.raadmin.acfs' on 'zeroserv01'\r\nCRS-2673: Attempting to stop 'ora.catalog.raosbadmin.acfs' on 'zeroserv01'\r\nCRS-2673: Attempting to stop 'ora.DELTA.dg' on 'zeroserv01'\r\nCRS-2673: Attempting to stop 'ora.CATALOG.dg' on 'zeroserv01'\r\nCRS-2673: Attempting to stop 'ora.LISTENER_SCAN2.lsnr' on 'zeroserv01'\r\nCRS-2673: Attempting to stop 'ora.cvu' on 'zeroserv01'\r\nCRS-2673: Attempting to stop 'ora.chad' on 'zeroserv01'\r\nCRS-2673: Attempting to stop 'ora.chad' on 'zeroserv02'\r\nCRS-2673: Attempting to stop 'ora.LISTENER.lsnr' on 'zeroserv01'\r\nCRS-2673: Attempting to stop 'ora.LISTENER_SCAN3.lsnr' on 'zeroserv01'\r\nCRS-2673: Attempting to stop 'ora.qosmserver' on 'zeroserv01'\r\nCRS-2677: Stop of 'ora.DELTA.dg' on 'zeroserv01' succeeded\r\nCRS-2677: Stop of 'ora.CATALOG.dg' on 'zeroserv01' succeeded\r\nCRS-2673: Attempting to stop 'ora.asm' on 'zeroserv01'\r\nCRS-2677: Stop of 'ora.cvu' on 'zeroserv01' succeeded\r\nCRS-2677: Stop of 'ora.asm' on 'zeroserv01' succeeded\r\nCRS-2673: Attempting to stop 'ora.ASMNET2LSNR_ASM.lsnr' on 'zeroserv01'\r\nCRS-2673: Attempting to stop 'ora.ASMNET1LSNR_ASM.lsnr' on 'zeroserv01'\r\nCRS-2677: Stop of 'ora.LISTENER_SCAN2.lsnr' on 'zeroserv01' succeeded\r\nCRS-2673: Attempting to stop 'ora.scan2.vip' on 'zeroserv01'\r\nCRS-2677: Stop of 'ora.LISTENER_SCAN3.lsnr' on 'zeroserv01' succeeded\r\nCRS-2673: Attempting to stop 'ora.scan3.vip' on 'zeroserv01'\r\nCRS-2677: Stop of 'ora.catalog.raosbadmin.acfs' on 'zeroserv01' succeeded\r\nCRS-2673: Attempting to stop 'ora.CATALOG.RAOSBADMIN.advm' on 'zeroserv01'\r\nCRS-2677: Stop of 'ora.CATALOG.RAOSBADMIN.advm' on 'zeroserv01' succeeded\r\nCRS-2677: Stop of 'ora.catalog.raadmin.acfs' on 'zeroserv01' succeeded\r\nCRS-2673: Attempting to stop 'ora.CATALOG.RAADMIN.advm' on 'zeroserv01'\r\nCRS-2677: Stop of 'ora.CATALOG.RAADMIN.advm' on 'zeroserv01' succeeded\r\nCRS-2673: Attempting to stop 'ora.proxy_advm' on 'zeroserv01'\r\nCRS-2677: Stop of 'ora.LISTENER.lsnr' on 'zeroserv01' succeeded\r\nCRS-2673: Attempting to stop 'ora.zeroserv01.vip' on 'zeroserv01'\r\nCRS-2677: Stop of 'ora.scan3.vip' on 'zeroserv01' succeeded\r\nCRS-2677: Stop of 'ora.qosmserver' on 'zeroserv01' succeeded\r\nCRS-2677: Stop of 'ora.scan2.vip' on 'zeroserv01' succeeded\r\nCRS-2677: Stop of 'ora.zeroserv01.vip' on 'zeroserv01' succeeded\r\nCRS-2677: Stop of 'ora.ASMNET1LSNR_ASM.lsnr' on 'zeroserv01' succeeded\r\nCRS-2677: Stop of 'ora.ASMNET2LSNR_ASM.lsnr' on 'zeroserv01' succeeded\r\nCRS-2677: Stop of 'ora.proxy_advm' on 'zeroserv01' succeeded\r\nCRS-2675: Stop of 'ora.chad' on 'zeroserv02' failed\r\nCRS-2679: Attempting to clean 'ora.chad' on 'zeroserv02'\r\nCRS-2675: Stop of 'ora.chad' on 'zeroserv01' failed\r\nCRS-2679: Attempting to clean 'ora.chad' on 'zeroserv01'\r\nCRS-2681: Clean of 'ora.chad' on 'zeroserv02' succeeded\r\nCRS-2681: Clean of 'ora.chad' on 'zeroserv01' succeeded\r\nCRS-2673: Attempting to stop 'ora.mgmtdb' on 'zeroserv01'\r\nCRS-2677: Stop of 'ora.mgmtdb' on 'zeroserv01' succeeded\r\nCRS-2673: Attempting to stop 'ora.MGMTLSNR' on 'zeroserv01'\r\nCRS-2677: Stop of 'ora.MGMTLSNR' on 'zeroserv01' succeeded\r\nCRS-2672: Attempting to start 'ora.MGMTLSNR' on 'zeroserv02'\r\nCRS-2672: Attempting to start 'ora.qosmserver' on 'zeroserv02'\r\nCRS-2672: Attempting to start 'ora.scan2.vip' on 'zeroserv02'\r\nCRS-2672: Attempting to start 'ora.scan3.vip' on 'zeroserv02'\r\nCRS-2672: Attempting to start 'ora.cvu' on 'zeroserv02'\r\nCRS-2672: Attempting to start 'ora.zeroserv01.vip' on 'zeroserv02'\r\nCRS-2676: Start of 'ora.cvu' on 'zeroserv02' succeeded\r\nCRS-2676: Start of 'ora.scan2.vip' on 'zeroserv02' succeeded\r\nCRS-2676: Start of 'ora.zeroserv01.vip' on 'zeroserv02' succeeded\r\nCRS-2672: Attempting to start 'ora.LISTENER_SCAN2.lsnr' on 'zeroserv02'\r\nCRS-2676: Start of 'ora.scan3.vip' on 'zeroserv02' succeeded\r\nCRS-2672: Attempting to start 'ora.LISTENER_SCAN3.lsnr' on 'zeroserv02'\r\nCRS-2676: Start of 'ora.MGMTLSNR' on 'zeroserv02' succeeded\r\nCRS-2672: Attempting to start 'ora.mgmtdb' on 'zeroserv02'\r\nCRS-2676: Start of 'ora.qosmserver' on 'zeroserv02' succeeded\r\nCRS-2676: Start of 'ora.LISTENER_SCAN2.lsnr' on 'zeroserv02' succeeded\r\nCRS-2676: Start of 'ora.LISTENER_SCAN3.lsnr' on 'zeroserv02' succeeded\r\nCRS-2676: Start of 'ora.mgmtdb' on 'zeroserv02' succeeded\r\nCRS-2672: Attempting to start 'ora.chad' on 'zeroserv02'\r\nCRS-2676: Start of 'ora.chad' on 'zeroserv02' succeeded\r\nCRS-2673: Attempting to stop 'ora.ons' on 'zeroserv01'\r\nCRS-2677: Stop of 'ora.ons' on 'zeroserv01' succeeded\r\nCRS-2673: Attempting to stop 'ora.net1.network' on 'zeroserv01'\r\nCRS-2677: Stop of 'ora.net1.network' on 'zeroserv01' succeeded\r\nCRS-2792: Shutdown of Cluster Ready Services-managed resources on 'zeroserv01' has completed\r\nCRS-2677: Stop of 'ora.crsd' on 'zeroserv01' succeeded\r\nCRS-2673: Attempting to stop 'ora.ctssd' on 'zeroserv01'\r\nCRS-2673: Attempting to stop 'ora.evmd' on 'zeroserv01'\r\nCRS-2673: Attempting to stop 'ora.storage' on 'zeroserv01'\r\nCRS-2677: Stop of 'ora.storage' on 'zeroserv01' succeeded\r\nCRS-2673: Attempting to stop 'ora.asm' on 'zeroserv01'\r\nCRS-2677: Stop of 'ora.ctssd' on 'zeroserv01' succeeded\r\nCRS-2677: Stop of 'ora.evmd' on 'zeroserv01' succeeded\r\nCRS-2677: Stop of 'ora.asm' on 'zeroserv01' succeeded\r\nCRS-2673: Attempting to stop 'ora.cssd' on 'zeroserv01'\r\nCRS-2677: Stop of 'ora.cssd' on 'zeroserv01' succeeded\r\nCRS-2673: Attempting to stop 'ora.diskmon' on 'zeroserv01'\r\nCRS-2677: Stop of 'ora.diskmon' on 'zeroserv01' succeeded\r\n[root@zeroserv01 ~]#<\/pre>\n
Stop\/Start CRS<\/h4>\n
[root@zeroserv01 ~]# \/u01\/app\/12.2.0.1\/grid\/bin\/crsctl stop crs -f\r\nCRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'zeroserv01'\r\nCRS-2673: Attempting to stop 'ora.gpnpd' on 'zeroserv01'\r\nCRS-2673: Attempting to stop 'ora.crf' on 'zeroserv01'\r\nCRS-2673: Attempting to stop 'ora.drivers.acfs' on 'zeroserv01'\r\nCRS-2673: Attempting to stop 'ora.mdnsd' on 'zeroserv01'\r\nCRS-2677: Stop of 'ora.drivers.acfs' on 'zeroserv01' succeeded\r\nCRS-2677: Stop of 'ora.gpnpd' on 'zeroserv01' succeeded\r\nCRS-2677: Stop of 'ora.crf' on 'zeroserv01' succeeded\r\nCRS-2673: Attempting to stop 'ora.gipcd' on 'zeroserv01'\r\nCRS-2677: Stop of 'ora.mdnsd' on 'zeroserv01' succeeded\r\nCRS-2677: Stop of 'ora.gipcd' on 'zeroserv01' succeeded\r\nCRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'zeroserv01' has completed\r\nCRS-4133: Oracle High Availability Services has been stopped.\r\n[root@zeroserv01 ~]#<\/pre>\n
[root@zeroserv01 ~]# \/u01\/app\/12.2.0.1\/grid\/bin\/crsctl start crs\r\nCRS-4123: Oracle High Availability Services has been started.\r\n[root@zeroserv01 ~]#<\/pre>\n
Check CRS Init<\/h4>\n
[root@zeroserv01 ~]# \/u01\/app\/12.2.0.1\/grid\/bin\/crsctl stat res -t -init\r\n--------------------------------------------------------------------------------\r\nName Target State Server State details\r\n--------------------------------------------------------------------------------\r\nCluster Resources\r\n--------------------------------------------------------------------------------\r\nora.asm\r\n 1 ONLINE ONLINE zeroserv01 Started,STABLE\r\nora.cluster_interconnect.haip\r\n 1 OFFLINE OFFLINE STABLE\r\nora.crf\r\n 1 ONLINE ONLINE zeroserv01 STABLE\r\nora.crsd\r\n 1 ONLINE ONLINE zeroserv01 STABLE\r\nora.cssd\r\n 1 ONLINE ONLINE zeroserv01 STABLE\r\nora.cssdmonitor\r\n 1 ONLINE ONLINE zeroserv01 STABLE\r\nora.ctssd\r\n 1 ONLINE ONLINE zeroserv01 ACTIVE:0,STABLE\r\nora.diskmon\r\n 1 ONLINE ONLINE zeroserv01 STABLE\r\nora.drivers.acfs\r\n 1 ONLINE ONLINE zeroserv01 STABLE\r\nora.drivers.oka\r\n 1 OFFLINE OFFLINE STABLE\r\nora.evmd\r\n 1 ONLINE ONLINE zeroserv01 STABLE\r\nora.gipcd\r\n 1 ONLINE ONLINE zeroserv01 STABLE\r\nora.gpnpd\r\n 1 ONLINE ONLINE zeroserv01 STABLE\r\nora.mdnsd\r\n 1 ONLINE ONLINE zeroserv01 STABLE\r\nora.storage\r\n 1 ONLINE ONLINE zeroserv01 STABLE\r\n--------------------------------------------------------------------------------\r\n[root@zeroserv01 ~]#\r\n<\/pre>\n
Other nodes<\/h4>\n
[root@zeroserv02 ~]# \/u01\/app\/12.2.0.1\/grid\/bin\/crsctl stop cluster -f\r\nCRS-2673: Attempting to stop 'ora.crsd' on 'zeroserv02'\r\nCRS-2790: Starting shutdown of Cluster Ready Services-managed resources on server 'zeroserv02'\r\nCRS-2673: Attempting to stop 'ora.DELTA.dg' on 'zeroserv02'\r\nCRS-2673: Attempting to stop 'ora.catalog.raadmin.acfs' on 'zeroserv02'\r\nCRS-2673: Attempting to stop 'ora.chad' on 'zeroserv02'\r\nCRS-2673: Attempting to stop 'ora.catalog.raosbadmin.acfs' on 'zeroserv02'\r\nCRS-2673: Attempting to stop 'ora.LISTENER.lsnr' on 'zeroserv02'\r\nCRS-2673: Attempting to stop 'ora.LISTENER_SCAN1.lsnr' on 'zeroserv02'\r\nCRS-2677: Stop of 'ora.DELTA.dg' on 'zeroserv02' succeeded\r\nCRS-2673: Attempting to stop 'ora.CATALOG.dg' on 'zeroserv02'\r\nCRS-2677: Stop of 'ora.CATALOG.dg' on 'zeroserv02' succeeded\r\nCRS-2673: Attempting to stop 'ora.asm' on 'zeroserv02'\r\nCRS-2677: Stop of 'ora.LISTENER.lsnr' on 'zeroserv02' succeeded\r\nCRS-2673: Attempting to stop 'ora.zeroserv02.vip' on 'zeroserv02'\r\nCRS-2677: Stop of 'ora.LISTENER_SCAN1.lsnr' on 'zeroserv02' succeeded\r\nCRS-2673: Attempting to stop 'ora.scan1.vip' on 'zeroserv02'\r\nCRS-2677: Stop of 'ora.catalog.raadmin.acfs' on 'zeroserv02' succeeded\r\nCRS-2673: Attempting to stop 'ora.CATALOG.RAADMIN.advm' on 'zeroserv02'\r\nCRS-2677: Stop of 'ora.CATALOG.RAADMIN.advm' on 'zeroserv02' succeeded\r\nCRS-2677: Stop of 'ora.catalog.raosbadmin.acfs' on 'zeroserv02' succeeded\r\nCRS-2673: Attempting to stop 'ora.CATALOG.RAOSBADMIN.advm' on 'zeroserv02'\r\nCRS-2677: Stop of 'ora.CATALOG.RAOSBADMIN.advm' on 'zeroserv02' succeeded\r\nCRS-2673: Attempting to stop 'ora.proxy_advm' on 'zeroserv02'\r\nCRS-2677: Stop of 'ora.asm' on 'zeroserv02' succeeded\r\nCRS-2673: Attempting to stop 'ora.ASMNET2LSNR_ASM.lsnr' on 'zeroserv02'\r\nCRS-2673: Attempting to stop 'ora.ASMNET1LSNR_ASM.lsnr' on 'zeroserv02'\r\nCRS-2677: Stop of 'ora.zeroserv02.vip' on 'zeroserv02' succeeded\r\nCRS-2677: Stop of 'ora.scan1.vip' on 'zeroserv02' succeeded\r\nCRS-2677: Stop of 'ora.ASMNET1LSNR_ASM.lsnr' on 'zeroserv02' succeeded\r\nCRS-2677: Stop of 'ora.ASMNET2LSNR_ASM.lsnr' on 'zeroserv02' succeeded\r\nCRS-2677: Stop of 'ora.proxy_advm' on 'zeroserv02' succeeded\r\nCRS-2677: Stop of 'ora.chad' on 'zeroserv02' succeeded\r\nCRS-2672: Attempting to start 'ora.scan1.vip' on 'zeroserv01'\r\nCRS-2672: Attempting to start 'ora.zeroserv02.vip' on 'zeroserv01'\r\nCRS-2676: Start of 'ora.zeroserv02.vip' on 'zeroserv01' succeeded\r\nCRS-2676: Start of 'ora.scan1.vip' on 'zeroserv01' succeeded\r\nCRS-2672: Attempting to start 'ora.LISTENER_SCAN1.lsnr' on 'zeroserv01'\r\nCRS-2676: Start of 'ora.LISTENER_SCAN1.lsnr' on 'zeroserv01' succeeded\r\nCRS-2673: Attempting to stop 'ora.ons' on 'zeroserv02'\r\nCRS-2677: Stop of 'ora.ons' on 'zeroserv02' succeeded\r\nCRS-2673: Attempting to stop 'ora.net1.network' on 'zeroserv02'\r\nCRS-2677: Stop of 'ora.net1.network' on 'zeroserv02' succeeded\r\nCRS-2792: Shutdown of Cluster Ready Services-managed resources on 'zeroserv02' has completed\r\nCRS-2677: Stop of 'ora.crsd' on 'zeroserv02' succeeded\r\nCRS-2673: Attempting to stop 'ora.ctssd' on 'zeroserv02'\r\nCRS-2673: Attempting to stop 'ora.evmd' on 'zeroserv02'\r\nCRS-2673: Attempting to stop 'ora.storage' on 'zeroserv02'\r\nCRS-2677: Stop of 'ora.storage' on 'zeroserv02' succeeded\r\nCRS-2673: Attempting to stop 'ora.asm' on 'zeroserv02'\r\nCRS-2677: Stop of 'ora.ctssd' on 'zeroserv02' succeeded\r\nCRS-2677: Stop of 'ora.evmd' on 'zeroserv02' succeeded\r\nCRS-2677: Stop of 'ora.asm' on 'zeroserv02' succeeded\r\nCRS-2673: Attempting to stop 'ora.cssd' on 'zeroserv02'\r\nCRS-2677: Stop of 'ora.cssd' on 'zeroserv02' succeeded\r\nCRS-2673: Attempting to stop 'ora.diskmon' on 'zeroserv02'\r\nCRS-2677: Stop of 'ora.diskmon' on 'zeroserv02' succeeded\r\n[root@zeroserv02 ~]#\r\n[root@zeroserv02 ~]#\r\n[root@zeroserv02 ~]#\r\n[root@zeroserv02 ~]# \/u01\/app\/12.2.0.1\/grid\/bin\/crsctl stop crs -f\r\nCRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'zeroserv02'\r\nCRS-2673: Attempting to stop 'ora.drivers.acfs' on 'zeroserv02'\r\nCRS-2673: Attempting to stop 'ora.mdnsd' on 'zeroserv02'\r\nCRS-2673: Attempting to stop 'ora.crf' on 'zeroserv02'\r\nCRS-2673: Attempting to stop 'ora.gpnpd' on 'zeroserv02'\r\nCRS-2677: Stop of 'ora.drivers.acfs' on 'zeroserv02' succeeded\r\nCRS-2677: Stop of 'ora.crf' on 'zeroserv02' succeeded\r\nCRS-2673: Attempting to stop 'ora.gipcd' on 'zeroserv02'\r\nCRS-2677: Stop of 'ora.mdnsd' on 'zeroserv02' succeeded\r\nCRS-2677: Stop of 'ora.gpnpd' on 'zeroserv02' succeeded\r\nCRS-2677: Stop of 'ora.gipcd' on 'zeroserv02' succeeded\r\nCRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'zeroserv02' has completed\r\nCRS-4133: Oracle High Availability Services has been stopped.\r\n[root@zeroserv02 ~]#\r\n[root@zeroserv02 ~]#\r\n[root@zeroserv02 ~]# \/u01\/app\/12.2.0.1\/grid\/bin\/crsctl start crs\r\nCRS-4123: Oracle High Availability Services has been started.\r\n[root@zeroserv02 ~]# \r\n[root@zeroserv02 ~]# \/u01\/app\/12.2.0.1\/grid\/bin\/crsctl stat res -t -init\r\n--------------------------------------------------------------------------------\r\nName Target State Server State details\r\n--------------------------------------------------------------------------------\r\nCluster Resources\r\n--------------------------------------------------------------------------------\r\nora.asm\r\n 1 ONLINE ONLINE zeroserv02 STABLE\r\nora.cluster_interconnect.haip\r\n 1 OFFLINE OFFLINE STABLE\r\nora.crf\r\n 1 ONLINE ONLINE zeroserv02 STABLE\r\nora.crsd\r\n 1 ONLINE ONLINE zeroserv02 STABLE\r\nora.cssd\r\n 1 ONLINE ONLINE zeroserv02 STABLE\r\nora.cssdmonitor\r\n 1 ONLINE ONLINE zeroserv02 STABLE\r\nora.ctssd\r\n 1 ONLINE ONLINE zeroserv02 ACTIVE:0,STABLE\r\nora.diskmon\r\n 1 ONLINE ONLINE zeroserv02 STABLE\r\nora.drivers.acfs\r\n 1 ONLINE ONLINE zeroserv02 STABLE\r\nora.drivers.oka\r\n 1 OFFLINE OFFLINE STABLE\r\nora.evmd\r\n 1 ONLINE ONLINE zeroserv02 STABLE\r\nora.gipcd\r\n 1 ONLINE ONLINE zeroserv02 STABLE\r\nora.gpnpd\r\n 1 ONLINE ONLINE zeroserv02 STABLE\r\nora.mdnsd\r\n 1 ONLINE ONLINE zeroserv02 STABLE\r\nora.storage\r\n 1 ONLINE ONLINE zeroserv02 STABLE\r\n--------------------------------------------------------------------------------\r\n[root@zeroserv02 ~]#<\/pre>\n
8 \u2013 Check if everything is UP<\/h3>\n
[root@zeroserv01 ~]# \/u01\/app\/12.2.0.1\/grid\/bin\/crsctl stat res -t\r\n--------------------------------------------------------------------------------\r\nName Target State Server State details\r\n--------------------------------------------------------------------------------\r\nLocal Resources\r\n--------------------------------------------------------------------------------\r\nob_dbfs\r\n OFFLINE OFFLINE zeroserv01 STABLE\r\n OFFLINE OFFLINE zeroserv02 STABLE\r\nora.ASMNET1LSNR_ASM.lsnr\r\n ONLINE ONLINE zeroserv01 STABLE\r\n ONLINE ONLINE zeroserv02 STABLE\r\nora.ASMNET2LSNR_ASM.lsnr\r\n ONLINE ONLINE zeroserv01 STABLE\r\n ONLINE ONLINE zeroserv02 STABLE\r\nora.CATALOG.RAADMIN.advm\r\n ONLINE ONLINE zeroserv01 STABLE\r\n ONLINE ONLINE zeroserv02 STABLE\r\nora.CATALOG.RAOSBADMIN.advm\r\n ONLINE ONLINE zeroserv01 STABLE\r\n ONLINE ONLINE zeroserv02 STABLE\r\nora.CATALOG.dg\r\n ONLINE ONLINE zeroserv01 STABLE\r\n ONLINE ONLINE zeroserv02 STABLE\r\nora.DELTA.dg\r\n ONLINE ONLINE zeroserv01 STABLE\r\n ONLINE ONLINE zeroserv02 STABLE\r\nora.LISTENER.lsnr\r\n ONLINE ONLINE zeroserv01 STABLE\r\n ONLINE ONLINE zeroserv02 STABLE\r\nora.catalog.raadmin.acfs\r\n ONLINE ONLINE zeroserv01 mounted on \/raacfs\/r\r\n aadmin,STABLE\r\n ONLINE ONLINE zeroserv02 mounted on \/raacfs\/r\r\n aadmin,STABLE\r\nora.catalog.raosbadmin.acfs\r\n ONLINE ONLINE zeroserv01 mounted on \/osbcat,S\r\n TABLE\r\n ONLINE ONLINE zeroserv02 mounted on \/osbcat,S\r\n TABLE\r\nora.chad\r\n ONLINE ONLINE zeroserv01 STABLE\r\n ONLINE ONLINE zeroserv02 STABLE\r\nora.net1.network\r\n ONLINE ONLINE zeroserv01 STABLE\r\n ONLINE ONLINE zeroserv02 STABLE\r\nora.ons\r\n ONLINE ONLINE zeroserv01 STABLE\r\n ONLINE ONLINE zeroserv02 STABLE\r\nora.proxy_advm\r\n ONLINE ONLINE zeroserv01 STABLE\r\n ONLINE ONLINE zeroserv02 STABLE\r\nrep_dbfs\r\n OFFLINE OFFLINE zeroserv01 STABLE\r\n OFFLINE OFFLINE zeroserv02 STABLE\r\n--------------------------------------------------------------------------------\r\nCluster Resources\r\n--------------------------------------------------------------------------------\r\nora.LISTENER_SCAN1.lsnr\r\n 1 ONLINE ONLINE zeroserv02 STABLE\r\nora.LISTENER_SCAN2.lsnr\r\n 1 ONLINE ONLINE zeroserv01 STABLE\r\nora.LISTENER_SCAN3.lsnr\r\n 1 ONLINE ONLINE zeroserv01 STABLE\r\nora.MGMTLSNR\r\n 1 ONLINE ONLINE zeroserv01 192.168.10.1 192.168\r\n .10.2,STABLE\r\nora.asm\r\n 1 ONLINE ONLINE zeroserv01 Started,STABLE\r\n 2 ONLINE ONLINE zeroserv02 Started,STABLE\r\nora.cvu\r\n 1 ONLINE ONLINE zeroserv01 STABLE\r\nora.mgmtdb\r\n 1 ONLINE ONLINE zeroserv01 Open,STABLE\r\nora.qosmserver\r\n 1 ONLINE ONLINE zeroserv01 STABLE\r\nora.scan1.vip\r\n 1 ONLINE ONLINE zeroserv02 STABLE\r\nora.scan2.vip\r\n 1 ONLINE ONLINE zeroserv01 STABLE\r\nora.scan3.vip\r\n 1 ONLINE ONLINE zeroserv01 STABLE\r\nora.zdlras.db\r\n 1 OFFLINE OFFLINE STABLE\r\n 2 OFFLINE OFFLINE Instance Shutdown,ST\r\n ABLE\r\nora.zeroserv01.vip\r\n 1 ONLINE ONLINE zeroserv01 STABLE\r\nora.zeroserv02.vip\r\n 1 ONLINE ONLINE zeroserv02 STABLE\r\n--------------------------------------------------------------------------------\r\n[root@zeroserv01 ~]#<\/pre>\n
9 \u2013 Start Databases<\/h3>\n
[oracle@zeroserv01 ~]$ srvctl start database -d zdlras\r\n[oracle@zeroserv01 ~]$<\/pre>\n
Clean up<\/h4>\n
Conclusion<\/h2>\n
\n