今天是2014-03-28，继续写RAC的内容，今天写的是如何恢复损坏的表决磁盘（votedisk）和OCR。 首先查看目前状态： [root@rac-two bin]# ./ocrcheckStatus of Oracle Cluster Registry is as follows : Version : 3 Total space (kbytes) : 262120 Used space
今天是2014-03-28，继续写RAC的内容，今天写的是如何恢复损坏的表决磁盘（votedisk）和OCR。
首先查看目前状态：
[root@rac-two bin]# ./ocrcheck
Status of Oracle Cluster Registry is as follows :
Version : 3
Total space (kbytes) : 262120
Used space (kbytes) : 3472
Available space (kbytes) : 258648
ID : 152256349
Device/File Name : +GIDG
Device/File integrity check succeeded
Device/File not configured
Device/File not configured
Device/File not configured
Device/File not configured
Cluster registry integrity check succeeded
Logical corruption check succeeded
[root@rac-two bin]# ./crsctl query css votedisk
## STATE File Universal Id File Name Disk group
-- ----- ----------------- --------- ---------
1. ONLINE af2fed5d6ac54f1bbf19e4982e364c57 (/dev/asm-diske) [GIDG]
2. ONLINE 34b65f2e3db14f23bf88e689b55058b7 (/dev/asm-diskd) [GIDG]
3. ONLINE 07bf3ce9af7d4ff6bf38d160255c671c (/dev/asm-diskc) [GIDG]
Located 3 voting disk(s).
[root@rac-two bin]# ./crsctl stat res -t
--------------------------------------------------------------------------------
NAME TARGET STATE SERVER STATE_DETAILS
--------------------------------------------------------------------------------
Local Resources
--------------------------------------------------------------------------------
ora.DATADG.dg
ONLINE ONLINE rac-one
ONLINE ONLINE rac-two
ora.GIDG.dg
ONLINE ONLINE rac-one
ONLINE ONLINE rac-two
ora.LISTENER.lsnr
ONLINE ONLINE rac-one
ONLINE ONLINE rac-two
ora.TESTDG.dg
OFFLINE OFFLINE rac-one
ONLINE OFFLINE rac-two
ora.asm
ONLINE ONLINE rac-one Started
ONLINE ONLINE rac-two Started
ora.gsd
OFFLINE OFFLINE rac-one
OFFLINE OFFLINE rac-two
ora.net1.network
ONLINE ONLINE rac-one
ONLINE ONLINE rac-two
ora.ons
ONLINE ONLINE rac-one
ONLINE ONLINE rac-two
ora.registry.acfs
ONLINE ONLINE rac-one
ONLINE ONLINE rac-two
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.LISTENER_SCAN1.lsnr
1 ONLINE ONLINE rac-two
ora.LISTENER_SCAN2.lsnr
1 ONLINE ONLINE rac-one
ora.LISTENER_SCAN3.lsnr
1 ONLINE ONLINE rac-one
ora.cvu
1 ONLINE ONLINE rac-one
ora.oc4j
1 ONLINE ONLINE rac-one
ora.rac-one.vip
1 ONLINE ONLINE rac-one
ora.rac-two.vip
1 ONLINE ONLINE rac-two
ora.rac.db
1 ONLINE ONLINE rac-two Open
2 ONLINE ONLINE rac-one Open
ora.scan1.vip
1 ONLINE ONLINE rac-two
ora.scan2.vip
1 ONLINE ONLINE rac-one
ora.scan3.vip
1 ONLINE ONLINE rac-one Copy after login
模拟votedisk和ocr损坏。
[root@rac-two bin]# dd if=/dev/zero of=/dev/asm-diske bs=1024 count=200;
dd if=/dev/zero of=/dev/asm-diskd bs=1024 count=200;
dd if=/dev/zero of=/dev/asm-diskc bs=1024 count=200;200+0 records in
200+0 records out
204800 bytes (205 kB) copied, 0.0884688 s, 2.3 MB/s
[root@rac-two bin]# dd if=/dev/zero of=/dev/asm-diskd bs=1024 count=200;
200+0 records in
200+0 records out
204800 bytes (205 kB) copied, 0.138348 s, 1.5 MB/s
[root@rac-two bin]# dd if=/dev/zero of=/dev/asm-diskc bs=1024 count=200;
200+0 records in
200+0 records out
204800 bytes (205 kB) copied, 0.103544 s, 2.0 MB/s
Copy after login
开始进行恢复：
停止crs资源。
[root@rac-two bin]# ./crsctl stop crs
CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'rac-two'
CRS-2673: Attempting to stop 'ora.crsd' on 'rac-two'
CRS-2790: Starting shutdown of Cluster Ready Services-managed resources on 'rac-two'
CRS-2673: Attempting to stop 'ora.LISTENER.lsnr' on 'rac-two'
CRS-2673: Attempting to stop 'ora.LISTENER_SCAN1.lsnr' on 'rac-two'
CRS-2673: Attempting to stop 'ora.GIDG.dg' on 'rac-two'
CRS-2673: Attempting to stop 'ora.registry.acfs' on 'rac-two'
CRS-2673: Attempting to stop 'ora.rac.db' on 'rac-two'
CRS-2677: Stop of 'ora.LISTENER.lsnr' on 'rac-two' succeeded
CRS-2673: Attempting to stop 'ora.rac-two.vip' on 'rac-two'
CRS-2677: Stop of 'ora.LISTENER_SCAN1.lsnr' on 'rac-two' succeeded
CRS-2673: Attempting to stop 'ora.scan1.vip' on 'rac-two'
CRS-2677: Stop of 'ora.rac-two.vip' on 'rac-two' succeeded
CRS-2672: Attempting to start 'ora.rac-two.vip' on 'rac-one'
CRS-2677: Stop of 'ora.registry.acfs' on 'rac-two' succeeded
CRS-2677: Stop of 'ora.scan1.vip' on 'rac-two' succeeded
CRS-2672: Attempting to start 'ora.scan1.vip' on 'rac-one'
CRS-2677: Stop of 'ora.rac.db' on 'rac-two' succeeded
CRS-2673: Attempting to stop 'ora.DATADG.dg' on 'rac-two'
CRS-2677: Stop of 'ora.DATADG.dg' on 'rac-two' succeeded
CRS-2676: Start of 'ora.rac-two.vip' on 'rac-one' succeeded
CRS-2676: Start of 'ora.scan1.vip' on 'rac-one' succeeded
CRS-2672: Attempting to start 'ora.LISTENER_SCAN1.lsnr' on 'rac-one'
CRS-2677: Stop of 'ora.GIDG.dg' on 'rac-two' succeeded
CRS-2673: Attempting to stop 'ora.asm' on 'rac-two'
CRS-2677: Stop of 'ora.asm' on 'rac-two' succeeded
CRS-2676: Start of 'ora.LISTENER_SCAN1.lsnr' on 'rac-one' succeeded
CRS-2673: Attempting to stop 'ora.ons' on 'rac-two'
CRS-2677: Stop of 'ora.ons' on 'rac-two' succeeded
CRS-2673: Attempting to stop 'ora.net1.network' on 'rac-two'
CRS-2677: Stop of 'ora.net1.network' on 'rac-two' succeeded
CRS-2792: Shutdown of Cluster Ready Services-managed resources on 'rac-two' has completed
CRS-2677: Stop of 'ora.crsd' on 'rac-two' succeeded
CRS-2673: Attempting to stop 'ora.crf' on 'rac-two'
CRS-2673: Attempting to stop 'ora.ctssd' on 'rac-two'
CRS-2673: Attempting to stop 'ora.evmd' on 'rac-two'
CRS-2673: Attempting to stop 'ora.asm' on 'rac-two'
CRS-2673: Attempting to stop 'ora.drivers.acfs' on 'rac-two'
CRS-2673: Attempting to stop 'ora.mdnsd' on 'rac-two'
CRS-2677: Stop of 'ora.crf' on 'rac-two' succeeded
CRS-2677: Stop of 'ora.evmd' on 'rac-two' succeeded
CRS-2677: Stop of 'ora.mdnsd' on 'rac-two' succeeded
CRS-2677: Stop of 'ora.asm' on 'rac-two' succeeded
CRS-2673: Attempting to stop 'ora.cluster_interconnect.haip' on 'rac-two'
CRS-2677: Stop of 'ora.ctssd' on 'rac-two' succeeded
CRS-2677: Stop of 'ora.cluster_interconnect.haip' on 'rac-two' succeeded
CRS-2673: Attempting to stop 'ora.cssd' on 'rac-two'
CRS-2677: Stop of 'ora.drivers.acfs' on 'rac-two' succeeded
CRS-2677: Stop of 'ora.cssd' on 'rac-two' succeeded
CRS-2673: Attempting to stop 'ora.gipcd' on 'rac-two'
CRS-2677: Stop of 'ora.gipcd' on 'rac-two' succeeded
CRS-2673: Attempting to stop 'ora.gpnpd' on 'rac-two'
CRS-2677: Stop of 'ora.gpnpd' on 'rac-two' succeeded
CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'rac-two' has completed
CRS-4133: Oracle High Availability Services has been stopped.
Copy after login
启动到独占模式且不启动ora.crsd:
[root@rac-two bin]# ./crsctl start crs -excl -nocrs
CRS-4123: Oracle High Availability Services has been started.
CRS-2672: Attempting to start 'ora.mdnsd' on 'rac-two'
CRS-2676: Start of 'ora.mdnsd' on 'rac-two' succeeded
CRS-2672: Attempting to start 'ora.gpnpd' on 'rac-two'
CRS-2676: Start of 'ora.gpnpd' on 'rac-two' succeeded
CRS-2672: Attempting to start 'ora.cssdmonitor' on 'rac-two'
CRS-2672: Attempting to start 'ora.gipcd' on 'rac-two'
CRS-2676: Start of 'ora.cssdmonitor' on 'rac-two' succeeded
CRS-2676: Start of 'ora.gipcd' on 'rac-two' succeeded
CRS-2672: Attempting to start 'ora.cssd' on 'rac-two'
CRS-2672: Attempting to start 'ora.diskmon' on 'rac-two'
CRS-2676: Start of 'ora.diskmon' on 'rac-two' succeeded
CRS-2676: Start of 'ora.cssd' on 'rac-two' succeeded
CRS-2672: Attempting to start 'ora.drivers.acfs' on 'rac-two'
CRS-2679: Attempting to clean 'ora.cluster_interconnect.haip' on 'rac-two'
CRS-2672: Attempting to start 'ora.ctssd' on 'rac-two'
CRS-2681: Clean of 'ora.cluster_interconnect.haip' on 'rac-two' succeeded
CRS-2672: Attempting to start 'ora.cluster_interconnect.haip' on 'rac-two'
CRS-2676: Start of 'ora.drivers.acfs' on 'rac-two' succeeded
CRS-2676: Start of 'ora.ctssd' on 'rac-two' succeeded
CRS-2676: Start of 'ora.cluster_interconnect.haip' on 'rac-two' succeeded
CRS-2672: Attempting to start 'ora.asm' on 'rac-two'
CRS-2676: Start of 'ora.asm' on 'rac-two' succeeded
Copy after login
创建磁盘组：
[root@rac-two bin]# su - grid
[grid@rac-two ~]$ sqlplus / as sysasm
SQL*Plus: Release 11.2.0.4.0 Production on Fri Mar 28 00:01:05 2014
Copyright (c) 1982, 2013, Oracle. All rights reserved.
Connected to:
Oracle Database 11g Enterprise Edition Release 11.2.0.4.0 - 64bit Production
With the Real Application Clusters and Automatic Storage Management options
SQL> create diskgroup gidg normal redundancy
2 disk '/dev/asm-diske','/dev/asm-diskd','/dev/asm-diskc' attribute 'au_size'='1M','compatible.asm'='11.2','compatible.rdbms'='11.2','compatible.advm'='11.2';
Diskgroup created.
SQL> select name,state from v$asm_diskgroup;
NAME STATE
------------------------------ -----------
DATADG DISMOUNTED
TESTDG DISMOUNTED
GIDG MOUNTED
SQL> exit
Disconnected from Oracle Database 11g Enterprise Edition Release 11.2.0.4.0 - 64bit Production
With the Real Application Clusters and Automatic Storage Management options
[grid@rac-two ~]$ exit
Copy after login
首先要恢复的是ocr
[root@rac-two bin]# ./ocrconfig -restore /u01/app/11.2.0/grid/cdata/rac-two-cluster/backup00.ocr
[root@rac-two bin]# ./ocrcheck
Status of Oracle Cluster Registry is as follows :
Version : 3
Total space (kbytes) : 262120
Used space (kbytes) : 3472
Available space (kbytes) : 258648
ID : 152256349
Device/File Name : +GIDG
Device/File integrity check succeeded
Device/File not configured
Device/File not configured
Device/File not configured
Device/File not configured
Cluster registry integrity check succeeded
Logical corruption check succeeded
Copy after login
恢复votedisk：
[root@rac-two bin]# ./crsctl query css votedisk
Located 0 voting disk(s).
[root@rac-two bin]# ./crsctl replace votedisk +GIDG
CRS-4602: Failed 27 to add voting file 58c5df27b4884f70bfb38fc4c93b8f72.
CRS-4602: Failed 27 to add voting file 51caa3f834964f5abf6efcfb90476d85.
CRS-4602: Failed 27 to add voting file 92b70ca3ea5d4f80bf9df66a8e975ba9.
Failed to replace voting disk group with +GIDG.
CRS-4000: Command Replace failed, or completed with errors.
Copy after login
提示失败，需要重启crs资源。
[root@rac-two bin]# ./crsctl stop crs
CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'rac-two'
CRS-2673: Attempting to stop 'ora.ctssd' on 'rac-two'
CRS-2673: Attempting to stop 'ora.asm' on 'rac-two'
CRS-2673: Attempting to stop 'ora.drivers.acfs' on 'rac-two'
CRS-2673: Attempting to stop 'ora.mdnsd' on 'rac-two'
CRS-2677: Stop of 'ora.mdnsd' on 'rac-two' succeeded
CRS-2677: Stop of 'ora.asm' on 'rac-two' succeeded
CRS-2673: Attempting to stop 'ora.cluster_interconnect.haip' on 'rac-two'
CRS-2677: Stop of 'ora.cluster_interconnect.haip' on 'rac-two' succeeded
CRS-2677: Stop of 'ora.drivers.acfs' on 'rac-two' succeeded
CRS-2677: Stop of 'ora.ctssd' on 'rac-two' succeeded
CRS-2673: Attempting to stop 'ora.cssd' on 'rac-two'
CRS-2677: Stop of 'ora.cssd' on 'rac-two' succeeded
CRS-2673: Attempting to stop 'ora.gipcd' on 'rac-two'
CRS-2677: Stop of 'ora.gipcd' on 'rac-two' succeeded
CRS-2673: Attempting to stop 'ora.gpnpd' on 'rac-two'
CRS-2677: Stop of 'ora.gpnpd' on 'rac-two' succeeded
CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'rac-two' has completed
CRS-4133: Oracle High Availability Services has been stopped.
[root@rac-two bin]# ./crsctl start crs -excl -nocrs
CRS-4123: Oracle High Availability Services has been started.
CRS-2672: Attempting to start 'ora.mdnsd' on 'rac-two'
CRS-2676: Start of 'ora.mdnsd' on 'rac-two' succeeded
CRS-2672: Attempting to start 'ora.gpnpd' on 'rac-two'
CRS-2676: Start of 'ora.gpnpd' on 'rac-two' succeeded
CRS-2672: Attempting to start 'ora.cssdmonitor' on 'rac-two'
CRS-2672: Attempting to start 'ora.gipcd' on 'rac-two'
CRS-2676: Start of 'ora.cssdmonitor' on 'rac-two' succeeded
CRS-2676: Start of 'ora.gipcd' on 'rac-two' succeeded
CRS-2672: Attempting to start 'ora.cssd' on 'rac-two'
CRS-2672: Attempting to start 'ora.diskmon' on 'rac-two'
CRS-2676: Start of 'ora.diskmon' on 'rac-two' succeeded
CRS-2676: Start of 'ora.cssd' on 'rac-two' succeeded
CRS-2672: Attempting to start 'ora.drivers.acfs' on 'rac-two'
CRS-2679: Attempting to clean 'ora.cluster_interconnect.haip' on 'rac-two'
CRS-2672: Attempting to start 'ora.ctssd' on 'rac-two'
CRS-2681: Clean of 'ora.cluster_interconnect.haip' on 'rac-two' succeeded
CRS-2672: Attempting to start 'ora.cluster_interconnect.haip' on 'rac-two'
CRS-2676: Start of 'ora.drivers.acfs' on 'rac-two' succeeded
CRS-2676: Start of 'ora.ctssd' on 'rac-two' succeeded
CRS-2676: Start of 'ora.cluster_interconnect.haip' on 'rac-two' succeeded
CRS-2672: Attempting to start 'ora.asm' on 'rac-two'
CRS-2676: Start of 'ora.asm' on 'rac-two' succeeded
[root@rac-two bin]# su - grid
[grid@rac-two ~]$ sqlplus / as sysasm
SQL*Plus: Release 11.2.0.4.0 Production on Fri Mar 28 00:13:30 2014
Copyright (c) 1982, 2013, Oracle. All rights reserved.
Connected to:
Oracle Database 11g Enterprise Edition Release 11.2.0.4.0 - 64bit Production
With the Real Application Clusters and Automatic Storage Management options
SQL> col name for a30
SQL> col path for a50
SQL> set linesize 200
SQL> select name,path from v$asm_disk;
no rows selected
SQL> alter system set asm_diskstring='/dev/asm-disk*';
System altered.
SQL> select name,path from v$asm_disk;
NAME PATH
------------------------------ --------------------------------------------------
/dev/asm-diskg
/dev/asm-diskd
/dev/asm-diskc
/dev/asm-diskf
/dev/asm-diskh
/dev/asm-diskm
/dev/asm-diskk
/dev/asm-diskj
/dev/asm-diski
/dev/asm-diskb
/dev/asm-diske
NAME PATH
------------------------------ --------------------------------------------------
/dev/asm-diskl
12 rows selected.
SQL> select name,state from v$asm_diskgroup;
NAME STATE
------------------------------ -----------
DATADG DISMOUNTED
GIDG DISMOUNTED
TESTDG DISMOUNTED
SQL> alter diskgroup gidg mount;
Diskgroup altered.
SQL> select name,state from v$asm_diskgroup;
NAME STATE
------------------------------ -----------
DATADG DISMOUNTED
GIDG MOUNTED
TESTDG DISMOUNTED
SQL> exit
Disconnected from Oracle Database 11g Enterprise Edition Release 11.2.0.4.0 - 64bit Production
With the Real Application Clusters and Automatic Storage Management options
[grid@rac-two ~]$ exit
logout
[root@rac-two bin]# ./ocrctl replace votedisk +GIDG
-bash: ./ocrctl: No such file or directory
[root@rac-two bin]# ./crsctl replace votedisk +GIDG
Successful addition of voting disk 4b55dbd068344fb6bfb89250b3586256.
Successful addition of voting disk 6b157dc173bd4f7ebf1846df01c23d5d.
Successful addition of voting disk 505d811a77d64f57bfa2797afba6a5f5.
Successfully replaced voting disk group with +GIDG.
CRS-4266: Voting file(s) successfully replacedCopy after login
至此完成了ocr和votedisk的恢复。
重启crs，查看状况：
[root@rac-two bin]# ./crsctl stop crs
CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'rac-two'
CRS-2673: Attempting to stop 'ora.mdnsd' on 'rac-two'
CRS-2673: Attempting to stop 'ora.drivers.acfs' on 'rac-two'
CRS-2673: Attempting to stop 'ora.ctssd' on 'rac-two'
CRS-2673: Attempting to stop 'ora.asm' on 'rac-two'
CRS-2677: Stop of 'ora.mdnsd' on 'rac-two' succeeded
CRS-2677: Stop of 'ora.asm' on 'rac-two' succeeded
CRS-2673: Attempting to stop 'ora.cluster_interconnect.haip' on 'rac-two'
CRS-2677: Stop of 'ora.ctssd' on 'rac-two' succeeded
CRS-2677: Stop of 'ora.drivers.acfs' on 'rac-two' succeeded
CRS-2677: Stop of 'ora.cluster_interconnect.haip' on 'rac-two' succeeded
CRS-2673: Attempting to stop 'ora.cssd' on 'rac-two'
CRS-2677: Stop of 'ora.cssd' on 'rac-two' succeeded
CRS-2673: Attempting to stop 'ora.gipcd' on 'rac-two'
CRS-2677: Stop of 'ora.gipcd' on 'rac-two' succeeded
CRS-2673: Attempting to stop 'ora.gpnpd' on 'rac-two'
CRS-2677: Stop of 'ora.gpnpd' on 'rac-two' succeeded
CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'rac-two' has completed
CRS-4133: Oracle High Availability Services has been stopped.
[root@rac-two bin]# ./crsctl start crs
CRS-4123: Oracle High Availability Services has been started.
[grid@rac-two ~]$ crsctl stat res -t
--------------------------------------------------------------------------------
NAME TARGET STATE SERVER STATE_DETAILS
--------------------------------------------------------------------------------
Local Resources
--------------------------------------------------------------------------------
ora.DATADG.dg
ONLINE ONLINE rac-two
ora.GIDG.dg
ONLINE ONLINE rac-two
ora.LISTENER.lsnr
ONLINE ONLINE rac-two
ora.TESTDG.dg
ONLINE OFFLINE rac-two
ora.asm
ONLINE ONLINE rac-two Started
ora.gsd
OFFLINE OFFLINE rac-two
ora.net1.network
ONLINE ONLINE rac-two
ora.ons
ONLINE ONLINE rac-two
ora.registry.acfs
ONLINE ONLINE rac-two
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.LISTENER_SCAN1.lsnr
1 ONLINE ONLINE rac-two
ora.LISTENER_SCAN2.lsnr
1 ONLINE ONLINE rac-two
ora.LISTENER_SCAN3.lsnr
1 ONLINE ONLINE rac-two
ora.cvu
1 ONLINE ONLINE rac-two
ora.oc4j
1 ONLINE ONLINE rac-two
ora.rac-one.vip
1 ONLINE INTERMEDIATE rac-two FAILED OVER
ora.rac-two.vip
1 ONLINE ONLINE rac-two
ora.rac.db
1 ONLINE ONLINE rac-two Open
2 ONLINE OFFLINE
ora.scan1.vip
1 ONLINE ONLINE rac-two
ora.scan2.vip
1 ONLINE ONLINE rac-two
ora.scan3.vip
1 ONLINE ONLINE rac-two
[grid@rac-two ~]$ Copy after login
在节点一重启crs资源，将其加入集群。
[root@rac-one bin]# ./crsctl start crs
CRS-4123: Oracle High Availability Services has been started.
[root@rac-one bin]# ./crsctl stat res -t
--------------------------------------------------------------------------------
NAME TARGET STATE SERVER STATE_DETAILS
--------------------------------------------------------------------------------
Local Resources
--------------------------------------------------------------------------------
ora.DATADG.dg
ONLINE ONLINE rac-one
ONLINE ONLINE rac-two
ora.GIDG.dg
ONLINE ONLINE rac-one
ONLINE ONLINE rac-two
ora.LISTENER.lsnr
ONLINE ONLINE rac-one
ONLINE ONLINE rac-two
ora.TESTDG.dg
OFFLINE OFFLINE rac-one
ONLINE OFFLINE rac-two
ora.asm
ONLINE ONLINE rac-one Started
ONLINE ONLINE rac-two Started
ora.gsd
OFFLINE OFFLINE rac-one
OFFLINE OFFLINE rac-two
ora.net1.network
ONLINE ONLINE rac-one
ONLINE ONLINE rac-two
ora.ons
ONLINE ONLINE rac-one
ONLINE ONLINE rac-two
ora.registry.acfs
ONLINE ONLINE rac-one
ONLINE ONLINE rac-two
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.LISTENER_SCAN1.lsnr
1 ONLINE ONLINE rac-one
ora.LISTENER_SCAN2.lsnr
1 ONLINE ONLINE rac-two
ora.LISTENER_SCAN3.lsnr
1 ONLINE ONLINE rac-two
ora.cvu
1 ONLINE ONLINE rac-two
ora.oc4j
1 ONLINE ONLINE rac-two
ora.rac-one.vip
1 ONLINE ONLINE rac-one
ora.rac-two.vip
1 ONLINE ONLINE rac-two
ora.rac.db
1 ONLINE ONLINE rac-two Open
2 ONLINE ONLINE rac-one Open
ora.scan1.vip
1 ONLINE ONLINE rac-one
ora.scan2.vip
1 ONLINE ONLINE rac-two
ora.scan3.vip
1 ONLINE ONLINE rac-two
[root@rac-one bin]# Copy after login
至此完成了ocr和votedisk的恢复。
THAT‘s all！！！！！
![[Web front-end] Node.js quick start](https://img.php.cn/upload/course/000/000/067/662b5d34ba7c0227.png)
