RAC 11gR2模拟OCR和Votedisk损坏恢复过程
1）破坏前的ocr和votedisk信息检查检查ocr自动备份[root@rac1~]#ocrconfig-showbackuprac22013/10/1309:45:30/u01/grid/product/11.2.0/cdata/rac-cluster/backu
1）破坏前的ocr和votedisk信息检查
检查ocr自动备份
[root@rac1 ~]# ocrconfig -showbackup
rac2 2013/10/13 09:45:30 /u01/grid/product/11.2.0/cdata/rac-cluster/backup00.ocr
rac2 2013/10/13 05:45:29 /u01/grid/product/11.2.0/cdata/rac-cluster/backup01.ocr
rac2 2013/10/13 01:45:28 /u01/grid/product/11.2.0/cdata/rac-cluster/backup02.ocr
rac2 2013/10/12 01:45:26 /u01/grid/product/11.2.0/cdata/rac-cluster/day.ocr
rac2 2013/09/28 02:55:56 /u01/grid/product/11.2.0/cdata/rac-cluster/week.ocr
PROT-25: Manual backups for the Oracle Cluster Registry are not available
这里有一个PROT-25的提示信息，手工备份时无效，所以破坏后直接用自动备份恢复
检查ocr磁盘信息
[root@rac1 ~]# ocrcheck
Status of Oracle Cluster Registry is as follows :
Version : 3
Total space (kbytes) : 262120
Used space (kbytes) : 2720
Available space (kbytes) : 259400
ID : 2026562699
Device/File Name : +OCRDATA
Device/File integrity check succeeded
Device/File not configured
Device/File not configured
Device/File not configured
Device/File not configured
Cluster registry integrity check succeeded
Logical corruption check succeeded
查看votedisk磁盘信息
[root@rac1 ~]# crsctl query css votedisk
## STATE File Universal Id File Name Disk group
-- ----- ----------------- --------- ---------
1. ONLINE 5c190e6ab4c04facbfdd4ca0e836a798 (ORCL:OCR1) [OCRDATA]
2. ONLINE abcc18afe6214fbcbfa02fad1c41b21b (ORCL:OCR2) [OCRDATA]
3. ONLINE 31e0a9df91514f73bf50a4e0a344af3d (ORCL:OCR3) [OCRDATA]
Located 3 voting disk(s).
可以看到OCR磁盘和votedisk都通过ASM OCRDATA磁盘组管理
查看ASM磁盘组 OCRDATA信息，我的OCRDATA磁盘组是有OCR1-3组成所以直接通过下列命令查询
[root@rac1 ~]# /etc/init.d/oracleasm querydisk -d OCR1
Disk "OCR1" is a valid ASM disk on device /dev/sda1[8,1]
[root@rac1 ~]# /etc/init.d/oracleasm querydisk -d OCR2
Disk "OCR2" is a valid ASM disk on device /dev/sdh1[8,113]
[root@rac1 ~]# /etc/init.d/oracleasm querydisk -d OCR3
Disk "OCR3" is a valid ASM disk on device /dev/sdb1[8,17]
由于ASM参数文件保存在OCRDATA下，所以对备份一份spfile
SQL> show parameter spfile;
NAME TYPE VALUE
------------------------------------ ----------- ------------------------------
spfile string +OCRDATA/rac-cluster/asmparame
terfile/registry.253.825083547
SQL> create pfile='/tmp/asmbak.ora' from spfile; --备份到/tmp/asmbak.ora
File created.
2）模拟损坏
使用dd命令破坏这几个磁盘,
[root@rac1 ~]# dd if=/dev/zero of=/dev/sda1 bs=1M count=10
10+0 records in
10+0 records out
10485760 bytes (10 MB) copied, 0.005454 seconds, 1.9 GB/s
[root@rac1 ~]# dd if=/dev/zero of=/dev/sdh1 bs=1M count=10
10+0 records in
10+0 records out
10485760 bytes (10 MB) copied, 0.00603 seconds, 1.7 GB/s
破坏后执行下面命令，发现各节点服务一切正常
[root@rac1 ~]# crs_stat -t
Name Type Target State Host
------------------------------------------------------------
ora.DATA.dg ora....up.type ONLINE ONLINE rac1
ora.FRA.dg ora....up.type ONLINE ONLINE rac1
ora....ER.lsnr ora....er.type ONLINE ONLINE rac1
ora....N1.lsnr ora....er.type ONLINE ONLINE rac1
ora.OCRDATA.dg ora....up.type ONLINE ONLINE rac1
ora.asm ora.asm.type ONLINE ONLINE rac1
ora.eons ora.eons.type ONLINE ONLINE rac1
ora.gsd ora.gsd.type OFFLINE OFFLINE
ora....network ora....rk.type ONLINE ONLINE rac1
ora.oc4j ora.oc4j.type OFFLINE OFFLINE
ora.ons ora.ons.type ONLINE ONLINE rac1
ora....SM1.asm application ONLINE ONLINE rac1
ora....C1.lsnr application ONLINE ONLINE rac1
ora.rac1.gsd application OFFLINE OFFLINE
ora.rac1.ons application ONLINE ONLINE rac1
ora.rac1.vip ora....t1.type ONLINE ONLINE rac1
ora....SM2.asm application ONLINE ONLINE rac2
ora....C2.lsnr application ONLINE ONLINE rac2
ora.rac2.gsd application OFFLINE OFFLINE
ora.rac2.ons application ONLINE ONLINE rac2
ora.rac2.vip ora....t1.type ONLINE ONLINE rac2
ora.ractest.db ora....se.type ONLINE ONLINE rac1
ora....ry.acfs ora....fs.type ONLINE ONLINE rac1
ora.scan1.vip ora....ip.type ONLINE ONLINE rac1
[root@rac1 ~]# ocrcheck
Status of Oracle Cluster Registry is as follows :
Version : 3
Total space (kbytes) : 262120
Used space (kbytes) : 2720
Available space (kbytes) : 259400
ID : 2026562699
Device/File Name : +OCRDATA
Device/File integrity check succeeded
Device/File not configured
Device/File not configured
Device/File not configured
Device/File not configured
Cluster registry integrity check succeeded
Logical corruption check succeeded
[root@rac1 ~]# crsctl query css votedisk
## STATE File Universal Id File Name Disk group
-- ----- ----------------- --------- ---------
1. ONLINE 5c190e6ab4c04facbfdd4ca0e836a798 (ORCL:OCR1) [OCRDATA]
2. ONLINE abcc18afe6214fbcbfa02fad1c41b21b (ORCL:OCR2) [OCRDATA]
3. ONLINE 31e0a9df91514f73bf50a4e0a344af3d (ORCL:OCR3) [OCRDATA]
Located 3 voting disk(s).
停止RAC1crs服务
[root@rac1 ~]# crsctl stop crs
CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'rac1'
CRS-2673: Attempting to stop 'ora.crsd' on 'rac1'
CRS-2790: Starting shutdown of Cluster Ready Services-managed resources on 'rac1'
CRS-2673: Attempting to stop 'ora.LISTENER.lsnr' on 'rac1'
CRS-2673: Attempting to stop 'ora.LISTENER_SCAN1.lsnr' on 'rac1'
CRS-2673: Attempting to stop 'ora.OCRDATA.dg' on 'rac1'
CRS-2673: Attempting to stop 'ora.ractest.db' on 'rac1'
CRS-2673: Attempting to stop 'ora.registry.acfs' on 'rac1'
CRS-2677: Stop of 'ora.registry.acfs' on 'rac1' succeeded
CRS-2677: Stop of 'ora.LISTENER.lsnr' on 'rac1' succeeded
CRS-2673: Attempting to stop 'ora.rac1.vip' on 'rac1'
CRS-2677: Stop of 'ora.LISTENER_SCAN1.lsnr' on 'rac1' succeeded
CRS-2673: Attempting to stop 'ora.scan1.vip' on 'rac1'
CRS-2677: Stop of 'ora.rac1.vip' on 'rac1' succeeded
CRS-2672: Attempting to start 'ora.rac1.vip' on 'rac2'
CRS-2677: Stop of 'ora.scan1.vip' on 'rac1' succeeded
CRS-2672: Attempting to start 'ora.scan1.vip' on 'rac2'
CRS-2676: Start of 'ora.rac1.vip' on 'rac2' succeeded
CRS-2676: Start of 'ora.scan1.vip' on 'rac2' succeeded
CRS-2672: Attempting to start 'ora.LISTENER_SCAN1.lsnr' on 'rac2'
CRS-2676: Start of 'ora.LISTENER_SCAN1.lsnr' on 'rac2' succeeded
CRS-2677: Stop of 'ora.OCRDATA.dg' on 'rac1' succeeded
CRS-2677: Stop of 'ora.ractest.db' on 'rac1' succeeded
CRS-2673: Attempting to stop 'ora.DATA.dg' on 'rac1'
CRS-2673: Attempting to stop 'ora.FRA.dg' on 'rac1'
CRS-2677: Stop of 'ora.FRA.dg' on 'rac1' succeeded
CRS-2677: Stop of 'ora.DATA.dg' on 'rac1' succeeded
CRS-2673: Attempting to stop 'ora.asm' on 'rac1'
CRS-2677: Stop of 'ora.asm' on 'rac1' succeeded
CRS-2673: Attempting to stop 'ora.ons' on 'rac1'
CRS-2673: Attempting to stop 'ora.eons' on 'rac1'
CRS-2677: Stop of 'ora.ons' on 'rac1' succeeded
CRS-2673: Attempting to stop 'ora.net1.network' on 'rac1'
CRS-2677: Stop of 'ora.net1.network' on 'rac1' succeeded
CRS-2677: Stop of 'ora.eons' on 'rac1' succeeded
CRS-2792: Shutdown of Cluster Ready Services-managed resources on 'rac1' has completed
CRS-2677: Stop of 'ora.crsd' on 'rac1' succeeded
CRS-2673: Attempting to stop 'ora.cssdmonitor' on 'rac1'
CRS-2673: Attempting to stop 'ora.ctssd' on 'rac1'
CRS-2673: Attempting to stop 'ora.evmd' on 'rac1'
CRS-2673: Attempting to stop 'ora.asm' on 'rac1'
CRS-2673: Attempting to stop 'ora.drivers.acfs' on 'rac1'
CRS-2673: Attempting to stop 'ora.mdnsd' on 'rac1'
CRS-2677: Stop of 'ora.cssdmonitor' on 'rac1' succeeded
CRS-2677: Stop of 'ora.evmd' on 'rac1' succeeded
CRS-2677: Stop of 'ora.mdnsd' on 'rac1' succeeded
CRS-2677: Stop of 'ora.ctssd' on 'rac1' succeeded
CRS-2677: Stop of 'ora.drivers.acfs' on 'rac1' succeeded
CRS-2677: Stop of 'ora.asm' on 'rac1' succeeded
CRS-2673: Attempting to stop 'ora.cssd' on 'rac1'
CRS-2677: Stop of 'ora.cssd' on 'rac1' succeeded
CRS-2673: Attempting to stop 'ora.gpnpd' on 'rac1'
CRS-2673: Attempting to stop 'ora.diskmon' on 'rac1'
CRS-2677: Stop of 'ora.gpnpd' on 'rac1' succeeded
CRS-2673: Attempting to stop 'ora.gipcd' on 'rac1'
CRS-2677: Stop of 'ora.gipcd' on 'rac1' succeeded
CRS-2677: Stop of 'ora.diskmon' on 'rac1' succeeded
CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'rac1' has completed
CRS-4133: Oracle High Availability Services has been stopped.
3） 故障定位
在此启动发现已经无法正常启动了，跟踪alter日志，（由于系统日志没有发现什么重要信息，这里就没贴出来）
[root@rac1 ~]# tail -f /u01/grid/product/11.2.0/log/rac1/alertrac1.log
........
[ohasd(28327)]CRS-2765:Resource 'ora.cssdmonitor' has failed on server 'rac1'.
2013-10-13 11:22:46.094
[cssd(28791)]CRS-1713:CSSD daemon is started in clustered mode
2013-10-13 11:22:46.178
[cssd(28791)]CRS-1637:Unable to locate configured voting file with ID 5c190e6a-b4c04fac-bfdd4ca0-e836a798; details at (:CSSNM00020:) in /u01/grid/product/11.2.0/log/rac1/cssd/ocssd.log
2013-10-13 11:22:46.179
[cssd(28791)]CRS-1637:Unable to locate configured voting file with ID abcc18af-e6214fbc-bfa02fad-1c41b21b; details at (:CSSNM00020:) in /u01/grid/product/11.2.0/log/rac1/cssd/ocssd.log
2013-10-13 11:22:46.179
Hot AI Tools
Undresser.AI Undress
AI-powered app for creating realistic nude photos
AI Clothes Remover
Online AI tool for removing clothes from photos.
Undress AI Tool
Undress images for free
Clothoff.io
AI clothes remover
AI Hentai Generator
Generate AI Hentai for free.
Hot Article
Hot Tools
Notepad++7.3.1
Easy-to-use and free code editor
SublimeText3 Chinese version
Chinese version, very easy to use
Zend Studio 13.0.1
Powerful PHP integrated development environment
Dreamweaver CS6
Visual web development tools
SublimeText3 Mac version
God-level code editing software (SublimeText3)
Hot Topics
How to solve the problem of mysql cannot open shared library
Mar 04, 2025 pm 04:01 PM
This article addresses MySQL's "unable to open shared library" error. The issue stems from MySQL's inability to locate necessary shared libraries (.so/.dll files). Solutions involve verifying library installation via the system's package m
Reduce the use of MySQL memory in Docker
Mar 04, 2025 pm 03:52 PM
This article explores optimizing MySQL memory usage in Docker. It discusses monitoring techniques (Docker stats, Performance Schema, external tools) and configuration strategies. These include Docker memory limits, swapping, and cgroups, alongside
How do you alter a table in MySQL using the ALTER TABLE statement?
Mar 19, 2025 pm 03:51 PM
The article discusses using MySQL's ALTER TABLE statement to modify tables, including adding/dropping columns, renaming tables/columns, and changing column data types.
Run MySQl in Linux (with/without podman container with phpmyadmin)
Mar 04, 2025 pm 03:54 PM
This article compares installing MySQL on Linux directly versus using Podman containers, with/without phpMyAdmin. It details installation steps for each method, emphasizing Podman's advantages in isolation, portability, and reproducibility, but also
What is SQLite? Comprehensive overview
Mar 04, 2025 pm 03:55 PM
This article provides a comprehensive overview of SQLite, a self-contained, serverless relational database. It details SQLite's advantages (simplicity, portability, ease of use) and disadvantages (concurrency limitations, scalability challenges). C
How do I configure SSL/TLS encryption for MySQL connections?
Mar 18, 2025 pm 12:01 PM
Article discusses configuring SSL/TLS encryption for MySQL, including certificate generation and verification. Main issue is using self-signed certificates' security implications.[Character count: 159]
Running multiple MySQL versions on MacOS: A step-by-step guide
Mar 04, 2025 pm 03:49 PM
This guide demonstrates installing and managing multiple MySQL versions on macOS using Homebrew. It emphasizes using Homebrew to isolate installations, preventing conflicts. The article details installation, starting/stopping services, and best pra
What are some popular MySQL GUI tools (e.g., MySQL Workbench, phpMyAdmin)?
Mar 21, 2025 pm 06:28 PM
Article discusses popular MySQL GUI tools like MySQL Workbench and phpMyAdmin, comparing their features and suitability for beginners and advanced users.[159 characters]
