节点2上crsd无法启动，数据库和监听无法自动启动，比如ocrconfig、ocrcheck以及srvct
CRSD进程在11g中的变化
在11.2中，CRSD进程不再是RAC中最关键的进程之一。
如果对10g RAC比较熟悉，应该清楚CRSD进程的重要性，Oracle在操作系统启动后，就是通过启动这个进程然后启动整个CLUSTER以及数据库的。
在11.2的RAC中，Oracle调整了ASM，使得OCR和VOT可以存储在ASM磁盘组中。ASM是CLUSTER所支持的一个组件，而CLUSTER启动所需的OCR和VOT却要放在ASM中，这其实要解决一个先有鸡还是先有蛋的问题。最终Oracle通过OHASD进程的方式解决了这个问题，而整个CLUSTER和ASM的架构也发生了重大的变化，OHASD进程取代了CRSD进程变成了RAC环境中最关键的进程。
而CRSD进程的重要性已经低到难以置信的地步，前两天在一个客户的11.2 RAC环境中发现，即使一个节点的CRSD进程没有启动，仍然可以手工启动数据库，且数据库可以正常访问。
导致的问题原因应该是节点2上访问OCR和VOT所在的磁盘组出现了错误，导致CRSD在多次尝试获取OCR中存储的信息失败后自动退出，从而使得节点2无法正常的启动。不过这时节点2上除了CRSD进程外，其他的CLUSTER进程已经完全启动，ASM实例也可以启动，这时可以手工启动节点2上的数据库。
节点2上ASM的alert有如下的错误信息：
Tue Jun 13 10:59:17 2017 Reconfiguration started (old inc 10, new inc 12) List of instances: 1 2 (myinst: 1) Global Resource Directory frozen Communication channels reestablished Master broadcasted resource hash value bitmaps Non-local Process blocks cleaned out Tue Jun 13 10:59:17 2017 LMS 0: 0 GCS shadows cancelled, 0 closed, 0 Xw survived Set master node info Submitted all remote-enqueue requests Dwn-cvts replayed, VALBLKs dubious All grantable enqueues granted Submitted all GCS remote-cache requests Fix write in gcs resources Reconfiguration complete Tue Jun 13 11:03:01 2017 IPC Send timeout detected. Sender: ospid 3173 [oracle@rac1 (PING)] Receiver: inst 2 binc 429480538 ospid 3190 Tue Jun 13 12:12:38 2017 NOTE: [ocrcheck.bin@rac1 (TNS V1-V3) 21461] opening OCR file Tue Jun 13 12:12:38 2017 NOTE: [ocrcheck.bin@rac1 (TNS V1-V3) 21461] opening OCR file Tue Jun 13 13:38:34 2017 MEMORY_TARGET defaulting to 1128267776. * instance_number obtained from CSS = 1, checking for the existence of node 0... * node 0 does not exist. instance_number = 1 Starting ORACLE instance (normal) Tue Jun 13 13:42:20 2017 WARNING: Waited 15 secs for write IO to PST disk 0 in group 1. WARNING: Waited 15 secs for write IO to PST disk 0 in group 1. WARNING: Waited 15 secs for write IO to PST disk 0 in group 2. WARNING: Waited 15 secs for write IO to PST disk 0 in group 2. WARNING: Waited 15 secs for write IO to PST disk 0 in group 3. WARNING: Waited 15 secs for write IO to PST disk 0 in group 3. WARNING: Waited 15 secs for write IO to PST disk 0 in group 4. WARNING: Waited 15 secs for write IO to PST disk 0 in group 4. WARNING: Waited 15 secs for write IO to PST disk 0 in group 5. WARNING: Waited 15 secs for write IO to PST disk 0 in group 5. WARNING: Waited 15 secs for write IO to PST disk 0 in group 6. WARNING: Waited 15 secs for write IO to PST disk 0 in group 6.
应该就是这个原因导致ocrd进程报错并退出。数据库可以正常打开，节点2上数据库和监听无法自动启动，vip也存在问题。此外节点2上需要ocr信息的工具都无法使用，比如ocrconfig、ocrcheck以及srvctl等。
目前还是没有解决，如有遇到的朋友可以