Environment： AIX 6.1 ，TWO node rac，oracle 10.2.0.4
Symptom ：
用户反映无法打开MES程序，连接到2号节点查看，发现sqlplus /as sysdba
回车后卡死，cancel不掉？重新开一个连接，到root账户下 crsctl check crs 命令也卡死，lsnrctl status 命令卡死，OS命令正常。
1号节点正常。
Alterlog有如下报错
Sat Nov 3 16:37:32 2012
Thread 2 advanced to log sequence 55778 (LGWR switch)
Current log# 8 seq# 55778 mem# 0: +SFC3DB_DATA/sfc3db/onlinelog/group_8.304.722515055
Current log# 8 seq# 55778 mem# 1: +SFC3DB_ARCH/sfc3db/onlinelog/group_8.264.722515055
Sat Nov 3 16:40:19 2012
Errors in file /apps/oracle/admin/sfc3db/bdump/sfc3db2_arc0_1843352.trc:
ORA-00240: control file enqueue held for more than 120 seconds
Sat Nov 3 16:42:29 2012
Errors in file /apps/oracle/admin/sfc3db/bdump/sfc3db2_arc1_565424.trc:
ORA-00240: control file enqueue held for more than 120 seconds
Sat Nov 3 16:47:00 2012
kkjcre1p: unable to spawn jobq slave process
Sat Nov 3 16:47:00 2012
Sat Nov 3 17:04:22 2012
ksvsubmit: Process(O001) creation failed
系统应该是无法分配新的进程了，所以这些命令都卡死， 但是当时OS free的内存还有10多G，也远没有达到最大进程数。
我最后的解决办法是直接reboot OS，重启后OK， 各位大神有遇到过这种情况的吗？需要什么LOG 我发上来。
如下是trc文件
sfc3rac2:/apps/oracle/admin/sfc3db/bdump$ cat /apps/oracle/admin/sfc3db/bdump/sfc3db2_cjq0_5861420.trc
Dump file /apps/oracle/admin/sfc3db/bdump/sfc3db2_cjq0_5861420.trc
Oracle Database 10g Enterprise Edition Release 10.2.0.4.0 - 64bit Production
With the Partitioning, Real Application Clusters, OLAP, Data Mining
and Real Application Testing options
ORACLE_HOME = /apps/oracle/product/10.2.0
System name: AIX
Node name: sfc3rac2
Release: 1
Version: 6
Machine: 00C8B9724C00
Instance name: sfc3db2
Redo thread mounted by this instance: 2
Oracle process number: 15
Unix process pid: 5861420, image: oracle@sfc3rac2 (CJQ0)
*** 2012-11-03 06:00:41.427
*** SERVICE NAMESYS$BACKGROUND) 2012-11-03 06:00:41.419
*** SESSION ID2192.1) 2012-11-03 06:00:41.419
GATHER_STATS_JOB: Stopped by Scheduler.
Consider increasing the maintenance window duration if this happens frequently.
The following objects/segments were not analyzed due to timeout:
*** 2012-11-03 16:31:39.479
Waited for process J000 to initialize for 60 seconds
*** 2012-11-03 16:31:39.479
Dumping diagnostic information for J000: