环境：p550（aix5.3-ML03）、HA（5.2 -MLIY58496）、fastT600 、RAC（oracle9i-ML06）
现象：
安装过程都没有错误，创建数据库也没有问题，问题就出在两边不能同时startup。当一节点startup之后，另外一个节点就不能startup（症状就是初始化SGA之后一直等待，不会出出现database mounted等信息，如果将已经startup的节点shutdown，这个就startup成功。）
无论哪个节点先startup，就肯定成功，后startup的就不成功。
不知道是怎么回事，困扰了我10多天了，寻求帮助，谢谢！
vitovito 回复于：2005-11-09 16:31:41
在alert日志中有如下信息：
ALTER DATABASE MOUNT
Fri Oct 28 16:20:50 2005
lmon registered with NM - instance id 2 (internal mem no 1)
Fri Oct 28 16:20:50 2005
Reconfiguration started
List of nodes: 1,
Global Resource Directory frozen
one node partition
Communication channels reestablished
Master broadcasted resource hash value bitmaps
Non-local Process blocks cleaned out
Resources and enqueues cleaned out
Resources remastered 0
0 GCS shadows traversed, 0 cancelled, 0 closed
0 GCS resources traversed, 0 cancelled
set master node info
Submitted all remote-enqueue requests
Update rdomain variables
Dwn-cvts replayed, VALBLKs dubious
All grantable enqueues granted
0 GCS shadows traversed, 0 replayed, 0 unopened
Submitted all GCS remote-cache requests
0 write requests issued in 0 GCS resources
0 PIs marked suspect, 0 flush PI msgs
yanbing 回复于：2005-11-09 16:55:40
不能同时startup通常原因是来源于磁盘的共享读写状态。
vitovito 回复于：2005-11-09 17:06:30
那意思说，HA 里的vg建立的又问题？还是？？？
yanbing 回复于：2005-11-09 17:11:58
我做rac的时候习惯上是不用ha里面的vg建立选项的。。。
不知道是不是凑巧回避了这种建立共享vg的问题。
有时间的话，可以试试啊！
vitovito 回复于：2005-11-09 19:34:44
斑竹，那你怎么样测试RAC，先建立一个vg，在一台机器上启动oracle，然后varyoffvg，在另一台机器上varyon，再启动数据库？
最终你还得利用HA带动vg呀，问题还是避免不了呀，我在那台机器上启动oracle都没有问题，就是不能同时启动，郁闷呀，问题还是无法回避呀！
yeebb 回复于：2005-11-10 09:05:19
用rda检查一下的安装
louis095 回复于：2005-11-10 18:30:21
我做的hacmp for rac , create share vg ----> concurrent mode , after 2 台node hacmp service up , 看到的 vg 都是 on 的 , 請試試 .
blue_stone 回复于：2005-11-11 15:37:29
设置cluster_interconnects初始化参数试一下
SIDPRIFIX1.cluster_interconnects='192.168.0.207'
SIDPRIFIX2.cluster_interconnects='192.168.0.208'
其中ip地址是本机用于rac interconnect的网络ip地址
vitovito 回复于：2005-11-11 16:05:50
引用：原帖由 blue_stone 于 2005-11-11 15:37 发表
设置cluster_interconnects初始化参数试一下
SIDPRIFIX1.cluster_interconnects='192.168.0.207'
SIDPRIFIX2.cluster_interconnects='192.168.0.208'
其中ip地址是本机用于rac interconnect的网络ip地址
其中ip地址是本机用于rac interconnect的网络ip地址，具体指的是什么地址？是不是在ha中增加？还是直接配单独的网卡，用交叉线直接连起来？
谢谢！
vitovito 回复于：2005-11-11 16:07:10
引用：原帖由 louis095 于 2005-11-10 18:30 发表
我做的hacmp for rac , create share vg ----> concurrent mode , after 2 台node hacmp service up , 看到的 vg 都是 on 的 , 請試試 .
不是on吧，应该是active吧
blue_stone 回复于：2005-11-11 16:28:58
rac 的心跳线的 ip地址
oracle建议通过交换机连接两台机器的心跳网络，用交叉线也可以
比如你的节点A的ip地址是192.168.0.207
节点B的ip地址是192.168.0.208
你就可以使用下面的语句修改spfile
alter system set cluster_interconnects='192.168.0.207' sid='SIDPRIFIX1' scope=spfile;
alter system set cluster_interconnects='192.168.0.208' sid='SIDPRIFIX2' scope=spfile;
shutdown immediate ;
vitovito 回复于：2005-11-11 17:23:58
引用：原帖由 blue_stone 于 2005-11-11 16:28 发表
rac 的心跳线的 ip地址
oracle建议通过交换机连接两台机器的心跳网络，用交叉线也可以
比如你的节点A的ip地址是192.168.0.207
节点B的ip地址是192.168.0.208
你就可以使用下面的语句修改spfile
alter syste ...
好，非常感谢！
下周三就可以验证了，我下周三去客户现场，如果是这问题就最好了，如果不是，我只能装AIX5.2+ha4.5+oracle9i（rac）了
jlhwlw 回复于：2005-11-11 20:56:03
照楼上给你说的做，肯定没问题，前几天我也遇到你的问题，ORACLE RAC需要高速的，HA用串口线做心跳线在RAC环境下就会出现一个能启动，另一个就不能启动
ggf0626 回复于：2005-11-11 23:16:27
如果是不能打开数据文件的话，解决办法有2个：
1、不要用enhanced concurrent vg，用普通的vg就行了。
或者
2、在ORACLE用户中加入环境变量
export PGSD_SUBSYS=grpsvcs
不知道是不是你这种情况，可以试一下。
[ 本帖最后由 ggf0626 于 2005-11-11 23:22 编辑 ]
wildhorse 回复于：2005-11-12 11:51:19
引用：原帖由 jlhwlw 于 2005-11-11 20:56 发表
照楼上给你说的做，肯定没问题，前几天我也遇到你的问题，ORACLE RAC需要高速的，HA用串口线做心跳线在RAC环境下就会出现一个能启动，另一个就不能启动
不是用串口做心跳的问题。心跳是HACMP用的。
vitovito 回复于：2005-11-14 13:39:12
谢谢各位，问题已经解决了，问题出在hacmp上，我正在整理文档，过2天把步骤都贴出来
genieup 回复于：2005-11-15 10:41:21
此问题可能是你没正确配置cluster_interconnects参数造成的,按10楼的建议里进行设置,rac interconnectrac的网络ip地址就是你/etc/hosts 的每个节点的主IP(BOOT1 IP)
vitovito 回复于：2005-11-17 14:07:54
1 写在前面的话
编写本文档要特别感谢北京现代集团的张维民及suwalau两位朋友，在他们的帮助下，才得以解决关键的问题。
2 需求分析
大唐王滩电厂在本次HACMP实施中采用双机并行的运行模式，即：
两台P550主机上安装Oracle 9i RAC产品，HACMP版本为5.2
3 环境说明
主机名 信息 备注
主机1 dtwtdb01 System Model: IBM,9113-550
Machine Serial Number: 10FAFEF
Processor Type: PowerPC_POWER5
Number Of Processors: 2
Processor Clock Speed: 1504 MHz
CPU Type: 64-bit
Kernel Type: 64-bit
LPAR Info: 1 10-FAFEF
Memory Size: 7808 MB
Disk:70G*2
AIX 5300-03
Hacmp：5.2并行版
主机2 dtwtdb02 System Model: IBM,9113-550
Machine Serial Number: 10FB0FF
Processor Type: PowerPC_POWER5
Number Of Processors: 2
Processor Clock Speed: 1504 MHz
CPU Type: 64-bit
Kernel Type: 64-bit
LPAR Info: 1 10-FB0FF
Memory Size: 7808 MB
Disk:70G*2
AIX 5300-03
Hacmp：5.2并行版
4 设计规划
4.1 IP及拓扑设计
cluster name dtwtcluster
node name dtwtdb01
ip label db1_boot db1_stdby db1_svr dtwtdb_tty1 db1_prv
network type ethernet ethernet ethernet rs232 ethernet
network attribute public public private serial private
ip alias disable disable disable disable
network name net_ether_02 net_ether_02 net_ether_02 net_rs232_01 net_ether_01
ip address 10.66.8.52 172.1.1.1 10.66.8.50 /dev/tty0 2.2.2.1
adapter function boot standby service serial private
adapter name en2 en3 en1
cluster name dtwtcluster
node name dtwtdb02
ip label db2_boot db2_stdby db2_svr dtwtdb_tty2 db2_prv
network type ethernet ethernet ethernet rs232 ethernet
network attribute public public private serial private
ip alias disable disable disable disable
network name net_ether_02 net_ether_02 net_ether_02 net_rs232_01 net_ether_01
ip address 10.66.8.53 172.1.1.2 10.66.8.51 /dev/tty0 2.2.2.2
adapter function boot standby service serial private
adapter name en2 en3 en1
4.2 资源组命名
Res.Grp.Name Node Rel. Part.Node Name
dtwtdb_res concurrent dtwtdb01 dtwtdb02
dtwtdb_res1 cascading dtwtdb01 dtwtdb02
dtwtdb_res2 cascading dtwtdb02 dtwtdb01
4.3 应用服务器建立
App Name Start Script Stop Script
dtwtapp /ha/start /ha/stop
4.4 配置资源组
Res. Grp.Name Service IP Volume Group Application Servers
dtwtdb_res oradatavg dtwtapp
dtwtdb_res1 db1_svc
dtwtdb_res2 db2_svc
4.5 hosts文件设计
127.0.0.1 loopback localhost # loopback (lo0) name/address
10.66.8.52 db1_boot dtwtdb01
10.66.8.50 db1_svr
172.1.1.1 db1_stdby
2.2.2.1 db1_prv
10.66.8.51 db2_svr
10.66.8.53 db2_boot dtwtdb02
172.1.1.2 db2_stdby
2.2.2.2 db2_prv
10.48.178.201 db1_console1
10.48.178.203 db2_console1
5 实施步骤
5.1 环境准备
序号 内容项 内容 备注
1. 安装HACMP软件 注意不要安装带PSSP的文件集（会导致启动数据库时不认裸设备）
2. 打HACMP补丁 IY58496 IY77379 IY73937 IY68370
3. 重起主机
4. 检查补丁 检查补丁是否已经打上
5. 修改/sbin/rc.boot ##Modified By Sinosoft For Hacmp Software Start
#nohup /usr/sbin/syncd 60 > /dev/null 2>&1 &
nohup /usr/sbin/syncd 10 > /dev/null 2>&1 &
##Modified By Sinosoft For Hacmp Software End
6. 修改water mark smit system -->
Change / Show Characteristics of Operating System
HIGH water mark for pending write I/Os per file [32]
LOW water mark for pending write I/Os per file [24]
5.2 IP地址几网络划分
步骤 节点 步骤 备注
1. dtwtdb01 在en2上建立boot网卡：
dtwtdb01/ 10.66.8.52/255.255.255.0
2. 在en3上建立standby网卡：
dtwtdb01/ 172.1.1.1/255.255.255.0
3. 在en1上建立private网卡：
dtwtdb01/ 2.2.2.1/255.255.255.0
4. dtwtdb02 在en2上建立boot网卡：
dtwtdb01/ 10.66.8.53/255.255.255.0
5. 在en3上建立standby网卡：
dtwtdb01/ 172.1.1.2/255.255.255.0
6. 在en1上建立private网卡：
dtwtdb01/ 2.2.2.2/255.255.255.0
7. dtwtdb01 修改/etc/hosts文件
建立/.rhosts文件
chmod 600 /.rhosts
8. dtwtdb02 修改/etc/hosts文件
建立/.rhosts文件
chmod 600 /.rhosts
9. 测试网络是否畅通(rlogin,ping等命令)
10. dtwtdb01 创建串口信息：
tty/rs232/terminal asynchrone/sa3/2/9600/none/8/1/vt100
11. dtwtdb02 创建串口信息：
tty/rs232/terminal asynchrone/sa3/2/9600/none/8/1/vt100
12. 测试心跳线是否正常
在dtwtdb02上：cat < /dev/tty0
在dtwtdb01上：cat /etc/hosts > /dev/tty0
如果在dtwtdb02上显示出相关内容，则配置正常
vitovito 回复于：2005-11-17 14:09:25
5.3 HACMP配置
5.3.1 让HACMP主动认双机配置
HACMP for AIX
Move cursor to desired item and press Enter.
Initialization and Standard Configuration
Extended Configuration
System Management (C-SPOC)
Problem Determination Tools
nitialization and Standard Configuration
Move cursor to desired item and press Enter.
Two-Node Cluster Configuration Assistant
Add Nodes to an HACMP Cluster
Configure Resources to Make Highly Available
Configure HACMP Resource Groups
Verify and Synchronize HACMP Configuration
HACMP Cluster Test Tool
Display HACMP Configuration
Configure Nodes to an HACMP Cluster (standard)
Type or select values in entry fields.
Press Enter AFTER making all desired changes.
[Entry Fields]
* Cluster Name [dtwtcluster]
New Nodes (via selected communication paths) [dtwtdb01]
[Entry Fields]
* Cluster Name [dtwtcluster]
New Nodes (via selected communication paths) [dtwtdb02]
Currently Configured Node(s) dtwtdb01
在这一过程中，HACMP会认出在/etc/hosts文件中配置的所有信息，并进行分类，配置如下：
Node:dtwtdb01
NetWork: net_ether_01
Adapter:db1_console1,db1_prv
NetWork: net_ether_02
Adapter:db1_boot,db1_stdby
====================================================================
Node:dtwtdb02
NetWork: net_ether_01
Adapter:db2_console1,db2_prv
NetWork: net_ether_02
Adapter:db2_boot,db2_stdby
5.3.2 修改HACMP的拓扑配置
5.3.2.1 删除db1_console1和db2_console1信息
HACMP for AIX
Move cursor to desired item and press Enter.
Initialization and Standard Configuration
Extended Configuration
System Management (C-SPOC)
Problem Determination Tools
Extended Configuration
Move cursor to desired item and press Enter.
Discover HACMP-related Information from Configured Nodes
Extended Topology Configuration
Extended Resource Configuration
Extended Event Configuration
Extended Performance Tuning Parameters Configuration
Security and Users Configuration
Snapshot Configuration
Export Definition File for Online Planning Worksheets
Extended Verification and Synchronization
HACMP Cluster Test Tool
Extended Topology Configuration
Move cursor to desired item and press Enter.
Configure an HACMP Cluster
Configure HACMP Nodes
Configure HACMP Sites
Configure HACMP Networks
Configure HACMP Communication Interfaces/Devices
Configure HACMP Persistent Node IP Label/Addresses
Configure HACMP Global Networks
Configure HACMP Network Modules
Configure Topology Services and Group Services
Show HACMP Topology
Configure HACMP Communication Interfaces/Devices
Move cursor to desired item and press Enter.
Add Communication Interfaces/Devices
Change/Show Communication Interfaces/Devices
Remove Communication Interfaces/Devices
Update HACMP Communication Interface with Operating System Settings
在列出的设备中，删除db1_console1和db2_console2
5.3.2.2 修改NetWork属性
Extended Topology Configuration
Move cursor to desired item and press Enter.
Configure an HACMP Cluster
Configure HACMP Nodes
Configure HACMP Sites
Configure HACMP Networks
Configure HACMP Communication Interfaces/Devices
Configure HACMP Persistent Node IP Label/Addresses
Configure HACMP Global Networks
Configure HACMP Network Modules
Configure Topology Services and Group Services
Show HACMP Topology
Move cursor to desired item and press Enter.
Add a Network to the HACMP Cluster
Change/Show a Network in the HACMP Cluster
Remove a Network from the HACMP Cluster
Select a Network to Change/Show
Move cursor to desired item and press Enter.
net_ether_01 (2.2.2.0/24)
net_ether_02 (10.66.8.0/24 172.1.1.0/24)
Change/Show an IP-Based Network in the HACMP Cluster
Type or select values in entry fields.
Press Enter AFTER making all desired changes.
[Entry Fields]
* Network Name net_ether_01
New Network Name []
* Network Type [ether]
* Netmask [255.255.255.0]
* Enable IP Address Takeover via IP Aliases [No]
IP Address Offset for Heartbeating over IP Aliases []
* Network attribute private
Change/Show an IP-Based Network in the HACMP Cluster
Type or select values in entry fields.
Press Enter AFTER making all desired changes.
[Entry Fields]
* Network Name net_ether_02
New Network Name []
* Network Type [ether]
* Netmask [255.255.255.0]
* Enable IP Address Takeover via IP Aliases [No]
IP Address Offset for Heartbeating over IP Aliases []
* Network attribute public(不变)
5.3.2.3 增加串口NetWork
Extended Topology Configuration
Move cursor to desired item and press Enter.
Configure an HACMP Cluster
Configure HACMP Nodes
Configure HACMP Sites
Configure HACMP Networks
Configure HACMP Communication Interfaces/Devices
Configure HACMP Persistent Node IP Label/Addresses
Configure HACMP Global Networks
Configure HACMP Network Modules
Configure Topology Services and Group Services
Show HACMP Topology
# Pre-defined Serial Device Types
XD_rs232
diskhb
rs232
Add a Serial Network to the HACMP Cluster
Type or select values in entry fields.
Press Enter AFTER making all desired changes.
[Entry Fields]
* Network Name [net_rs232_01]
* Network Type rs232
COMMAND STATUS
Command: OK stdout: yes stderr: no
Before command completion, additional instructions may appear below.
Network net_ether_01 is private
Network net_ether_01 is private
vitovito 回复于：2005-11-17 14:10:21
5.3.2.4 增加串口的设备
Extended Topology Configuration
Move cursor to desired item and press Enter.
Configure an HACMP Cluster
Configure HACMP Nodes
Configure HACMP Sites
Configure HACMP Networks
Configure HACMP Communication Interfaces/Devices
Configure HACMP Persistent Node IP Label/Addresses
Configure HACMP Global Networks
Configure HACMP Network Modules
Configure Topology Services and Group Services
Show HACMP Topology
Configure HACMP Communication Interfaces/Devices
Move cursor to desired item and press Enter.
Add Communication Interfaces/Devices
Change/Show Communication Interfaces/Devices
Remove Communication Interfaces/Devices
Update HACMP Communication Interface with Operating System Settings
Add Discovered Communication Interface and Devices
Add Pre-defined Communication Interfaces and Devices
Communication Interfaces
Communication Devices
Select a Device Network
Move cursor to desired item and press Enter.
net_rs232_01
Add a Communication Device
Type or select values in entry fields.
Press Enter AFTER making all desired changes.
[Entry Fields]
* Device Name [dtwt_tty1]
* Network Type rs232
* Network Name net_rs232_01
* Device Path [/dev/tty0]
* Node Name [dtwtdb01]
Add a Communication Device
Type or select values in entry fields.
Press Enter AFTER making all desired changes.
[Entry Fields]
* Device Name [dtwt_tty2]
* Network Type rs232
* Network Name net_rs232_01
* Device Path [/dev/tty0]
* Node Name [dtwtdb02]
5.3.2.5 增加service label
Extended Configuration
Move cursor to desired item and press Enter.
Discover HACMP-related Information from Configured Nodes
Extended Topology Configuration
Extended Resource Configuration
Extended Event Configuration
Extended Performance Tuning Parameters Configuration
Security and Users Configuration
Snapshot Configuration
Export Definition File for Online Planning Worksheets
Extended Verification and Synchronization
HACMP Cluster Test Tool
Extended Resource Configuration
Move cursor to desired item and press Enter.
HACMP Extended Resources Configuration
Configure Resource Group Run-Time Policies
HACMP Extended Resource Group Configuration
HACMP Extended Resources Configuration
Move cursor to desired item and press Enter.
Configure HACMP Applications
Configure HACMP Service IP Labels/Addresses
Configure HACMP Tape Resources
Configure HACMP Communication Adapters and Links
Configure Custom Disk Methods
Customize Resource Recovery
Configure Resource Distribution Preferences
Configure HACMP Service IP Labels/Addresses
Move cursor to desired item and press Enter.
Add a Service IP Label/Address
Change/Show a Service IP Label/Address
Remove Service IP Label(s)/Address(es)
Select a Service IP Label/Address type
Move cursor to desired item and press Enter.
Configurable on Multiple Nodes
Bound to a Single Node
Network Name
Move cursor to desired item and press Enter. ?
net_ether_01 (2.2.2.0/24) ?
net_ether_02 (10.66.8.0/24 172.1.1.0/24)
Add a Service IP Label/Address configurable on Multiple Nodes (extended)
Type or select values in entry fields.
Press Enter AFTER making all desired changes.
[Entry Fields]
* IP Label/Address db_srv1
* Network Name net_ether_02
Alternate HW Address to accompany IP Label/Address []
Add a Service IP Label/Address configurable on Multiple Nodes (extended)
Type or select values in entry fields.
Press Enter AFTER making all desired changes.
[Entry Fields]
* IP Label/Address db_srv2
* Network Name net_ether_02
Alternate HW Address to accompany IP Label/Address []
vitovito 回复于：2005-11-17 14:11:02
5.3.3 配置资源组信息
5.3.3.1 增加app server
HACMP Extended Resources Configuration
Move cursor to desired item and press Enter.
Configure HACMP Applications
Configure HACMP Service IP Labels/Addresses
Configure HACMP Tape Resources
Configure HACMP Communication Adapters and Links
Configure Custom Disk Methods
Customize Resource Recovery
Configure Resource Distribution Preferences
Configure HACMP Application Servers
Move cursor to desired item and press Enter.
Add an Application Server
Change/Show an Application Server
Remove an Application Server
Add Application Server
Type or select values in entry fields.
Press Enter AFTER making all desired changes.
[Entry Fields]
* Server Name [dtwtapp]
* Start Script [/ha/start]
* Stop Script [/ha/stop]
Application Monitor Name(s)
5.3.3.2 增加资源组
Extended Resource Configuration
Move cursor to desired item and press Enter.
HACMP Extended Resources Configuration
Configure Resource Group Run-Time Policies
HACMP Extended Resource Group Configuration
HACMP Extended Resource Group Configuration
Move cursor to desired item and press Enter.
Add a Resource Group
Change/Show a Resource Group
Change/Show Resources and Attributes for a Resource Group
Remove a Resource Group
Show All Resources by Node or Resource Group
Add a Resource Group (extended)
Type or select values in entry fields.
Press Enter AFTER making all desired changes.
[Entry Fields]
* Resource Group Name [dtwtres]
* Participating Nodes (Default Node Priority) [dtwtdb01 dtwtdb02]
Startup Policy Online On All Available Nodes
Fallover Policy Bring Offline (On Error Node Only)
Fallback Policy Never Fallback
Add a Resource Group (extended)
Type or select values in entry fields.
Press Enter AFTER making all desired changes.
[Entry Fields]
* Resource Group Name [dtwtres1]
* Participating Nodes (Default Node Priority) [dtwtdb01 dtwtdb02]
Startup Policy Online On Home Node Only
Fallover Policy Fallover To Next Priority Node In The Lis>
Fallback Policy Fallback To Higher Priority Node In The L>
Add a Resource Group (extended)
Type or select values in entry fields.
Press Enter AFTER making all desired changes.
[Entry Fields]
* Resource Group Name [dtwtres2]
* Participating Nodes (Default Node Priority) [dtwtdb02 dtwtdb01]
Startup Policy Online On Home Node Only
Fallover Policy Fallover To Next Priority Node In The Lis>
Fallback Policy Fallback To Higher Priority Node In The L>
5.3.3.3 修改资源组属性
HACMP Extended Resource Group Configuration
Move cursor to desired item and press Enter.
Add a Resource Group
Change/Show a Resource Group
Change/Show Resources and Attributes for a Resource Group
Remove a Resource Group
Show All Resources by Node or Resource Group
逐个修改三个资源组信息：
dtwtres
dtwtres1
dtwtres2
Change/Show All Resources and Attributes for a Resource Group
Type or select values in entry fields.
Press Enter AFTER making all desired changes.
[TOP] [Entry Fields]
Resource Group Name dtwtres
Participating Nodes (Default Node Priority) dtwtdb01 dtwtdb02
Startup Policy Online On All Available Nodes
Fallover Policy Bring Offline (On Error Node Only)
Fallback Policy Never Fallback
Concurrent Volume Groups [datavg]
Use forced varyon of volume groups, if necessary false
Automatically Import Volume Groups false
Application Servers [dtwtapp]
Tape Resources []
Raw Disk PVIDs []
Disk Fencing Activated false
Fast Connect Services []
Communication Links []
Workload Manager Class []
Miscellaneous Data []
Change/Show All Resources and Attributes for a Resource Group
Type or select values in entry fields.
Press Enter AFTER making all desired changes.
[TOP] [Entry Fields]
Resource Group Name dtwtres1
Participating Nodes (Default Node Priority) dtwtdb01 dtwtdb02
Startup Policy Online On Home Node Only
Fallover Policy Fallover To Next Priority Node In The Lis>
Fallback Policy Fallback To Higher Priority Node In The L>
Fallback Timer Policy (empty is immediate) []
Service IP Labels/Addresses [db1_svr]
Application Servers []
Volume Groups []
Change/Show All Resources and Attributes for a Resource Group
Type or select values in entry fields.
Press Enter AFTER making all desired changes.
[TOP] [Entry Fields]
Resource Group Name dtwtres2
Participating Nodes (Default Node Priority) dtwtdb02 dtwtdb01
Startup Policy Online On Home Node Only
Fallover Policy Fallover To Next Priority Node In The Lis>
Fallback Policy Fallback To Higher Priority Node In The L>
Fallback Timer Policy (empty is immediate) []
Service IP Labels/Addresses [db2_svr]
Application Servers []
Volume Groups []
vitovito 回复于：2005-11-17 14:11:45
5.3.4 同步HACMP配置
Extended Configuration
Move cursor to desired item and press Enter.
Discover HACMP-related Information from Configured Nodes
Extended Topology Configuration
Extended Resource Configuration
Extended Event Configuration
Extended Performance Tuning Parameters Configuration
Security and Users Configuration
Snapshot Configuration
Export Definition File for Online Planning Worksheets
Extended Verification and Synchronization
HACMP Cluster Test Tool
HACMP Verification and Synchronization
Type or select values in entry fields.
Press Enter AFTER making all desired changes.
[Entry Fields]
* Verify, Synchronize or Both [Both]
* Automatically correct errors found during [No]
verification?
* Force synchronization if verification fails? [No]
* Verify changes only? [No]
* Logging [Standard]
5.4 启动HACMP
smitty clstart
Start Cluster Services
Type or select values in entry fields.
Press Enter AFTER making all desired changes.
[Entry Fields]
* Start now, on system restart or both now Start Cluster Services on these nodes [dtwtdb01]
BROADCAST message at startup? true
Startup Cluster Information Daemon? false
Reacquire resources after forced down ? false
以同样方式在dtwtdb02节点运行
5.5 创建并发VG，并加入资源组dtwtdb_res
5.5.1 创建VG
smitty cl_convg
Concurrent Volume Groups
Move cursor to desired item and press Enter.
List All Concurrent Volume Groups
Create a Concurrent Volume Group
Create a Concurrent Volume Group with Data Path Devices
Set Characteristics of a Concurrent Volume Group
Import a Concurrent Volume Group
Mirror a Concurrent Volume Group
Unmirror a Concurrent Volume Group
Node Names
Move cursor to desired item and press Esc+7.
ONE OR MORE items can be selected.
Press Enter AFTER making all selections.
> dtwtdb01
> dtwtdb02
步骤 节点 步骤 备注
1. dtwtdb01 创建vg：datavg，采用smitty cl_convg命令
2. dtwtdb01 创建裸设备信息,并修改字符流设备的权限
用smitty cl_conlv进行创建裸设备
创建结束后，VG的信息显示如下：
# lsvg -L datavg
VOLUME GROUP: datavg VG IDENTIFIER: 00cfafef00004c000000010737d159aa
VG STATE: active PP SIZE: 256 megabyte(s)
VG PERMISSION: read/write TOTAL PPs: 399 (102144 megabytes)
MAX LVs: 256 FREE PPs: 331 (84736 megabytes)
LVs: 22 USED PPs: 68 (17408 megabytes)
OPEN LVs: 0 QUORUM: 2
TOTAL PVs: 1 VG DESCRIPTORS: 2
STALE PVs: 0 STALE PPs: 0
ACTIVE PVs: 1 AUTO ON: no
Concurrent: Enhanced-Capable Auto-Concurrent: Disabled
VG Mode: Concurrent
Node ID: 1 Active Nodes: 2
MAX PPs per VG: 32512
MAX PPs per PV: 1016 MAX PVs: 32
LTG size (Dynamic): 1024 kilobyte(s) AUTO SYNC: no
HOT SPARE: no BB POLICY: relocatable
vitovito 回复于：2005-11-17 14:12:40
6 附录
6.1 相关文件
6.1.1 /etc/hosts
以dtwtdb01的配置为例
127.0.0.1 loopback localhost dtwtdb01# loopback (lo0) name/address
10.66.8.52 db1_boot dtwtdb01
10.66.8.50 db1_svr
172.1.1.1 db1_stdby
2.2.2.1 db1_prv
10.66.8.51 db2_svr
10.66.8.53 db2_boot dtwtdb02
172.1.1.2 db2_stdby
2.2.2.2 db2_prv
10.48.178.201 db1_console1
10.48.178.203 db2_console1
6.1.2 /.rhosts
该文件的权限为600
db1_boot
db2_boot
db1_svr
db2_svr
db1_stdby
db2_stdby
dtwtdb01
dtwtdb02
db1_prv
db2_prv
6.1.3 /usr/es/sbin/cluster/etc/rhosts
该文件的权限为600
10.66.8.50
10.66.8.51
10.66.8.52
10.66.8.53
172.1.1.1
172.1.1.2
2.2.2.1
2.2.2.2
6.1.4 /usr/sbin/cluster/netmon.cf
10.66.8.50
10.66.8.51
10.66.8.52
10.66.8.53
172.1.1.1
172.1.1.2
2.2.2.1
2.2.2.2
6.1.5 创建lv的pp数目列表
dbcntrl1 1
dbcntrl2 1
dbcntrl3 1
dbsystem 8
dbusers 8
dbdrsys 4
dbtools 4
dbtemp 8
dbundotbs1 4
dbundotbs2 4
dblog11 1
dblog12 1
dblog21 1
dblog22 1
dbindx 4
dbcwmlite 1
dbexample 4
dboemrepo 4
dbspfile 1
dbsrvmconf 2
dbxdb 4
dbodm 1
vitovito 回复于：2005-11-17 14:14:09
6.2 问题集锦
6.2.1 AIX 5.3不支持hacmp 4.5版本
配置完HA之后，同步HA时报错：
ERROR: Could not get netmask from interface db1_boot on node dtwtdb01.
ERROR: Could not get netmask from interface db1_sdby on node dtwtdb01.
ERROR: Could not get netmask from interface db2_boot on node dtwtdb02.
ERROR: Could not get netmask from interface db2_sdby on node dtwtdb02.
尝试了无数次之后，还是报错，经过查证，是版本不匹配的原因，详见下表：
AIX 4.3.3 AIX 5.1 (32-bit) AIX 5.1 (64-bit) AIX 5.2 AIX 5.3
HACMP 4.4 YES YES
NO NO NO
HACMP 4.4.1 YES YES
NO NO NO
HACMP/ES 4.4.1 YES YES
YES
NO NO
HACMP 4.5 NO YES NO YES
NO
HACMP/ES 4.5 NO YES YES
YES
NO
HACMP 5.1 NO YES
YES
YES
YES
HACMP 5.2 NO YES
YES
YES
YES
HACMP 5.3 NO NO
NO
YES
YES
vitovito 回复于：2005-11-17 14:15:00
6.2.2 创建数据库时没有权限访问裸设备
解决方法：
在安装oracle rac时oracle无法正常访问并行裸设备。
export PGSD_SUBSYS=grpsvcs 在oracle用户下的.profile里加入即可解决oracle无法并行访问raw lv
或者不安装VSD用的pssp rsct.basic.sp &rsct.compat.basic.sp和所有的VSD安装包
6.2.3 HACMP同步失败
报错信息如下：
ERROR: Verification of Cluster Topology for RSCT failed.
See "/var/ha/log/topsvcs.default" for detailed information.
WARNING: File 'netmon.cf' is missing or empty on node dtwtdb01. This file is needed for a cluster with the single-adapter network net_rs232_01. Please create 'netmon.cf' file on node dtwtdb01 as described in 'HACMP Planning and Installation Guide'.
WARNING: File 'netmon.cf' is missing or empty on node dtwtdb02. This file is needed for a cluster with the single-adapter network net_rs232_01. Please create 'netmon.cf' file on node dtwtdb02 as described in 'HACMP Planning and Installation Guide'.
解决方法：
到两个节点的/usr/sbin/cluster目录下检查是否有netmon.cf文件，如果没有，请增加该文件，并增加内容；如果有，请检查内容，内容如下：
192.168.129.1
192.168.129.2
172.168.129.1
172.168.129.2
10.66.8.50
10.66.8.51
6.2.4 不能两个节点同时mount问题需要考虑的问题
 问题现象
在启动数据库实例的时候，在第一个节点启动Oracle没有问题，在启动第二个节点时，Oracle数据库不能到open状态；如果这时将第一个节点shutdown，第二个节点会自动open；也就是说：哪个节点先启动，就能到open状态；后启动的实例需要等第一个节点释放资源后才能启动成功。
vitovito 回复于：2005-11-17 14:15:37
 问题分析
从日志文件分析，List of nodes: 0,和List of nodes: 1,可以看出，数据库启动的时候并不能认出两个节点，说明两台主机的通讯存在问题，因此，问题应该出在HACMP的配置中。
 问题解决
通过对HA的配置分析，发现原先配置的方式有问题，做了下面的调整：
1． 采用HACMP自己认网络配置后加以修改的方式
2． 不用IP别名的方式，修改在4.3.2.2中有说明
3． 配置私有网络，作为RAC的通讯网络，具体在4.2、4.3.2.2节中有说明
vitovito 回复于：2005-11-17 14:17:37
好麻烦，由于论坛有限制，只能一点一点粘贴，如果有需要，我可以发word文档，请留下联系方式
awksed 回复于：2005-11-17 17:46:03
给我发一份吧，[email]hbyl@sohu.com[/email], 谢谢
zhoujianbo 回复于：2005-11-18 10:25:10
给我发一份吧，[email]jianbozhou@tom.com[/email], 谢谢
yanbing 回复于：2005-11-18 15:38:17
vitovito兄弟辛苦了！！！谢谢！！！
cc1016 回复于：2005-11-18 16:50:28
兄弟辛苦了，我也要一份！
[email]caochao@yeah.net[/email]
huaweihz 回复于：2005-11-18 17:14:49
兄弟辛苦了，我也要一份！
[email]huaweihz@163.com[/email]
sailorxue 回复于：2005-11-21 15:42:40
建议用HACMP5.1，以前我有弄过，HACMP5.2没弄好，改用HACMP5.1就OK
blue_stone 回复于：2005-11-21 18:29:27
请也给我一份
谢谢
[email]blue_stone@xinhuanet.com[/email]
johnmm 回复于：2005-11-21 19:17:05
给我来一份。谢谢哥们。辛苦了。[email]wsjmax2002@126.com[/email]
killer911 回复于：2005-11-21 20:44:13
[email]aixguard@yahoo.com.cn[/email]谢谢
flyabout 回复于：2005-11-22 14:18:48
把9206卸掉试试
SmartFox365 回复于：2005-11-22 14:48:31
我也要一份,谢谢楼主.
[email]smartfox365@126.com[/email]
jlhwlw 回复于：2005-11-22 15:24:26
给我也来一份,谢谢! [email]jlhwlw@126.com[/email]
顺便说一下,楼主你做得太复杂了吧,其实你的实际应用在HA里还可以做得更简单,用IP别名的方式也是可以实现你的需求的
[ 本帖最后由 jlhwlw 于 2005-11-22 15:28 编辑 ]
goldenperson 回复于：2005-11-22 15:48:58
ding
申请一份！！谢谢
[email]goldenperson@163.com[/email]
RZC 回复于：2005-11-22 15:59:30
也给我一份：[email]rqz@citiz.net[/email]，谢谢！
fishsample2 回复于：2005-11-23 10:14:21
也发我一份，[email]k81g@21cn.com[/email]
另外我想问一下，添加本机网卡资源组的时候，为什么要两个节点都加进去，不是加本机的节点就可以了吗？
难道SVC IP还要切换到另外的节点？
Add a Resource Group (extended)
Type or select values in entry fields.
Press Enter AFTER making all desired changes.
[Entry Fields]
* Resource Group Name [dtwtres1]
* Participating Nodes (Default Node Priority) [dtwtdb01 dtwtdb02]
Startup Policy Online On Home Node Only
Fallover Policy Fallover To Next Priority Node In The Lis>
Fallback Policy Fallback To Higher Priority Node In The L>
Add a Resource Group (extended)
Type or select values in entry fields.
Press Enter AFTER making all desired changes.
[Entry Fields]
* Resource Group Name [dtwtres2]
* Participating Nodes (Default Node Priority) [dtwtdb02 dtwtdb01]
Startup Policy Online On Home Node Only
Fallover Policy Fallover To Next Priority Node In The Lis>
Fallback Policy Fallback To Higher Priority Node In The L>
yy_lee8888 回复于：2005-11-23 10:29:23
请给我一份
[email]yy_lee8888@163.com[/email]
vitovito 回复于：2005-11-23 10:39:51
截至到44楼，全部发过了，请各位查收
bpmf 回复于：2005-11-23 11:42:08
相当不错，谢谢！
[email]bobo2000@vip.sina.com[/email]
dong_jh 回复于：2005-11-23 12:01:56
多半不是INTER_CONNECT的原因。你的DB_BLOCK_SIZE是不是16K？还有INIT中的db_file_multiblock_read_count是不是改成32了，如果是，我可以告诉你原因。
我也曾被这个问题搞了好几天。
[ 本帖最后由 dong_jh 于 2005-11-23 12:03 编辑 ]
dong_jh 回复于：2005-11-23 12:14:54
The information in this article applies to:
Oracle Server - Enterprise Edition - Version: 9.2.0 to 9.2.0
This problem can occur on any platform.
Oracle Database
Symptoms
When starting the second instance of a RAC Cluster, The second node will hang indefinitely until the first node is stopped. Starting the second node first will result in the same behaviour for the first node. In essence, only one node can be brought up.
Changes
This would mostly occur in 9i so change that could trigger the problem is
1) Upgrade to 920
2) Applying the 9202 or 9203 patchset
Cause
If udp_sendspace and udp_recvspace is less than db_block_size * db_file_multiblock_read_count then there is a chance that some cr blocks are not received on the second instance which results in this hang
Fix
Check the udp send and recv parameters
For example on AIX we can get this values using
no -o udp_sendspace
no -o udp_recvspace
On increasing these values the problem does not reproduce.
udp_recvspace = 65536
udp_sendspace = 65536
解决办法：
加大udp_sendspace（用no –o udp_sendspace更改，并加入到rc.net，5L也可以使用¬-p参数使重启后还生效），如果还有问题，请把db_file_multiblock_read_count改小
ipzhw 回复于：2005-11-24 11:34:51
[email]ipzhw@sina.com[/email]
谢谢！
cnbill 回复于：2005-11-25 16:07:28
很不错，能给我一份吗？
[email]huangdr@hsdchina.com[/email]
Thank you very much!
testtest111 回复于：2005-11-28 10:09:28
能给我一份吗？
[email]jihui.lee@gmail.com[/email]
maihailong 回复于：2005-11-28 10:55:40
给我一份，谢谢
[email]maihailong@hotmail.com[/email]
chinadns 回复于：2005-11-28 16:39:17
一直没机会搞把rac·······
想死我了···
谢谢楼主
[email]xiongh@mcm.com.cn[/email]
Proton_wd 回复于：2005-11-28 17:00:35
也给我一份吧 谢谢 谢谢
[email]andywd_1982@yahoo.com[/email]
ztlsunny 回复于：2005-11-30 09:10:40
楼主也送我一份吧。。。
好久没过话了。。[email]ztlsunny@163.com[/email]
谢谢。。。。
coollinli 回复于：2005-11-30 10:40:15
[email]coollinli@sina.com.cn[/email]
多谢了。
霍夫 回复于：2005-12-05 09:18:25
[email]huofu-ie@126.com[/email]
谢谢楼主拉
kaka_wang 回复于：2005-12-05 09:58:53
好东西，发一份给我,[email]xiaofeng_wang1981@tom.com[/email],谢谢！
arken woo 回复于：2005-12-05 16:09:01
辛苦了，写了这么多！好人呀，鼓励好人好事！
[email]wusx007@hotmail.com[/email]
bin__bin 回复于：2005-12-06 11:42:53
要双机都要启动,并同时都能访问到rac, 就必须把共享vg 配置成concurrent mode. 否则就只能单机访问.
uoku 回复于：2005-12-07 15:10:46
能给我一份嘛？谢谢！
[email]uokuliu@gmail.com[/email]
cleva 回复于：2006-01-06 22:12:30
谢谢楼主，请发给我一份谢谢.cleva@263.net
aixnewer 回复于：2006-01-09 14:53:48
辛苦了哥们
[email]zhan-qi@hotmail.com[/email]
wxws2002 回复于：2006-01-09 15:39:31
我也申请一份，[email]chiminfeng@gmail.com[/email]
whuang9527 回复于：2006-01-12 19:37:18
写得很精彩，辛苦了，楼主！
能给我一份吗？
[email]huangweifeng123@sohu.com[/email]
jianan 回复于：2006-01-12 20:29:47
[email]yinan_tang@126.com[/email]
谢谢了!
wxws2002 回复于：2006-02-08 11:40:05
[email]chiminfeng@gmail.com[/email]
fly2000year 回复于：2006-02-08 12:19:55
很不错，能给我一份吗？
[email]fly2000year@163.com[/email]
Thank you very much!
ecwwf 回复于：2006-02-08 12:38:09
引用：原帖由 louis095 于 2005-11-10 18:30 发表
我做的hacmp for rac , create share vg ----> concurrent mode , after 2 台node hacmp service up , 看到的 vg 都是 on 的 , 請試試 .
两如机器的VG应是concurrent 状态才对巴？
ecwwf 回复于：2006-02-08 12:45:02
有空发一份给我谢谢 [email]ecwwf@163.com[/email]
followmezhangxu 回复于：2006-02-09 11:49:52
[email]zhangxu@runbest.com.cn[/email]
niuhh 回复于：2006-02-09 13:58:50
给我一份，万分感激 [email]niuhonghong@gmail.com[/email]
AIXCGS 回复于：2006-02-09 17:21:39
给我一份，万分感激 [email]danny@cgs.com.tw[/email]
totoro009 回复于：2006-02-15 15:02:31
请给我发一份 谢谢 [email]sam5765@sina.com.cn[/email]
mi6007 回复于：2006-02-16 19:18:27
呵呵,来的有点晚搭个末班车.兄弟辛苦了
[email]ciafbimi62003@yahoo.com[/email]
myfy 回复于：2006-02-22 13:48:10
呵呵，不知道我还搭不搭得上。谢谢先。[email]myfy@163.com[/email]
novmcgrady 回复于：2006-03-10 14:07:21
也给我一份把
[email]zhaohui1128@163.com[/email]
dummysky 回复于：2006-03-14 13:32:36
楼主,我要一份文档.dummysky@gmail.com
freebug 回复于：2006-03-26 20:55:19
楼主，可以给我发一份么？
[email]freebug@163.com[/email]
maihailong 回复于：2006-03-27 11:10:48
[email]mhlmhl@tom.com[/email]可以给我一份吗，谢谢
lanzhiyang 回复于：2006-03-30 12:00:03
可以给我发一份么？
[email]lanzhiyang@163.com[/email]
我也遇到过这个问题，但是厂家给解决了。不知道原因，不知哪位能否发一份，小弟想学习学习
Arian 回复于：2006-03-31 11:34:16
可以给我一份么？我现在有差不多配置的设备，小弟我想先学习学习。
[email]sn19817@163.com[/email]
谢谢！！
ghost529 回复于：2006-04-09 11:12:26
可发我一份吗，也想学习一下[email]ghost529@163.com[/email]
bffans 回复于：2006-04-09 11:24:12
[email]sleepsheep79@yahoo.com.cn[/email]
shfjaxl 回复于：2006-04-12 15:27:57
俺也需要[email]xupj@vip.winzheng.com[/email]
4369 回复于：2006-04-13 10:12:46
麻烦您给我发一份，谢谢
[email]dhb36@163.com[/email]
mis123 回复于：2006-04-13 13:17:10
麻烦给我一份呢 [email]fzhenhua@sohu.com[/email]
shue 回复于：2006-05-27 20:56:31
[email]degree98@sina.com.cn[/email]
未来的路在何方 回复于：2006-05-28 01:55:40
也发给我一份吧,[email]wangzhenyu@tjhq.com[/email]
promisures 回复于：2006-05-29 15:56:29
我来晚了，感谢兄弟，麻烦发一份给我
[email]xu_ning@126.com[/email]
3KU
boe 回复于：2006-05-30 14:45:56
我一点点copy下来了，很详细，谢谢了
dishanqian 回复于：2006-07-04 11:37:52
麻烦给我一份
[email]liyuefu@yahoo.com.cn[/email]
sturm 回复于：2006-07-05 12:29:17
我也一点点copy下来了，不过还是希望发给我一份word的
[email]sturm@126.com[/email]
谢谢
smile_pp 回复于：2006-07-27 11:29:34
我也一点点copy下来了，不过也希望发给我一份word的
[email]pqf@sina.com[/email]
谢谢
plmplmplm 回复于：2006-07-27 12:40:40
就是你的共享存储有问题
具体可以msn联系我
novmcgrady 回复于：2006-08-11 17:46:04
大哥，把文章发给小弟把，多谢
邮箱：[email]zhaohui@strongit.com.cn[/email]
sliny 回复于：2006-08-15 10:47:28
想分享人家的经验,连COPY都想省!!!!!::))
whbwer 回复于：2006-11-12 00:04:01
请发发给我一份。谢谢大哥了。[email]whbwer@163.com[/email]
whbwer 回复于：2006-11-12 00:10:55
如果有就发一份给我。谢谢。[email]whbwer@163.com[/email]
971726 回复于：2006-11-16 15:49:37
不能完全COPY下来，发一份给兄弟吧，谢谢。
[email]ericxyh@hotmail.com[/email]
anthony.xiong 回复于：2006-12-12 17:37:46
给我一份，谢谢！
[email]anthony.xiong@163.com[/email]
wangdc 回复于：2006-12-19 11:56:29
谢谢，请给我发一份。[email]mrwanducao@yahoo.com.cn[/email]
crazyman2046 回复于：2006-12-28 14:15:01
也给我来一份吧，3KQ
[email]xusongxu@126.com[/email]
mi6007 回复于：2006-12-30 14:36:30
希望不是太晚，谢谢楼主
[email]hsc_root@yahoo.com.cn[/email]
houyingjun 回复于：2007-01-12 11:32:43
我也想要一份
email：[email]hyj@nankang.com.cn[/email]
dradhzn 回复于：2007-01-12 23:56:57
[email]nathanhz@gmail.com[/email]
thanks
chinamylinux 回复于：2007-09-27 14:00:38
什么时候能够特别熟练的操作这套东东咧？？加油！！
chinamylinux 回复于：2007-09-27 14:01:42
[email]zhangrui_nana@163.com[/email]来一份！！
gigo521 回复于：2007-09-28 09:51:24
受教育了
xp1223 回复于：2007-09-28 11:21:36
整理文档是相当的累人的，楼主好样的
reecole 回复于：2007-09-29 12:13:05
印象中以前做rac的时候用ha来创建共享卷组是有问题的，具体出了什么问题不记得了。。 - -||
我们一般的做法是在一个节点上建卷组、裸设备、建库，然后在另外一个节点上importvg，然后再在两个节点上把卷组属性改成cuncurrent，clstart的时候两边卷组就都varyon了，状态是cuncurrent
meilixueshan 回复于：2007-09-30 16:39:31
我以前见过这个问题：
你检查一下/etc线的oratab文件，将最后一行关于是否启动RAC的参数修改为YES即可
guojz 回复于：2007-10-19 16:32:58
我也一点点copy下来了，不过也希望发给我一份word的
[email]guojingze@163.com[/email]
谢谢!:)
neoniu2718 回复于：2007-10-21 10:36:29
晚来一大步,[email]ngd@shangage.com[/email] , thanks
金牌小卧底 回复于：2007-11-28 12:17:11
好贴 ~ :mrgreen:
cssweb234 回复于：2008-07-01 12:08:18
回帖支持一下。看了这么多回帖，最后楼主整理成文档。不错啊。看得真值。
ie_e 回复于：2008-07-01 17:35:47
邮箱:Chj-[email]2@126.com[/email]
|
