Clusterware startup sequence for Oracle 11g R2
Understanding how the clusterware startup occurs is critical to the diagnosis and resolution of Oracle RAC problems.
In Unix and Linux operating systems, there is a master daemon process named INIT that functions to start up additional system background processes. The INIT process first spawns the init.ohasd process, which in turn starts up the Oracle High Availability Services Daemon (OHASD). In turn, the OHASD daemon then spawns additional Clusterware processes at each startup level as shown next:
Level 1—OHASD spawns:
Cssdagent: Agent responsible for spawning CSSD
Orarootagent: Agent responsible for managing all root-owned ohasd resources
Oraagent: Agent responsible for managing all Oracle-owned ohasd resources
cssdmonitor: Monitors CSSD and node health (along wth the cssdagent)
Level 2—OHASD rootagent spawns:
Cluster Ready Services Daemon (CRSD)—primary daemon responsible for managing cluster resources
Cluster Time Synchronization Services Daemon (CTSSD)
Diskmon—provides disk monitoring services
ASM Cluster File System (ACFS) Drivers
During the second level of startup for Clusterware, the oraagent spawns the following Clusterware processes for 11g R2:
MDNSD: Used for DNS lookup
GIPCD: Used for inter-process and inter-node communication
GPNPD: Grid Plug and Play Profile Daemon
EVMD: Event Monitor Daemon
ASM: Resource for monitoring ASM instances
Level 3—CRSD spawns:
orarootagent: Agent responsible for managing all root-owned CRSD resources
oraagent: Agent responsible for managing all Oracle-owned CRSD resources
Level 4—CRSD rootagent spawns:
Network resource: To monitor the public network
SCAN VIP(s): Single Client Access Name Virtual IPs
Node VIPs: One per node
ACFS Registery: For mounting ASM Cluster File system
GNS VIP (optional): VIP for GNS
During this phase for Clusterware startup with 11g R2, the oraagent spawns the following processes:
ASM Resouce: ASM Instance(s) resource
Diskgroup: Used for managing/monitoring ASM diskgroups
DB Resource: Used for monitoring and managing the DB and instances
SCAN Listener: Listener for single client access name, listening on SCAN VIP
Listener: Node listener listening on the Node VIP
Services: Used for monitoring and managing services
ONS: Oracle Notification Service
eONS: Enhanced Oracle Notification Service
GSD: For 9i backward compatibility
GNS (optional): It is a grid naming service that performs name resolution
Log file locations for Oracle 11g RAC and ASM
The important Clusterware daemon logs are located under the
<GRID_HOME>/log/<nodename> directory. There are additional logfiles located under the <GRID_HOME>/log/<nodename>directory as listed next:
alert<NODENAME>.log - look here first for most clusterware issues
./admin:
./agent:
./agent/crsd:
./agent/crsd/oraagent_oracle:
./agent/crsd/ora_oc4j_type_oracle:
./agent/crsd/orarootagent_root:
./agent/ohasd:
./agent/ohasd/oraagent_oracle:
./agent/ohasd/oracssdagent_root:
./agent/ohasd/oracssdmonitor_root:
./agent/ohasd/orarootagent_root:
./client:
./crsd:
./cssd:
A daemon is a long-running background process that response for service requests. The term originated with Unix, but most operating systems use daemons in some form or another. In Unix, the names of daemons conventionally end in "d". Some examples include inetd , httpd , nfsd , sshd , named , and lpd .
---------
Don’t let this picture scare you too much. You aren’t responsible for managing all of these processes, that is the Clusterware’s job!
Short summary of the startup sequence: INIT spawns init.ohasd (with respawn) which in turn starts the OHASD process (Oracle High Availability Services Daemon). This daemon spawns 4 processe
Level 1: OHASD Spawns:
Short summary of the startup sequence: INIT spawns init.ohasd (with respawn) which in turn starts the OHASD process (Oracle High Availability Services Daemon). This daemon spawns 4 process.
Level 1: OHASD Spawns:
cssdagent – responsible for I/O fencing. Killing this process would cause node reboot. Stops,start and checks the status of occsd.bin daemon
orarootagent – Agent responsible for managing all root owned ohasd resources.
oraagent – Agent responsible for managing all oracle owned ohasd resources.
cssdmonitor – It monitors(via oclsomon functionality) OCSSD(it is nothing but cssd) process hangs and node hangs (via OPROCD funcationality) .
Level 2: OHASD rootagent spawns:
CRSD – This process is responsible for start, stop, monitor and failover of resource. It maintains OCR and also restarts the resources when the failure occurs.
CRS daemon has two modes of running
1) reboot mode -> in reboot mode , when crsd starts , all the resources are restarted.
2) restart mode -> in restart mode , when crsd starts , the resources are started as these were before the shutdown.
CTSSD – Cluster Time Synchronization Services Daemon - Provides Time Management in a cluster for Oracle Clusterware
Diskmon
ACFS (ASM Cluster File System) Drivers
Level 2: OHASD oraagent spawns:
MDNSD – Used by Grid Plug and Play to locate profiles in the cluster, as well as by GNS to perform name resolution. The mDNS process is a background process on Linux and UNIX and on Windows.
GIPCD – Used for inter-process and inter-node communication (A support daemon that enables Redundant Interconnect Usage.)
GPNPD – Provides access to the Grid Plug and Play profile, and coordinates updates to the profile among the nodes of the cluster to ensure that all of the nodes have the most recent profile.
EVMD – Distributes and communicates some cluster events to all of the cluster members so that they are aware of the cluster changes
ASM – Resource for monitoring ASM instances
Level 3: CRSD spawns:
orarootagent – A specialized oraagent process that helps crsd to manage resources owned by root, such as the network, and the Grid virtual IP address.
oraagent – Agent responsible for managing all oracle owned crsd resources.
Level 4: CRSD rootagent spawns:
Network resource – To monitor the public network
SCAN VIP(s) – Single Client Access Name Virtual IPs
Node VIPs – One per node
ACFS Registery – For mounting ASM Cluster File System
GNS VIP (optional) – VIP for GNS
Level 4: CRSD oraagent spawns:
ASM Resouce – ASM Instance(s) resource
Diskgroup – Used for managing/monitoring ASM diskgroups.
DB Resource – Used for monitoring and managing the DB and instances
SCAN Listener – Listener for single client access name, listening on SCAN VIP
Listener – Node listener listening on the Node VIP
Services – Used for monitoring and managing services
ONS – Oracle Notification Service
eONS – Enhanced Oracle Notification Service
GSD – For 9i backward compatibility
GNS (optional) – Handles requests sent by external DNS servers, performing name resolution for names defined by the cluster.
Understanding how the clusterware startup occurs is critical to the diagnosis and resolution of Oracle RAC problems.
In Unix and Linux operating systems, there is a master daemon process named INIT that functions to start up additional system background processes. The INIT process first spawns the init.ohasd process, which in turn starts up the Oracle High Availability Services Daemon (OHASD). In turn, the OHASD daemon then spawns additional Clusterware processes at each startup level as shown next:
Level 1—OHASD spawns:
Cssdagent: Agent responsible for spawning CSSD
Orarootagent: Agent responsible for managing all root-owned ohasd resources
Oraagent: Agent responsible for managing all Oracle-owned ohasd resources
cssdmonitor: Monitors CSSD and node health (along wth the cssdagent)
Level 2—OHASD rootagent spawns:
Cluster Ready Services Daemon (CRSD)—primary daemon responsible for managing cluster resources
Cluster Time Synchronization Services Daemon (CTSSD)
Diskmon—provides disk monitoring services
ASM Cluster File System (ACFS) Drivers
During the second level of startup for Clusterware, the oraagent spawns the following Clusterware processes for 11g R2:
MDNSD: Used for DNS lookup
GIPCD: Used for inter-process and inter-node communication
GPNPD: Grid Plug and Play Profile Daemon
EVMD: Event Monitor Daemon
ASM: Resource for monitoring ASM instances
Level 3—CRSD spawns:
orarootagent: Agent responsible for managing all root-owned CRSD resources
oraagent: Agent responsible for managing all Oracle-owned CRSD resources
Level 4—CRSD rootagent spawns:
Network resource: To monitor the public network
SCAN VIP(s): Single Client Access Name Virtual IPs
Node VIPs: One per node
ACFS Registery: For mounting ASM Cluster File system
GNS VIP (optional): VIP for GNS
During this phase for Clusterware startup with 11g R2, the oraagent spawns the following processes:
ASM Resouce: ASM Instance(s) resource
Diskgroup: Used for managing/monitoring ASM diskgroups
DB Resource: Used for monitoring and managing the DB and instances
SCAN Listener: Listener for single client access name, listening on SCAN VIP
Listener: Node listener listening on the Node VIP
Services: Used for monitoring and managing services
ONS: Oracle Notification Service
eONS: Enhanced Oracle Notification Service
GSD: For 9i backward compatibility
GNS (optional): It is a grid naming service that performs name resolution
Log file locations for Oracle 11g RAC and ASM
The important Clusterware daemon logs are located under the
<GRID_HOME>/log/<nodename> directory. There are additional logfiles located under the <GRID_HOME>/log/<nodename>directory as listed next:
alert<NODENAME>.log - look here first for most clusterware issues
./admin:
./agent:
./agent/crsd:
./agent/crsd/oraagent_oracle:
./agent/crsd/ora_oc4j_type_oracle:
./agent/crsd/orarootagent_root:
./agent/ohasd:
./agent/ohasd/oraagent_oracle:
./agent/ohasd/oracssdagent_root:
./agent/ohasd/oracssdmonitor_root:
./agent/ohasd/orarootagent_root:
./client:
./crsd:
./cssd:
A daemon is a long-running background process that response for service requests. The term originated with Unix, but most operating systems use daemons in some form or another. In Unix, the names of daemons conventionally end in "d". Some examples include inetd , httpd , nfsd , sshd , named , and lpd .
---------
Clusterware Startup Sequence
The following is the Clusterware startup sequence (image from the “Oracle Clusterware Administration and Deployment Guide):
Don’t let this picture scare you too much. You aren’t responsible for managing all of these processes, that is the Clusterware’s job!
Short summary of the startup sequence: INIT spawns init.ohasd (with respawn) which in turn starts the OHASD process (Oracle High Availability Services Daemon). This daemon spawns 4 processe
Level 1: OHASD Spawns:
- cssdagent – Agent responsible for spawning CSSD.
- orarootagent – Agent responsible for managing all root owned ohasd resources.
- oraagent – Agent responsible for managing all oracle owned ohasd resources.
- cssdmonitor – Monitors CSSD and node health (along wth the cssdagent).
Level 2: OHASD rootagent spawns:
- CRSD – Primary daemon responsible for managing cluster resources.
- CTSSD – Cluster Time Synchronization Services Daemon
- Diskmon
- ACFS (ASM Cluster File System) Drivers
Level 2: OHASD oraagent spawns:
- MDNSD – Used for DNS lookup
- GIPCD – Used for inter-process and inter-node communication
- GPNPD – Grid Plug & Play Profile Daemon
- EVMD – Event Monitor Daemon
- ASM – Resource for monitoring ASM instances
Level 3: CRSD spawns:
- orarootagent – Agent responsible for managing all root owned crsd resources.
- oraagent – Agent responsible for managing all oracle owned crsd resources.
Level 4: CRSD rootagent spawns:
- Network resource – To monitor the public network
- SCAN VIP(s) – Single Client Access Name Virtual IPs
- Node VIPs – One per node
- ACFS Registery – For mounting ASM Cluster File System
- GNS VIP (optional) – VIP for GNS
Level 4: CRSD oraagent spawns:
- ASM Resouce – ASM Instance(s) resource
- Diskgroup – Used for managing/monitoring ASM diskgroups.
- DB Resource – Used for monitoring and managing the DB and instances
- SCAN Listener – Listener for single client access name, listening on SCAN VIP
- Listener – Node listener listening on the Node VIP
- Services – Used for monitoring and managing services
- ONS – Oracle Notification Service
- eONS – Enhanced Oracle Notification Service
- GSD – For 9i backward compatibility
- GNS (optional) – Grid Naming Service – Performs name resolution
Short summary of the startup sequence: INIT spawns init.ohasd (with respawn) which in turn starts the OHASD process (Oracle High Availability Services Daemon). This daemon spawns 4 process.
Level 1: OHASD Spawns:
cssdagent – responsible for I/O fencing. Killing this process would cause node reboot. Stops,start and checks the status of occsd.bin daemon
orarootagent – Agent responsible for managing all root owned ohasd resources.
oraagent – Agent responsible for managing all oracle owned ohasd resources.
cssdmonitor – It monitors(via oclsomon functionality) OCSSD(it is nothing but cssd) process hangs and node hangs (via OPROCD funcationality) .
Level 2: OHASD rootagent spawns:
CRSD – This process is responsible for start, stop, monitor and failover of resource. It maintains OCR and also restarts the resources when the failure occurs.
CRS daemon has two modes of running
1) reboot mode -> in reboot mode , when crsd starts , all the resources are restarted.
2) restart mode -> in restart mode , when crsd starts , the resources are started as these were before the shutdown.
CTSSD – Cluster Time Synchronization Services Daemon - Provides Time Management in a cluster for Oracle Clusterware
Diskmon
ACFS (ASM Cluster File System) Drivers
Level 2: OHASD oraagent spawns:
MDNSD – Used by Grid Plug and Play to locate profiles in the cluster, as well as by GNS to perform name resolution. The mDNS process is a background process on Linux and UNIX and on Windows.
GIPCD – Used for inter-process and inter-node communication (A support daemon that enables Redundant Interconnect Usage.)
GPNPD – Provides access to the Grid Plug and Play profile, and coordinates updates to the profile among the nodes of the cluster to ensure that all of the nodes have the most recent profile.
EVMD – Distributes and communicates some cluster events to all of the cluster members so that they are aware of the cluster changes
ASM – Resource for monitoring ASM instances
Level 3: CRSD spawns:
orarootagent – A specialized oraagent process that helps crsd to manage resources owned by root, such as the network, and the Grid virtual IP address.
oraagent – Agent responsible for managing all oracle owned crsd resources.
Level 4: CRSD rootagent spawns:
Network resource – To monitor the public network
SCAN VIP(s) – Single Client Access Name Virtual IPs
Node VIPs – One per node
ACFS Registery – For mounting ASM Cluster File System
GNS VIP (optional) – VIP for GNS
Level 4: CRSD oraagent spawns:
ASM Resouce – ASM Instance(s) resource
Diskgroup – Used for managing/monitoring ASM diskgroups.
DB Resource – Used for monitoring and managing the DB and instances
SCAN Listener – Listener for single client access name, listening on SCAN VIP
Listener – Node listener listening on the Node VIP
Services – Used for monitoring and managing services
ONS – Oracle Notification Service
eONS – Enhanced Oracle Notification Service
GSD – For 9i backward compatibility
GNS (optional) – Handles requests sent by external DNS servers, performing name resolution for names defined by the cluster.
No comments:
Post a Comment