3
Real Application Clusters Software Architecture

This chapter describes the Oracle Real Application Clusters-specific architectural components. Some of these components are supplied with the Oracle database software while others are vendor-specific. Topics in this chapter include descriptions of the following Real Application Clusters components:

• The Operating System-Dependent Clusterware
• Real Application Clusters Shared Disk Components
• Real Application Clusters-Specific Daemon and Instance Processes
• The Global Cache and Global Enqueue Service

The Operating System-Dependent Clusterware

Real Application Clusters processing uses operating system-dependent (OSD) clusterware to access the operating system and for cluster-related service processing such as communicating information about instance startup and shutdown. Vendors provide the OSD clusterware for UNIX operating systems, and Oracle provides the OSD clusterware for Windows NT and Windows 2000 operating systems. The OSD has the following subcomponents:

• The Cluster Manager
• The Node Monitor
• The Interconnect

The Cluster Manager

The cluster manager (CM) oversees internode messaging that travels over the interconnect to coordinate internode operations. The cluster manager also provides a global view of the cluster and the nodes and instances that are members of it. The cluster manager also controls cluster membership.

The Node Monitor

The cluster manager includes a subset of functionality known as a node monitor. The node monitor polls the status of each resource in the cluster including the nodes, the interconnect hardware and software, and the shared disks. In the Oracle-supplied Cluster Manager for Windows, the node monitor also polls the Oracle instances.

The cluster manager informs clients and the Oracle server when the status of cluster resources change. This is because Real Application Clusters manages cluster membership by reconfiguring the cluster database when a joining instance registers with the cluster manager or when an existing instance disconnects from it.

The node monitor also serves the cluster manager by:

• Providing node management interface modules
• Discovering and tracking the membership states of the nodes by providing a common view of node membership across the cluster
• Detecting and diagnosing changes in the states of active nodes and communicating information about those change events

The Interconnect

The interprocess communication (IPC) software, or interconnect, as described in Chapter 2, is another key OSD component. The IPC controls messaging among the nodes. Real Application Clusters also uses the IPC to transfer data blocks between instances.

Real Application Clusters Shared Disk Components

Real Application Clusters databases have the same components as single-instance Oracle databases. These include one or more control files, a set of online redo log files, the optional archive log files, datafiles, and so on. Therefore, you must provide shared disk access for each control file and datafile and for each online redo log member of every redo log group. You must also configure shared disks for the undo tablespace datafiles to use the recommended automatic undo management feature.

For Windows NT and Windows 2000 only, you must provide shared access to a voting or quorum disk on which Oracle stores cluster configuration information. You can place this disk on a cluster file system (CFS) or on a raw device. The node monitor uses the quorum disk configuration information to manage the cluster configuration.

The Oracle configuration and administrative tools also require access to cluster configuration data stored on shared disks. You must configure a shared disk resource to use the Database Configuration Assistant (DBCA), Oracle Enterprise Manager (EM), and the Server Control (SRVCTL) command-line administrative utility. On Windows NT and Windows 2000, the voting disk and configuration data share the same disk resource.

Parameter file management is simplified in Real Application Clusters if you use the server parameter file. Store this file on a shared disk to administer global and instance-specific parameter settings in one file.

Real Application Clusters-Specific Daemon and Instance Processes

This section describes the Real Application Clusters-specific daemon and instance processes under the following headings:

• The Global Services Daemon
• Instance Processes Specific to Real Application Clusters

The Global Services Daemon

The Global Services Daemon (GSD) runs on each node with one GSD process per node. The GSD coordinates with the cluster manager to receive requests from clients such as the DBCA, EM, and the SRVCTL utility to execute administrative job tasks such as instance startup or shutdown. The GSD is not an Oracle instance background process and is therefore not started with the Oracle instance.

Instance Processes Specific to Real Application Clusters

A Real Application Clusters database has the same processes as single-instance Oracle databases such as process monitor (PMON), database writer (DBWRn), log writer (LGWR), and so on. There are also additional Real Application Clusters-specific processes as shown in Figure 3-1. The exact names of these processes and the trace files that they create are platform-dependent.

• Global Cache Service Processes (LMSn), where n ranges from 0 to 9 depending on the amount of messaging traffic, control the flow of messages to remote instances and manage global data block access. LMSn processes also transmit block images between the buffer caches of different instances. This processing is part of the Cache Fusion feature.
• The Global Enqueue Service Monitor (LMON) monitors global enqueues and resources across the cluster and performs global enqueue recovery operations. Enqueues are shared memory structures that serialize row updates.
• The Global Enqueue Service Daemon (LMD) manages global enqueue and global resource access. Within each instance, the LMD process manages incoming remote resource requests.
• The Lock Process (LCK) manages non-Cache Fusion resource requests such as library and row cache requests.
• The Diagnosability Daemon (DIAG) captures diagnostic data about process failures within instances. The operation of this daemon is automated and it updates an alert log file to record the activity that it performs.

Figure 3-1 Real Application Clusters-Specific Instance Processes

Text description of the illustration sps81034.gif

The Global Cache and Global Enqueue Service

The Global Cache Service (GCS) and Global Enqueue Service (GES) are integrated components of Real Application Clusters that coordinate simultaneous access to the shared database and to shared resources within the database. These services maintain consistency and data integrity. The GCS and GES on each instance, as well as the Cluster Manager, use the IPC to communicate between instances and within the cluster. This section describes the following features of the GCS and GES:

• Application Transparency
• Global Resource Directory with Distributed Architecture
• Resource Mastering and Affinity
• GCS and GES Interaction with the Cluster Manager

Application Transparency

The coordination of access to resources that is performed by the GCS and GES is transparent to applications. Applications in Real Application Clusters use the same concurrency mechanisms as in single-instance Oracle databases.

Global Resource Directory with Distributed Architecture

The GCS and GES maintain a Global Resource Directory to record information about resources. The Global Resource Directory resides in memory, is distributed throughout the cluster, and is available to all active instances. In this distributed architecture, each node participates in the management of information in the directory. This distributed scheme provides fault tolerance and enhanced runtime performance.

The GCS and GES ensure the integrity of the Global Resource Directory even if multiple nodes fail. The shared database is always accessible if at least one instance is active after recovery is completed. The fault tolerance of the resource directory also enables Real Application Clusters instances to start and stop at any time, in any order.

Resource Mastering and Affinity

The Global Cache and Global Enqueue Services maintain information about each resource within a cluster. The GCS and GES nominate one instance to manage all information about a particular resource. This instance is called the resource master. The GCS evaluates resource mastering periodically and changes the resource master based on data access patterns. This reduces network traffic as well as resource acquisition time.

GCS and GES Interaction with the Cluster Manager

The GCS and GES operate independently of the cluster manager. However, these services rely on the cluster manager for timely and correct information about the statuses of the instances in the cluster. If these services cannot obtain the information they need from a particular instance, then Oracle shuts down the unresponsive instance. This ensures the integrity of Real Application Clusters databases because each instance must be aware of all other active instances to coordinate shared disk access.