4 Managing and Operating Middle-tier High Availability

4.2.1.1 Associating an Instance with an OracleAS Database-based Farm

If you have not already done so during the Oracle Application Server installation process, you can associate an application server instance with an OracleAS Database-based Farm:

For an OracleAS Database-based Farm, do the following to add an Oracle Application Server instance to the OracleAS Farm:

1. Navigate to the Application Server Control Console Instance Home Page.

2. In the Home area, select the Infrastructure link and follow the instructions for associating an application server instance with an Oracle Application Server Infrastructure.

4.2.1.2 Associating an Instance with an OracleAS File-based Farms

This section covers the following topics:

• Section 4.2.1.2.1, "Creating an OracleAS File-based Farm Repository Host"

• Section 4.2.1.2.2, "Adding Instances to an OracleAS File-based Farm"

4.2.1.3 Using the Application Server Control Console Create Cluster Page

Using the Application Server Control Console Farm Home Page, you can create a new DCM-Managed OracleAS Cluster.

From the Farm Home page, create a new DCM-Managed OracleAS Cluster as follows:

1. Click the Farm link to navigate to the Farm Home Page.

   
   

   Note: Application Server Control Console shows the Farm Home Page when an Oracle Application Server instance is part of a farm.

   
   

2. Click Create Cluster to display the Create Cluster page (Figure 4-5).

   Figure 4-5 Create Cluster Page

   
   Description of "Figure 4-5 Create Cluster Page"
   
   

3. Enter a name for the new cluster and click Create. Cluster names within a farm must be unique.

   A confirmation page appears.

4. Click OK to return to the Farm Home Page.

   The Farm Home page shows the cluster in the Clusters area. The new cluster is empty and does not include any Oracle Application Server instances. Use the Join Cluster button on the Farm Home page to add instances to the cluster.

4.2.5.1 Configuration and HttpSession Replication

One of the benefits of using OracleAS Clusters managed through file-based or database-based repositories (also known as DCM-managed clusters) is that the creation of a cluster through DCM automatically triggers the propagation of configuration across all the participants in the cluster. This enables the deployment of applications to all the Oracle Application Server instances in a cluster in a single step. DCM propagates the EAR or WAR file and replicates the configuration to all the nodes that are part of that cluster.

This configuration replication feature is commonly misunderstood and associated with the replication of "runtime state" in a cluster. Configuration replication is achieved with DCM. Replication of HttpSession state in a J2EE deployment is achieved through IP multicast and serialization mechanisms in OracleAS Cluster (OC4J).

The creation of OracleAS Cluster through Application Server Control triggers both mechanisms: configuration replication with DCM and HttpSession replication between the OC4J containers that "reside" in the Oracle Application Server instances that are part of the cluster. The session replication mechanism, however, is totally independent of the configuration replication. Only three configuration parameters are needed to enable the replication of the HttpSession object across OC4J instances:

• the multicast address and port

• the island definition

• the "distributable" tag in the web.xml deployment descriptor for the application

Based on this, it is possible to participate in an "HttpSession replication group" without belonging to the same configuration group. The procedure described in this section (intended to maintain state across redeployments of application to an OracleAS Cluster) is based on the separation of these two concepts. The procedure does not apply to OracleAS Cluster (OC4J) that is manually configured. For this type of cluster, the deployment is done node by node. In these cases, the session will be maintained automatically as long as there is enough time (between the deployments to each instance in the cluster) for replication to take place.

4.2.5.2 Scenario

The procedure described later in this section assumes the following scenario:

One OracleAS Farm containing two Oracle Application Server instances (I1 and I2) configured in one cluster (cluster1) sharing state (that is, configured inside the same OC4J island). This cluster can be managed either through a file-based or database-based repository.

One application ("app") is deployed on this cluster. The application is deployed on one OC4J instance (OC4J_Clustered). The application is already deployed and is at version 0. Some state is being stored in the HttpSession. The goal is to deploy a new version of the application (version 1) while maintaining the state. The two Oracle HTTP Servers in the Oracle Application Server instances are load-balanced by an external load balancer (typically OracleAS Web Cache or a third-party hardware load balancer).

Figure 4-8 shows the initial scenario.

Figure 4-8 Initial Scenario

Description of "Figure 4-8 Initial Scenario"

4.2.5.3 Procedure

1. Using Application Server Control Console, stop OC4J and Oracle HTTP Server in one of the instances (I2, for example) in the cluster. This ensures that all requests will get routed to the one single node that will maintain the state.

2. Using Application Server Control Console, remove the stopped instance (I2) from the cluster.

   1. In Application Server Control Console, click "Cluster1".

   2. Select the "I2" instance.

   3. Click Remove.

   At this point, although you have removed I2 from the cluster and stopped the OC4J_Clustered component in I2, the replication configuration still remains the same and the two instances are still sharing the same IP multicast group.

   Figure 4-9 shows the configuration at this point.

   Figure 4-9 Instance I2 Stopped and Removed from Cluster1

   
   Description of "Figure 4-9 Instance I2 Stopped and Removed from Cluster1"
   
   

3. Using Application Server Control Console, deploy a new version of the application to the I2 instance, which is no longer in the cluster. See Figure 4-10.

   Figure 4-10 Instance I2 with New Version of the Application

   
   Description of "Figure 4-10 Instance I2 with New Version of the Application"
   
   

4. Create a separate cluster (Cluster2) and add the standalone instance (I2) to it. See Figure 4-11.

   Figure 4-11 Instance I2 Added to a New Cluster, Cluster2

   
   Description of "Figure 4-11 Instance I2 Added to a New Cluster, Cluster2"
   
   

5. Start the I2 instance in the second cluster (Cluster2). At this time, because the instances are still using the same IP multicast group and same island definition for the OC4J instance, replication between the two separate clusters is still taking place. See Figure 4-12.

   Figure 4-12 Instance I2 Started Up

   
   Description of "Figure 4-12 Instance I2 Started Up"
   
   

6. Stop the I1 instance in the first cluster (Cluster1).

7. Remove the I1 instance from the first cluster (Cluster1).

   Figure 4-13 shows the scenario at this point.

   Figure 4-13 Instance I1 Stopped and Removed from Cluster1

   
   Description of "Figure 4-13 Instance I1 Stopped and Removed from Cluster1"
   
   

8. Join instance I1, which is stopped, to the second cluster (Cluster2). DCM will automatically deploy the new version of the application (version 1) in the joining instance (I1).

   Figure 4-14 Instance I1 Added to Cluster2, App Is now Version 1

   
   Description of "Figure 4-14 Instance I1 Added to Cluster2, App Is now Version 1"
   
   

9. Start the first instance I1.

   Figure 4-15 Both Instances Running and Using the New Version of the Application

   
   Description of "Figure 4-15 Both Instances Running and Using the New Version of the Application"
   
   

You now have two nodes with the new version of the application. The HttpSession information has been maintained across deployments. No state has been lost in the process.

4.2.5.4 Automation of the Procedure Using DCM Scripts

To automate the procedure above, you can use DCM scripts in each of the involved instances. The following scripts provide a sample implementation and can be customized with the following parameters:

appname - the name of the application being deployed and updated

app.ear - the full path location to the new EAR file being deployed

instance_n - the name of the first Oracle Application Server instance that will be updated

last_instance - the name of the last Oracle Application Server instance that will be updated

cluster1 - the name of the cluster where the application is originally deployed

cluster2 - the name of the cluster used for session migration

dcmscript_instance_n

stop -i instance_n -ct OC4J
stop -i instance_n -co HTTP_Server -ct HTTP_Server
leaveCluster -i instance_n
undeployApplication -a appname
deployApplication -f app.ear -a appname
joinCluster -cl cluster2 -i instance_n
start -i instance_n

dcmscript_last_instance

stop -i last_instance
leaveCluster -i last_instance
joinCluster -cl cluster2 -i last_instance
start -i instance_1

The procedure involves running first dcmscript_instance_n in each of the instances that participate in the cluster and that will be updated before the last instance. After this is done in all the instances except the last one, run dcmscript_last_instance from the last instance. To run these scripts:

> cd ORACLE_HOME/dcm/bin
> dcmctl shell -f filename

where filename is the name of the DCM script (dcmscript_instance_n or dcmscript_last_instance).

For future stateful deployments, the operations would require to switch roles between the original cluster and the session holder cluster. This can be achieved by creating a couple of additional scripts.

dcmscript_instance_n_odd_run

stop -i instance_n -ct OC4J
stop -i instance_n -co HTTP_Server -ct HTTP_Server
leaveCluster -i instance_n
undeployApplication -a appname
deployApplication -f app.ear -a appname
joinCluster -cl cluster1 -i instance_n
start -i instance_n

dcmscript_last_instance_odd_run

stop -i last_instance
leaveCluster -i last_instance
joinCluster -cl cluster1 -i last_instance
start -i instance_1

The next deployment with state would require running the first set of scripts, the following deployment would use the odd_run scripts, and so on.

4.2.5.5 Additional Considerations

If the cluster contains more than two instances, it is necessary to move the instances between the two clusters in the same way I1 was moved in the example above (you would have to apply steps 6 through 9 for each one of the instances in the cluster).

cluster1 and cluster2 switch roles in subsequent deployments. After you finish moving the last instance as described in the procedure, simply leave the empty cluster for future use.

If the new version of the application modifies the signature of the objects that are added to the session (for example, it added new attributes to these objects), the container will automatically trigger the load of the new version of the classes involved in the cluster. This means that the session will be lost whenever the signature is changed. Other modifications to the servlets, JSPs, and classes included in the redeployment will be assimilated by the instances without any loss in the HttpSession state.

4.2.6.1 Using and Configuring mod_oc4j Load Balancing

Using DCM-Managed OracleAS Clusters, the Oracle HTTP Server module mod_oc4j load balances requests to OC4J processes. The Oracle HTTP Server, using mod_oc4j configuration options, supports different load balancing policies. By specifying load balancing policies DCM-Managed OracleAS Clusters provide performance benefits along with failover and high availability, depending on the network topology and host machine capabilities.

By default, mod_oc4j uses weights to select a node to forward a request to. Each node uses a default weight of 1. A node's weight is taken as a ratio compared to the weights of the other available nodes to define the number of requests the node should service compared to the other nodes in the DCM-Managed OracleAS Cluster. Once a node is selected to service a particular request, by default, mod_oc4j uses the roundrobin policy to select OC4J processes on the node. If an incoming request belongs to an established session, the request is forwarded to the same node and the same OC4J process that started the session.

The mod_oc4j load balancing policies do not take into account the number of OC4J processes running on a node when calculating which node to send a request to. Node selection is based on the configured weight for the node, and its availability.

To modify the mod_oc4j load balancing policy, use the Oc4jSelectMethod and Oc4jRoutingWeight configuration directives in the mod_oc4j.conf file.

Using Application Server Control Console, configure the mod_oc4j.conf file as follows:

1. Select the HTTP_Server component from the System Components area of an instance home page.

2. Click the Administration link on the HTTP_Server page.

3. Click the Advanced Server Properties link on the Administration page.

4. On the Advanced Server Properties page, select the mod_oc4j.conf link from the Configuration Files area.

5. On the Edit mod_oc4j.conf page, within the <IfModule mod_oc4j.c> section, add or edit the directives Oc4jSelectMethod and Oc4jRoutingWeight to select the desired load balancing option.

> dcmctl updateconfig -ct ohs
> opmnctl @cluster:<cluster_name> restartproc ias-component=HTTP_Server process-type=HTTP_Server

4.2.6.2 Configuring Oracle HTTP Server Instance-Specific Parameters

You can modify the Oracle HTTP Server ports and listening addresses on the Server Properties Page, which can be accessed from the Oracle HTTP Server Home Page. You can modify the virtual host information by selecting a virtual host from the Virtual Hosts section on the Oracle HTTP Server Home Page.

Table 4-3 shows the Oracle HTTP Server instance-specific parameters.

4.2.6.3 Configuring mod_plsql With Real Application Clusters

This section covers the following:

• Section 4.2.6.3.1, "Configuring Detection and Cleanup of Dead Connections"

• Section 4.2.6.3.2, "Using Oracle Directory for Lookups"

4.2.7.1 How the Common Configuration Is Established

The order in which Oracle Application Server instances are added to the DCM-Managed OracleAS Cluster is significant. The common configuration that will be replicated across the DCM-Managed OracleAS Cluster is established by the first Oracle Application Server instance added to the cluster. The configuration of the first Oracle Application Server instance added is inherited by all Oracle Application Server instances that subsequently join the DCM-Managed OracleAS Cluster.

The common configuration includes all cluster-wide configuration information— namely, DCM-Managed OracleAS Cluster and Oracle Application Server instance attributes, such as components configured. For example, if the first Oracle Application Server instance to join the cluster has four OC4J instances, then the common configuration includes those four OC4J instances and the applications deployed on them. OC4J Instances that subsequently join the DCM-Managed OracleAS Cluster replicate the OC4J instances and their deployed applications. (In addition, when the Oracle Application Server instance joins the DCM-Managed OracleAS Cluster, DCM removes any OC4J components that do not match the common configuration). Furthermore, changes to one Oracle Application Server instance in the DCM-Managed OracleAS Cluster, such as adding new OC4J instances or removing OC4J instances, are replicated across the DCM-Managed OracleAS Cluster; the components configured are part of the replicated cluster-wide, common configuration.

When the last Oracle Application Server instance leaves a DCM-Managed OracleAS Cluster, the DCM-Managed OracleAS Cluster becomes an empty DCM-Managed OracleAS Cluster, and the next Oracle Application Server instance to joins the DCM-Managed OracleAS Cluster provides a new common configuration for the DCM-Managed OracleAS Cluster.

4.2.7.2 Parameters Excluded from the Common Configuration: Instance-Specific Parameters

Some parameters only apply to a given Oracle Application Server instance or computer; these parameters are instance-specific parameters. DCM does not propagate instance-specific parameters to the Oracle Application Server instances in a DCM-Managed OracleAS Cluster. When you change an instance-specific parameter, if you want the change to apply across the DCM-Managed OracleAS Cluster, you must apply the change individually to each appropriate Oracle Application Server instance.

log-file
process-module

id
ORACLE_HOME
ORACLE_CONFIG_HOME

port.range
start
stop
ping
restart
process-set

module-data/category/data[id='config-file']
ias-component/module-data/category/data[id='config-file']
ias-component/process-type/ module-data/category/data[id='config-file']
ias-component/process-type/ process-set/module-data/category/data[id='config-file']

[id='dcm-daemon']
[id='WebCache']
[id='OID']
[id='IASPT']
[id='wireless']
[id='Discoverer']
[id='LogLoader']
[id='Custom']

4.3.2.1 Selecting the Instance to Use for a OracleAS File-based Farm Repository Host

An important consideration for using DCM-Managed OracleAS Clusters with a OracleAS File-based Farm is determining which Oracle Application Server instance is the repository host.

Consider the following when selecting the repository host for an OracleAS File-based Farm:

• When the repository host instance is temporarily unavailable, a DCM-Managed OracleAS Cluster that uses a OracleAS File-based Farm is still able to run normally, but it cannot update any configuration information.

• Because the Oracle Application Server instance that is the repository host instance stores and manages the DCM-Managed OracleAS Cluster configuration information in its file system, the repository host instance should use mirrored or RAID disks. Disk mirroring improves the availability of the DCM-Managed OracleAS Cluster.

• When the repository host instance is not available, read-only configuration operations are not affected on any Oracle Application Server instances that are running. The OracleAS Farm cluster-wide configuration information is distributed and managed through local Java Object Cache.

• When the repository host instance is not available, operations that attempt to change configuration information in the file-based repository will generate an error. These operations must be delayed until the repository host instance is available, or until the repository host instance is relocated to another application server instance within the OracleAS File-based Farm.

4.3.2.2 Protecting Against the Loss of a Repository Host

Using an OracleAS File-based Farm, one instance in the farm is designated as the repository host. The repository host holds configuration information for all instances in the OracleAS File-based Farm. Access to the repository host is required for all configuration changes, write operations, for instances in the OracleAS File-based Farm. However, instances have local configuration caches to perform read operations, where the configuration is not changing.

In the event of the loss of the repository host, any other instance in the OracleAS File-based Farm can take over as the new repository host if an exported copy of the old repository hosts is available. You should make regular backups of the repository host, and save the backups on a separate system.

4.3.2.3 Impact of Repository Host Unavailability

When the repository host is unavailable, only read-only operations are allowed. No configuration changes are allowed. If an operation is attempted that requires updates to the repository host, such as use of the updateConfig command, dcmctl reports an error message. For example:

ADMN-100205
Base Exception:
The DCM repository is not currently available. The OracleAS 10g instance,
"myserver.mydomain.com", is using a cached copy of the repository information.
This operation will update the repository, therefore the repository must be available.

If the repository host is permanently down, or unavailable for the long-term, then the repository host should be relocated. If the restored repository is not recent, local instance archives can be applied to bring each instance up to a newer state.

4.3.2.4 Impact of Non-Repository Host Unavailability

When the instances in a DCM-Managed OracleAS Cluster, other than the repository host instance, are down, all other instances can still function properly. If an instance is experiencing a short-term outage, the instance automatically updates its configuration information when it becomes available again.

If an instance is permanently lost, this will have no affect on other instances in the OracleAS File-based Farm. However, to maintain consistency, it will be necessary to delete all records pertaining to the lost instance.

To delete configuration information for a lost instance, use the following command:

> dcmctl destroyInstance

4.3.2.5 Updating and Checking the State of Local Configuration

It is important that all configuration changes complete successfully, and that all instances in a cluster are "In Sync". The local configuration information must match the information stored in the repository. DCM does not know about manual changes to configuration files, and such changes could make the instances in a cluster have an In Sync status of false.

Use the following dcmctl command to return a list of all managed components with their In Sync status:

> dcmctl getState -cl cluster_name

The In Sync status of true implies that the local configuration information for a component is the same as the information that is stored in the repository.

If you need to update the file-based repository with changed, local information, use the dcmctl command updateConfig, as follows,

> dcmctl updateconfig
> dcmctl getstate -cl cluster_name

Use the resyncInstance command to update local information with information from the repository. For example:

> dcmctl resyncinstance

By default this command only updates configuration information for components whose In Sync status is false. Use the -force option to update all components, regardless of their In Sync status.

4.3.2.6 Performing Administration on a DCM-Managed OracleAS Cluster

During planned administrative downtimes, with a DCM-Managed OracleAS Cluster using an OracleAS File-based Farm that runs on multiple hosts with sufficient resources, you can perform administrative tasks while continuing to handle requests. This section describes how to relocate the repository host in a DCM-Managed OracleAS Cluster, while continuing to handle requests.

These procedures are useful for performing administrative tasks on a DCM-Managed OracleAS Cluster, such as the following:

• Relocating the repository for repository host node decommission.

• Applying required patches to the DCM-Managed OracleAS Cluster.

• Applying system upgrades, changes, or patches that require a system restart for a host in the DCM-Managed OracleAS Cluster.

Use the following steps to relocate the repository host in a DCM-Managed OracleAS Cluster.

1. Issue the following DCM command, on UNIX systems:

   > cd $ORACLE_HOME/dcm/bin
   > dcmctl exportRepository -f file
   
   

   On Windows systems:

   > cd %ORACLE_HOME%\dcm\bin
   > dcmctl exportRepository -f file
   
   

   
   

   Note: After this step, do not perform configuration or administration commands that would change the configuration. Otherwise those changes will not be copied when the repository file is imported to the new repository host.

   
   

2. Stop the administrative system, including Enterprise Manager and the DCM daemon in each instance of the OracleAS File-based Farm, except for the instance that is going to be the new repository host.

   On UNIX systems use the following commands on each instance in the cluster:

   > $ORACLE_HOME/bin/emctl stop iasconsole
   > $ORACLE_HOME/opmn/bin/opmnctl stopproc ias-component=dcm-daemon
   
   

   On Windows systems use the following commands on each instance in the cluster:

   > %ORACLE_HOME%\bin\emctl stop iasconsole
   > %ORACLE_HOME%\opmn\bin\opmnctl stopproc ias-component=dcm-daemon
   

   At this point, the DCM-Managed OracleAS Cluster can still handle requests.

3. Import the saved repository on the host that is to be the repository host instance.

   On UNIX systems, use the following commands:

   > cd $ORACLE_HOME/dcm/bin/
   > dcmctl importRepository -file filename
   
   

   On Windows systems, use the following commands:

   > cd %ORACLE_HOME%\dcm\bin\
   > dcmctl importRepository -file filename
   
   

   filename is the name of the file you specified in the exportRepository command.

   While importRepository is active, the DCM-Managed OracleAS Cluster can still handle requests.

   
   

   Note: The importRepository command issues a prompt that specifies that the system that is the currently hosting the repository must be shutdown. However, only the dcm-daemon on the system that is currently hosting the repository must be shutdown, and not the entire system.

   
   

4. Use the following command to start all components on the new repository host. Do not perform administrative functions at this time.

   On UNIX systems:

   > $ORACLE_HOME/opmn/bin/opmnctl startall
   
   

   On Windows systems:

   > %ORACLE_HOME%\opmn\bin\opmnctl startall
   
   

5. On the system that was the repository host, indicate that the instance is no longer the host by issuing the following command,

   > dcmctl repositoryRelocated
   
   

6. Start Application Server Control Console on the new repository host instance. The repository has now been relocated, and the new repository instance now handles requests.

   On UNIX systems use the following commands on each instance in the cluster:

   > $ORACLE_HOME/bin/emctl start iasconsole
   
   

   On Windows systems use the following commands on each instance in the cluster:

   > %ORACLE_HOME%\bin\emctl start iasconsole
   
   

7. Shut down the Oracle Application Server instance associated with the old repository host, using the following commands:

   On UNIX systems:

   > $ORACLE_HOME/opmn/bin/opmnctl stopall
   
   

   On Windows systems:

   > %ORACLE_HOME%\opmn\bin\opmnctl startall
   
   

You can now perform the required administrative tasks on the old repository host system, such as the following.

• Applying required patches to the repository host system in the DCM-Managed OracleAS Cluster

• Decommissioning the node

• Applying system upgrades, changes, or patches that require a system restart for the DCM-Managed OracleAS Cluster

After completing the administrative tasks on the system that was the repository host, if you want to switchback the repository host, you need to perform these steps again.

4.3.2.7 Best Practices for Repository Backups

When you export repository files and archives, keep the files in known locations and back up the exports and archives regularly. It is also recommended that exported repositories be available to non-repository instances, not only as a backup means but also for availability. If the repository instance becomes unavailable, a new instance can become the new repository host but only if an exported repository file is available.

Oracle Application Server does not provide an automated repository backup procedure. However, to assure that you can recover from loss of configuration data, you need to put a repository backup plan in place. Perform repository backups regularly and frequently, and perform a repository backup after any configuration changes or topology changes where instances are added or removed.

Save repository backups to different nodes that are available to other nodes in the OracleAS File-based Farm.

4.3.2.8 Best Practices for Managing Instances in OracleAS File-based Farms

When you add or remove an instance from an OracleAS File-based Farm, all the managed processes on that instance are stopped. If you want the instance to be available, then after performing the leave or join farm operation, restart the instance.

It is recommended that you back up the local configuration before leaving or joining an OracleAS File-based Farm. For example, use the following commands to create and export an archive:

> dcmctl createarchive -arch myarchive -comment "Archive before leaving farm"
> dcmctl exportarchive -arch myarchive -f /archives/myarchive

Archives are portable across OracleAS File-based Farm. When an instance joins a new farm it can apply archives created on a previous farm.

4.4.2.1 Creating or Deleting OC4J Instances in an OracleAS Cluster (OC4J)

You can create a new OC4J instance on any Oracle Application Server instance within a DCM-Managed OracleAS Cluster, and the OC4J instance will be propagated to all Oracle Application Server instances across the cluster.

To create an OC4J instance, do the following:

1. Navigate to any application server instance within the DCM-Managed Oracle Application Server Cluster.

2. Select Create OC4J Instance under the System Components area. This displays the Create OC4J instance page.

3. Enter a name in the OC4J Instance name field.

4. Select Create.

   Oracle Application Server creates the instances and then DCM propagates the new OC4J instance across the DCM-Managed OracleAS Cluster.

A new OC4J instance is created with the name you provided. This OC4J instance shows up on each application server instance across the cluster, in the System Components section.

To delete an OC4J instance, select the checkbox next to the OC4J instance you wish to delete, then select Delete OC4J Instance. DCM propagates the OC4J removal across the cluster.

4.4.2.2 Deploying Applications on an OracleAS Cluster (OC4J)

In DCM-Managed OracleAS Cluster, when you deploy an application to one application server instance, the application is propagated to all application server instances across the cluster.

To deploy an application across a cluster, do the following:

1. Select the cluster you want to deploy the application to.

2. Select any application server instance from within the cluster.

3. Select an OC4J instance on the application server instance where you want to deploy the application.

4. Deploy the application to the OC4J instance using either Application Server Control Console or dcmctl commands.

   DCM then propagates the application across the DCM-Managed Oracle Application Server Cluster.

   
   

   See Also: Oracle Application Server Containers for J2EE User's Guide for details on deploying applications to an OC4J instance

   
   

4.4.2.3 Configuring Web Application State Replication with OracleAS Cluster (OC4J)

To assure that Oracle Application Server maintains, across DCM-Managed OracleAS Cluster, the state of stateful Web applications you need to configure state replication for the Web applications.

To configure state replication for stateful Web applications, do the following:

1. Select the Administration link on the OC4J Home Page.

2. Select the Replication Properties link in the Instance Properties area.

3. Scroll down to the Web Applications section. Figure 4-16 shows this section.

   Figure 4-16 Web State Replication Configuration

   
   Description of "Figure 4-16 Web State Replication Configuration"
   
   

4. Select the Replicate session state checkbox.

   Optionally, you can provide the multicast host IP address and port number. If you do not provide the host and port for the multicast address, it defaults to host IP address 230.0.0.1 and port number 9127. The host IP address must be between 224.0.0.2 through 239.255.255.255. Do not use the same multicast address for both HTTP and EJB multicast addresses.

   
   

   Note: When choosing a multicast address, ensure that the address does not collide with the addresses listed in: http://www.iana.org/assignments/multicast-addresses Also, if the low order 23 bits of an address is the same as the local network control block, 224.0.0.0 – 224.0.0.255, then a collision may occur. To avoid this problem, provide an address that does not have the same bits in the lower 23 bits of the address as the addresses in this range.

   
   

5. Add the <distributable/> tag to all web.xml files in all Web applications. If the Web application is serializable, you must add this tag to the web.xml file.

   The following shows an example of this tag added to web.xml:

   <web-app>
     <distributable/>
     <servlet>
     ...
     </servlet>
   </web-app>
   

Figure 4-17 Application Server Control Console Properties Page for Setting Load on Startup

Description of "Figure 4-17 Application Server Control Console Properties Page for Setting Load on Startup"

4.4.2.4 Configuring EJB Application State Replication with OracleAS Cluster (OC4J-EJB)

To create an EJB cluster, also known as OracleAS Cluster (OC4J-EJB), you specify the OC4J instances that are to be involved in the cluster, configure each of them with the same multicast address, username, and password, and deploy the EJB, which is to be clustered, to each of the nodes in the cluster.

EJBs involved in a OracleAS Cluster (OC4J-EJB) cannot be sub-grouped in an island. Instead, all EJBs within the cluster are in one group. Also, only session beans are clustered.

The state of all beans is replicated at the end of every method call to all nodes in the cluster using a multicast topic. Each node included in the OracleAS Cluster (OC4J-EJB) is configured to use the same multicast address.

The concepts for understanding how EJB object state is replicated within a cluster are described in the Oracle Application Server Containers for J2EE Enterprise JavaBeans Developer's Guide.

To configure EJB replication, do the following:

1. Click the Administration link on the OC4J Home Page.

2. Click the Replication Properties link in the Instance Properties area.

3. In the EJB Applications section, select the Replicate State checkbox.

   Figure 4-18 shows this section.

   Figure 4-18 EJB State Replication Configuration

   
   Description of "Figure 4-18 EJB State Replication Configuration"
   
   

4. Provide the username and password, which is used to authenticate itself to other hosts in the OracleAS Cluster (OC4J-EJB). If the username and password are different for other hosts in the cluster, they will fail to communicate. You can have multiple username and password combinations within a multicast address. Those with the same username/password combinations are considered a unique cluster.

   Optionally, you can provide the multicast host IP address and port number. If you do not provide the host and port for the multicast address, it defaults to host IP address 230.0.0.1 and port number 9127. The host IP address must be between 224.0.0.2 through 239.255.255.255. Do not use the same multicast address for both Web Application and EJB multicast addresses.

   
   

   Note: When choosing a multicast address, ensure that the address does not collide with the addresses listed in: http://www.iana.org/assignments/multicast-addresses Also, if the low order 23 bits of an address is the same as the local network control block, 224.0.0.0 – 224.0.0.255, then a collision may occur. To avoid this, provide an address that does not have the same bits in the lower 23 bits of the address as the addresses in this range.

   
   

5. Configure the type of EJB replication in the orion-ejb-jar.xml file within the JAR file. See Section 4.4.2.5, "Configuring Stateful Session Bean Replication for OracleAS Cluster (OC4J-EJB)s" for details. You can configure these within the orion-ejb-jar.xml file before deployment or add this through the Application Server Control Console screens after deployment. To add this after deployment, drill down to the JAR file from the application page.

4.4.2.5 Configuring Stateful Session Bean Replication for OracleAS Cluster (OC4J-EJB)s

For stateful session beans, you may have to modify the orion-ejb-jar.xml file to add the state replication configuration. Because you configure the replication type for the stateful session bean within the bean deployment descriptor, each bean can use a different type of replication.

Stateful session beans require state to be replicated among nodes. In fact, stateful session beans must send all their state between the nodes, which can have a noticeable effect on performance. Thus, the following replication modes are available to you to decide on how to manage the performance cost of replication:

<session-deployment replication="EndOfCall" .../>

<session-deployment replication="VMTermination" .../>

4.4.3.1 Configuring OC4J Islands and OC4J Processes

To provide a redundant environment and to support high availability using DCM-Managed OracleAS Clusters, you need to configure multiple OC4J processes within each OC4J instance.

In DCM-Managed OracleAS Cluster, state is replicated in OC4J islands with the same name within OC4J instances and across instances in the DCM-Managed OracleAS Cluster. To assure high availability, with stateful applications, OC4J island names within an OC4J instance must be the same in corresponding OC4J instances across the DCM-Managed OracleAS Cluster. It is your responsibility to make sure that island names match where session state replication is needed in a DCM-Managed OracleAS Cluster.

The number of OC4J processes on an OC4J instance within a DCM-Managed OracleAS Cluster is an instance-specific parameter because different hosts running application server instances in the DCM-Managed OracleAS Cluster could each have different capabilities, such as total system memory. Thus, it could be appropriate for a DCM-Managed OracleAS Cluster to contain application server instances that each run different numbers of OC4J processes within an OC4J instance.

To modify OC4J islands and the number of processes each OC4J island contains, do the following:

1. Click the Administration link on the OC4J Home Page of the application server instance of interest in the DCM-Managed OracleAS Cluster.

2. Click Server Properties in the Instance Properties area.

3. Scroll down to the Multiple VM Configuration section (Figure 4-19). This section defines the islands and the number of OC4J processes that should be started on this application server instance in each island.

   Figure 4-19 OC4J instance Island and Number of Processes Configuration

   
   Description of "Figure 4-19 OC4J instance Island and Number of Processes Configuration"
   
   

4. Create any islands for this OC4J instance within the cluster by clicking Add Another Row. You enter a name for each island in the Cluster(OC4J) Name field. In the Number of Processes field, you designate how many OC4J processes should be started within each island.

4.4.3.2 Configuring Port Numbers and Command Line Options

Figure 4-20 shows the section where you can modify OC4J ports and set command-line options.

To modify OC4J ports or command-line options, do the following:

1. Click the Administration link on the OC4J Home Page of the Oracle Application Server instance of interest in the cluster.

2. Click Server Properties in the Instance Properties area.

3. Scroll down to the Multiple VM Configuration section. This section defines the ports and the command line options for OC4J and for the JVM that runs OC4J processes.

Figure 4-20 shows the Ports and Command line options areas on the Server Properties page.

Figure 4-20 OC4J Ports and Command Line Options Configuration

Description of "Figure 4-20 OC4J Ports and Command Line Options Configuration"

4.5.1.1 Configuration and Deployment Changes for OracleAS Cold Failover Cluster (Middle-Tier)

In separate Oracle home installations, any applications deployed or any configuration changes made to the middle-tier installation should be made on both nodes of the cold failover cluster. This needs to be ensured by the administrator managing the environment.

Application deployment is applied, as with any other middle-tier environment. To deploy applications on the passive node, bring up the node and then deploy the application. For the J2EE installation of OC4J and Oracle HTTP Server, the application deployment is like any other multiple middle-tier environment. The passive node can be brought up during the deployment phase and the application deployment can be done on this node. Similarly, applications can be deployed on the active node.

ORACLE_HOME/discoverer/.reg_key.dc
ORACLE_HOME/discoverer/.reg_key.dc.bak
ORACLE_HOME/discoverer/util/pref.txt

For single Oracle home installations, application deployments and configuration changes need to done only on the current node.

4.5.1.2 Backup and Recovery for OracleAS Cold Failover Cluster (Middle-Tier)

For separate Oracle home installations, you should back up both nodes of the OracleAS Cold Failover Cluster (Middle-Tier). The procedure for this remains the same as for any other middle tier and is documented in the Oracle Application Server Administrator's Guide. Each installation needs to be backed up. During restoration, each backup can only be restored to the host it was backed up from. It should not be restored to the other node.

For single Oracle home installations, the middle tier backup and restore operations need to be done from just the current active node.

4.5.1.3 Using Application Server Control Console for OracleAS Cold Failover Cluster (Middle-Tier)

For separate Oracle home installations, to monitor or manage a node using Application Server Control Console, log in to the console using the physical hostname of the current active node. The Application Server Control Console processes can be up and running on both nodes of the cluster simultaneously. When changes to the environment are made, including configuration changes or application deployments, perform the changes on both nodes of the OracleAS Cold Failover Cluster (Middle-Tier).

For single Oracle home installations, to monitor or manage a OracleAS Cold Failover Cluster (Middle-Tier) deployment using Application Server Control Console, log in to the console using the virtual hostname.

4.5.2.1 Manual Failover for OracleAS Cold Failover Cluster (Middle-Tier)

For single Oracle home installations, the failover process to make the formerly passive node the new active node includes the following steps:

1. Stop all middle-tier services on the currently active node, if the node is still available.

2. Fail over the virtual IP to the new active node.

3. Fail over the shared disk on which the shared Oracle home resides and fail over the components to the new active node.

4. Start the middle-tier services on the new active node.

For separate Oracle home installations, the failover process to make the formerly passive node the new active node includes the following steps:

1. Stop all middle-tier services on current active node, if the node is still available.

2. Fail over the virtual IP to the new active node.

3. Fail over the components to the new active node.

4. Start the middle-tier services on the new active node.

4.5.2.2 Manual Failover for the Virtual IP in OracleAS Cold Failover Cluster (Middle-Tier)

Perform the following steps to fail over the virtual IP in OracleAS Cold Failover Cluster (Middle-Tier):

1. Stop all Oracle Application Server processes on the failed node, if possible.

   On UNIX systems:

   > $ORACLE_HOME/opmn/bin/opmnctl stopall
   
   

   > On Windows systems,

   %ORACLE_HOME%\opmn\bin\opmnctl stopall
   
   

2. Stop Oracle Application Server Administration processes on the failed node, if possible. On UNIX systems:

   > $ORACLE_HOME/bin/emctl stop iasconsole
   > $ORACLE_HOME/bin/emctl stop agent
   
   

   On Windows systems,

   > %ORACLE_HOME%\bin\emctl stop iasconsole
   > %ORACLE_HOME%\bin\opmnctl stop agent
   
   

3. Fail over the virtual IP from the failed node to the new active node.

   On Sun SPARC Solaris:

   1. If the failed node is usable, login as root and execute the following command (on the failed node):

      # ifconfig <interface_name> removeif <virtual_IP>
      
      

   2. Login as root on the new active node and execute the command:

      # ifconfig <interface_name> addif <virtual_IP> up
      
      
      

   On Linux:

   1. If the failed node is usable, login as root on the failed node and execute the following command:

      # /sbin/ifconfig <interface_name> down
      
      

   2. Login as root on the new active node and execute the command:

      # ifconfig <interface_name> netmask <netmask> <virtual_IP> up
      
      

   On Windows:

   On the failed node, move the group that was created using Oracle Fail Safe as follows:

   1. Start up Oracle Fail Safe Manager.

   2. Right-click the group that was created during the Oracle Application Server middle-tier installation and select "Move to different node".

   
   

   Note: If OracleAS JMS is using file-persistence, fail over the shared disk as well.

   
   

4.5.2.3 Manual Failover of Components for OracleAS Cold Failover Cluster (Middle-Tier)

After performing the failover steps for the virtual IP on the new active node, perform the following steps to fail over on the OracleAS Cold Failover Cluster (Middle-Tier) system. Perform the following steps to stop and start Oracle Application Server processes:

1. Stop Oracle Application Server processes on the new active node and start OPMN only.

   Execute the following commands on UNIX systems:

   > $ORACLE_HOME/opmn/bin/opmnctl stopall
   > $ORACLE_HOME/opmn/bin/opmnctl start
   
   

2. Stop Oracle Application Server Administration processes on the new active node, using the following commands

   On UNIX systems:

   > $ORACLE_HOME/bin/emctl stop iasconsole
   > $ORACLE_HOME/bin/emctl stop agent
   
   

   On Windows systems,

   > %ORACLE_HOME%\bin\emctl stop iasconsole
   > %ORACLE_HOME%\bin\opmnctl stop agent
   
   

3. On the current active node, execute the following commands.

   On UNIX systems:

   > $ORACLE_HOME/opmn/bin/opmnctl stopall
   > $ORACLE_HOME/opmn/bin/opmnctl startall
   
   

   On Windows systems:

   > %ORACLE_HOME%\opmn\bin\opmnctl stopall
   > %ORACLE_HOME%\opmn\bin\opmnctl startall
   
   

4. If you use Application Server Control Console, start Oracle Application Server Administration processes on the current active node using the following commands.

   On UNIX systems:

   > $ORACLE_HOME/bin/emctl start agent
   > $ORACLE_HOME/bin/emctl start iasconsole
   
   

   On Windows systems,

   > %ORACLE_HOME%\bin\emctl start agent
   > %ORACLE_HOME%\bin\opmnctl start iasconsole
   
   

4.5.2.4 Manual Failover of OracleAS Cluster (OC4J-JMS)

If you are using OracleAS Cluster (OC4J-JMS), and the system fails abnormally, you may need to perform additional failover steps such as removing lock files for OracleAS JMS file-based persistence.