D Troubleshooting Upgrade

D.1.1 Cannot Expand an Installation Type During an Upgrade

After installing a middle tier, you may decide that you need a component that is in a larger middle tier. For example, you installed a J2EE and Web Cache middle tier, and you realize you need OracleAS Portal.

Problem

You want to expand your installation type while upgrading the middle tier to Oracle Application Server 10g Release 2 (10.1.2).

Solution

Expanding the installation type and upgrading to 10g Release 2 (10.1.2) are two separate steps. For example, to upgrade your 10g Release 2 (9.0.4) J2EE and Web Cache installation to a 10g Release 2 (10.1.2) Portal and Wireless installation, do the following:

1. Upgrade the J2EE and Web Cache instance to Oracle Application Server 10g Release 2 (10.1.2).

2. Use Oracle Universal Installer and the 10g Release 2 (10.1.2) installation procedure to expand the installation type to Portal and Wireless.

D.1.2 Problems Using Middle Tier with Upgraded Infrastructure

Problem

After upgrading an OracleAS Infrastructure, all the middle tiers generate errors when attempting to connect to the upgraded Infrastructure.

Solution

This problem occurs when you upgrade the OracleAS Metadata Repository before you upgrade the middle tiers that depend on the repository. To fix the problem, you must do one of the following:

• Upgrade the middle tiers so they are the same version as the OracleAS Metadata Repository.

• Revert the OracleAS Metadata Repository to its original version by restoring a backup of the OracleAS Metadata Repository; then, upgrade the middle tiers before upgrading the OracleAS Metadata Repository.

  
  

  See Also: Section 1.4, "Upgrade Rules to Follow"

  
  

D.1.3 Cannot Upgrade to a High Availability Environment During an Upgrade

Oracle Application Server 10g Release 2 (10.1.2) provides documentation for configuring your Oracle Application Server environment for high availability.

Problem

How do I upgrade my Oracle Application Server to a high availability environment?

Solution

Implementing a high available environment for Oracle Application Server and upgrading to 10g Release 2 (10.1.2) are two separate steps:

• To upgrade your Oracle Application Server installation to Oracle Application Server 10g Release 2 (10.1.2), use the instructions in this guide.

• To configure you environment for high availability, see the Oracle Application Server High Availability Guide.

D.1.4 Installing a 10g Release 2 (10.1.2) Middle Tier Against a Release 2 (9.0.2) Infrastructure

Problem

Is it possible to install a 10g Release 2 (10.1.2) middle tier against a Release 2 (9.0.2) Infrastructure?

Solution

Yes, but before you install the 10g Release 2 (10.1.2) middle tier, you must make a required change to the Release 2 (9.0.2) Oracle Internet Directory.

D.1.5 Locating the Upgrade Documentation for Your Platform

Problem

You cannot find the Upgrade documentation specific to your platform (for example, Sun Solaris, Linux, or Windows).

Solution

Carefully review the contents of the Oracle Application Server documentation page on OTN:

http://www.oracle.com/technology/documentation/appserver10g.html

Note that platform specific documentation is listed separately. For example, the Oracle Application Server documentation for the Sun Solaris platform is available in the Solaris platform-specific documentation library.

Do not attempt to use the Upgrade documentation for a different platform if the platform-specific information you are looking for is unavailable.

If the upgrade documentation for a particular platform is not available, it is likely that platform is not yet supported for the current Oracle Application Server release. You can obtain more information about supported releases on Oracle MetaLink, http://metalink.oracle.com.

D.1.6 Decommissioned Application Server Instances Still Visible on Farm Page in Application Server Control Console

Problem

After upgrading from Oracle Application Server Release 2 (9.0.2) or 10g (9.0.4) to 10g Release 2 (10.1.2), the source application server instances still appear on the Application Server Control Console Farm page.

Solution

Before you deinstall a source Oracle home that has been successfully upgrded to 10g Release 2 (10.1.2), be sure to remove the source Oracle home from its OracleAS Farm by using the instructions in Section 4.9.3, "Removing the Source Oracle Home from the OracleAS Farm".

D.1.7 'Could not start process' Error When Starting Oracle Application Server Services

Problem

After upgrade, when attempting to start the Oracle Application Server 10g Release 2 (10.1.2) services in the Windows Services Control panel, a "could not start process" error occurs and the status of the service is shown as "Starting."

Solution

In most cases you can ignore the error message because, in fact, the Application Server Control and Oracle Process Manager and Notification Server services have started successfully. Wait a few minutes, and then close and reopen the Services control panel to verify that the services have started; or, display the Application Server Control Console to verify that the Application Server Control and the Oracle Application Server components have started.

D.1.8 Enterprise Manager Web Site Port is Changed After Upgrade From Release 2 (9.0.2)

Problem

After upgrading from Oracle Application Server Release 2 (9.0.2), the URL for the Enterprise Manager Web site does not work.

Solution

After upgrading from Release 2 (9.0.2), you cannot use the same URL to manage your application server with Oracle Enterprise Manager. Instead, after upgrade to 10g Release 2 (10.1.2), you must use the new Application Server Control URL to display the Application Server Control Console.

For more information, see Section 4.6.1, "About Port Values and the portlist.ini File After Upgrade".

D.1.9 OracleAS Web Cache Fails to Start Due to XML Parsing Error After Configuring Upgraded Middle Tier for SSL

Problem

After upgrading the middle tier, if you configure the upgraded middle tier for SSL, OracleAS Web Cache fails to start. Errors in the log file indicate an XML parsing error; for example:

[03/Aug/2005:14:32:45 -0700] [alert 13109] XML parsing error in
@ /private/iasinst/upg/1012p2/bi/webcache/webcache.xml. Error code
@ 213: LSX-00213: only 0 occurrences of particle "HOSTREF", minimum is 1

Solution

This problem can be caused when an empty OracleAS Web Cache site is created in the source Oracle home. To fix this problem:

1. Locate and open the following configuration file in the destination Oracle home with a text editor:

   DESTINATION_ORACLE_HOME\webcache\webcache.xml
   
   

2. Locate the following element in the webcache.xml file:

   <VIRTUALHOSTMAP NAME="nodename.domain" PORT="4445" EXCLUDE="NONE" QUEUESIZE="0">
   
   

3. Verify that each VIRTUALHOSTMAP element includes a corresponding HOSTREF entry, as in the following example:

   <VIRTUALHOSTMAP NAME="hostname.domain" PORT="4445" EXCLUDE="NONE" QUEUESIZE="0">
         <HOSTREF HOSTID="hostid"/>
   </VIRTUALHOSTMAP
   
   

4. If any of the VIRTUALHOSTMAP elements do not include a HOSTREF element, then do the following:

   1. Locate and open the corresponding webcache.xml file in the source Oracle home.

   2. Locate the equivalent VIRTUALHOSTMAP element in the file.

   3. Copy the entire VIRTUALHOSTMAP element (including the HOSTREF element).

   4. Paste the entire VIRTUALHOSTMAP element into the webcache.xml file in the destination ORACLE_HOME.

   5. Save and close both webcache.xml files.

5. Try starting OracleAS Web Cache.

D.2.1 Problems Encountered When Running the Oracle Internet Directory Upgrade Assistant From Oracle Universal Installer

The Oracle Internet Directory upgrade assistant is one of the assistants that run near the end of the 10g Release 2 (10.1.2) installation procedure when you are upgrading an OracleAS Identity Management installation.

You can get information about the cause of Oracle Internet Directory upgrade assistant errors by looking at the following log file:

ORACLE_HOME\ldap\log\oidca.log

Problem 1

The upgrade assistant log file (oidca.log) reports the following:

OID processes are currently running

This is a result of some Oracle Internet Directory or Oracle Directory Integration and Provisioning processes not being shut down properly in the source Oracle Home.

Solution

Without exiting the Intaller, open another terminal window and shut down all the processes in the source Oracle Home. Then, retry the Oracle Internet Directory upgrade assistant from the Oracle Universal Installer configuration assistants page.

Without exiting the Installer, shut down all the processes in the source Oracle Home and then retry the Oracle Internet Directory configuration assistant from the Oracle Universal Installer configuration assistants page.

If you are upgrading from 10g (9.0.4), you can ensure that the proper processes are shut down by using the Oracle Process Manager and Notification Server (OPMN) command utility to start and then stop all processes in the Oracle home.For example, use the following procedure to start all the processes and then shut down all the processess. Note carefully whether or not any errors appear when you are starting or shutting down the processes:

1. Start all the processes in the OracleAS Identity Management 10g (9.0.4) source Oracle home using the following command:

   SOURCE_ORACLE_HOME\opmn\bin\opmnctl startall
   
   

2. Make sure that no errors occurred during startup and make sure that the Oracle Internet Directory server is up and running.

   To verify that Oracle Internet Directory is running, enter one of the following commands.

   If you are running Oracle Internet Directory on a non-secure port:

   SOURCE_ORACLE_HOME\bin\ldapbind -p Non-SSL_port -h
   
   

   If you are running Oracle Internet Directory on a secure port:

   SOURCE_ORACLE_HOME\bin\ldapbind -p SSL_port -h -U 1
   
   

   These commands should return a "bind successful" message.

   
   

   See Also: "Syntax for LDIF and Command-Line Tools" in the Oracle Internet Directory Administrator's Guide for more information about the ldapbind utility

   
   

3. Enter the following command to stop all the processes:

   SOURCE_ORACLE_HOME\opmn\bin\opmnctl stopall
   
   

4. Use the ldapbind commmand again to verify that Oracle Universal Installer is down and not running.

5. Rerun the Oracle Internet Directory upgrade assistant.

Problem 2

The Oracle Internet Directory upgrade fails with "table or view does not exist" error.

Solution

This problem occurs when the Oracle Internet Directory upgrade assistant is run against a 9.2.0.x OracleAS Metadata Repository containing a 9.2.0.x Oracle Internet Directory.

The solution is to do the following:

1. Create table imcfgregistry in the 9.2.0.x Oracle Internet Directory database repository by running the following SQL statement as ODS schema:

   CREATE TABLE imcfgregistry (Component  VARCHAR2(255),
      instMode VARCHAR2(255),
      IASInstance VARCHAR2(255))
      TABLESPACE OLTS_DEFAULT MONITORING;
   
   

2. Retry the Oracle Internet Directory upgrade assistant from the Oracle Universal Installer configuration assistants screen.

Problem 3

The Oracle Internet Directory configuration assistant fails during the Configuration Assistants phase of the OracleAS Identity Management upgrade with Oracle Universal Installer.

Solution

Check the contents of the following configuration file in the destination Oracle home and verify that the file contains the correct SERVICE_NAME entry for your Metadata Repository. If the value assigned to this entry is incorrect, enter the correct name, save the file, and retry the assistant.

DESTINATION_ORACLE_HOME\network\admin\tnsnames.ora

D.2.2 Problem Stopping Processes in Source Oracle Home During OracleAS Identity Management Upgrade

Problem

When you run the Oracle Universal Installer to upgrade OracleAS Identity Management, a popup dialog notifies you that the installer will shut down some processes in the source Oracle home.

After the installer performs the shutdown, it checks that Oracle Internet Directory is stopped. If Oracle Internet Directory is not stopped for some reason, installer will display another popup dialog notifying you of the problem.

Solution

Examine the following log file to determine the cause of the problem:

DESTINATION_ORACLE_HOME\cfgtoollogs\shutdownprocesses.log

Resolve the problem and then manually stop Oracle Internet Directory in the source Oracle home. Once Oracle Internet Directory is stopped, continue with the OracleAS Identity Management upgrade by clicking Continue in Oracle Universal Installer.

D.2.3 Database Listener Errors When Running Configuration Assistants During OracleAS Identity Management Upgrade

Problem

Oracle Universal Installer invokes configuration assistants at the end of the OracleAS Identity Management upgrade. Some of the configuration assistants require an Oracle Database 10g database listener to connect to the database. If an Oracle Database 10g database listener is not available, those configuration assistants fail.

Often, when this problem occurs, one of the following errors (or a similar error) appears in the installation log files:

java.lang.Exception: Error: Database Listener is down. Please start listener
and make sure database is up before running this script.

OR

java.sql.SQLException: Listener refused the connection with the followingerror:
ORA-12500, TNS:listener failed to start a dedicated server process 

Solution

The installer normally starts an Oracle Database 10g database listener in the destination Oracle home. However, if an Oracle9i Database listener is already running, then the installer fails to start the Oracle Database 10g (10.1.0.2) database listener.

The most common cause of this problem is that you missed the instruction in a pop-up dialog during the installation. This pop-up message indicates during the interview phase of the installation that there is a running database listener running and that you should stop the listener manually before proceeding.

To correct the problem, stop the existing Oracle9i (9.0.1.3) listener, and then start the database listener in the destination Oracle home, as follows:

1. Set the ORACLE_HOME environment variable to point to the destination Oracle home of upgrade.

2. Change directory to bin directory of the destination Oracle home.

3. Run the lsnrctl start command to start the listener.

After the Oracle Database 10g database listener is running, continue with the OracleAS Identity Management upgrade by clicking Retry on the Configuration Assistants page in Oracle Universal Installer.

D.2.4 Oracle Directory Integration and Provisioning Configuration Assistant Fails with Time Synchronization Error

Problem

While upgrading a 10g (9.0.4) distributed OracleAS Identity Management environment, the Oracle Directory Integration and Provisioning configuration assistant fails.

When you check the following log file, you notice an error message indicating that there is time difference of at least 250 seconds between two of the OracleAS Identity Management hosts:

DESTINATION_ORACLE_HOME\ldap\log\oidmon.log

Specifically, the error message says:

Time Difference of at least 250 sec found between DIP_hostname and OID_hostname.
Please sync the time between DIP-hostname and OID-hostname

Solution

Synchronize the system clocks on all nodes where the OracleAS Identity Management components reside so they are running within 250 seconds of each other.

When synchronizing the system clocks, make sure the clocks are set to the same time zone.

After you synchronize the clocks, rerun the Oracle Directory Integration and Provisioning configuration assistant.

D.3.1 Deciding When to Upgrade an Infrastructure Database

Problem

When a newer version of Oracle Database is announced, should I upgrade the OracleAS Metadata Repository database to the new database version?

Solution

In general, use caution when upgrading your Infrastructure database to a new database version. Check Oracle Metalink (http://metalink.oracle.com) for posted articles or announcements that confirm that the database version and upgrade has been tested and is supported for an existing OracleAS Metadata Repository database.

D.3.2 Verifying the Progress of the Database Upgrade Assistant During OracleAS Identity Management Upgrade

Problem

Oracle Universal Installer invokes Database Upgrade Assistant at the end of OracleAS Identity Management Upgrade. Database Upgrade Assistant may take a long time depending on the size and contents of the database. The installer shows progress of the Database Upgrade Assistant by displaying percentage numbers, but no details about the progress are shown on the Configuration Assistants screen in Oracle Universal Installer.

Solution

If you would like to obtain more detailed information about the progress of the Database Upgrade Assistant, examine the log files generated by the Database Upgrade Assistant. The log files reside in:

drive_letter_for_install_disk:\admin\SID\upgrade

In this example, replace SID with the system identifier of the database in the source Oracle home.

To obtain the timestamps of the different stages of the database upgrade, search for the string "COMP_TIME" in the log files. For example,

find "COMP_TIME" *.log

grep ^COMP_TIME *.log

The output of the command identifies each stage of the database upgrade, as well as a timestamp for each stage. For example:

Oracle_Server.log:COMP_TIMESTAMP DBUPG__BGN 2004-12-16 10:11:00 2453356 36660
Oracle_Server.log:COMP_TIMESTAMP UTLIP__END 2004-12-16 10:12:58 2453356 36778
Oracle_Server.log:COMP_TIMESTAMP CATALG_BGN 2004-12-16 10:27:44 2453356 37664
Oracle_Server.log:COMP_TIMESTAMP CATPROC    2004-12-16 11:18:45
Oracle_Server.log:COMP_TIMESTAMP RDBMS      2004-12-16 11:21:50
Oracle_Server.log:COMP_TIMESTAMP JAVAVM     2004-12-16 12:27:24
Oracle_Server.log:COMP_TIMESTAMP XML        2004-12-16 12:41:17
Oracle_Server.log:COMP_TIMESTAMP CATJAVA    2004-12-16 12:45:03
Oracle_Server.log:COMP_TIMESTAMP CONTEXT    2004-12-16 12:49:17
Oracle_Server.log:COMP_TIMESTAMP XDB        2004-12-16 12:56:32
Oracle_Server.log:COMP_TIMESTAMP OWM        2004-12-16 13:01:14
Oracle_Server.log:COMP_TIMESTAMP AMD        2004-12-16 13:11:04
Oracle_Server.log:COMP_TIMESTAMP ORDIM      2004-12-16 13:43:34
Oracle_Server.log:COMP_TIMESTAMP SDO        2004-12-16 13:52:30
Oracle_Server.log:COMP_TIMESTAMP WK         2004-12-16 13:56:24
Oracle_Server.log:COMP_TIMESTAMP DBUPG_END  2004-12-16 14:10:39
PostUpgrade.log:COMP_TIMESTAMP UTLRP_BGN  2004-12-16 14:12:32
PostUpgrade.log:COMP_TIMESTAMP UTLRP_END  2004-12-16 15:29:47

D.3.3 Performance Issues When Using the Metadata Repository Upgrade Assistant (MRUA) to Upgrade the OracleAS Portal Schema

Problem 1

The first time you run MRUA, the OracleAS Portal component (or plug-in) appears to hang or stop working while upgrading the OracleAS Portal schema.

Problem 2

If you run MRUA after the OracleAS Portal schema has already been upgraded, the OracleAS Portal MRUA plug-in, unlike other component MRUA plug-ins, does not immediately report the fact that OracleAS Portal has already been upgraded.

Solution

In most cases, no solution is necessary. Simply wait until the OracleAS Portal component of MRUA finishes processing.The OracleAS Portal component of MRUA performs an extensive sequence of pre-checks that can take several minutes to finish.

In addition, the OracleAS Portal component of MRUA executes for a much longer period of time than any of the other plug-ins. As a result, it can take up to 40 mintes to run.

If Portal plug-in appears to be hanging, review the log files in the following directory to determine whether or not portal upgrade is progressing:

MRUA_HOME\temp\portal

Do not abort MRUA during the OracleAS Portal schema upgrade without first verifying that Portal upgrade has actually stopped processing.

D.3.4 Database Upgrade Assistant Failure During OracleAS Identity Management Upgrade

Oracle Universal Installer invokes Database Upgrade Assistant at the end of OracleAS Identity Management Upgrade. If the Database Upgrade Assistant fails, you can examine the log files generated by the Database Upgrade Assistant. The log files reside in:

drive_letter_for_install_disk:\admin\SID\upgrade

In this example, replace SID with the system identifier of the database in the source Oracle home.

Examine the log files and determine the cause of the failure. In most cases, it is not possible to retry the Database Upgrade Assistant. Instead, you will need to restore the source Oracle home and the database files to their state before the OracleAS Identity Management Upgrade. After the restoration, make sure that the problems which caused the Database Upgrade Assistant to fail are resolved. Then run OracleAS Identity Management Upgrade again.

D.3.5 Problem Upgrading Oracle Workflow and OracleAS Portal Schemas with MRUA

Problem

When running MRUA, both the Oracle Workflow and OracleAS Portal upgrades fail and result in invalid versions.

The following error appears in the log files:

EXP-00056: ORACLE error 942 encountered
ORA-00942: table or view does not exist
EXP-00000: Export terminated unsuccessfully
Ending export at Mon May 30 02:59:21 2005

Solution

This error can be caused by providing an incorrect value for the -oracle_home argument on the MRUA command line. When running MRUA be sure to enter the Oracle home of the OracleAS Metadata Repository. This is especially important when upgrading a non-colocated Infrastructure where the OracleAS Metadata Repository is in its own Oracle home.