5 Managing Log Files

5.1.1 Understanding Log File Data and Naming

Several Oracle Application Server components use Oracle Diagnostic Logging (ODL). Using ODL, log file naming and the format of the contents of log files conforms to an Oracle standard and the diagnostic messages are written in XML. Some Oracle Application Server components do not use ODL, and write their diagnostic messages using a component specific text format.

Regardless of the format of the messages that are stored in log files, ODL or text based, you can view log files using the Application Server Control Console, or you can download log files to your local client and view them using another tool (for example a text editor, or another file viewing utility).

This section covers the following topics:

• ODL Message Formatting and ODL Log File Naming

• Log File Messages by Component

  
  

  Note: Some Oracle Application Server components do not support ODL. Other components support ODL, but do not enable ODL by default.

  
  

5.1.2 Using a Log Repository

The Application Server Control Console supports viewing diagnostic messages from a Log Repository. A Log Repository can be file-based or stored in a database, and contains messages collected from multiple diagnostic log files across components. The Log Repository does not contain messages from access or trace log files because access logs and trace logs are verbose and do not contain diagnostic information.

The Oracle Application Server Log Loader component initializes and updates the data in a Log Repository. After the Log Loader starts, it stores information from diagnostic log files to the Log Repository at regular intervals.

Using a Log Repository consolidates Oracle Application Server log file data. Then, you can use the Application Server Control Console to easily search and view log file data generated by multiple components. Using a Log Repository can speed up the diagnostic process and reduce the resources required to support Oracle Application Server.

5.1.3 Configuring Component Logging Options

Administrators configure logging options to manage and limit the logging information that Oracle Application Server components generate and save.

The logging configuration options include:

• Specifying log file names and pathnames: Most Oracle Application Server components let you specify the directory for storing diagnostic log files. Specifying the diagnostic logging directory allows administrators to manage system and network resources.

• Limiting log file size: As Oracle Application Server components run and generate diagnostic messages, the size of the log files increases. Oracle Application Server components use one of several strategies to deal with log file size. Some components allow log files to keep increasing in size; in this case, it is the administrator's responsibility to monitor and clean up the log files. Other components, including OC4J, let you specify configuration options that limit how much log file data is collected and saved.

• Using log file archiving: Certain Oracle Application Server components let you specify configuration options to control the size of diagnostic logging directories. This lets you determine a maximum size for the directories containing a component's log files. When the maximum size is reached, older logging information is deleted before newer logging information is saved.

• Setting component logging levels: Certain Oracle Application Server components, including Oracle HTTP Server, allow administrators to configure logging levels. By configuring logging levels, the number of messages saved to diagnostic log files can be reduced. For example, you can set the logging level so that the system only reports and saves critical messages.

  
  

  See Also: Oracle Application Server component documentation for information on setting logging configuration options

  
  

5.2.1 Listing Log Files for Components

You can list the log files for individual components from the Application Server Control Console. To list the log files, perform the following steps:

1. Select the Logs link on the Application Server Control Console. The View Logs page is displayed.

2. To view all components, click Move All to move all available components to Selected Components. To view some components, select them in the Available Components box and click Move.

3. Click Search to list the log files for the selected components.

4. After the search returns, the Results section shows log file information such as the name of the component associated with a log file and a link to the log file.

Figure 5-1 shows the Application Server Control Console View Logs page after a search.

Figure 5-1 Enterprise Manager View Logs Search Results

Description of "Figure 5-1 Enterprise Manager View Logs Search Results"

5.2.2 Listing Log Files from Oracle Application Server Components Pages

After you select a system component link on the Application Server Control Console Home page, you can view the log files for the selected component by clicking Logs at the top of the page. When you click Logs, the Application Server Control Console searches for the log files associated with the current component. Then, you can view the log files on the resulting View Logs page by selecting the links in the Log Files column in the Results table.

For example, if you click Logs on the HTTP Server Home page, Enterprise Manager searches for the log files associated with the Oracle HTTP Server and displays the View Logs page with a list of Oracle HTTP Server log files in the Results table.

When you select the Logs link from a component page, the log file pages include a Return to link at the bottom of each page. The Return to link returns you to the component page from which you selected the Logs link.

5.2.3 Using Log Files Advanced Search

You can filter the search for log files by certain log file attributes by using the Advanced Search page of the Application Server Control Console.

You can list log files using a search filter by performing the following steps:

1. Select the Logs link on an Application Server Control Console page. The View Logs page is shown.

2. Click Advanced Search to display the View Logs Advanced Search page. The Advanced Search page lets you list log files for Oracle Application Server components and enables you to filter the search for log files by certain log file attributes.

3. Select the desired components from the Available Components box and click Move or Move All to move components to the Selected Components box.

4. Select a field from the Log File Attribute list.

5. Click Add Row to add a row for the selected log file attribute.

6. Enter the desired search value in the Value field.

7. If you want to select additional fields with values, click Add Another Row and enter additional values.

8. Click Search to perform the search. When the search returns, the Results section shows log files with matching fields.

To obtain more information on filtering using log file attributes, click the information icon next to the Log File Attribute list.

Figure 5-2 shows the Advanced Search Filter By Log File Attributes selection box, with the Log File Attribute list and the Add Another Row button.

Figure 5-2 Log Files Advanced Search Filter By Log File Attributes

Description of "Figure 5-2 Log Files Advanced Search Filter By Log File Attributes"

5.3.1 Getting Started with Log Repository

The Log Repository must contain diagnostic messages before you can search the Log Repository. The Log Repository is initialized and updated by the Log Loader component, and the Log Loader must be started before you can search the Log Repository. The Log Loader is not started automatically; you must start the Log Loader to make sure that the Log Repository is updated.

5.3.2 Searching Log Repository with Simple Search

To search for diagnostic log entries in the Log Repository using simple search criteria, do the following:

1. Select the Logs link on an Application Server Control Console page. The View Logs page is shown.

2. From the View Logs page, click Search Log Repository.

3. Select components from the Available Components box, then click Move or Move All to move the selected components to the Selected Components box. This step is optional.

4. Use the default selections, or select the available search and result display options. The online help describes the available search and display options for the Search Log Repository page.

   
   

   Note: The Message Type selection box includes the Unknown option. Some components do not include a message type when the component writes log file entries. These messages are loaded into the Log Repository with Unknown specified as the message type.

   
   

5. Click Search to search for messages in the Log Repository that match the constraints you specify. When the search returns, the Results section shows the matching diagnostic log messages from the Log Repository.

   Figure 5-3 shows the Search Log Repository page.

Figure 5-3 Search Log Repository Page

Description of "Figure 5-3 Search Log Repository Page"

See Section 5.3.4, "Viewing Repository Log Entry Details" for information about viewing the log entry details.

5.3.3 Searching Log Repository with Advanced Search

With the Advanced Search feature, you can select from a set of repository query fields that can restrict the list of log message entries to those that apply to your criteria.

Take the following steps:

1. Select the Logs link on an Application Server Control Console page. The View Logs page is shown.

2. From the View Logs page, click Search Log Repository, then click Advanced Search.

3. On the Search Log Repository Advanced Search page, use the Filter By Log Entry Fields box to select log message fields and values to search. Select multiple fields by clicking Add Another Row. When you specify values for multiple fields, the search only returns results that match all of the specified constraints. The online help describes the available search and display options for the Search Log Repository page.

4. Use the default selections, or specify search and result date range and message type options by making selections and entering constraints.

5. Click Search to search for messages in the Log Repository that match the selection constraints. When the search returns, the Results section shows the matching log entries.

Figure 5-4 shows the Advanced Search Log Repository Filter By Log Entry Fields section.

Figure 5-4 Search Log Repository Advanced Search Filter By Log Entry Fields

Description of "Figure 5-4 Search Log Repository Advanced Search Filter By Log Entry Fields"

5.3.4 Viewing Repository Log Entry Details

To view a log entry, either select the link shown in the Time field of the Results area on the View Logs page, or select entries in the Select field and then click View Details. The Log Entry Details page is displayed, as shown in Figure 5-5. It displays information about the log entry, including the Message Type, Component, the Message Text, and optionally the Execution Context ID (ECID).

Figure 5-5 Log Repository Log Entry Details Page

Description of "Figure 5-5 Log Repository Log Entry Details Page"

5.3.5 Using Regular Expressions with Log Repository Search

Regular expression matching is applied when the check box in the Regular Expression field is selected on the Log Repository Simple Search or Advanced Search page. On the Simple Search page, the Regular Expression check box is under the Message Text field. On the Advanced Search page, the Regular Expression check box is in the Filter by Log Entry Fields box. Using a regular expression in a search enables you to enter a pattern description to match strings for a Log Repository search.

The Log Repository search uses the Apache Jakarta regular expression engine, which uses "*" for a string of characters, "?" for a single character, and supports boundary matches, including "^" for a match only at the beginning of an entry, and "$" for a match only at the end of an entry, and special characters, including "\t" for Tab, "\n" for newline, "\r" for return, and "\f" for form feed.

5.4.1 Correlating Messages Across Log Files and Components

Certain Oracle Application Server components provide message correlation information for diagnostic messages. Message correlation information helps those viewing diagnostic messages determine relationships between messages across components. The Execution Context ID (ECID) is a globally unique identifier associated with a thread of execution. The ECID helps you to use log file entries to correlate messages from one application or across application server components. By searching related messages using the message correlation information, multiple messages can be examined and the component that first generates a problem can be identified (this technique is called first-fault component isolation). Message correlation data can help establish a clear path for a diagnostic message across components, within which errors and related behavior can be understood.

When you view an entry on the Log Entry Details page in the Application Server Control Console, if the Execution Context ID field is available, it displays the Execution Context ID as a link. Selecting the Execution Context ID link shows you all the diagnostic messages in the Log Repository with the same execution context ID.

You can use the ECID to track requests as they move through Oracle Application Server.

The ECID takes the following format:

request_id, sequence_number

• The request_id is a unique string that is associated with each request.

• The sequence_number represents the hop number of the request, as it passes through Oracle Application Server (or through the component).

  For example, OracleAS Web Cache assigns an initial sequence number of 0 to a request (when OracleAS Web Cache handles the request). After that, the sequence number is incremented as the request moves through Oracle Application Server components.

Table 5-2 lists the Oracle Application Server components that provide message correlation information (using an ECID), and if a component supports message correlation, but it is not enabled by default.

5.4.2 Diagnosing Component Problems

When an Oracle Application Server component has a problem, you can isolate and determine the cause of the problem by viewing the diagnostic messages. The following general techniques can assist you in accomplishing this task:

• Search for errors, or warnings, related to the problem

• Correlate the errors across components

• Correlate the errors across a time interval

• Perform component-based analysis

Using a Log Repository can make searching for the root cause of a problem much easier. A Log Repository consolidates log file data and enables you to easily search, correlate, and view log file data that is generated by multiple Oracle Application Server components. A Log Repository correlates cross-component information by time, and correlates events that occur in a cascading fashion. Once a problem is isolated to a particular component in the repository, then, if needed, the problem can be further analyzed by examining the component-specific diagnostic files.

5.5.1 Starting and Stopping Log Loader

You can use the controls on the Application Server Control Console Log Loader page to start and stop the Log Loader.

To start the Log Loader, perform the following steps:

1. Select the Logs link on any Application Server Control Console page.

2. From the View Logs page, select the Search Log Repository link.

3. Select Log Loader.

4. On the Log Loader page, click Start.

5. On the confirmation page, you can click one of the following:

   • Cancel: Cancels the operation.

   • Start: Starts the Log Loader, but it does not load any existing log messages from component log files. Only messages that are added to the component logs after the Log Loader is started are added to the Log Repository.

   • Start and Load Existing Logs: Starts the Log Loader and loads all existing log messages from component log files. Any messages that are added to the component logs after the Log Loader is started are also added to the Log Repository.

5.5.2 Enabling and Disabling Log Loader

On the Log Loader page, the Enable button enables the Log Loader. By default, when you first install Oracle Application Server, the Log Loader is enabled, but not started. When you disable the Log Loader, Enterprise Manager stops the Log Loader and the Log Loader component does not appear in the list of components on the View Logs page.

When you enable the Log Loader, the Log Loader component appears in the components list on the View Logs page, but it is not started.

5.5.3 Updating the Log Configuration

When the Log Loader starts, it loads configuration information about the component log files it will use as sources for the diagnostic messages that are stored in the Log Repository. (This includes information on the location and format of the log files).Most log configuration files are installed when Oracle Application Server components are configured. The log configuration files for HTTP Server, OPMN, OC4J, and the Log Loader are generated when the Log Loader is initially started.If configuration changes are made that effect the location of diagnostic log files for these components, click Update Log Configuration on the Log Loader page to regenerate the log configuration files for these components. This ensures the Log Loader is loading the correct set of logs into the Log Repository.

5.5.4 Setting Log Loader Properties

You can set Log Loader properties from the Log Loader page. To navigate to the Log Loader page:

1. Select the Logs link on any Application Server Control Console page.

2. From the View Logs page, select the Search Log Repository link.

3. Click Log Loader.

4. Select the Log Loader Properties link in the Administration section. The Log Loader Properties page includes fields showing the current values for the Log Loader properties.

To change the Log Loader properties, perform the following steps:

1. Enter updated values in the appropriate fields on the Log Loader Properties page.

2. Click Apply to apply the new values.

Figure 5-6 shows the Application Server Control Console Log Loader Properties page.

Figure 5-6 Log Loader Properties Page

Description of "Figure 5-6 Log Loader Properties Page"

The Application Server Control Console online help includes detailed information on the Log Loader Properties fields.

5.5.5 Understanding Log Loader Diagnostic Messages

The Log Loader logs its diagnostic messages, including errors, to its log file. Diagnostic messages might include errors encountered due to an incorrect configuration, or errors that occur while the Log Loader is reading data from a log file or is writing data to the log repository.

The common Log Loader problems include:

• Errors in the Log Loader configuration file (ORACLE_HOME/diagnostics/config/logloader.xml). Errors in the configuration file usually prevent the Log Loader from running. Such errors need to be corrected before the Log Loader can work properly.

• Configuration errors that occur when a component's registration file contains errors (ORACLE_HOME/diagnostics/config/registration/*.xml). Errors in the registration files do not prevent the Log Loader from running but may prevent the contents of certain log files from being loaded in the repository. Typically, there are two common types of registration file errors:

  • XML syntax errors that prevent the file from being parsed. If such errors are encountered, the Log Loader completely ignores the contents of the file.

  • A wrong path specified for a configuration file. If the Log Loader cannot find a log file at the specified path, it issues a Warning level diagnostic message. This does not always indicate an error. For example, it is possible that the component that generates that log was not active when the Log Loader started and the log file had not been created yet. The Log Loader continues to look for the log file and starts reading messages when the log file is created.

• Errors may occur while the Log Loader is reading messages from a log file. If the log file includes contents that cannot be read or parsed, then the Log Loader issues a log message indicating that it cannot read part of the contents of the file. In this case, the Log Loader attempts to recover from the error and continue to read the Log File.

• Errors may occur when writing messages to the repository (for example, a disk error). This type of error may indicate a problem that may require attention from the system administrator to correct the problem.

• The Log Loader produces an error message when it skips reading log files because a log file exceeds the currently specified maximum load size. The maximum load size can be specified on the Log Loader properties page.

  In this case, the Log Loader logs an error message in the following format:

  Size of data to be read from log /logfile exceeds threshold of x bytes. Skipping y_skipped bytes and moving to end of log.
  
  

  This message indicates the size of data to be read exceeds the specified maximum load size x, and that the Log Loader is skipping to the end of the log file. The error message provides information on the name of the log file /logfile, and the number of bytes skipped y_skipped.

5.6.1 Using the printlogs Tool to View Log Messages

The printlogs tool is a command-line alternative to the Application Server Control Console for viewing log messages. The printlogs tool supports a variety of options for gathering and filtering log messages, and prints the results to standard output in a single format. For example, you can use printlogs to:

• Read log messages from the Log Repository or individual log files

• Filter log messages according to timestamp or log field value

• Print log messages in ODL or text format

• Sort log messages by field

• Report the number of log messages of a specified type

• Run in a continuous loop, printing log reports and sleeping for a specified amount of time

  
  

  See Also: Appendix F for more information about the printlogs tool

  
  

5.6.2 Understanding ODL Messages and ODL Log Files

This section covers the following topics:

• ODL Message Contents

• ODL Log File Naming

<MESSAGE>
  <HEADER>
    <TSTZ_ORIGINATING>2002-04-01T18:38:48.058-08:00</TSTZ_ORIGINATING>
    <ORG_ID>oracle.com</ORG_ID>
    <COMPONENT_ID>OHS</COMPONENT_ID>
    <HOSTING_CLIENT_ID>0.0.255.255</HOSTING_CLIENT_ID>
    <MSG_TYPE TYPE="ERROR"></MSG_TYPE>
    <MSG_LEVEL>17</MSG_LEVEL>
    <HOST_ID>test-perf9</HOST_ID>
    <HOST_NWADDR>0.0.255.255</HOST_NWADDR>
    <MODULE_ID>apache_core</MODULE_ID>
    <PROCESS_ID>5713</PROCESS_ID>
  </HEADER>
  <CORRELATION_DATA>
    <EXEC_CONTEXT_ID>
      <UNIQUE_ID>1017715128:255..255.255.88:5713:0:1</UNIQUE_ID>
      <SEQ>1</SEQ>
    </EXEC_CONTEXT_ID>
  </CORRELATION_DATA>
  <PAYLOAD>
    <MSG_TEXT>File does not exist:
 /files/Apache/docs/images/java-apache-project.gif
    </MSG_TEXT>
 </PAYLOAD>
</MESSAGE>

File                        Size

log.xml                     10002
log9872.xml                 15000
log9873.xml                 15000
log9874.xml                 15000
log9875.xml                 15000
log9876.xml                 15000

5.6.3 Understanding Log Loader Log File Format Conversion

The Log Loader reads logs in several different formats and it converts the contents of non-ODL logs to ODL format. In most cases, the resulting ODL log record will contain only a timestamp and the message text from the original log entry. Values for other ODL message fields, such as COMPONENT_ID and MODULE_ID can be provided in the log registration file for each log, so that these values are set to all log records parsed from the log. The Log Loader attempts to determine the severity or level of each non-ODL log and generate an appropriate ODL message type. However, in many cases, if the severity or level cannot be determined, the resulting ODL log record will have the message type set to UNKNOWN.

The Log Loader can even read unformatted logs, which may not even contain timestamp values. This is the case for several logs in the ORACLE_HOME/opmn/logs directory which contain redirected output from Oracle Application Server processes managed by Oracle Process Manager and Notification Server (OPMN). When log entries do not contain a timestamp, the Log Loader sets the timestamp to the value of the "last known timestamp" for that log. The value of the last known timestamp is determined according to the following rules:

1. The initial value of the last known timestamp is zero. Note that whenever adding a log record to the repository, a zero value timestamp is converted to the current time.

2. If the Log Loader finds an OPMN generated timestamp, it sets the last known timestamp with its value.

3. When the Log Loader reaches the end of the log, it sets the last known timestamp with the current time. If the Log Loader is running regularly, such as once every five minutes, this results in timestamps that are approximate to the actual time the message was written, within a five minute range. If the Log Loader is not run frequently, the value of these timestamps could be inaccurate.

   
   

   Note: The OC4J redirected logs found in the ORACLE_HOME/opmn/logs directory are not treated as unformatted logs, because each line in the OC4J logs contains a timestamp. Most other logs in this directory are treated as unformatted logs, and have timestamps assigned according to the preceding rules.

   
   

5.6.4 Component Diagnostic Log File Registration

The Application Server Control Console and the Log Loader read Oracle Application Server component diagnostic registration files to determine names, locations, and additional configuration information about diagnostic log files. The directory ORACLE_HOME/diagnostics/config/registration contains the diagnostic log file registration files.

Oracle Application Server components may have multiple registration files in the configuration registration directory.

The format for the registration files includes a Oracle Application Server component ID, and extension, .xml. Table 5-4 lists the Oracle Application Server Components and their associated Component IDs.

5.6.5 Configuring Components to Produce ODL Messages and ECIDs

Table 5-5 lists the Oracle Application Server components that support ODL messages but that generate text messages by default. By making configuration changes, you can configure these components to produce ODL messages and for OC4J, an ECID.

This section covers the following topics:

• Configuring Oracle HTTP Server to Produce ODL Messages

• Configuring OC4J to Produce ODL Messages

• Configuring OC4J to Produce ECIDs

See Table 5-1 for the complete list of Oracle Application Server components that produce ODL messages.

ORACLE_HOME/Apache/Apache/logs

ORACLE_HOME/j2ee/instance_name/log
ORACLE_HOME/j2ee/application-deployments/application_name/application.log

5.6.6 Creating and Managing a Diagnostic Message Database Repository

You can use SQL scripts to create and manage a database repository for diagnostic messages. By creating a database repository for diagnostic messages, you can search, view, and correlate diagnostic messages across multiple Oracle Application Server instances.

Use the SQL scripts described in the following sections to create and manage a repository for diagnostic messages. The scripts are located in the following directory:

• On UNIX:

  ORACLE_HOME/diagnostics/admin
  
  

• On Windows:

  ORACLE_HOME\diagnostics\admin
  
  

The database that hosts the Log Repository can be an Oracle9i database or an Oracle Database 10g database.

SQL> @ORACLE_HOME/diagnostics/admin/dmrep_cleanup.sql 7 DAY

SQL> connect dmrepusr
...
SQL> @ORACLE_HOME/diagnostics/admin/dmrep_drop.sql

SQL> connect sys as sysdba
...
SQL> drop user dmrepusr;
SQL> drop tablespace dmrep including contents and datafiles;

<repository>
   <xml_repository
      path="diagnostics/repository" 
      encoding="utf-8" segmentSize="5242880" maxSize="52428800"/>
</repository>

(UNIX) ORACLE_HOME/dcm/bin/dcmctl updateConfig -ct logloader
(Windows) ORACLE_HOME\dcm\bin\dcmctl updateConfig -ct logloader

5.6.7 Limitations and Configuration

Note the following limitations and configuration issues with log files:

• The Logs link in the Application Server Control Console gives you an integrated view of many Oracle Application Server component log files. However, certain log files are only available at the component level. Oracle Application Server components use the directory ORACLE_HOME/diagnostics/config/registration to make their log files visible to the Application Server Control Console. Some Oracle Application Server component log files are not exposed through Application Server Control Console pages.

• If you shut down the Log Loader after the database is shutdown, and you restart Log Loader after the database is restarted, some messages may be reloaded to the repository database twice.

  Usually, Log Loader saves its state after each load. However, if the database is shutdown first, Log Loader does not save its state. When it restarts, it resets its state to the end of the last successful load and will repeat any load that was unsuccessful. If the repository database was shutdown in the middle of that load, some of the records may have been written to the repository database, but the entire load will be repeated.