7 Fetchlets

OS Command Fetchlets

The operating system (OS) command fetchlets allow you to obtain metric data by executing OS commands (either individually or from scripts) that return a standard out (stdout) data stream.

Three OS command fetchlets are available:

• OS Fetchlet (raw)

• OSLines Fetchlet (split into lines)

• OSLineToken Fetchlet (tokenized lines)

OS Fetchlet

The OS Fetchlet executes a given OS command and returns the command's output in a single cell table.

Input Parameters

Table 7-1 OS Fetchlet Input Parameters

Parameter	Type	Description	Use
command	string	Operating system command to be executed.	required
ENVname	string	OSFetchlet parameters starting with "ENV" appear in the execution environment for the command as name environment variables	zero or more
errStartsWith	String	When defined, this property allows you to define a custom prefix for error messages. If this property is not defined, the OSFetchlet defaults to "em_error=" as the message prefix.	Optional
script	string	Specifies the script to be executed if command property only provides an interpretor. For example, command might consist of "perl." script is then used to specify the particular perl script to be run. Although scripts can be specified directly from the command parameter, using the script parameter adds to stylistic clarity and readability when defining a target type metadata file.	Optional
args	string	A property that is taken as one or more arguments to the command and script properties. Although args can be specified directly from the command parameter, using the args parameter adds to stylistic clarity and readability when defining a target type metadata file.	Optional
separateErrorStream	boolean	If an error occurs while executing a command, this property instructs the fetchlet whether to return both the stdout and stderr to the user as an error message. When set to TRUE, only stderr output is sent to the user as an error message when there is a command error. When set to FALSE (default value), both the stdout and the stderr are sent to the user as an error message upon command failure.	Optional. (TRUE/FALSE)
em_metric_timeout	integer	Metric timeout period (in seconds). After the timeout period has finished, the Management Agent returns a timeout exception and terminates any child processes that may have been created. The Management Agent DOES NOT kill any of the grandchild processes. Specify "-1" for no timout period.	Optional

Example

You want to obtain metric data by executing the UNIX echo command.

Executing the command from the shell environment, you would enter:

echo Line 1|some more|even more\nLine 2\n\nLine 4|a little more|\n|Line 5\n|Line 6|\n|Line 7|again|\nLine 8|the|end 

The echo command produces the following standard output:

Line 1|some more|even more 
Line 2 

Line 4|a little more| 
|Line 5 
|Line 6| 
|Line 7|again|
Line 8|the|end 

Using the OS Fetchlet with the given example command.

The fetchlet returns the following 1 x 1 table:

Figure 7-1 Table Returned by the OS Fetchlet

Description of "Figure 7-1 Table Returned by the OS Fetchlet"

The raw output of the OS command is returned. Any standard error output is appended to the standard output.

Error Handling

Any problems encountered launching the command (For example, the command program no longer exists) results in an oracle.sysman.emSDK.emd.fetchlet.MetricSourceException wrapping a java.io.IOException. If the command exits with a non-zero exit value, the fetchlet throws an oracle.sysman.emSDK.emd.fetchlet.MetricSourceException wrapping an oracle.sysmand.emd.fetchlets.CommandFailedException.

Notes

Commands are NOT executed as if they are being run in a shell. This means that common shell symbols do not work, including piping, output redirection, and backgrounding.

Commands cannot read from standard input.

The fetchlet blocks and waits for the command to finish.

OSLines Fetchlet (split into lines)

The OS Lines Fetchlet executes a given OS command and tokenizes the OS command's output. The output is tokenized by lines. The fetchlet returns the tokens in a single column table. The nth row in the table represents the nth line in the OS command's output.

To get the raw, untokenized output of an OS command, use the OS Fetchlet. To get the output of an OS command tokenized by lines and each line tokenized by a given delimiter, see the OS Line Token Fetchlet.

Input Parameters

Table 7-2 OSLines Fetchlet Input Parameters

Parameter	Type	Description	Use
command	string	Operating system command to be executed.	required
startsWith	string	Only lines starting with this string are included in the result.	optional; default = "" (all lines are included)
ENVname	string	Parameters starting with "ENV" appear in the execution environment for the command as name environment variables	zero or more of these
errStartsWith	string	When defined, this property allows you to define a custom prefix for error messages. If this property is not defined, the OSFetchlet defaults to "em_error=" as the message prefix.	Optional
script	string	Specifies the script to be executed if command property only provides an interpretor. For example, command might consist of "perl." script is then used to specify the particular perl script to be run. Although scripts can be specified directly from the command parameter, using the script parameter adds to stylistic clarity and readability when defining a target type metadata file.	Optional
args	string	A property that is taken as one or more arguments to the command and script properties. Although args can be specified directly from the command parameter, using the args parameter adds to stylistic clarity and readability when defining a target type metadata file.	Optional
separateErrorStream	boolean	If an error occurs while executing a command, this property instructs the fetchlet to whether to return both the stdout and stderr to the user as an error message. When set to TRUE, only stderr output is sent to the user as an error message when there is a command error. When set to FALSE (default value), both the stdout and the stderr are sent to the user as an error message upon command failure.	Optional. (TRUE/FALSE)
em_metric_timeout	integer	Metric timeout period (in seconds). After the timeout period has finished, the Management Agent returns a timeout exception and terminates any child processes that may have been created. The Management Agent DOES NOT kill any of the grandchild processes. Specify "-1" for no timout period.	Optional

Example

Let's take the following Unix command:

echo Line 1|some more|even more\nLine 2\n\nLine 4|a little more|\n|Line 5\n|Line 6|\n|Line 7|again|\nLine 8|the|end 

It produces the following output:

Line 1|some more|even more  Line 2 

Line 4|a little more|  |Line 5  |Line 6|  |Line 7|again|  Line 8|the|end 

Running OSLinesFetchlet with the given example command produces the following single column table.

Figure 7-2 Table Returned by the OS LINES Fetchlet

Description of "Figure 7-2 Table Returned by the OS LINES Fetchlet"

Note that without content, "\n" results in a blank line inserted between Line 2 and Line 4.

Notes: Commands are NOT executed as if they are being run in a shell. This means that common shell symbols do not work, including piping, output redirection, and backgrounding. Commands cannot read from standard input. The fetchlet blocks and waits for the command to finish. The standard output of the command is captured and the standard error is captured and appended to the standard output. Lines are tokenized using "\n".

OSLineToken Fetchlet (tokenized lines)

The OS Line Token Fetchlet executes a given OS command and tokenizes the OS command's output. The output is tokenized first by lines, and then each line is tokenized by a given delimiter set. The fetchlet returns the tokens in a table. The nth row in the table represents the nth line in the OS command's output. The nth column in the table represents the nth token in a line as determined by the given delimiter set.

To get the raw, untokenized output of an OS command, see the OS Fetchlet.

Input Parameters

Table 7-3 OSLineToken Fetchlet Input Parameters

Parameter	Type	Description	Use
command	string	Operating system command to be executed.	required
delimiter	string	Set of characters that act as delimiters to tokenize the lines	optional; default = "" (just breaks output into lines)
startsWith	string	Only lines starting with this string are included in the result	optional; default = "" (all lines are included)
ENVname	string	Parameters starting with "ENV" appear in the execution environment for the command as name environment variables	zero or more of these
errStartsWith	String	When defined, this property allows you to define a custom prefix for error messages. If this property is not defined, the OSFetchlet defaults to "em_error=" as the message prefix.	Optional
script	string	Specifies the script to be executed if command property only provides an interpretor. For example, command might consist of "perl." The script is then used to specify the particular perl script to be run. Although scripts can be specified directly from the command parameter, using the script parameter adds to stylistic clarity and readability when defining a target type metadata file.	Optional
args	string	A property that is taken as one or more arguments to the command and script properties. Although args can be specified directly from the command parameter, using the args parameter adds to stylistic clarity and readability when defining a target type metadata file.	Optional
separateErrorStream	boolean	If an error occurs while executing a command, this property instructs the fetchlet to whether to return both the stdout and stderr to the user as an error message. When set to TRUE, only stderr output is sent to the user as an error message when there is a command error. When set to FALSE (default value), both the stdout and the stderr are sent to the user as an error message upon command failure.	Optional. (TRUE/FALSE)
em_metric_timeout	integer	Metric timeout period (in seconds). After the timeout period has finished, the Management Agent returns a timeout exception and terminates any child processes that may have been created. The Management Agent DOES NOT kill any of the grandchild processes. Specify "-1" for no timout period.	Optional

Example

Let's take the following Unix command:

echo Line 1|some more|even more\nLine 2\n\nLine 4|a little more|\n|Line 5\n|Line 6|\n|Line 7|again|\nLine 8|the|end 

It produces the following output:

Line 1|some more|even more 
Line 2 

Line 4|a little more| 
|Line 5 
|Line 6| 
|Line 7|again| 
Line 8|the|end 

Running OSLineTokenFetchlet with the given example command and a single character "|" for the delimiter generates the following table:

Figure 7-3 Table Returned by the OS Token Lines Fetchlet

Description of "Figure 7-3 Table Returned by the OS Token Lines Fetchlet"

Error Handling

Any problem launching the command (unable to find the command program) results in a oracle.sysman.emSDK.emd.fetchlet.MetricSourceException wrapping a java.io.IOException.

If the command exits with a non-zero exit value, the fetchlet throws a oracle.sysman.emSDK.emd.fetchlet.MetricSourceException wrapping a oracle.sysmand.emd.fetchlets.CommandFailedException.

Notes

Commands are NOT executed as if they are being run in a shell. This means that common shell symbols do not work, including piping, output redirection, and backgrounding.

The fetchlet promptly closes the input stream to the running command.

The fetchlet blocks and waits for the command to finish.

Lines are tokenized using "\n".

The delimiter can be a single character or a set of characters. For example, it can be "|+_", if the line should be broken up by pipes, pluses, and underscores. If two or more delimiters are together in the output text, such as "||" or "+|+", then it is as if there are empty string tokens between them. These empty strings get columns in the result table. It is NOT considered that there are empty strings preceding a delimiter that starts a line or following a delimiter that ends a line.

In order to express non-printable characters in the delimiter set (such as tabs) in XML, use "&#xHH;" where H is the hexadecimal identifier for the character.

SQL Fetchlet

The SQL Fetchlet executes a given SQL statement on a given database as a given user and returns the table result.

Input Parameters

Notes

The SQL Fetchlet uses the Oracle Call Interface (OCI) from libclntsh.so.

The SQL statement or PL/SQL block can be specified either through the STATEMENT property, or via a file whose name is provided through the FILENAME property.

The SQL fetchlet supports input parameters. Input and output parameters are indicated in the SQL/PLSQL text in the usual way, by using ":<number>". Input parameters can be used to bind values to both SQL queries and PL/SQL blocks.

Input parameter values are specified as properties of the form SQLINPARAM<position>. There can be any number of input parameters. The input parameters need to be scalar: input parameters of type ARRAY and STRUCT are not supported.

The SQL fetchlet supports the execution of anonymous PL/SQL blocks (which may call other functions or procedures) to retrieve data. When executing a block of PL/SQL, data is returned to the fetchlet by means of an OUT parameter. There can be exactly one out parameter. It must be of type SQL_CURSOR (a PL/SQL REF CURSOR), or it must be a named type that represents an array of objects. In the latter case, each field of the object represents one column of the table; and each object instance in the array represents one complete row in the table. The OUT parameter position and type are specified by means of the properties SQLOUTPARAMPOS and SQLOUTPARAMTYPE. If an OUT parameter is specified, then the fetchlet assumes it is executing PL/SQL and treats the STATEMENT property as an anonymous PL/SQL block.

If no OUT parameter is specified, the fetchlet assumes that it is executing a SQL query.

Note that all input parameters to the SQL fetchlet are strings. This means that all other datatypes will have to be converted to strings. This is straightforward for datatypes such as numbers, but not, for example, dates and timestamps. You can pass an absolute date or timestamp by passing a character representation of the value (using a DateFormat class). There is no way currently to pass in a date function, such as SYSDATE or SYSDATE+1. In such case, you could embed the date argument directly in the SQL, for example:

begin func1(:1, :2, SYSDATE); end;

The other caveat is passing null arguments to a procedure. Consider the following SQL:

STATEMENT=begin func1(:1,:2); end;
SQLINPARAM1=null
SQLOUTPARAMPOS=2
SQLOUTPARAMTYPE=fooret

Assume that the first argument is intended to be a varchar2. By parameterizing it and passing 'null' as the first argument, what we are really doing is passing the *string* 'null' to the argument, and not a null value. If you intend to pass a null value, do the following:

STATEMENT=begin func1(null, :1); end;
SQLOUTPARAMPOS=1
SQLOUTPARAMTYPE=fooret

Examples

The following properties execute a query (get all users) with no parameters:

MachineName=skini-pc 
Port=1521 
SID=o817 
UserName=scott 
password=tiger 
STATEMENT=select * from all_users; 
NUMROWS=30 

The following properties execute a query (get the first few objects of a specified type owned by a specified user) with input parameters:

MachineName=skini-pc 
Port=1521 
SID=o817 
UserName=scott 
password=tiger 
STATEMENT=select * from all_objects where owner=:1 and object_type=:2 and rownum<:3tt>
NUMROWS=30 
SQLINPARAM1=SYSTEM 
SQLINPARAM2=INDEX 
SQLINPARAM3=10 

The following example executes a PL/SQL procedure that returns a cursor and has input parameters:

MachineName=skini-pc
Port=1521
SID=o817
UserName=scott
password=tiger
STATEMENT=begin :1 := skini_junk.func1(:2); end;
NUMROWS=30
SQLINPARAM2=SYSTEM
SQLOUTPARAMPOS=1
SQLOUTPARAMTYPE=sql_cursor

The following example specifies a PL/SQL procedure that returns an array of strings:

MachineName=skini-pc
Port=1521
SID=o817
UserName=scott
password=tiger
STATEMENT=begin skini_junk.newproc(:1,:2); end;
NUMROWS=30
SQLINPARAM1=SYSTEM
SQLOUTPARAMPOS=2
SQLOUTPARAMTYPE=my_string_array

The following example specifies a PL/SQL package that returns an array of structures:

MachineName=skini-pc
Port=1521
SID=o817
UserName=scott
password=tiger
STATEMENT=begin :1 := skini_junk.func2(:2,:3,:4,:5,:6); end;
NUMROWS=30
SQLINPARAM2=somename
SQLINPARAM3=someplace
SQLINPARAM4=someanimal
SQLINPARAM5=something
SQLINPARAM6=22
SQLOUTPARAMPOS=1
SQLOUTPARAMTYPE=my_struct_array

The PL/SQL used in the examples above is provided below for reference:

create or replace type my_type as Object (
    name varchar2(128),
    place varchar2(128),
    animal integer,
    thing number,
    thing2 number);
/
create or replace type my_struct_array as table of my_type;
/

create or replace type my_string_array as table of varchar2(3000);
/

create or replace type my_int_array as table of integer;
/

create or replace package skini_junk as

type Jcr is ref cursor;

function func1(username in varchar2) return Jcr;
function func2(name varchar2, place varchar2, animal integer,
             thing number, thing2 number) return my_struct_array;
procedure newproc(name varchar2, outArray OUT my_string_array);
procedure newproc2(numrows in varchar2, outArray OUT my_int_array);

end skini_junk;
/


create or replace package body skini_junk as

function func1(username in varchar2) return Jcr is
cr Jcr;
begin
    open cr for select object_name, object_type, status from all_objects where
         owner=upper(username);

     return cr;
end;

function func2(name varchar2, place varchar2, animal integer,
             thing number, thing2 number) return my_struct_array IS
ret my_struct_array := my_struct_array();

begin
    ret.extend(50);

    for i in 1..50 loop
        ret(i) := my_type(name || i,
                          place || i,
                          animal+i,
                          thing+i,
                          thing2+i);
    end loop;
    return ret;
end;

procedure newproc(name varchar2, outArray OUT my_string_array) IS
begin
    outArray := my_string_array();
    outArray.extend(100);

    for i in 1..100 loop
        outArray(i) := name || i;
    end loop;
end;

procedure newproc2(numrows in varchar2, outArray OUT my_int_array) IS
begin
    outArray := my_int_array();
    outArray.extend(numrows);
    for i in 1..numrows loop
        outArray(i) := i;
    end loop;
end;

end skini_junk;
/

SNMP Fetchlet

An object identifier (OID) corresponds to either a MIB variable instance or a MIB variable with multiple instances. Given a list of (OIDs), the SNMP Fetchlet polls an SNMP agent on a given host for corresponding instances.

Input Parameters

Error Handling

MissingParameterException is thrown if either hostname or OIDS is not given. FetchletException is thrown if TABLE is not equal to either TRUE or FALSE, an I/O error occurs while sending/receiving SNMP messages to/from the agent, or the agent responds with an SNMP error.

Notes

The table returned by the fetchlet will contain a column for every OID in OIDS. If input OIDs correspond to single variable instances, the table will have just one row with those instances. On the other hand, if the OIDs correspond to variables with multiple instances, each column in the table will contain instances for its OID and each row will correspond to a different subidentifier. (A subidentifier is an OID extension that uniquely identifies a particular variable instance for some MIB variable.) OIDS must contain either all OIDs with subidentifiers or all OIDs without the subidentifiers.

For example, to request the variable instances for the three OIDs: sysDescr, sysUpTime, and sysName, OIDS would have to be "1.3.6.1.2.1.2.1.1.1.0 1.3.6.1.2.1.2.1.1.3.0 1.3.6.1.2.1.2.1.1.5.0". In this case, all OIDs contain the instance subidentifier, ".0". The return table would appear as follows (the actual values may be different):

Figure 7-4 SNMP Fetchlet

Description of "Figure 7-4 SNMP Fetchlet"

Alternatively, assume that some MIB contains the following 3 columns and 4 instances:

Figure 7-5 SNMP Fetchlet: Columns 3 and 4 Content

Description of "Figure 7-5 SNMP Fetchlet: Columns 3 and 4 Content"

To construct a table with 3 columns corresponding to ifDescr, ifInOctets, and ifOutOctets, OIDS would have to look as follows

"1.3.6.1.2.1.2.1.2.2.1.2 1.3.6.1.2.1.2.1.2.2.1.10 1.3.6.1.2.1.2.1.2.2.1.16"

The fetchlet would return:

Figure 7-6 SNMP Fetchlet:ifDescr, ifInOctets, and ifOutOctets OIDS

Description of "Figure 7-6 SNMP Fetchlet:ifDescr, ifInOctets, and ifOutOctets OIDS"

The rows correspond to subidentifiers 1,2,3,4 respectively.

Any OID in OIDS can be appended with another placement OID. The variable instances for the placement OID do not appear in the returned table. Instead, they determine the place for the variable instances of the original OID within a column. In particular, for every variable instance I with subidentifier S in the set of instances for the original OID, (a) there must exist a variable instance X with subidentifier S in the set of instances corresponding to the placement OID, and (b) X is used as the subidentifier for the instance I.

For example, consider a MIB containing the following 3 columns, each with 4 variable instances:

Figure 7-7 SNMP Fetchlet: MIB Content with 4 Variable Instances

Description of "Figure 7-7 SNMP Fetchlet: MIB Content with 4 Variable Instances"

To construct a table containing ifDescr and ipAdEntNetMask, OID of ipAdEntIfIndex would have to be used as the placement OID to "align" the columns. Thus, the OIDS input to the fetchlet would be "1.3.6.1.2.1.2.1.2.2.1.2 1.3.6.1.2.1.2.1.4.20.1.3*1.3.6.1.2.1.2.1.4.20.1.2". The fetchlet output will be as follows:

Figure 7-8 SNMP Fetchlet: Table Containing ifDescr and ipAdEntNetMask

Description of "Figure 7-8 SNMP Fetchlet: Table Containing ifDescr and ipAdEntNetMask"

If OIDS were "1.3.6.1.2.1.2.1.2.2.1.2 1.3.6.1.2.1.2.1.4.20.1.3" for the previous example, the output would be as follows:

Figure 7-9 SNMP Fetchlet: Alternate OID

Description of "Figure 7-9 SNMP Fetchlet: Alternate OID"

URL Timing Fetchlet

The URL Timing Fetchlet gets the contents of a given URL, timing not only the base page source but any frames or images in the page as well.

Input Parameters

Metric Columns and Output Modes

The format of information and specific metric information returned are controlled by the "output_format" and "metrics" input parameters. The following table lists the format categories and the metrics (columns) returned by each. For a description of available metric columns, see Table 7-8, "URLTIMING Fetchlet: Metric Columns"

Metric Columns

The following table shows the metric columns returned by the URLTIMING fetchlet.

Example

Let's take the following URL:

url0=http://www.oracle.com/ 

With the input parameter output_format=summary, the fetchlet returns the following table (minus the headers on the columns):

Figure 7-10 Summary Output Format

Description of "Figure 7-10 Summary Output Format"

With output_format = summary and metrics = total_response_time, status, status_description the fetchlet returns the following table (minus the headers on the columns):

Figure 7-11 Summary Output Format with Specified Metric Columns

Description of "Figure 7-11 Summary Output Format with Specified Metric Columns"

With output_format = summary and metrics = total_response_time, status, status_description the fetchlet returns the following table (minus the headers on the columns) and the server is giving error:

Figure 7-12 Summary Output Format with Specified Metric Columns and Internal Server Error

Description of "Figure 7-12 Summary Output Format with Specified Metric Columns and Internal Server Error"

Let's take the following URL:

url0=http://www.oracle.com/
url1=http://nedc.us.oracle.com/

With the output_format=summary, the fetchlet returns the following table (minus the headers on the columns). Here the numbers are time taken for fetching both the urls.

Figure 7-13 Summary Output Format for Two URLs

Description of "Figure 7-13 Summary Output Format for Two URLs"

With the output_format=detailed, the fetchlet returns the following table (minus the headers on the columns):

Figure 7-14 Detailed Output for Two URLs

Description of "Figure 7-14 Detailed Output for Two URLs"

With the output_format=repeat_column, the fetchlet returns the following table (minus the headers on the columns):

Figure 7-15 Repeat Column Output Format

Description of "Figure 7-15 Repeat Column Output Format"

Error Handling

Metric error if the URL parameter is missing, malformed, or if the metric cannot be computed.

Notes

The time required to perform a retry will be added on to the total time of the page. For example, if two retries are performed and then a success occurs, the total page time will be the time of the page that succeeded plus the time it took for the two retries to fail.

Proposed usage:

• For basic monitoring:

  Use url0=<URL to be monitored> , output_mode=summary and specify metrics=status, computed_response_time, status_description

• For getting all columns:

  Use url0=<url to be monitored> , output_mode=summary

Dynamic Monitoring Service (DMS) Fetchlet

The Dynamic Monitoring Service (DMS) fetchlet contacts an Application Server (AS) and then collects the metrics instrumented by the DMS.

The DMS allows application and system developers to measure and export customized, component-specific performance metrics. The Oracle Management Agent allows software components to import runtime performance data into Oracle Enterprise Manager Grid Control.

The DMS Fetchlet is an Oracle Management Agent plug-in module that allows the Management Agent to import the performance data that is exported by the DMS. Using the DMS fetchlet, any component that is instrumented using DMS API calls may share its performance data with Enterprise Manager Grid Control.

/* first create the nouns*/
  Noun n1 = Noun.create("/myExample/myComponent/noun1", "MyType");
  Noun n2 = Noun.create("/myExample/myComponent/noun2", "MyType");

  /* next, create the Sensors */
  PhaseEvent pe1 = PhaseEvent.create(n1, "criticalPhase", "a critical interval");
  PhaseEvent pe2 = PhaseEvent.create(n2, "criticalPhase", "a critical interval");
  Event e1 = Event.create(n1, "importantEvt", "an important event");
  Event e2 = Event.create(n2, "importantEvt", "an important event");

  /* here is a third set that shows the use of Noun.setType(String) */
  PhaseEvent pe3 = PhaseEvent.create( 
                               "/myExample/myComponent/noun3/criticalPhase", 
                               "a critical interval");
  Event e3 = PhaseEvent.create( 
                               "/myExample/myComponent/noun3/importantEvt", 
                               "an important event");
  Noun n3 = Noun.get("/myExample/myComponent/noun3");
  n3.setType("MyType");

http://YOUR_AS_HOST:YOUR_AS_PORT/YOUR_SERVLET_PATH/AggreSpy?format=targetmetadata 

<Target Type='oracle_dms' NAME='DMS_YOUR-IAS-HOST_YOUR-IAS-PORT' VERSION='2.0'>
 <Property NAME='host' VALUE='YOUR_IAS_HOST' />
 <Property NAME='port' VALUE='YOUR_IAS_PORT' />
 <Property NAME='dmsPath' VALUE='YOUR_SERVLET_PATH' />
</Target>

>$ORACLE_HOME/bin/emctl reload

http://<YOUR_AGENT_HOST>:<YOUR_AGENT_PORT>/emd/browser/main 

HTTP Data Fetchlets

The HTTP data fetchlets obtain the contents of a URL and returns the contents of the URL as data. Three fetchlets are available:

• URL Fetchlet

• URL Lines

• URL Lines Token

URL Fetchlet (raw)

The URL Fetchlet gets the contents of a given URL and returns the contents of the URL in a single cell table.

To get the output of a URL tokenized by lines and each line tokenized by a given delimiter, see the URL Line Token Fetchlet.

Input Parameters

Table 7-10 URL Fetchlet Input Parameters

Name	Description	Use
url	URL to retrieve the contents of	required
proxyHost	proxy host through which to make the URL connection.	optional
proxyPort	proxy port through which to make the URL connection.	optional

Example

Let's take the following URL:

http://localhost/nhcities.txt 

It has the following contents:

Line 1: Nashua, Keene,

Line 2: Concord

Line 3: , Conway, Manchester, Milford, Brookline,

Line 4:

Line 5: Hollis, Meredith

Now let's run the URL Fetchlet with the given URL. The fetchlet returns the following one-by-one table:

Figure 7-16 URL Fetchet Output

Description of "Figure 7-16 URL Fetchet Output"

The raw contents of the URL is returned.

Error Handling

MissingParameterException if URL parameter is missing. FetchletException if the URL is malformed or an I/O error occurs in retrieving the content of the URL.

URL Lines Fetchlet (split into lines)

The URL Fetchlet gets the contents of a given URL and tokenizes the contents of the URL. The output is tokenized by lines. The fetchlet returns the tokens in a single column table. The nth row in the table represents the nth line of the URL contents.

Note: To get the raw, untokenized contents of a URL, see the URL Fetchlet. To get the contents of a URL tokenized by lines and each line tokenized by a given delimiter, see the URL Line Token Fetchlet.

Table 7-11 URL Lines Fetchlet Input Parameters

Name	Description	Use
url	URL to retrieve the contents of	required
proxyHost	proxy host through which to make the URL connection.	optional
proxyPort	proxy port through which to make the URL connection.	optional
startsWith	only lines starting with this string are included in the result	optional; default = "" (all lines are included)

Example

Let's take the following URL:

http://localhost/nhcities.txt 

It has the following contents:

Line 1: Nashua, Keene,

Line 2: Concord

Line 3: , Conway, Manchester, Milford, Brookline,

Line 4:

Line 5: Hollis, Meredith

Now let's run the URL Fetchlet with the given URL.

The fetchlet returns the following table:

Figure 7-17 URL LInes Fetchlet Output

Description of "Figure 7-17 URL LInes Fetchlet Output"

Error Handling

MissingParameterException if URL parameter is missing.

FetchletException if the URL is malformed or an I/O error occurs in retrieving the content of the URL.

Notes

Lines are tokenized using "\n".

URL Line Token Fetchlet (tokenized lines)

The URL Fetchlet gets the contents of a given URL and tokenizes the contents of the URL. The output is tokenized first by lines, and then each line is tokenized by a given delimiter set. The fetchlet returns the tokens in a table. The nth row in the table represents the nth line of the URL content. The nth column in the table represents the nth token in a line as determined by the given delimiter set.

To get the raw, untokenized contents of a URL, see the URL Fetchlet.

Table 7-12 URL Line Token Fetchlet Input Parameters

Name	Description	Use
url	URL to retrieve the contents of	required
delimiter	set of characters that act as delimiters to tokenize the lines	optional; default = "" (just breaks output into lines)
proxyHost	proxy host through which to make the URL connection.	optional
proxyPort	proxy port through which to make the URL connection.	optional
startsWith	only lines starting with this string are included in the result	optional; default = "" (all lines are included)

Example

Let's take the following URL:

http://localhost/nhcities.txt 

It has the following contents:

Line 1: Nashua, Keene,

Line 2: Concord

Line 3: , Conway, Manchester, Milford, Brookline,

Line 4:

Line 5: Hollis, Meredith

Now let's run the URL Fetchlet with the given URL and a single character "," for the delimiter.

The fetchlet returns the following table:

Figure 7-18 URL Token Lines Output

Description of "Figure 7-18 URL Token Lines Output"

Error Handling

MissingParameterException if URL parameter is missing.

FetchletException if the URL is malformed or an I/O error occurs in retrieving the content of the URL.

Notes

Lines are tokenized using "\n".

The delimiter can be a single character or a set of characters. For example, it can be "|+_", if the line should be broken up by pipes, pluses, and underscores. If two or more delimiters are together in the output text, such as "||" or "+|+", then it is as if there are empty string tokens between them. These empty strings get columns in the result table. It is NOT considered that there are empty strings preceding a delimiter that starts a line or following a delimiter that ends a line.

In order to express non-printable characters in the delimiter set (such as tabs) in XML, use "&#xHH;" where H is the hexadecimal identifier for the character.

URLXML Fetchlet

The URL XML Fetchlet obtains the XML content of a given URL, and extracts information based on a given pattern. A pattern is a list of "chunks" of XML to match against. The return table is a table with a column for each grabber (*) in the pattern in order and a row each time the pattern chunk matches in the XML content.

Input Parameters

Example

Let's take the following URL:

http://localhost/urlxmltestfile.xml

It has the following content:

<?xml version="1.0"?>
<testfile>
    <test>Simple text</test>
    <test><level>A little more complex</level></test>
    <test></test>
    <notatest></notatest>
    <test>Yet more complexity<level>Even a little more complex</level>Will it ever stop?</test>
    <test1>must match<level>extract me!</level></test1>
    <test1>must match here<level>extract me, too!</level></test1>
</testfile>

Running the URL XML Fetchlet with the given URL and the pattern:

<testfile><test>*<level>*</level></test></testfile>

returns the following table:

Figure 7-19 URL XML Fetchlet Output

Description of "Figure 7-19 URL XML Fetchlet Output"

Error Handling

MissingParameterException if URL or pattern parameters are missing.

A FetchletException is generated if:

• The URL is malformed.

• An I/O error occurs in retrieving the content of the URL.

• The URL contents or pattern contains invalid XML.

Notes

Setting the proxy host and/or port changes these settings for the java.net package for the whole Java environment and is not thread-safe if the proxy settings are changing.

WBEM Fetchlet

The WBEM fetchlet accesses a CIMOM and retrieves requested information using the specified CIM class. The CIM class is mapped to a Management Repository table metric. The name of the CIM class is the name of the table metric that is returned, and the properties defined for the CIM class are used to name the table columns for the metric. The properties of interest must be specified during metric definition.

The fetchlet returns the instances that have been instantiated for the CIM class as rows of the Management Repository table metric.

Input Parameters

Error Handling

The following types of errors have been identified for the WBEM fetchlet.

MissingParameterException occurs when:

• No CIM Class parameters match.

Fetchlet exception occurs when:

• The class name is not found in the CIMOM namespace.

• The namespace is not found.

• The connection to the CIMOM does not have valid credentials.

• The connection to the CIMOM failed because the CIMOM was not running.

• The CIM class property does not exist

• An unsupported operation was specified

• No properties were specified.

Notes

Ports: Some CIMOM client interfaces expose the port that the CIMOM is listening on while some clients do not. To cover both cases, the port is exposed as an optional input parameter that defaults to port 5988. This is the default Pegasus CIMOM listener port. The Java API that is provided through Sun's Wbem Services does not expose the CIMOM port.

Protocols: Most CIMOMs support either an RMI or HTTP protocol for communicating with the CIMOM. The testing that has been done shows that the HTTP protocol is not as stable, and in some cases, not fully implemented in the CIMOM. Because of this, the protocol currently defaults to RMI. The actual parameters for the WBEM Services CIMOM for the protocol are: CIMClient.CIM_RMI or CIMClient.CIM_XML.

Fetchlet Operations: The WBEM APIs are very flexible at allowing clients to traverse the class hierarchies that are defined and their associations. At this point in time, the options on accessing CIM data from an EMD are restricted to counting, getting the properties of classes, and CIMOM status. These are the more important operations that need to be performed for monitoring. As additional requirements come in, we can add new operations to support them if necessary. For the prototype, only the count operation has been implemented.

Authentication: Most CIMOMs provide APIs to support authentication through a user identity mechanism. The majority of the CIMOMs have not implemented the API, so this capability is really a no-op. In any case, we've supplied the capability in the fetchlet so that as CIMOM implementations catch up with the standard, we'll have the necessary support in place.

Examples

The Wbem fetchlet supports three basic operations. At this point, the fetchlet only handles one operation at a time, so you cannot mix count, status, and value operations within a single fetchlet call. The following example shows how to write the metadata for a COUNT operation:

<Metric NAME="Load" TYPE="TABLE">
    <Display>
       <Label NLSID="wbem_cimom_load">Load</Label>
     </Display>
     <TableDescriptor>
       <ColumnDescriptor NAME="Active Clients" TYPE="NUMBER" IS_KEY="FALSE">
           <Display>
             <Label NLSID="wbem_cimom_active_clients">Active CIMOM Clients</Label>
           </Display>
         </ColumnDescriptor>
     </TableDescriptor>  
    <QueryDescriptor FETCHLET_ID="Wbem">
       <Property NAME="username" SCOPE="GLOBAL">guest</Property>
       <Property NAME="password" SCOPE="GLOBAL">guest</Property>
       <Property NAME="CIMClassname" SCOPE="GLOBAL">EX_SFLProvider</Property>
       <Property NAME="operation" SCOPE="GLOBAL">COUNT</Property> 
    </QueryDescriptor>
  </Metric>

The FETCHLET_ID is identified as Wbem. Property names are passed to the fetchlet for the required parameters username, password, and CIMClassname. The operation is identified as COUNT.

The following example shows how to implement a Response Status metric to determine whether the CIMOM is running or not. It returns a value of 1 if the connection to the CIMOM is successful, otherwise 0.

<Metric NAME="Response" TYPE="TABLE">
     <Display>
       <Label NLSID="wbem_cimon_response">Response</Label>
     </Display>
     <TableDescriptor>
       <ColumnDescriptor NAME="Status" TYPE="NUMBER" IS_KEY="FALSE">
           <Display>
             <Label NLSID="wbem_cimom_response_status">Status</Label>
           </Display>
         </ColumnDescriptor> 
    </TableDescriptor> 
    <QueryDescriptor FETCHLET_ID="Wbem">
       <Property NAME="username" SCOPE="GLOBAL">guest</Property>
       <Property NAME="password" SCOPE="GLOBAL">guest</Property>
       <Property NAME="operation" SCOPE="GLOBAL">STATUS</Property>
     </QueryDescriptor>
   </Metric>

The default operation is the VALUES operation. It is used to fetch the values of a class that is defined in the CIMOM.

In the final example, the EX_Teacher class is accessed and fetches the name column. Name is the key of the class and of the new metric being defined, so the IS_KEY property is set to true. The CIM class properties will be mapped to the Enterprise Manager columns in the order that they are specified in the properties property. In this case, there is only 1 property - Name.

<Metric NAME="EX_Teacher" TYPE="TABLE">
     <Display>
       <Label NLSID="wbem_EX_Teacher">EX_Teacher Class</Label>
     </Display>
     <TableDescriptor>
       <ColumnDescriptor NAME="Name" TYPE="STRING" IS_KEY="TRUE">
           <Display>
             <Label NLSID="wbem_ex_teacher_name">Name</Label>
           </Display>
         </ColumnDescriptor>
     </TableDescriptor>
     <QueryDescriptor FETCHLET_ID="Wbem">
       <Property NAME="username" SCOPE="GLOBAL">guest</Property>
       <Property NAME="password" SCOPE="GLOBAL">guest</Property>
       <Property NAME="CIMClassname" SCOPE="GLOBAL">EX_Teacher</Property>
       <Property NAME="properties" SCOPE="GLOBAL">Name</Property>
     </QueryDescriptor>
   </Metric>

If multiple properties are fetched for a class, semi-colons should separate them. The properties should be provided in the order that the column descriptors are specified for the metric table definition.

<Metric NAME="EX_SFLProvider" TYPE="TABLE">
     <Display>
       <Label NLSID="wbem_EX_SFLProvider">EX_SFLProvider Class</Label>
     </Display>
     <TableDescriptor>
       <ColumnDescriptor NAME="Name" TYPE="STRING" IS_KEY="TRUE">
           <Display>
             <Label NLSID="wbem_ex_sfl_name">Name</Label>
           </Display>
         </ColumnDescriptor>
       <ColumnDescriptor NAME="Win" TYPE="NUMBER" IS_KEY="FALSE">
           <Display>
             <Label NLSID="wbem_ex_sfl_win">Win</Label>
           </Display>
         </ColumnDescriptor>
       <ColumnDescriptor NAME="Lost" TYPE="NUMBER" IS_KEY="FALSE">
           <Display>
             <Label NLSID="wbem_ex_sfl_lost">Lost</Label>
           </Display>
         </ColumnDescriptor>
     </TableDescriptor>
     <QueryDescriptor FETCHLET_ID="Wbem">
       <Property NAME="username" SCOPE="GLOBAL">guest</Property>
       <Property NAME="password" SCOPE="GLOBAL">guest</Property>
       <Property NAME="CIMClassname" SCOPE="GLOBAL">EX_SFLProvider</Property>
       <Property NAME="properties" SCOPE="GLOBAL">Name;Win;Lost</Property>
     </QueryDescriptor>
   </Metric>

JDBC Fetchlet

Call-level interfaces such as JDBC permit external access to SQL database manipulation and update commands. The Java Database Connectivity (JDBC) fetchlet allows you to execute common JDBC commands and obtain their response time for any type of database.

Input Parameters

jdbc:oracle:thin:@hostname:port

jdbc:mysql://hostname:port

OJMX/SOAP Fetchlet

The OJMX fetchlet communicates with the JMX agent on the managed J2EE server and performs the specified operations.

Input Parameters

Output

The OJMX fetchlet returns an object of type MetricResult that contains the information retrieved.