Purpose of this post is to share how you can debug ONS  - FAN events notification if you need to setup Weblogic / Data Source to be aware of the ONS  FAN events

ONS is a very useful option if you are using Oracle RAC architecture but the prior actual configuration of GridLink in the Weblogic make sure that your ONS working from the local and remote side because it may lead to an issue in the Weblogic and high CPU utilization

 Also, check Metalink for a bug related to the jdbc driver and ONS feature because it may require to patch Weblogic prior to implementing it.

 Please check the below reference link in order to get familiar with ONS and what has been changed in 18c compared to 12c

Oracle Clusterware 18c introduces the following features & enhancements compare to 12.2 ( as per the documentation )

 

• Cluster Domain architectures enhancements:
• Conversions, from Standalone Cluster to Member Cluster, and from one storage access configuration to another for Database Member Clusters.

• Oracle ACFS Remote Service: enable Oracle ACFS file systems on local Member Clusters utilizing remotely hosted ASM storage services on the Domain Services Cluster.
• Single network support for Application Member Clusters.

• Shared SCAN – the optional election of SCAN VIP’s on one cluster to act as the SCAN VIP’s for other clusters, reducing the number of IP addresses required by using a single SCAN VIP setup for multiple clusters.

• Node VIP-less Clusters – optionally, deploy new clusters without Node VIP’s, thus freeing up cluster resources and reducing the number of IP addresses required.
• Cross-Cluster Dependency Proxies – support for Clusterware resource dependencies that span clusters.

 

18c Changes 

https://www.oracle.com/technetwork/database/database-technologies/clusterware/overview/oracle-clusterware-18c-4392486.pdf

 

12c Fast Application Notification in 12c

http://www.oracle.com/technetwork/database/options/clustering/overview/fastapplicationnotification12c-2538999.pdf

Please check this document and follow the instruction in order to avoid an issue with configuration 

 

How to Monitor FAN events

https://blogs.oracle.com/weblogicserver/monitoring-fan-events

 

Testing WLS and ONS Configuration

https://blogs.oracle.com/weblogicserver/testing-wls-and-ons-configuration

 

 In case of GI upgrade from 12.1 to 12.2

Recommended upgrade of Oracle Notification Server (ONS) to 12.2 from an earlier version 12.1 Grid Infrastructure (Doc ID 2376962.1)

Testing:

LOCAL on database server

export CRS_HOME=/opt/app/18.0.0/grid export CLASSPATH="$CRS_HOME/jdbc/lib/ojdbc8.jar:$CRS_HOME/opmn/lib/ons.jar:." [grid@aixrac1 ~]$ /opt/app/18.0.0/grid/jdk/bin/javac ~/fanWatcher.java Note: /home/grid/fanWatcher.java uses or overrides a deprecated API. Note: Recompile with -Xlint:deprecation for details. [grid@aixrac1 ~]$   [grid@aixrac1 ~]$ /opt/app/18.0.0/grid/jdk/bin/javac -Xlint:deprecation -nowarn ~/fanWatcher.java home/grid/fanWatcher.java:76:  warning: [deprecation] Subscriber(String,String) in Subscriber has been deprecated s = new Subscriber(eventType, "");  // subscribe to service events only ^ /home/grid/fanWatcher.java:103:  warning: [deprecation] receive(boolean) in Subscriber has been deprecated Notification e = s.receive(true);  // blocking wait for notification receive ^ /home/grid/fanWatcher.java:108:  warning: [deprecation] print() in Notification has been deprecated e.print(); ^  3 warnings

 

Once you have compiled java you should see fanWatcher.class file which has been created

 

[grid@aixrac1 ~]$ ls -la | grep fanWatcher.class -rw-r--r--. 1 grid oinstall 5812 Aug 14 22:55 fanWatcher.class [grid@aixrac1 ~]$

 

Now you need to set ONS in debug mode /opt/app/18.0.0/grid/opmn/bin/onsctl set target=debug comp='ons[all,!workers,!servers]' /opt/app/18.0.0/grid/opmn/bin/onsctli reload /opt/app/18.0.0/grid/opmn/bin/onsctli query target=debug /opt/app/18.0.0/grid/opmn/bin/onsctli query No target for set or query request

 

If you see this message, try to stop and start ONS

 

[grid@aixrac1 ~]$ /opt/app/18.0.0/grid/opmn/bin/onsctl stop onsctl stop shutting down ons daemon ...   [grid@aixrac1 ~]$ /opt/app/18.0.0/grid/opmn/bin/onsctl start onsctl start: ons started    [grid@aixrac1 ~]$ /opt/app/18.0.0/grid/opmn/bin/onsctl debug HTTP/1.1 200 OK Connection: close Content-Type: text/html Response: == aixrac1:6200 31573 (2) 18/08/14 23:04:16 == Build: ONS_18.1.0.0.0_LINUX.X64_171214.2047 2017/12/15 6:9:53 UTC Home: /opt/app/18.0.0/grid

 

 

 You have now 7 connections

Now you can execute java program

[grid@aixrac1 ~]$ /opt/app/18.0.0/grid/jdk/bin/java -Doracle.ons.oraclehome=$CRS_HOME fanWatcher crs Subscribing to events of type:

 

if you have the above message, you can run again debug to check if there are new connection allocated

 

[grid@aixrac1 ~]$ /opt/app/18.0.0/grid/opmn/bin/onsctl debug

 

 

as you can see connecting now is 8 which means that you have a new client which is our program

Now you are ready to test ONS service by triggering event as stop or start instance or service which is part of the Cluster

Also, keep in mind that ONS should be tested from Application server - Weblogic in order to make sure that connection works as expected and you don’t any FW issue in regards to the port for ONS

In case of FW issue, you should add special rules in the FW

 

for Weblogic testing - please check document - Testing WLS and ONS Configuration from the above list, because there are some options which you may need to configure or to be aware like wallet and it really depends on what kind of architecture you have :)

 

You can follow the below documents because there are a different approach for one or other situation

https://docs.oracle.com/middleware/1212/wls/JDBCA/gridlink_datasources.htm#JDBCA461

http://www.oracle.com/us/products/middleware/cloud-app-foundation/weblogic/wls-db12c-integration-wp-2765860.pdf

 

 Supported AGL Data Source URL Formats

 

AGL data sources only support long format JDBC URLs. The supported long format pattern is:

jdbc:oracle:thin:@(DESCRIPTION=(ADDRESS_LIST=(ADDRESS=(PROTOCOL=TCP)(HOST=[SCAN_VIP])(PORT=[SCAN_PORT])))(CONNECT_DATA=(SERVICE_NAME=[SERVICE_NAME])))

Easy Connect (short) format URLs are not supported for AGL data sources. The following is an example of an Easy Connect URL pattern that is not supported for use with AGL data sources:

jdbc:oracle:thin:[SCAN_VIP]:[SCAN_PORT]/[SERVICE_NAME]

 

Recommendations for AGL Data Source URLs

The following section provides general recommendations when creating AGL data source URLs.

• Use a single DESCRIPTION. Avoid a DESCRIPTION_LIST to avoid connection delays.

• Use one ADDRESS_LIST for each RAC cluster or DataGuard database.

• Enter RETRY_COUNT, RETRY_DELAY, CONNECT_TIMEOUT at the DESCRIPTION level so that all ADDRESS_LIST entries use the same value.

• RETRY_DELAY specifies the delay, in seconds, between the connection retries. This attribute is new in the Oracle 12.1.0.2 release.

• RETRY_COUNT is used to specify the number of times an ADDRESS the list is traversed before the connection attempt is terminated. The default value is 0. When using SCAN listeners with FAILOVER=on, setting RETRY_COUNT to a value of 2 means that if you had 3 SCAN IP addresses, each would be traversed three times each, resulting in a total of nine connect attempts (3 * 3).

• Specify LOAD_BALANCE=on for each address list to balance the SCAN addresses.

• The service name should be a configured application service, not a PDB or administration service.

• CONNECT_TIMEOUT is used to specify the overall time used to complete the Oracle Net connect. Set CONNECT_TIMEOUT=90 or higher to prevent login storms. For JDBC driver 12.1.0.2 and earlier, CONNECT_TIMEOUT is also used for the TCP/IP connection timeout for each address in the URL. When considering TCP/IP connections, a shorter CONNECT_TIMEOUT is preferred though secondary to overall timeout requirements.

• Do not set the oracle.net.CONNECT_TIMEOUT driver property on the data source because it is overridden by the URL property.

 

 

If you have done some of the tests from the above process and you have issue, error or just feedback add in the below as comments