EBS 12.2 adcfgclone.pl on APPS Tier failed with ERROR: Script failed, exit code 255 when creating new wls domain.

In: 12.2 version, Rapidclone

When attempting to clone in APPS Tier, the following error occurs.

ERROR:-

Creating new WLS domain.

Running /data02/app/xxx/fs1/FMW_Home/oracle_common/bin/pasteConfig.sh -javaHome /data02/app/xxx/fs1/EBSapps/comn/clone/FMW/t2pjdk -al /data02/app/xxx/fs1/EBSapps/comn/clone/FMW/WLS/EBSdomain.jar -tdl /data02/app/xxx/fs1/FMW_Home/user_projects/domains/EBS_domain_xxx -tmw /data02/app/xxx/fs1/FMW_Home -mpl /data02/app/xxx/fs1/EBSapps/comn/clone/FMW/WLS/plan/moveplan.xml -ldl /data02/app/xxx/fs1/inst/apps/xxx_xxx/admin/log/clone/wlsT2PApply -silent true -domainAdminPassword /data02/app/xxx/fs1/EBSapps/comn/clone/FMW/tempinfo.txt

Script Executed in 4712761 milliseconds, returning status 255

Script failed, exit code 255

 

CLONE2021-03-04_xx-xx-xxPM.error

SEVERE : Feb 20, 2015 3:48:12 PM - ERROR - CLONE-20362  Connection to AdminServer failed.

SEVERE : Feb 20, 2015 3:48:12 PM - CAUSE - CLONE-20362  AdminServer start failed. Make sure that AdminServer parameters specified in the move plan are correct.

SEVERE : Feb 20, 2015 3:48:12 PM - ACTION - CLONE-20362  Check the AdminServer logs.

oracle.as.t2p.exceptions.FMWT2PPasteConfigException: PasteConfig failed. Make sure that the move plan and the values specified in moveplan are correct.

at oracle.as.clone.cloner.component.J2EEComponentApplyCloner.doClone(J2EEComponentApplyCloner.java:262)

at oracle.as.clone.cloner.Cloner.doFinalClone(Cloner.java:61)

at oracle.as.clone.request.ApplyCloneRequest.applyArchive(ApplyCloneRequest.java:188)

at oracle.as.clone.request.ApplyCloneRequest._clone(ApplyCloneRequest.java:69)

at oracle.as.clone.process.CloningExecutionProcess.execute(CloningExecutionProcess.java:131)

at oracle.as.clone.process.CloningExecutionProcess.execute(CloningExecutionProcess.java:114)

at oracle.as.clone.client.CloningClient.executeT2PCommand(CloningClient.java:238)

at oracle.as.clone.client.CloningClient.main(CloningClient.java:115)

SEVERE : Feb 20, 2015 3:48:12 PM - ERROR - CLONE-20237   Restoring the sourceid "J2EECOMPONENT@EBS_domain" has failed.

SEVERE : Feb 20, 2015 3:48:12 PM - CAUSE - CLONE-20237   An internal operation failed.

SEVERE : Feb 20, 2015 3:48:12 PM - ACTION - CLONE-20237   Check the clone log and error file for more details.

 

CLONE2021-03-04_xx-xx-xxPM.log

FINE : Feb 20, 2015 3:48:12 PM - [GenericPasteConfigSteps:executeOnlineScripts] AdminServer is down. Cannot execute online config changes.

FINE : Feb 20, 2015 3:48:12 PM - CLONE-22264   Time taken to restore sourceid "J2EECOMPONENT@EBS_domain_gouf" is "4,465,671" milliseconds.

FINE : Feb 20, 2015 3:48:12 PM - [ApplyCloneRequest:applyArchive] One cloner object was not executed properly, so rest were not executed.

INFO : Feb 20, 2015 3:48:12 PM - CLONE-21037   Time taken to restore all cloners from archive was "4,465" seconds.

INFO : Feb 20, 2015 3:48:12 PM - CLONE-21006   Total time taken by T2P process was 4,465 seconds.

CHANGES:-

Run adcfgclone on AP Tier.

CAUSE:-

This issue is reported by Bug 18752606 - 12.2.3 IS FAILING CREATING NEW WLS DOMAIN but No root cause was found.

SOLUTION:-

==>Back up the inventory.xml file to inventory.xml.bk

==>Remove (rm -rf) the inst directory on the target run file system

==>Remove (rm -rf) the FMW_Home directory on the target run file system

==>Ensure that the above Oracle Homes are unregistered from the inventory "/xxx/APPS/oraInventory/ContentsXML/inventory.xml" : /APPS/fs1/FMW_Home/Oracle_EBS-app1 , /APPS/fs1/FMW_Home/webtier                                                                                                                         /APPS/fs1/FMW_Home/oracle_common

 ==>From a brand new session(terminal) where nothing has been sourced, navigate to your COMMON_TOP/clone/bin directory

==>Again make sure there are no EBS specific env variables sourced on this session(terminal) and run perl adcfgclone.pl appsTier

FS_CLONE Fails With java.lang.Exception: Unable to get free port for server.

After applying R12.AD.C.Delta.9 and R12.TXK.C.Delta.9 in our EBS instance, Below error occured while doing fs_clone.

$ adop phase=fs_clone fails

adop.log shows:

[EVENT] Running fs_clone on admin node: [<node_name>].

..

[ERROR] Node: "<node_name>" Status: "failed"

[ERROR] adop phase=fs_clone failed on Node: "<node_name>"

[UNEXPECTED]Error occurred running "perl /data02/app/applmgr/122/instance/fs1/EBSapps/appl/ad/12.0.0/patch/115/bin/txkADOPPreparePhaseSynchronize.pl -contextfile=/data02/app/applmgr/122/instance/fs1/inst/apps/instance_<node_name>/appl/admin/instance_<node_name>1.xml -patchcontextfile=/data02/app/applmgr/122/instance/fs2/inst/apps/instance_<node_name>/appl/admin/instance_<node_name>.xml -promptmsg=hide -console=off -mode=create -sync_mode=copy -sessionid=42 -timestamp=20170622_155032 -outdir=/data02/app/applmgr/122/instance/fs_ne/EBSapps/log/adop/42/20170622_155032/fs_clone/<node_name>"

[UNEXPECTED]occurred during CLONE Patch File System from Run File System, running command: "perl /data02/app/applmgr/122/instance/fs1/EBSapps/appl/ad/12.0.0/patch/115/bin/txkADOPPreparePhaseSynchronize.pl -contextfile=/data02/app/applmgr/122/instance/fs1/inst/apps/instance_<node_name>/appl/admin/instance_<node_name>.xml -patchcontextfile=/data02/app/applmgr/122/instance/fs2/inst/apps/instance_<node_name>/appl/admin/instance_<node_name>.xml -promptmsg=hide -console=off -mode=create -sync_mode=copy -sessionid=42 -timestamp=20170622_155032 -outdir=/data02/app/applmgr/122/instance/fs_ne/EBSapps/log/adop/42/20170622_155032/fs_clone/<node_name>".

fs_clone/node-prod-01/TXK_SYNC_create/txkADOPPreparePhaseSynchronize.log shows:

======================================

Inside copyCloneLogsToFSNE()...

======================================

Creating the directory: /data02/app/applmgr/122/instance/fs_ne/EBSapps/log/adop/42/20170622_155032/fs_clone/<node_name>/TXK_SYNC_create/fsclone_apply

Copying the directory

---------------------

SOURCE : /data02/app/applmgr/122/instance/fs1/inst/apps/instance_<node_name>/admin/log/clone

TARGET : /data02/app/applmgr/122/instance/fs_ne/EBSapps/log/adop/42/20170622_155032/fs_clone/<node_name>/TXK_SYNC_create/fsclone_apply

/data02/app/applmgr/122/instance/fs1/EBSapps/comn/adopclone_<node_name>/bin/adclone.pl did not go through successfully.

LOG DIRECTORY: /data02/app/applmgr/122/instance/fs_ne/EBSapps/log/adop/42/20170622_155032/fs_clone/<node_name>/TXK_SYNC_create/fsclone_apply.

*******FATAL ERROR*******

PROGRAM : (/data02/app/applmgr/122/instance/fs1/EBSapps/appl/ad/12.0.0/patch/115/bin/txkADOPPreparePhaseSynchronize.pl)

TIME : Thu Jun 22 16:30:37 2017

FUNCTION: main::runFSCloneApply [ Level 1 ]

ERRORMSG: /data02/app/applmgr/122/instance/fs1/EBSapps/comn/adopclone_<node_name>/bin/adclone.pl did not go through successfully.

/fs_clone/<node_name>/TXK_SYNC_create/fsclone_apply/FSCloneApplyAppsTier_06221627.log shows:

END: Created backup...

Backup applications domain directory if exists

Directory /data02/app/applmgr/122/<sid>/fs2/FMW_Home/user_projects/applications/EBS_domain_<sid> does not exist.

START: Creating new WLS domain.

Running /data02/app/applmgr/122/<sid>/fs2/FMW_Home/oracle_common/bin/pasteConfig.sh -javaHome /data02/app/applmgr/122/<sid>/fs2/EBSapps/comn/util/jdk -al /data02/app/applmgr/122/<sid>/fs1/EBSapps/comn/adopclone_<node_name>/FMW/WLS/EBSdomain.jar -tdl /data02/app/applmgr/122/<sid>/fs2/FMW_Home/user_projects/domains/EBS_domain_<sid> -tmw /data02/app/applmgr/122/<sid>/fs2/FMW_Home -mpl /data02/app/applmgr/122/<sid>/fs1/EBSapps/comn/adopclone_<node_name>/FMW/WLS/plan/moveplan.xml -ldl /data02/app/applmgr/122/<sid>/fs1/inst/apps/<sid>_<node_name>1/admin/log/clone/wlsT2PApply -silent true -debug true -domainAdminPassword /data02/app/applmgr/122/<sid>/fs1/EBSapps/comn/adopclone_<node_name>/FMW/tempinfo.txt

Script Executed in 169799 milliseconds, returning status 255

ERROR: Script failed, exit code 255

START: Inside exitClone....

Updating status INCOMPLETE for ApplywlsTechStack

START: Updating status INCOMPLETE for action ApplywlsTechStack

END: Updated status INCOMPLETE for action ApplywlsTechStack

/fs_clone/<node_name>/TXK_SYNC_create/fsclone_apply/wlsT2PApply/CLONE2017-06-22_16-27-46_1106077216.log shows:

FINE : Jun 22, 2017 16:30:34 - [PortUtil:isPortAvailable] Port 5047 reserved: From the port range 5001-5050 by server oafm_server13

FINE : Jun 22, 2017 16:30:34 - [PortUtil:isPortAvailable] Port 5048 reserved: From the port range 5001-5050 by server forms-c4ws_server13

FINE : Jun 22, 2017 16:30:34 - [PortUtil:isPortAvailable] Port 5049 reserved: From the port range 5001-5050 by server oacore_server14

FINE : Jun 22, 2017 16:30:34 - [PortUtil:isPortAvailable] Port 5050 reserved: From the port range 5001-5050 by server forms_server14

FINE : Jun 22, 2017 16:30:34 - [J2EEGenericValidationUtil:getTemporaryPortFromPortRange] Unable to get free port for server oafm_server14 from port range 5001-5050

FINE : Jun 22, 2017 16:30:34 - [CloneLogger:unSetSystemOutErrToCloneLogger] Invoking unsetSystemOutErrToCloneLogger 1

FINE : Jun 22, 2017 16:30:34 - [CloneLogger:unSetSystemOutErrToCloneLogger] Invoking unsetSystemOutErrToCloneLogger 2

FINE : Jun 22, 2017 16:30:34 - CLONE-22264 Time taken to restore sourceid "J2EECOMPONENT@EBS_domain_ebsprod" is "141,611" milliseconds.

FINE : Jun 22, 2017 16:30:34 - [ApplyCloneRequest:applyArchive] One cloner object was not executed properly, so rest were not executed.

INFO : Jun 22, 2017 16:30:34 - CLONE-21037 Time taken to restore all cloners from archive was "141" seconds.

INFO : Jun 22, 2017 16:30:34 - CLONE-21006 Total time taken by T2P process was 141 seconds.

INFO : Jun 22, 2017 16:30:34 - CLONE-21276

---------------------------------------------------

T2P Summary Begin

---------------------------------------------------

Error Message :1

Jun 22, 2017 16:30:34 - SEVERE - CLONE-20937 "pasteConfig" operation of J2EE domain failed. Check clone log and error files for more details.

Error Message :2

Jun 22, 2017 16:30:34 - ERROR - CLONE-20237 Restoring the sourceid "J2EECOMPONENT@EBS_domain_ebsprod" has failed.

Jun 22, 2017 16:30:34 - CAUSE - CLONE-20237 An internal operation failed.

Jun 22, 2017 16:30:34 - ACTION - CLONE-20237 Check the clone log for more details.

---------------------------------------------------

T2P Summary End

---------------------------------------------------

/fs_clone/<node_name>/TXK_SYNC_create/fsclone_apply/wlsT2PApply/CLONE2017-06-22_16-27-46_1106077216.error shows:

java.lang.Exception: Unable to get free port for server oafm_server14 from port range 5001-5050

at oracle.as.clone.cloner.component.j2ee.util.J2EEGenericValidationUtil.getTemporaryPortFromPortRange(J2EEGenericValidationUtil.java:443)

at oracle.as.clone.cloner.component.j2ee.util.J2EEGenericValidationUtil.validateAndGetServerPort(J2EEGenericValidationUtil.java:322)

at oracle.as.clone.cloner.component.j2ee.util.J2EEGenericValidationUtil.validateServerConfig(J2EEGenericValidationUtil.java:171)

at oracle.as.clone.cloner.component.j2ee.config.GenericPasteConfigSteps.getServerConfig(GenericPasteConfigSteps.java:922)

at oracle.as.clone.cloner.component.j2ee.config.GenericPasteConfigSteps.executeGenericPasteConfig(GenericPasteConfigSteps.java:385)

at oracle.as.clone.cloner.component.J2EEComponentApplyCloner.doClone(J2EEComponentApplyCloner.java:259)

at oracle.as.clone.cloner.Cloner.doFinalClone(Cloner.java:63)

at oracle.as.clone.request.ApplyCloneRequest.applyArchive(ApplyCloneRequest.java:198)

at oracle.as.clone.request.ApplyCloneRequest._clone(ApplyCloneRequest.java:77)

at oracle.as.clone.process.CloningExecutionProcess.execute(CloningExecutionProcess.java:131)

at oracle.as.clone.process.CloningExecutionProcess.execute(CloningExecutionProcess.java:114)

at oracle.as.clone.client.CloningClient.executeT2PCommand(CloningClient.java:236)

at oracle.as.clone.client.CloningClient.main(CloningClient.java:124)

 

CHANGES:-

Both AD/TXK Delta 9+ are applied as per the Document: Oracle E-Business Suite Applications DBA and Technology Stack Release Notes for R12.AD.C.Delta.9 and R12.TXK.C.Delta.9 (Doc ID 2233485.1). This patch introduces context variable s_wls_t2p_temp_port_range which defaults to 5001-5050.

ROOT CAUSE:-

The context variable s_wls_t2p_temp_port_range which is too small a range if instance has moderate to large number of managed servers.

You can check the value of this setting:

SELECT extractValue(XMLType(TEXT),'//host[@oa_var="s_hostname"]') s_hostname,extractValue(XMLType(TEXT),'//WLST2PTEMP_PORT_RANGE') WLST2PTEMP_PORT_RANGE from fnd_oam_context_files where name not in ('TEMPLATE','METADATA') and (status is null or status !='H') and EXTRACTVALUE(XMLType(TEXT),'//file_edition_type') in ('patch') and CTX_TYPE = 'A' ORDER by s_hostname;

 

SOLUTION:-

Specify a larger range for s_wls_t2p_temp_port_range in apps tier context files and ensure autoconfig completes without issue.

WebLogic Managed Server Does Not Start with error "The server xxxx is in an incompatible state".

If managed server was incorrectly stopped unexpectedly, trying to start it using the node manager could lead to the following error:

You can not start the managed server using the admin console because of this.

Messages:-

Message icon - Warning The server xxxx  is in an incompatible state.

Message icon - Warning All of the servers selected are currently in a state which is incompatible with this operation or are not associated with a running Node Manager or you are not authorized to perform the action requested. No action will be performed.

"The server xxxx is in an incompatible state"

The likely cause of this error is that the status of of WL server reflected by node manager, is not correct.

To solve the issue you need to follow the next steps:

Check that the instance is not running using ps -ef. If it is, use kill -9 <pid> to stop the running instance.

Remove the state file:

If the managed server name is “Server-Name”, then go to the managed server machine, delete the following:

$DOMAIN_HOME/servers/<Server-Name>/data/nodemanage/<Server-Name>.state

<Server-Name>.state is the node manager file containing instance information and status.

After deleting this file, the managed server can be remotely started from the Admin Console.

WebLogic Failed to Start with BEA-160197 Error Weblogic.Descriptor.DescriptorException: Unmarshaller Failed

In a WebLogic Server 10.3.6 installation,  server instances failed to start with the following error:

WebLogic Failed to Start with BEA-160197 Error Weblogic.Descriptor.DescriptorException: Unmarshaller Failed

<Error> <J2EE> <BEA-160197> <Unable to load descriptor java.net.URLClassLoader@6cc57fe2/null of module null. The error is weblogic.descriptor.DescriptorException: Unmarshaller failed

at weblogic.descriptor.internal.MarshallerFactory$1.createDescriptor(MarshallerFactory.java:161)

at weblogic.descriptor.BasicDescriptorManager.createDescriptor(BasicDescriptorManager.java:323)

at weblogic.application.descriptor.AbstractDescriptorLoader2.getDescriptorBeanFromReader(AbstractDescriptorLoader2.java:788)

at weblogic.application.descriptor.AbstractDescriptorLoader2.createDescriptorBean(AbstractDescriptorLoader2.java:409)

at weblogic.application.descriptor.AbstractDescriptorLoader2.loadDescriptorBeanWithoutPlan(AbstractDescriptorLoader2.java:759)

at weblogic.application.descriptor.AbstractDescriptorLoader2.loadDescriptorBean(AbstractDescriptorLoader2.java:768)

at weblogic.jdbc.module.JDBCDeploymentHelper.getJDBCDataSourceBean(JDBCDeploymentHelper.java:186)

at weblogic.jdbc.module.JDBCDeploymentHelper.createJDBCDataSourceDescriptor(JDBCDeploymentHelper.java:51)

at weblogic.management.mbeans.custom.JDBCSystemResource.loadDescriptor(JDBCSystemResource.java:60)

at weblogic.management.mbeans.custom.ConfigurationExtension.getExtensionRoot(ConfigurationExtension.java:178)

at weblogic.management.mbeans.custom.JDBCSystemResource.getJDBCResource(JDBCSystemResource.java:45)

at weblogic.management.mbeans.custom.JDBCSystemResource._postCreate(JDBCSystemResource.java:50)

at weblogic.management.configuration.JDBCSystemResourceMBeanImpl._postCreate(JDBCSystemResourceMBeanImpl.java:355)

at weblogic.descriptor.internal.AbstractDescriptorBean._postCreate(AbstractDescriptorBean.java:670)

at weblogic.management.configuration.DomainMBeanImpl.setJDBCSystemResources(DomainMBeanImpl.java:11820)

at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)

at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

at java.lang.reflect.Method.invoke(Method.java:606)

at com.bea.staxb.runtime.internal.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:48)

at com.bea.staxb.runtime.internal.RuntimeBindingType$BeanRuntimeProperty.setValue(RuntimeBindingType.java:539)

at com.bea.staxb.runtime.internal.AttributeRuntimeBindingType$QNameRuntimeProperty.fillCollection(AttributeRuntimeBindingType.java:381)

at com.bea.staxb.runtime.internal.MultiIntermediary.getFinalValue(MultiIntermediary.java:52)

at com.bea.staxb.runtime.internal.AttributeRuntimeBindingType.getFinalObjectFromIntermediary(AttributeRuntimeBindingType.java:140)

at com.bea.staxb.runtime.internal.UnmarshalResult.unmarshalBindingType(UnmarshalResult.java:200)

at com.bea.staxb.runtime.internal.UnmarshalResult.unmarshalDocument(UnmarshalResult.java:169)

at com.bea.staxb.runtime.internal.UnmarshallerImpl.unmarshal(UnmarshallerImpl.java:65)

at weblogic.descriptor.internal.MarshallerFactory$1.createDescriptor(MarshallerFactory.java:150)

at weblogic.descriptor.BasicDescriptorManager.createDescriptor(BasicDescriptorManager.java:323)

at weblogic.management.provider.internal.DescriptorManagerHelper.loadDescriptor(DescriptorManagerHelper.java:68)

at weblogic.management.provider.internal.RuntimeAccessImpl$IOHelperImpl.parseXML(RuntimeAccessImpl.java:690)

at weblogic.management.provider.internal.RuntimeAccessImpl.parseNewStyleConfig(RuntimeAccessImpl.java:270)

at weblogic.management.provider.internal.RuntimeAccessImpl.<init>(RuntimeAccessImpl.java:115)

at weblogic.management.provider.internal.RuntimeAccessService.start(RuntimeAccessService.java:41)

at weblogic.t3.srvr.ServerServicesManager.startService(ServerServicesManager.java:461)

at weblogic.t3.srvr.ServerServicesManager.startInStandbyState(ServerServicesManager.java:166)

at weblogic.t3.srvr.T3Srvr.initializeStandby(T3Srvr.java:881)

at weblogic.t3.srvr.T3Srvr.startup(T3Srvr.java:568)

at weblogic.t3.srvr.T3Srvr.run(T3Srvr.java:469)

at weblogic.Server.main(Server.java:71)

Caused by: com.bea.xml.XmlException: java.lang.IllegalArgumentException: 0 is not a legal value for MaxCapacity. It does not fall in the range [1, 2147483647]

at com.bea.staxb.runtime.internal.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:54)

at com.bea.staxb.runtime.internal.RuntimeBindingType$BeanRuntimeProperty.setValue(RuntimeBindingType.java:539)

at com.bea.staxb.runtime.internal.ByNameRuntimeBindingType$ElementQNameProperty.fill(ByNameRuntimeBindingType.java:351)

at com.bea.staxb.runtime.internal.LiteralUnmarshalResult.extractAndFillElementProp(LiteralUnmarshalResult.java:159)

at com.bea.staxb.runtime.internal.ByNameUnmarshaller.deserializeContents(ByNameUnmarshaller.java:51)

at com.bea.staxb.runtime.internal.AttributeUnmarshaller.unmarshalIntoIntermediary(AttributeUnmarshaller.java:47)

at com.bea.staxb.runtime.internal.LiteralUnmarshalResult.unmarshalElementProperty(LiteralUnmarshalResult.java:184)

at com.bea.staxb.runtime.internal.LiteralUnmarshalResult.extractAndFillElementProp(LiteralUnmarshalResult.java:156)

at com.bea.staxb.runtime.internal.ByNameUnmarshaller.deserializeContents(ByNameUnmarshaller.java:51)

at com.bea.staxb.runtime.internal.AttributeUnmarshaller.unmarshalIntoIntermediary(AttributeUnmarshaller.java:47)

at com.bea.staxb.runtime.internal.UnmarshalResult.unmarshalBindingType(UnmarshalResult.java:199)

at com.bea.staxb.runtime.internal.UnmarshalResult.unmarshalDocument(UnmarshalResult.java:169)

at com.bea.staxb.runtime.internal.UnmarshallerImpl.unmarshal(UnmarshallerImpl.java:65)

at weblogic.descriptor.internal.MarshallerFactory$1.createDescriptor(MarshallerFactory.java:150)

... 39 more

Caused by: java.lang.IllegalArgumentException: 0 is not a legal value for MaxCapacity. It does not fall in the range [1, 2147483647]

at weblogic.descriptor.beangen.LegalChecks.checkInRange(LegalChecks.java:80)

at weblogic.j2ee.descriptor.wl.JDBCConnectionPoolParamsBeanImpl.setMaxCapacity(JDBCConnectionPoolParamsBeanImpl.java:173)

at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)

at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

at java.lang.reflect.Method.invoke(Method.java:606)

at com.bea.staxb.runtime.internal.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:48)

... 52 more

.>

<Critical> <WebLogicServer> <BEA-000362> <Server failed. Reason: [Management:141266]Parsing Failure in config.xml: weblogic.management.ManagementRuntimeException: weblogic.application.ModuleException: >

<Notice> <WebLogicServer> <BEA-000365> <Server state changed to FAILED>

<Error> <WebLogicServer> <BEA-000383> <A critical service failed. The server will shut itself down>

<Notice> <WebLogicServer> <BEA-000365> <Server state changed to FORCE_SHUTTING_DOWN>

Maximum Capacity in one of the JDBC connection pools for the SOADataSource was set to zero which caused the error. Verifying the JDBC configuration files, we found that SOADataSource-jdbc.xml has max-capacity set to 0 as shown below:

<jdbc-connection-pool-params>

<initial-capacity>0</initial-capacity>

<max-capacity>0</max-capacity>          

<capacity-increment>1</capacity-increment>

<connection-creation-retry-frequency-seconds>10</connection-creation-retry-frequency-seconds>

<test-frequency-seconds>300</test-frequency-seconds>

<test-connections-on-reserve>true</test-connections-on-reserve>

<test-table-name>SQL SELECT 1 FROM DUAL</test-table-name>

<remove-infected-connections>false</remove-infected-connections>

<seconds-to-trust-an-idle-pool-connection>0</seconds-to-trust-an-idle-pool-connection>

</jdbc-connection-pool-params>

To solve the issue, follow the below steps :-

1.  Take a backup of '<Domain_Home>/config/jdbc/SOADataSource-jdbc.xml' (This is the JDBC configuration file that had MaxCapacity value as 0)

2.  Edit SOADataSource-jdbc.xml to make max-capacity in the file to 20 and save.  (The value of MaxCapacity should be set to value equal to or greater than 1, this value is specific to customer environment.)

3.  Restart WebLogic Server to see it come up RUNNING without errors.

How To Recreate the Oracle E-Business Suite (EBS) WLS (Weblogic Server) Domain

To recreate EBS WLS domain, please do the following after taking a full backup of your environment:

1. In the Database Tier exactly do the following:-

     ==> Log into the database tier.

     ==> Source the environment.

     ==> Ensure that database and listener are up and running.

2. In Applications Tier  exactly do the following:-

    ==> Source the RUN file system environment.

    ==> Stop all middle tier services using $ADMIN_SCRIPTS_HOME/adstpall.sh.

    ==> Ensure that no process is running from RUN file system.

    ==> Take a backup of the context file.  Update s_adminservertimeout context variable value to 4000 and run AutoConfig.

   

Execute below command:

 $ $FND_TOP/bin/txkrun.pl -script=ChkEBSDependecies -server=ALL_SERVER

3. Recreate the domain by following below steps:  

        $ cd $FND_TOP/patch/115/bin

        $ perl txkEBSDomainConfig.pl

   ==> Start the services and check the WLS domain is fine. For example, you might want to start the  WLS console and check all the expected managed servers are up and running.

Note: When running txkEBSDomainConfig.pl you will be prompted "Enter the server start mode for the domain [DEFAULT - prod]:".Choose the default value of prod. Oracle Web Logic Server can be run in two different modes, either prod or dev, and the recommended option is to choose prod as it is safer. The dev mode allows to deploy unsigned java classes but is quicker than prod.

Cause: cmlmcm() failed due to ORA-20100: ORA-20100: Temporary file creation for FND_FILE failed.

Cause: cmlmcm() failed due to ORA-20100: ORA-20100: Temporary file creation for FND_FILE failed.

Directory &FILE_DIR is invalid, contact your system administrator.

This error was being thrown when one instance was trying to create .tmp file

To resolve this issue,shutdown both the apps and db services of one instance and we have to set utl_file_dir properly.

Ensure that the directories specified in the utl_file_dir parameter in the V$PARAMETER table are writeable by the Oracle owner.

The profile option CSI: Log File Path must be the first entry in the utl_file_dir parameter.

Make sure that $APPLPTMP and $APPLTMP are set to this same value.

Verify that the profile 'CSI: Log File Path' has a valid directory path (NO trailing slash) and that it is defined in the 'utl_file_dir' parameter

set '/usr/tmp' and changed the ownership of this dir to user owner of this instance

Logon to database as sysdba

Create pfile from spfile

                                           

SQL> alter system set utl_file_dir='/usr/tmp','/data01/orahome/appsutil/outbound/DEV_xxxapps','/usr/tmp' scope=spfile;

System altered.

modified UTL_FILE_DIR parameter's first entry from '/usr/tmp' ;

Created spfile from pfile

Brought up the db and listener

Now modified the $APPLPTMP variable in TEST_oratest.xml file as  '/usr/tmp'

Run the autoconfig on appsTier.

Brought up the apps services

Retested the issue and it was resolved