Sajid's Oracle DBA Blog: June 2021

How to execute empty patching cycle, without applying any patch.

On: 12.2 version, we will execute the emptly patching cycle.

Can a cutover be run, from fs1 to fs2 outside of patching cycle? For testing purposes, without applying any patch, can a cutover to switch filesystem from fs1 to fs2?

How to execute empty patching cycle, without applying any patch?

SOLUTION:-

Earlier prepare phase was failed. To resolve it need to be truncate the

below tables.

truncate table applsys.fnd_nodes;

truncate table applsys.fnd_oam_context_files;

truncate table applsys.adop_valid_nodes;

If on AD/TXK 8 or higher:

truncate table applsys.ad_nodes_config_status;

Run autoconfig on DB and appsTier.

The steps to run an empty patch cycle is:

Quick Version:

$ cd <EBS_ROOT>

$ source EBSapps.env run

$ adop phase=prepare,finalize,cutover

$ source EBSapps.env run

$ adop phase=cleanup

Complete Version:

$ cd <EBS_ROOT>

$ source EBSapps.env run

$ adop phase=prepare

$ adop phase=actualize_all

$ adop phase=finalize finalize_mode=full

$ adop phase=cutover

$ source EBSapps.env run

$ adop phase=cleanup cleanup_mode=full

How to run autoconfig on patch file system in r12.2

In order to run autoconfig on patch file system successfully.We have to disable trigger ebs_logon on run file system.

[applmgr@svrapps scripts]$ sqlplus system

SQL*Plus: Release 10.1.0.5.0 - Production on Mon Jun 28 12:17:06 2021

Enter password:

Connected to:

Oracle Database 11g Enterprise Edition Release 11.2.0.4.0 - 64bit Production

With the Partitioning, OLAP, Data Mining and Real Application Testing options

SQL> show user

USER is "SYSTEM"

SQL> alter trigger ebs_logon disable;

Trigger altered.

Source the patch file sytem environment.

. /oracle/EBSFS/fs2/EBSapps/appl/APPSTEST_svrapps.env patch

[applmgr@svrapps scripts]$echo $FILE_EDITION

patch

[applmgr@svrapps scripts]$sh adautocfg.sh

Enter the APPS user password:

The log file for this session is located at: /oracle/EBSFS/fs2/inst/apps/TEST_svrapps/admin/log/06281218/adconfig.log

wlsDomainName: EBS_domain

WLS Domain Name is VALID.

AutoConfig is configuring the Applications environment...

AutoConfig will consider the custom templates if present.

Using CONFIG_HOME location : /oracle/EBSFS/fs2/inst/apps/TEST_svrapps

Classpath : /oracle/EBSFS/fs2/FMW_Home/Oracle_EBS-app1/shared-libs/ebs-appsborg/WEB-INF/lib/ebsAppsborgManifest.jar:/oracle/EBSFS/fs2/EBSapps/comn/java/classes

Using Context file : /oracle/EBSFS/fs2/inst/apps/TEST_svrapps/appl/admin/TEST_svrapps.xml

Context Value Management will now update the Context file

Updating Context file...COMPLETED

Attempting upload of Context file and templates to database...COMPLETED

Configuring templates from all of the product tops...

Configuring AD_TOP........COMPLETED

Configuring FND_TOP.......COMPLETED

Configuring ICX_TOP.......COMPLETED

Configuring MSC_TOP.......COMPLETED

Configuring IEO_TOP.......COMPLETED

Configuring BIS_TOP.......COMPLETED

Configuring CZ_TOP........COMPLETED

Configuring SHT_TOP.......COMPLETED

Configuring AMS_TOP.......COMPLETED

Configuring CCT_TOP.......COMPLETED

Configuring WSH_TOP.......COMPLETED

Configuring CLN_TOP.......COMPLETED

Configuring OKE_TOP.......COMPLETED

Configuring OKL_TOP.......COMPLETED

Configuring OKS_TOP.......COMPLETED

Configuring CSF_TOP.......COMPLETED

Configuring IBY_TOP.......COMPLETED

Configuring JTF_TOP.......COMPLETED

Configuring MWA_TOP.......COMPLETED

Configuring CN_TOP........COMPLETED

Configuring CSI_TOP.......COMPLETED

Configuring WIP_TOP.......COMPLETED

Configuring CSE_TOP.......COMPLETED

Configuring EAM_TOP.......COMPLETED

Configuring GMF_TOP.......COMPLETED

Configuring PON_TOP.......COMPLETED

Configuring FTE_TOP.......COMPLETED

Configuring ONT_TOP.......COMPLETED

Configuring AR_TOP........COMPLETED

Configuring AHL_TOP.......COMPLETED

Configuring IES_TOP.......COMPLETED

Configuring OZF_TOP.......COMPLETED

Configuring CSD_TOP.......COMPLETED

Configuring IGC_TOP.......COMPLETED

AutoConfig completed successfully.

Make sure autoconfig completed successfully.

Now enable the login trigger on run filesystem.

SQL> alter trigger ebs_logon enable;

adop prepare phase failing with the error ![UNEXPECTED]Nodes are listed in the ADOP_VALID_NODES table but not in the FND_NODES table.

In this post we will fix the error '[UNEXPECTED]Nodes are listed in the ADOP_VALID_NODES table but not in the FND_NODES table'.

[applmgr@sa**apps ~]$ adop phase=prepare

Enter the APPS password:

Enter the SYSTEM password:

Enter the WLSADMIN password:

Validating credentials.

Initializing.

Run Edition context : /oracle/EBSFS/fs1/inst/apps/UAT_sam**apps/appl/admin/UAT_sam**apps.xml

Patch edition context: /oracle/EBSFS/fs2/inst/apps/UAT_sam**apps/appl/admin/UAT_sam***apps.xml

Patch file system free space: 482.94 GB

Validating system setup.

[UNEXPECTED]Nodes "erpbackup" are listed in the ADOP_VALID_NODES table but not in the FND_NODES table.

[UNEXPECTED]To correct this, run AutoConfig on nodes "erpbackup"

[UNEXPECTED]Error determining whether this is a multi-node instance

[STATEMENT] Please run adopscanlog utility, using the command

"adopscanlog -latest=yes"

to get the list of the log files along with snippet of the error message corresponding to each log file.

adop exiting with status = 2 (Fail)

Solution:-

Please ensure you have taken a valid backup of your DB & APPS Tier before doing below action plan.

1. Backup the fnd_oam_context_files, fnd_nodes, and adop_valid_nodes tables in the EBS env:

sqlplus apps/apps

create table fnd_oam_context_files_bkp1 as select * from fnd_oam_context_files;

create table fnd_nodes_bk1 as select * from fnd_nodes;

create table adop_valid_nodes_bk1 as select * from adop_valid_nodes;

2. Truncate the following tables:

truncate table applsys.fnd_oam_context_files;

truncate table applsys.fnd_nodes;

truncate table applsys.adop_valid_nodes;

3. Run AutoConfig on the DB tier

4. Run Autoconfig on the run file system.

5. Run Autoconfig on the patch file system

Before running Autoconfig on the patch file system the ebs_login trigger MUST be disabled

After the successful completion of Autoconfig the ebs_login trigger MUST be re-enabled.

This needs to be done as the SYSTEM schema user(SYS).

a. Disable the ebs_login trigger using the following SQL.

SQL> conn system

Enter password:

Connected.

SQL> show user

USER is "SYSTEM"

SQL>

SQL> alter trigger ebs_logon disable;

Trigger altered.

b. At this time Run autoconfig with the patch env sourced.

c. Enable the ebs_login trigger using the following SQL.

SQL> show user

USER is "SYSTEM"

SQL> alter trigger ebs_logon enable;

6. Check if entries are appropriate in the tables and if it looks good, you can go ahead with the prepare phase.

frm-92101 there was a failure in the forms server during startup. invalid configuration

After successful cloning the EBS instance. We just login in ebs and lounched the forms but unable to do that . Because of below error.

Cause:

The cause is not unusual with Linux, or Oracle, or Oracle Apps. It’s that Oracle Forms is looking for a library file. But the file doesn’t exist in the expected place.

In this case, libXm.so.2 can’t be found.

Solution:

Check this directory and below mentioned files "/usr/X11R6/lib".

/usr/lib/libMrm.so.3.0.2

/usr/lib/libMrm.so.4.0.1

/usr/lib/libUil.so.3.0.2

/usr/lib/libUil.so.4.0.1

/usr/lib/libXm.so.3.0.2

/usr/lib/libXm.so.4.0.1

If its not there then create it below mentioned manner,

1. login as root:

mkdir -p /usr/X11R6/lib

2. Make Links To The Original Files:

cd /usr/X11R6/lib

ln -s /usr/lib/libMrm.so.3.0.2 .

ln -s /usr/lib/libMrm.so.4.0.1 .

ln -s /usr/lib/libUil.so.3.0.2 .

ln -s /usr/lib/libUil.so.4.0.1 .

ln -s /usr/lib/libXm.so.3.0.2 .

ln -s /usr/lib/libXm.so.4.0.1 .

Make Links With An Abbreviated Name:

ln -s /usr/lib/libXm.so.3.0.2 libXm.so.3

ln -s /usr/lib/libXm.so.4.0.1 libXm.so.4

ln -s /usr/lib/libXm.so.4.0.1 libXm.so

ln -s /usr/lib/libUil.so.4.0.1 libUil.so.4

ln -s /usr/lib/libUil.so.3.0.2 libUil.so.3

ln -s /usr/lib/libUil.so.4.0.1 libUil.so

ln -s /usr/lib/libMrm.so.4.0.1 libMrm.so.4

ln -s /usr/lib/libMrm.so.3.0.2 libMrm.so.3

ln -s /usr/lib/libMrm.so.4.0.1 libMrm.so

ln -s /usr/lib/libMrm.so.3.0.2 libMrm.so.2

ln -s /usr/lib/libUil.so.3.0.2 libUil.so.2

ln -s /usr/lib/libXm.so.3.0.2 libXm.so.2

Retest the issue again, hope it will open without FRM 92101 error.

ORA-29702: error occurred in Cluster Group Service operation

While trying to login in oracle database, after that ORA-29702 error coming.

$sqlplus / as sysdba

SQL*Plus: Release 11.2.0.4.0 Production on Thu Jun 10 10:29:23 2021

Connected to an idle instance.

SQL> startup pfile=initPHASE2.ora

ORA-29702: error occurred in Cluster Group Service operation

SQL>

SQL> shut immediate

ORA-01034: ORACLE not available

ORA-27101: shared memory realm does not exist

Linux-x86_64 Error: 2: No such file or directory

SQL>

To resolve this error , need to be do the same as below.

Solution:-

cd $ORACLE_HOME/rdbms/lib

make -f ins_rdbms.mk rac_off ioracle

make -f ins_rdbms.mk ioracle

Check it now, able to login successfully.

sqlplus / as sysdba

SQL*Plus: Release 11.2.0.4.0 Production on Thu Jun 10 10:34:28 2021

Connected to an idle instance.

SQL> startup

ORACLE instance started.

Total System Global Area 2137886720 bytes

Fixed Size 2254952 bytes

Variable Size 553650072 bytes

Database Buffers 1560281088 bytes

Redo Buffers 21700608 bytes

Database mounted.

Database opened.

WARNING: Could not find all the required OS utilities in the $PATH. Please review the checks above

[applmgr@****apps bin]$ perl adcfgclone.pl appsTier

Redwood Shores, California, USA

Oracle E-Business Suite Rapid Clone

Version 12.2

adcfgclone Version 120.63.12020000.65

***********************************************************

In AD-TXK Delta 7, we recommend you clone the run and patch

file systems in a single operation using the 'dualfs' option.

Separate cloning of the run and patch file systems will be deprecated

************************************************************

Enter the APPS password :

Enter the Weblogic AdminServer password :

Do you want to add a node (yes/no) [no] : no

which: no g++ in (/oracle/EBSFS/fs1/EBSapps/comn/clone/bin/../jre/bin:/usr/lib64/qt-3.3/bin:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/home/applmgr/bin)

Checking for make... found - /usr/bin/make

Checking for ld... found - /usr/bin/ld

Checking for gcc... found - /usr/bin/gcc

Checking for g++...

Unable to locate 'g++' utility in path

Checking for ar... found - /usr/bin/ar

Unable to locate all utilities with system path.

PATH = /oracle/EBSFS/fs1/EBSapps/comn/clone/bin/../jre/bin:/usr/lib64/qt-3.3/bin:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/home/applmgr/bin

WARNING: Could not find all the required OS utilities in the $PATH. Please review the checks above

Solution:-

#yum install gcc-c++.x86_64

which g++

/usr/bin/g++

[applmgr@****apps bin]$ g++ --version

g++ (GCC) 4.4.7 20120313 (Red Hat 4.4.7-23.0.1)

This is free software; see the source for copying conditions. There is NO

warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

[applmgr@****apps bin]$ perl adcfgclone.pl appsTier dualfs

Redwood Shores, California, USA

Oracle E-Business Suite Rapid Clone

Version 12.2

adcfgclone Version 120.63.12020000.65

Enter the APPS password :

Enter the Weblogic AdminServer password :

Do you want to add a node (yes/no) [no] : no

Running: Context clone...

Log file located at /oracle/EBSFS/fs1/EBSapps/comn/clone/bin/CloneContext_0610161855.log

Provide the values required for creation of the new APPL_TOP Context file.

Target System Hostname (virtual or normal) [***apps]:

Target System Database SID : TEST

Target System Database Server Node [***apps] : ***apps

Target System Database Domain Name [***-****.com] :

Target System Base Directory : /oracle/EBSFS

Target System Base Directory set to /oracle/EBSFS

Target System Current File System Base set to /oracle/EBSFS/fs1

Target System Other File System Base set to /oracle/EBSFS/fs2

Target System Fusion Middleware Home set to /oracle/EBSFS/fs1/FMW_Home

Target System Other File System Fusion Middleware Home set to /oracle/EBSFS/fs2/FMW_Home

Target System Web Oracle Home set to /oracle/EBSFS/fs1/FMW_Home/webtier

Target System Other File System Web Oracle Home set to /oracle/EBSFS/fs2/FMW_Home/webtier

Target System Appl TOP set to /oracle/EBSFS/fs1/EBSapps/appl

Target System Other File System Appl TOP set to /oracle/EBSFS/fs2/EBSapps/appl

Target System COMMON TOP set to /oracle/EBSFS/fs1/EBSapps/comn

Target System Other File System COMMON TOP set to /oracle/EBSFS/fs2/EBSapps/comn

Target System Instance Home Directory [/oracle/EBSFS] :

Target System Current File System Instance Top set to /oracle/EBSFS/fs1/inst/apps/UAT_samarthapps

Do you want to preserve the Display [***test:0.0] (y/n) : n

Target System Display [***apps:0.0] :

Target System Root Service [enabled] :

Target System Web Entry Point Services [enabled] :

Target System Web Application Services [enabled] :

Target System Batch Processing Services [enabled] :

Target System Other Services [disabled] :

Do you want the target system to have the same port values as the source system (y/n) [y] ? : n

Target System Port Pool [0-99] : 1

Checking the port pool 1

done: Port Pool 1 is free

Report file located at /oracle/EBSFS/fs1/inst/apps/***_****apps/admin/out/portpool.lst

ERROR: Failed to execute FMW pre-requisite check

In this post we will identify the cloning issue and fix accordingly.

$perl adcfgclone.pl appsTier dualfs

Target System Port Pool [0-99] : 2

Checking the port pool 2

done: Port Pool 2 is free

Report file located at /oracle/EBSFS/fs2/inst/apps/UAT_samarthapps/admin/out/portpool.lst

The new APPL_TOP context file has been created :

/oracle/EBSFS/fs2/inst/apps/UAT_samarthapps/appl/admin/UAT_samarthapps.xml

Check Clone Context logfile /oracle/EBSFS/fs1/EBSapps/comn/clone/bin/CloneContextPatch_0610164000.log for details.

FMW Pre-requisite check log file location : /oracle/EBSFS/fs1/EBSapps/comn/clone/FMW/logs/prereqcheck.log

Running: FMW pre-req check...

ERROR: Failed to execute FMW pre-requisite check

ERROR: Run file system configuration has failed

Check the logfile /oracle/EBSFS/fs1/inst/apps/UAT_samarthapps/admin/log/clone/run/RCloneApplyAppstier_06101635.log

ERROR: Failed to configure the target system,

please check the logfile in : /oracle/EBSFS/fs1/inst/apps/UAT_samarthapps/admin/log/clone

Error while executing /oracle/EBSFS/fs1/EBSapps/comn/clone/bin/alter_ebs_logon_trigger.sql

Failed to enable EBS_LOGON trigger

prerequisite log:-

Check Name:Packages

Check Description:This is a prerequisite condition to test whether the packages recommended for installing the product are available on t

Checking for binutils-2.20.51.0.2-5.11.el6-x86_64; found binutils-2.20.51.0.2-5.42.el6-x86_64. Passed

Checking for compat-libcap1-1.10-1-x86_64; Not found. Failed <<<<

Checking for compat-libstdc++-33-3.2.3-69.el6-x86_64; found compat-libstdc++-33-3.2.3-69.el6-x86_64. Passed

Checking for compat-libstdc++-33-3.2.3-69.el6-i686; found compat-libstdc++-33-3.2.3-69.el6-i686. Passed

Checking for libgcc-4.4.4-13.el6-i686; found libgcc-4.4.7-23.0.1.el6-i686. Passed

Checking for libgcc-4.4.4-13.el6-x86_64; found libgcc-4.4.7-23.0.1.el6-x86_64. Passed

Checking for libstdc++-4.4.4-13.el6-x86_64; found libstdc++-4.4.7-23.0.1.el6-x86_64. Passed

Checking for libstdc++-4.4.4-13.el6-i686; found libstdc++-4.4.7-23.0.1.el6-i686. Passed

Checking for libstdc++-devel-4.4.4-13.el6-x86_64; found libstdc++-devel-4.4.7-23.0.1.el6-x86_64. Passed

Checking for sysstat-9.0.4-11.el6-x86_64; found sysstat-9.0.4-33.0.1.el6_9.1-x86_64. Passed

Checking for gcc-4.4.4-13.el6-x86_64; found gcc-4.4.7-23.0.1.el6-x86_64. Passed

Checking for gcc-c++-4.4.4-13.el6-x86_64; found gcc-c++-4.4.7-23.0.1.el6-x86_64. Passed

Checking for glibc-2.12-1.7.el6-i686; found glibc-2.12-1.212.0.3.el6_10.3-i686. Passed

Checking for glibc-2.12-1.7.el6-x86_64; found glibc-2.12-1.212.0.3.el6_10.3-x86_64. Passed

Checking for glibc-devel-2.12-1.7.el6-x86_64; found glibc-devel-2.12-1.212.0.3.el6_10.3-x86_64. Passed

Checking for glibc-devel-2.12-1.7.el6; found glibc-devel-2.12-1.212.0.3.el6_10.3-x86_64. Passed

Checking for libaio-0.3.107-10.el6-x86_64; found libaio-0.3.107-10.el6-x86_64. Passed

Checking for libaio-devel-0.3.107-10.el6-x86_64; found libaio-devel-0.3.107-10.el6-x86_64. Passed

Check complete. The overall result of this check is: Failed <<<<

Error:-

compat-libcap1-1.10-1-x86_64; Not found. Failed <<<<

Solution:-

rpm -Uvh compat-libcap1-1.10-1.x86_64.rpm

Preparing... ########################################### [100%]

1:compat-libcap1 ########################################### [100%]

After that restest the issue.

txkWfClone.sh exited with status 127 ERROR: txkWfClone.sh execution failed, exit code 127

$perl adcfgclone.pl appsTier dualfs

Redwood Shores, California, USA

Oracle E-Business Suite Rapid Clone

Version 12.2

adcfgclone Version 120.63.12020000.65

Enter the APPS password :

Enter the Weblogic AdminServer password :

Do you want to add a node (yes/no) [no] : no

Running: Context clone...

Log file located at /oracle/EBSFS/fs1/EBSapps/comn/clone/bin/CloneContext_0612142654.log

Provide the values required for creation of the new APPL_TOP Context file.

Target System Hostname (virtual or normal) [samarthapps] :

Target System Database SID : UAT

Target System Database Server Node [samarthapps] : uatapps

Target System Database Domain Name [***-***.com] :