Tuesday, November 18, 2014

Schedule Exachk using OEM and command Line | Exadata healthcheck OEM

Links to this post
There is New Plugin to Install and schedule Exachk on Exadata/ Exalogic system Through 12c Grid control.

Plugin has to first deploy on OMS server and then Exadata / Exalogic Agents.

12c Home page setup (right upper corner) -> extensibility -> Plugin -> Engineered system -> Oracle Exadata healthcheck.

Once it deploy on 12c Grid OMS and Exadata agent.
You can create Monitoring templates for Exadata Healthcheck Metrics.

To schedule Using command Line.

Download Latest Exachk from
Oracle Exadata Database Machine exachk or HealthCheck (Doc ID 1070954.1).
Exachk Health-Check Tool for Exalogic (Doc ID 1449226.1)


Note : Exachk must run from Enterprise Vserver in Virtual Exalogic Environment.

---check Version

./exachk -v
EXACHK  VERSION: 12.1.0.2.1

---Set User equivelency between DB node to storage cell and IB switches 

./exachk -initpresetup   

---setup auto restart and Exachk Deamon. 

./exachk -initsetup
Setting up exachk auto restart functionality using inittab
Starting exachk daemon. . .  .

---Check auto restart and deamon status . 

./exachk -initcheck
Auto restart functionality is configured.
exachk daemon is running. PID : 00000

---schedule exachk. 

AUTORUN_SCHEDULE * * * *       :- Automatic run at specific time in daemon mode.
                 - - - -
                 ? ? ? ?
                 ? ? ? +----- day of week (0 - 6) (0 to 6 are Sunday to Saturday)
                 ? ? +---------- month (1 - 12)
                 ? +--------------- day of month (1 - 31)
                 +-------------------- hour (0 - 23)

./exachk -set "AUTORUN_SCHEDULE=23 * * 0;AUTORUN_FLAGS= -a -o v;NOTIFICATION_EMAIL=firstname.lastname@company.com;PASSWORD_CHECK_INTERVAL=1;"

---List all parameter for exachk. 

./exachk -get all

Sunday, November 2, 2014

Exadata Patching | Upgrade Exadata

Links to this post
This is High Level step by step Instruction for APR-2014 (11.2.0.4)  Exadata QFSDP.

It is divided in Three major Part.

-Upgrade Exadata Database server RPMs
-Upgrade Storage server Image and Infiniband Switch.
-Upgrade Grid Home and Oracle Home .

1  ExadataDatabaseServer 
  • This Part required Reboot and shutdown of CRS on Local Node. We will apply This patch In rolling. 
  • Database Node gets updates from script provide along with Exadata QFSDP zip file.   
  • Check usage of script. 
  • dbnodeupdate.sh: Exadata Database Server Patching using the DB Node Update Utility (Doc ID 1553103.1)

1.1 Inflate dbnodeupdate script comes with QFSDP

1.2 Upgrade Compute Node From Local Yum zip file.

This command upgrade / and reboot Compute Node at end Of Patch. If we include –s it will stop CRS in this Node.  

cd <patch_location>/18370227/Infrastructure/ExadataDBNodeUpdate/3.20
./dbnodeupdate.sh -u -l /u01/patches/YUM/p17809253_112330_Linux-x86-64.zip –s –v 

1.3 Post –relink of Compute Node once server comes back online. 

This command will relink GI and OH and also start CRS on That Node. 

./dbnodeupdate.sh -c


2 ExadataStorageServer  and InfiniBandSwitch

2.1 Download Patch manager Plugin  from 1487339.1  Metalink Note. 

2.2 Preparing Exadata Cells for Patch Application

Execute below command as Root from Compute Node. 
Generate rsa/dsa Key on Compute Node. 

ssh-keygen -t rsa
ssh-keygen -t dsa
Push key to all cell 
dcli -l root -g cell_group –k
dcli -g cell_group -l root 'hostname -i'

2.3 Set DISK REPAIR TIME on ASM disks. 

select dg.name,a.value from v$asm_diskgroup dg, v$asm_attribute a where dg.group_number=a.group_number and a.name='disk_repair_time';
alter diskgroup diskgroup_name set attribute 'disk_repair_time'='3.6h';

2.4 Turn off DB services on compute Node for NON-ROLLING patch. 

NOTE: THIS PART APPLY ONLY IF YOU DECIDE TO APPLY PATCH IN NON-ROLLING FASHION , DO NOT SHUT-DOWN IF PATCH IS ROLLING. 

Run below command as ROOT. From compute Node.  

dcli -g dbs_group -l root "/u01/app/11.2.0/grid/bin/crsctl stop crs -f"
dcli -g dbs_group -l root "ps -ef | grep grid" 
dcli -g cell_group -l root "cellcli -e alter cell shutdown services all"

2.5 Pre-check on Patch on Cell storage using patch manager.

cd <patchlocation>/18370227/Infrastructure/11.2.3.3.0/ExadataStorageServer_InfiniBandSwitch/patch_11.2.3.3.0.131014.1

./patchmgr -cells ~/cell_group -reset_force

./patchmgr -cells cell_group -patch_check_prereq [-rolling] [-ignore_alerts] [-smtp_from "addr" -smtp_to "addr1 addr2 addr3 ..."]

2.6 Patch cell by Patch Utility. 

If the prerequisite checks pass, then start patch application. Use -rolling option if you plan to use rolling updates. Use the -ignore_alerts option to ignore any open hardware alerts on the cells, and continue. Use the -smtp_from, -smtp_to options to set an e-mail address to receive patchmgr alert messages, and continue.

./patchmgr -cells cell_group -patch [-rolling] [-ignore_alerts] [-smtp_from "from_email_address"] [-smtp_to "to_email_address1   to_email_address2 ..."]

2.7 Check any Grid disk are inactive or offline. 

dcli -g ~/cell_group -l root                                          \
"cat /root/attempted_deactivated_by_patch_griddisks.txt | grep -v     \
ACTIVATE | while read line; do str=\`cellcli -e list griddisk where   \
name = \$line attributes name, status, asmmodestatus\`; echo \$str |  \
grep -v \"active ONLINE\"; done"

2.8 Run ibswitch pre-requisite 

./patchmgr –ibswitches -upgrade -ibswitch_precheck 

2.9 Apply patch on IBSWITCH.

cd <patch location>/18370227/Infrastructure/11.2.3.3.0/ExadataStorageServer_InfiniBandSwitch/patch_11.2.3.3.0.131014.1

./patchmgr –ibswitches -upgrade 

3 Database and Grid Home Upgrade. 

3.1 Distribute GI and OH patch to NFS or /tmp 
3.1 Install Latest OPatch and Oplan. 
3.2  Generate steps to Patch GI using Oplan 

<$GRID_HOME>/OPatch/oplan generateApplySteps <patch location>/18370227/database/11.2.0.4.6_QDPE_Apr2014/18371656
<$GRID_HOME>/OPatch/oplan generateRollbackSteps <patch location>/18370227/database/11.2.0.4.6_QDPE_Apr2014/18371656

3.3 Create OCM file 

dcli -g ~/dbs_group -l oracle $ORACLE_HOME/OPatch/ocm/bin/emocmrsp –output /home/oracle

3.4 Follow Oplan generated File Instruction. 

Thursday, October 30, 2014

Exalogic Patching - Virtual

Links to this post
This is procedure to apply Exalogic Virtual 2.0.6.1.1 April 2014 Patchset.

1 Set up the PSU
2 Run precheck for Patch
3 Create Rack History File.
4 Apply Patch on Exalogic Control servers.
5 Upgrade QDR InfiniBand (NM2-GW) Gateway Switches.
6 Update the Compute Node Base Image ROLLING.
7 Patch the OVMM, PC1, and PC2 Templates
8 Upgrade the Guest vServer  (VN02)
9 Observation
10 Reference

Important Notes About the Upgrade Procedure


Always run ExaPatch from the PSU bundle directory (where the  exapatch_descriptor.py file is located) using the full path to   exapatch from any compute node within the rack.
Do not run ExaPatch directly on the compute node being patched.
ExaPatch patches the NM2-GW and NM2-36P switches one switch at a time
(rolling upgrade). The switch running the master subnet manager is
patched after patching all the non-master switches.
The upgrade procedure supports upgrading the compute nodes one node at   a time (rolling upgrade). Upgrading one node at a time ensures that the hosted services and applications are not disrupted.
Oracle recommends that these patches be applied to a test or a   nonproduction system before it is applied to the production system. The total time taken for patching the test system can be used as a   baseline for scheduling the maintenance windows to patch the production system.
Perform the patching by following the steps exactly as documented in this readme.


--------------------------------------------
1 Set up the PSU
--------------------------------------------

~~~~~~~~

1. Log in to the compute node as root.
2. cd /exalogic-lcdata/patches
3. Add execute permissions for psuSetup.sh using the chmod command. In the following example,
      execute permissions for all users is added:
      # chmod a+x psuSetup.sh
4.  Run the script:
./psuSetup.sh ZFS_IP_Address [--mountonly] [--unmountonly] [--remount] [--verbose] [--force ] [--help]

[root@DummyCN01 patches]# ./psuSetup.sh 1.1.1.10

INFO: Pre-requiste Check...
INFO: Checking for Python version...
INFO: Python version check... succeeded
INFO: Checking for Root permissions...
INFO: /exalogic-lcdata is already mounted from 1.1.1.10
INFO: /exalogic-lctools is already mounted from 1.1.1.10
INFO: Extracting PSU Bundle data to /exalogic-lcdata. This will take a few minutes
ExaBR 1.1 (build 5951)
ExaBR 1.1 (build 5951)
INFO: /exalogic-lctools Version: 14.1 and expatch Version: 1.2.1 is already installed
INFO: Installation complete.
INFO: ExaPatch is installed in /exalogic-lctools/bin/exapatch
INFO: PSU is installed in /exalogic-lcdata/patches/Virtual/18178980/
#####

--------------------------------------------
2 Run precheck for Patch
--------------------------------------------

~~~~~~~~

1) Copy config file. 
cp /exalogic-lcdata/patches/Virtual/18178980/Infrastructure/2.0.6.1.1/exapatch_descriptor.py ./exalogic-lctools/bin
2) Cd to exapatch directory 
cd /exalogic-lcdata/patches/Virtual/18178980/Infrastructure/2.0.6.1.1/exalogic-lctools/bin
3) Run Exapatch pre-Patch check.
[root@DummyCN01 bin]# exapatch -a prePatchCheck
Logging to file /var/log/exapatch_20150730200807.log
log file: /var/log/exapatch_20200730200807.log
#####

Check authentication from Exapatch

The following prerequisites must be fulfilled on all Exalogic Control  vServers before they can be upgraded to version 12.1.4 b2500
 
When updating the Exalogic Control services, ExaPatch must be run on a compute node with TCP/IP access to all Exalogic Control vServers.
All the Exalogic Control vServers must be running. Verify that access to vServer-EC-OVMM and to the two vServer-EC-EMOC-PC is OK by running
   the following ExaPatch command:
         [root@compute-node]# /exalogic-lctools/bin/exapatch -a checkAuthentication
Log in to the Exalogic Control BUI and make sure that assets(switches storage) are managed by a single ProxyController(PC) at a time. If any of the assets appear to be managed by both the ProxyControllers, refer
           to the Troubleshooting section.
Back up the Exalogic Control Stack using the ExaBR tool, as described in Section 4.1 of Oracle Exalogic Elastic Cloud Backup and Recovery Guide Using ExaBR


-------------------------------------------------
3) Create Rack History File.
-------------------------------------------------

~~~~~~~~
[root@DummyCN01 bin]# ./exapatch -a getHistory
"/exalogic-lcdata/inventory/rack_history.xml"
INFO: Creating rack history file: /exalogic-lcdata/inventory/rack_history.xml
INFO: Updated rack history file: /exalogic-lcdata/inventory/rack_history.xml
#####

Make sure each component is in rack_history file.
WARNING: unable to update patch history: Unable to find unique identifier "xxyyzz123" for Compute-Node 1.1.557.66 in the rack history file.

-------------------------------------------------
4 ) Apply Patch on Exalogic Control servers.
-------------------------------------------------

1. Cd to Patch directory                                                                                       .
Cd /exalogic-lcdata/patches/1.78980/Infrastructure/2.0.6.1.1
2. Run the ExaPatch tool as follows:

~~~~~~~~
[root@DummyCN01 2.0.6.1.1]# /exalogic-lctools/bin/exapatch -a runExtension -p Exalogic_Control/emoc_patch_extension.py exapatch_descriptor.py
Logging to file /var/log/exapatch_20200804100747.log
Enter Compute-Node root password:
Enter ILOM-ComputeNode root password:
Enter ILOM-ZFS root password:
Enter vServer-EC-EMOC-PC 1.1.557.74 root password:
Enter vServer-EC-EMOC-PC 1.1.557.75 root password:
Enter EMOC-PC-service 1.1.557.74 root password:
Enter EMOC-PC-service 1.1.557.75 root password:
INFO: EMOC-PC-service 1.1.557.74 successfully completed all pre-patch checks
INFO: EMOC-PC-service 1.1.557.75 successfully completed all pre-patch checks
INFO: EMOC-EC-service 1.1.558.21 successfully completed all pre-patch checks
Upgrading PC software on EMOC-PC-service host 1.1.557.74 from version:
        12.1.4.2330
WARNING: unable to update patch history: RackHistory source file "/exalogic-lcdata/inventory/rack_history.xml" does not exist.
Upgrading PC software on EMOC-PC-service host 1.1.557.75 from version:
        12.1.4.2330
WARNING: unable to update patch history: RackHistory source file "/exalogic-lcdata/inventory/rack_history.xml" does not exist.
INFO: EMOC-PC-service 1.1.557.74 successfully completed all post-patch checks
Completed upgrade of PC software on EMOC-PC-service host 1.1.557.74 new version:
        12.1.4.2500
INFO: EMOC-PC-service 1.1.557.75 successfully completed all post-patch checks
Completed upgrade of PC software on EMOC-PC-service host 1.1.557.75 new version:
        12.1.4.2500
Upgrading EC software on EMOC-EC-service host 1.1.558.21 from version:
        12.1.4.2330
INFO: uploading patch bundle to vServer
INFO: running upgrade scripts
WARNING: unable to update patch history: RackHistory source file "/exalogic-lcdata/inventory/rack_history.xml" does not exist.
Completed upgrade of EC software on EMOC-EC-service host 1.1.558.21 new version:
        12.1.4.2500
INFO: EMOC-EC-service 1.1.558.21 successfully completed all post-patch checks
#####

-------------------------------------------------
5) Upgrade QDR InfiniBand (NM2-GW) Gateway Switches.
-------------------------------------------------

~~~~~~~~
----KILL user process.
Log in as root and run the following command:
[root@nm2gw-ib01 ~]# ps -ef | grep ssh


----Reduce the timeout for idle root sessions by editing sshd_config:
 
[root@nm2gw-ib01 ~]# vi /etc/ssh/sshd_config
-- 8< -- 
ClientAliveInterval 60
ClientAliveCountMax 3
-- 8< -- 

Restart the sshd service:
    
---[root@nm2gw-ib01 ~]# service sshd restart 
Stopping sshd:                                             [  OK  ]
Starting sshd:                                             [  OK  ]

---Verify that only one target is displayed by the sessions command, as shown in the following example:

[root@nm2gw-ib01 ~]# spsh
-> show /SP/sessions
/SP/sessions
Targets:
120350 (current)
      Properties:
      Commands:
      cd
      show
----Edit the timeout for the ILOM session:

 -> set /SP/cli timeout=1
 Set 'timeout' to '1'

---Get gwinstance value .
showgwconfig

----Updating the Firmware of the NM2-GW Switches

This step Approximately  41 Min.

Login into compute Node and Cd to Patch directory
Cd /exalogic-lcdata/patches/1.78980/Infrastructure/2.0.6.1.1
Apply Patch

    [root@compute-node]# /exalogic-lctools/bin/exapatch -a patch nm2-gw

[root@DummyCN01 2.0.6.1.1]# /exalogic-lctools/bin/exapatch -a patch nm2-gw
Logging to file /var/log/exapatch_20200804105003.log
INFO: NM2-GW-IB-Switch 1.1.557.72 pre-patch checks may run for approximately 10 minutes.
INFO: NM2-GW-IB-Switch 1.1.557.72 successfully completed all pre-patch checks
Upgrading InfiniBand software on NM2-GW-IB-Switch host 1.1.557.72 from version:
        SUN DCS gw version: 2.1.3-4
        Build time: Aug 28 2013 18.6:06
        FPGA version: 0x34
        SP board info:
        Hardware Revision: 0x0007
        Firmware Revision: 0x0000
        BIOS version: SUN0R100
        BIOS date: 06/22/2010
INFO: IB switch upgrade can take 10-15 minutes.
Completed upgrade of InfiniBand software on NM2-GW-IB-Switch host 1.1.557.72 new version:
        SUN DCS gw version: 2.1.4-1
        Build time: Jan.7 2020 11:18:57
        FPGA version: 0x34
        SP board info:
        Hardware Revision: 0x0007
        Firmware Revision: 0x0000
        BIOS version: SUN0R100
        BIOS date: 06/22/2010
INFO: NM2-GW-IB-Switch 1.1.557.72 post-patch checks may run for approximately 10 minutes.
INFO: NM2-GW-IB-Switch 1.1.557.72 successfully completed all post-patch checks
INFO: Changing master NM2-GW-IB-Switch 1.1.557.71 to standby.
INFO: NM2-GW-IB-Switch 1.1.557.71 pre-patch checks may run for approximately 10 minutes.
INFO: NM2-GW-IB-Switch 1.1.557.71 successfully completed all pre-patch checks
Upgrading InfiniBand software on NM2-GW-IB-Switch host 1.1.557.71 from version:
        SUN DCS gw version: 2.1.3-4
        Build time: Aug 28 2013 18.6:06
        FPGA version: 0x34
        SP board info:
        Hardware Revision: 0x0007
        Firmware Revision: 0x0000
        BIOS version: SUN0R100
        BIOS date: 06/22/2010
INFO: IB switch upgrade can take 10-15 minutes.
Completed upgrade of InfiniBand software on NM2-GW-IB-Switch host 1.1.557.71 new version:
        SUN DCS gw version: 2.1.4-1
        Build time: Jan.7 2020 11:18:57
        FPGA version: 0x34
        SP board info:
        Hardware Revision: 0x0007
        Firmware Revision: 0x0000
        BIOS version: SUN0R100
        BIOS date: 06/22/2010
INFO: NM2-GW-IB-Switch 1.1.557.71 post-patch checks may run for approximately 10 minutes.
INFO: NM2-GW-IB-Switch 1.1.557.71 successfully completed all post-patch checks
2020-08-04T10:50 2020-08-04T11:30:55
#####

-----If exalogic and exadata is connected via spine switch verify-topology will not work , Use ibnetdiscover instead

If the output does not contain all the NM2-GW switches in the rack, verify-topology will fail. To fix this do below.

~~~~~~~~
[root@DummyCN01 bin]# verify-topology
[ Exalogic Machine Infiniband Cabling Topology Verification Tool ]
[ERROR] switch xxyyxxzzz909090 does not configured to set ib node description.
[root@DummyCN01 bin]# ll /opt/exalogic.tools/tools/idgen.sh
-rwxr--r-- 1 root root 869 Oct 18  2013 /opt/exalogic.tools/tools/idgen.sh
[root@DummyCN01 bin]# /opt/exalogic.tools/tools/idgen.sh
spawn ssh-keygen -t dsa
Generating public/private dsa key pair.
Enter file in which to save the key (/root/.ssh/id_dsa):
/root/.ssh/id_dsa already exists.
Overwrite (y/n)? y
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /root/.ssh/id_dsa.
Your public key has been saved in /root/.ssh/id_dsa.pub.
The key fingerprint is:
root@DummyCN01.domain.com
[root@DummyCN01 bin]# /opt/exalogic.tools/tools/dcli -c 1.1.555.71,1.1.555.72 -k
root@1.1.555.72's password:
root@1.1.555.71's password:
1.1.555.71: ssh key added
1.1.555.72: ssh key added
[root@DummyCN01 bin]# /opt/exalogic.tools/tools/dcli -c 1.1.555.71,1.1.555.72 -f /opt/exalogic.tools/tools/network_tools/switch_node_desc_config.tgz -d /tmp
[root@DummyCN01 bin]# /opt/exalogic.tools/tools/dcli -c 1.1.555.71 -f /opt/exalogic.tools/tools/network_tools/remote_config_switch_node_desc.sh -d /tmp "/tmp/remote_config_switch_node_desc.sh 1"
1.1.555.71: exalogic/
1.1.555.71: exalogic/rc.local
1.1.555.71: exalogic/config_ib_desc.sh
1.1.555.71: exalogic/ib_set_node_desc.sh
1.1.555.71: Successfully configured switch node description
[root@DummyCN01 bin]# /opt/exalogic.tools/tools/dcli -c 1.1.555.72 -f /opt/exalogic.tools/tools/network_tools/remote_config_switch_node_desc.sh -d /tmp "/tmp/remote_config_switch_node_desc.sh 2"
1.1.555.72: exalogic/
1.1.555.72: exalogic/rc.local
1.1.555.72: exalogic/config_ib_desc.sh
1.1.555.72: exalogic/ib_set_node_desc.sh
1.1.555.72: Successfully configured switch node description
#####

-------------------------------------------------
6) Update the Compute Node Base Image ROLLING.
-------------------------------------------------

Note: Updating Base Image on Compute Node does not required reboot of Compute Node.

-----Compute nodes can be patched in the following ways:
Rolling: Patch one node at a time
o This method applies on one node at a time, patching it. With this method, only one node is being patched at any point in time and the other nodes can continue to provide services, but the patching process takes more time.
Parallel: Patch multiple nodes simultaneouly
o This method applies compute node patches/updates across multiple nodes in parallel. Using ExaPatch, you can patch a subset of nodes at a time or patch all the nodes in the Exalogic rack.
It is recommended that you patch one compute node, verify that everything works as expected, and then attempt to patch multiple compute nodes in parallel.

-----Pre-requiste

Before upgrading:
Ensure that the compute node base image is at v2.0.6.1.0.
There is no need to backup the compute nodes. If there is a need to restore a compute node, install the base image and use the Exalogic Configuration Utility (ECU) to configure it.
Ensure that at least 80 MB of free space exists in the root (/) partition. You can free up some disk space by running yum clean all or by deleting files that are not longer needed in the /tmp directory. Do not delete files in the directories: /var/log/xen, /var/tmp/exalogic or /var/tmp/ebi_conf.pre20611.bak. In the /var/log directory, do not delete ExaPatch log files or the file called ebi_20611.log.


------Setup Patch on all Nodes

Transfer psuSetup.sh from the downloaded location to this machine:
    [root@compute-node2]# scp root@compute-node1:~/psuSetup.sh .

Run the psuSetup.sh while specifying ZFS active head IPoIB address and with --mountonly option:
    [root@compute-node2]# ./psuSetup.sh 1.1.555.120 --mountonly

First upgrade compute Node one of the Node.

~~~~~~~~
[root@DummyCN01 2.0.6.1.1]# /exalogic-lctools/bin/exapatch -a patch cn -h 1.1.557.68
Logging to file /var/log/exapatch_202008041.750.log
Enter Compute-Node root password:
INFO: Compute-Node 1.1.557.68 successfully completed all pre-patch checks
Upgrading Base Image software on Compute-Node host 1.1.557.68 from version:
        Image version       : 2.0.6.1.0
        Image build version : 228455
        BIOS                : 25010600-07/08/2013
        OFED                : OFED-IOV-1.5.5-1.0.054
        InfiniBand card     : 2.11.1282
        Disk controller     : 12.12.0-.78
Completed upgrade of Base Image software on Compute-Node host 1.1.557.68 new version:
        Image version       : 2.0.6.1.1
        Image build version : 228455
        BIOS                : 25010600-07/08/2013
        OFED                : OFED-IOV-1.5.5-1.0.054
        InfiniBand card     : 2.11.1282
        Disk controller     : 12.12.0-.78
INFO: Compute-Node 1.1.557.68 successfully completed all post-patch checks
#####


Now Upgrade compute Node rest of The Node.

~~~~~~~~
[root@DummyCN02 2.0.6.1.1]#/exalogic-lctools/bin/exapatch -a patch cn -h 1.1.557.63 -h 1.1.557.64 -h 1.1.557.65 -h 1.1.557.66 -h 1.1.557.67
Logging to file /var/log/exapatch_20200804124409.log
Enter Compute-Node root password:
        Image build version : 228455
        BIOS                : 25010600-07/08/2013
        OFED                : OFED-IOV-1.5.5-1.0.054
        InfiniBand card     : 2.11.1282
        Disk controller     : 12.12.0-.78
Upgrading Base Image software on Compute-Node host 1.1.557.63 from version:
        Image version       : 2.0.6.1.0
        Image build version : 228455
        BIOS                : 25010600-07/08/2013
        OFED                : OFED-IOV-1.5.5-1.0.054
        InfiniBand card     : 2.11.1282
        Disk controller     : 12.12.0-.78
WARNING: unable to update patch history: Unable to find unique identifier "xyxyxyxy" for Compute-Node 1.1.557.66 in the rack history file.
Completed upgrade of Base Image software on Compute-Node host 1.1.557.64 new version:
        Image version       : 2.0.6.1.1
        Image build version : 228455
        BIOS                : 25010600-07/08/2013
        OFED                : OFED-IOV-1.5.5-1.0.054
        InfiniBand card     : 2.11.1282
        Disk controller     : 12.12.0-.78
INFO: Compute-Node 1.1.557.64 successfully completed all post-patch checks


#####

----Verify upgrade.
This process takes less than 5 minutes. The main update to compute node is ksplice and version update. This does not cause a reboot on the compute node being upgraded. You may connect to the serial console through ILOM and monitor the upgrade process.
After all the compute nodes, other than the one on which ExaPatch is running, have been upgraded, log in to the second compute node (cn02) and run ExaPatch to upgrade the first (cn01) compute node.
Verify the new compute node base image version in the ExaPatch output:

~~~~~~~~

[root@DummyCN01 ~]# dcli -g cn_group.txt 'imageinfo | grep "Image version"'
DummyCN01: Image version       : 2.0.6.1.1

Get Version of Image History. 

[root@DummyCN01 ~]# dcli -g cn_group.txt 'imagehistory | grep "Image version"'
DummyCN01: Image version       : 2.0.6.1.1
DummyCN01: Image version       : 2.0.6.1.0

#####



-------------------------------------------------
7 Patch the OVMM, PC1, and PC2 Templates
-------------------------------------------------
--- Pre-requisite
Reboot Control Vserver Stack
While patching Vserver or Guest Vserver , measure progress from another session via xm console <id>
All Exalogic Control vServers must be running when the patching procedure starts.


----Apply Patch .


[root@compute-node1]# /exalogic-lctools/bin/exapatch -a patch ectemplates
Patching each vServer template can take at least 25 minutes. The total duration for this step is at least 75 minutes. During the patching process, the progress can be tracked by using xm console to log in to the console of the vServer being patched. If you are tracking the progress, after each vServer reboot you should run xm console again to reconnect to the vServer console.

~~~~~~~~
[root@DummyCN01 2.0.6.1.1]# /exalogic-lctools/bin/exapatch -a patch ectemplates
Logging to file /var/log/exapatch_2020080.73044.log
Enter vServer-EC-EMOC-PC 1.1.557.74 root password:
Enter vServer-EC-EMOC-PC 1.1.557.75 root password:
Enter EMOC-PC-service 1.1.557.74 root password:
Enter EMOC-PC-service 1.1.557.75 root password:
INFO: vServer-EC-OVMM 1.1.558.21 successfully completed all pre-patch checks
Upgrading Base Template on vServer-EC-OVMM host 1.1.558.21 from version:
        2.0.6.1.1
Completed upgrade of Base Template on vServer-EC-OVMM host 1.1.558.21 new version:
        2.0.6.1.1
INFO: vServer-EC-OVMM 1.1.558.21 successfully completed all post-patch checks
Completed patching VM template vServer-EC-OVMM 1.1.558.21
INFO: vServer-EC-EMOC-PC 1.1.557.74 successfully completed all pre-patch checks
Starting to patch VM template vServer-EC-EMOC-PC 1.1.557.74
Upgrading Base Template on vServer-EC-EMOC-PC host 1.1.557.74 from version:
        2.0.6.1.0
Completed upgrade of Base Template on vServer-EC-EMOC-PC host 1.1.557.74 new version:
        2.0.6.1.1
INFO: vServer-EC-EMOC-PC 1.1.557.74 successfully completed all post-patch checks
Completed patching VM template vServer-EC-EMOC-PC 1.1.557.74
INFO: vServer-EC-EMOC-PC 1.1.557.75 successfully completed all pre-patch checks
Starting to patch VM template vServer-EC-EMOC-PC 1.1.557.75
Upgrading Base Template on vServer-EC-EMOC-PC host 1.1.557.75 from version:
        2.0.6.1.0
Completed upgrade of Base Template on vServer-EC-EMOC-PC host 1.1.557.75 new version:
        2.0.6.1.1
INFO: vServer-EC-EMOC-PC 1.1.557.75 successfully completed all post-patch checks
Completed patching VM template vServer-EC-EMOC-PC 1.1.557.75
#####

----Work around for Failed exapatch, while updating EC template.

Refer to April 2020 PSU Upgrade - Known Issues (Doc ID.673860.1)

~~~~~~~~
1. On the EC vServer, back up /etc/nsswitch.conf 
2. comment out nis, nisplus, ldap in /etc/nsswitch.conf 
3, go to /opt/egbt_upgrade/BaseTemplate/2.0.6.0.1/scripts/ 
Run./egbt_upgrade.sh --unattended 
4. reboot 
5. run force upgrade from the compute node 
/exalogic-lctools/bin/exapatch -force -a patch ectemplates 
6. check the imageinfo, imagehistory on the vServer to make sure its April PSU. 
7. restore /etc/nsswitch.conf 
#####

-------------------------------------------------
8 Upgrade the Guest vServer  (VN02)
-------------------------------------------------


Log in to the first compute node and run the following command:

~~~~~~~~
[root@compute-node1]# /exalogic-lctools/bin/exapatch -a patch vserver -h <vserverIP1> [-h <vServerIPn>]
To patch multiple guest vServers, you can specify multiple -h vserverIP options.
The following is a sample of the output displayed on the console while patching from 2.0.6.0.0.:
Logging to file /var/log/exapatch_20200302134.7.log
Enter root password for guest vServers: 
INFO: vServer-Generic xx.xx.xx.xx successfully completed all pre-patch checks
Upgrading Base Template on vServer-Generic host xx.xx.xx.xx from version:
      2.0.6.0.0
Completed patching VM templates.     
#####

-------------------------------------------------
9 Observation
-------------------------------------------------

After upgrading each component , exapatch update rack_hisotry.xml file.
If –l option is not provided , then exapatch will generate logfile  itself.
Non master IB switch gets upgraded first, then non master switch changes  to master and then patch applies to Master.
Would like to see if  it disconnection existing connection while “ INFO: Changing master NM2-GW-IB-Switch 1.1.557.71 to standby.”
May be Gwinstance value get increase
Base Image update does not reboot compute Node or Migrate VM on it.
WARNING: unable to update patch history: Unable to find unique identifier "1337FML021" for Compute-Node 1.1.557.66 in the rack history file.
While patching Vserver or Guest Vserver , measure progress from another session via xm console <id>
All exapatch command must run from /exalogic-lcdata/patches/Virtual/1.78980/Infrastructure/2.0.6.1.1  dir and must be fully qualified path till exapatch.


-------------------------------------------------
10 Reference
-------------------------------------------------

Exalogic April 2014 PSU - 2.0.6.1.1 (Linux - Physical - Exalogic X4-2) Infrastructure Upgrade Guide (Doc ID 1638838.1)
Refer to April 2014 PSU Upgrade - Known Issues (Doc ID 1673860.1)
Exalogic Infrastructure PSU Patching - Troubleshooting Guide (Doc ID 1590392.1)
Exalogic Infrastructure PSU Upgrade - ExaPatch Guide (Doc ID 1612143.1)

Thursday, August 7, 2014

Goldengate Filter data based on Before image and and after Value in Extract.

Links to this post
Below is example to filter data based on Existing value or before image in column and New updated value .

This is used in 11.2 OGG. In OGG 12C use @BEFORE to Get before image.

So X_COMMAND will appear as "REFRESH" If status change from Inactive to Active.

X_COMMAND=@IF (((@STRFIND(before.STATUS, "Inactive") > 0 ) and (@STRFIND(STATUS, "Active") >0 )) > 0 , "REFRESH", @CASE(@GETENV("GGHEADER", "OPTYPE"), "SQL COMPUPDATE", "UPDATE", "PK UPDATE", "UPDATE"))


You can also use this code for compare Exist and new value.
((@STRFIND(before.STATUS, "Inactive") > 0 ) and (@STRFIND(STATUS, "Active") >0 )) > 0 . 

Monday, June 30, 2014

Goldengate Filter data using Lookup Table and SQLEXEC .

Links to this post

This is true that You can use multiple SQLEXEC in Goldengate.

Below code shows How to filter data from SOURCE Table using Lookup table and get filter value dynamically .

This code check If Old value ORG_ID for source_table is exist in LOOKUP_PARAMETER_TAB.
to filter using lookup table ,I used FILTER and get lookup value from SQLEXEC.if value match it execute another SQLEXEC where it
execute stored procedure and Get dynamic parameters using @GETVAL function and pass to Procedure.
TABLE "INV"."SOURCE_TABLE",FILTER (ORG_ID = @GETVAL(lookup.XYZ)),SQLEXEC (ID lookup, QUERY "select distinct (MASTER_ORG_ID) XYZ from ABC.LOOKUP_PARAMETER_TAB mp  where mp.MASTER_ORG_ID= :p_organization_id_X", PARAMS (p_organization_id_X=ORG_ID),BEFOREFILTER , TRACE),SQLEXEC (ID ABC.PACKAGE_TO_EXEC.PROCEDURE_TO_EXEC,SPNAME ABC.PACKAGE_TO_EXEC.PROCEDURE_TO_EXEC,PARAMS (P_DBNAME=@GETENV("DBENVIRONMENT","DBNAME"),P_TRANSACTION_ID=@GETENV("TRANSACTION", "XID"),P_COMMAND=@CASE(@GETENV("GGHEADER", "OPTYPE"), "INSERT", "INSERT", "SQL COMPUPDATE", "UPDATE", "PK UPDATE", "UPDATE","DELETE","DELETE"),P_INVENT_ITEM_ID=INVENT_ITEM_ID,P_ORGANIZATION_ID=ORG_ID),AFTERFILTER, TRACE);

Tuesday, June 17, 2014

Goldengate Stored procedure parameter in SQLEXEC with GETENV

Links to this post

Here GG is extracting ABC.DUMMY table and while It does that, It also does following.

- check Where ORGANIZATION_ID =83,
- if true above execute XYZ.DUMMY_PKG (PACKAGE) to DUMMY_PROCEDURE_NAME (procedure),GG has bug for fully qualify Name of schema and Packagename as prefix.
-while executing above Procedure , It gets value run-time and pass value to procedure.

~database Name using GETENV("DBENVIRONMENT","DBNAME").
~Object Name
~Inventory_item_id (comes within procedure )
~Get Command type INSERT/UPDATE/DELETE ,again case function wrote to convert SQL COMPUPDATE / PKUPDATE to UPDATE string.
~Get Transaction XID of database.

TABLE ABC.DUMMY,WHERE (ORGANIZATION_ID = 83),SQLEXEC (ID XYZ.DUMMY_PKG.DUMMY_PROCEDURE_NAME,SPNAME XYZ.DUMMY_PKG.DUMMY_PROCEDURE_NAME,PARAMS (P_DBNAME=@
GETENV("DBENVIRONMENT","DBNAME"),P_OBJECT_NAME='DUMMY',P_INVENTORY_ITEM_ID=INVENTORY_ITEM_ID,P_ORGANIZATION_ID=ORGANIZATION_ID,P_COMMAND=@CASE(@GETENV("GGHEADER", "OPTYPE"), "INSERT", "INSERT", "SQL COMPUPDATE", "UPDATE", "PK UPDATE", "UPDATE","DELETE","DELETE"),P_TRANSACTION_ID=@GETENV("TRANSACTION", "XID")));

Friday, May 30, 2014

ExaLogic | List VM on Compute Node.

Links to this post

Part of ELLC Tool, Exapatch has variety of Commands that can be Handy for Admin also.

we can query Exapatch to see How many VMs are running on which Compute Nodes.

/exalogic-lctools/bin/exapatch -a listVMs
Logging to file /var/log/exapatch_20140529151107.log
Compute-Node: 10.10.100.101:
        000aaa00000600004995bfe5e81dcbdf (ExalogicControlOpsCenterPC1)
        000aaa0000060000d299f9c94afe41da (ExalogicControl)

Compute-Node: 10.10.100.102:
        000aaa00000600001f012b58d1fb85b6 (dummyvn02)
        000aaa0000060000cc95187b5e532439 (ExalogicControlOpsCenterPC2)

Compute-Node: 10.10.100.103:
        000aaa0000060000063eb61a02ae11c1 (dummyvn04)
        000aaa000006000021013dc081e56c98 (dummyvn18)

Compute-Node: 10.10.100.104:
        000aaa00000600006955f56fefc87683 (dummyvn01)

Compute-Node: 10.10.100.105:
        000aaa00000600007f2ee56ea19ed478 (dummyvn09)

Compute-Node: 10.10.100.106:
        000aaa0000060000268c78fb303c0e25 (dummyvn03)

Compute-Node: 10.10.100.107:
        000aaa00000600003056e4719030bef3 (dummyvn50)

Compute-Node: 10.10.100.108:
        000aaa000006000044490976c7f0c9a6 (dummyvn08)


The second Best use of Exapatch is get Version of Whole Exalogic Stack components.

/exalogic-lctools/bin/exapatch -a getVersion