Professional Documents
Culture Documents
Applies to:
SPARC SuperCluster T4-4 Full Rack - Version All Versions and later
SPARC SuperCluster T4-4 - Version All Versions and later
Exadata X3-2 Hardware - Version All Versions and later
Oracle Exadata Storage Server Software - Version 11.2.3.2.0 to 11.2.3.2.0 [Release 11.2]
Exadata X3-2 Quarter Rack - Version All Versions and later
Information in this document applies to any platform.
Exadata Storage Server gives a warning of unsupported firmware during boot if Flash F40 cards with
different firmware versions are detected.
Symptoms
Exadata Storage Server with image 11.2.3.2.0 gives a warning of unsupported firmware during boot if Flash
F40 cards with different firmware versions are detected, for example after a F40 card replacement:
[WARNING] The hardware and firmware are not supported. See details below
[PCISlot:CardId:MinCardFw:CardModel:DOMId:OSDevice:MinDOMfw:DOMCount]
Requires:
Found:
[WARNING] The hardware and firmware are not supported. See details above
Cause
This issue is caused because Flash F40 firmware was updated in the factory on the Exadata prior to
shipping, with firmware 08.05.01.00 and FMOD firmware UI39 (SW1.1), and a F40 card was installed with
older firmware 06.05.10.00 and FMOD firmware TI35 (SW1.0). Exadata image 11.2.3.2.0 supports running
the later firmware but does not have the firmware files in it to upgrade the older card. The warning is
generated due to having different firmware on different F40 cards in the same system.
Solution
Update Exadata Storage Server image to v11.2.3.2.1 or later that includes Flash F40 card firmware
08.05.01.00 and FMOD firmware UI39 (SW1.1). Refer to MOS Note 888828.1 for details.
If this is not possible at this time, the following solution may be used:
The Flash F40 card with older firmware should be upgraded to SW1.1 manually using the firmware and
'ddcli' utility contained in patch 16005846. This should be done on all down-rev Flash F40 cards so that the
firmware matches on all 4 Flash F40 cards in the Exadata Storage Server. The cell services will need to be
offlined first to ensure the system is not doing I/O to the Flash F40 card being upgraded.
2. Copy the patch to the Exadata Storage Server, make a directory "p16005846" and unzip the patch file
into this directory.
4. Offline the Exadata Storage Server from ASM prior to flash updating the Flash F40 card.
a. By default, ASM drops a disk shortly after it is taken offline; however, you can set the
DISK_REPAIR_TIME attribute to prevent this operation by specifying a time interval to
repair the disk and bring it back online. The default DISK_REPAIR_TIME attribute value of
3.6h should be adequate for this procedure.
i. To check repair times for all mounted disk groups - log into the ASM instance on the
DB node and perform the following query:
SQL> select dg.name,a.value from v$asm_diskgroup
dg, v$asm_attribute a where dg.group_number=a.group_number and
a.name='disk_repair_time';
ii. The disks should be offline for no more than 1 hour for this procedure, so if you are
not set for the 3.6h default value and are set for less than 1 hour, then adjust the
parameter by issuing the command below as an example:
b. Next you will need to check if ASM will be OK if the grid disks go OFFLINE. The following
command should return 'Yes' for the grid disks being listed:
c. If one or more disks return asmdeactivationoutcome='No', you should wait for some time and
repeat step #2. Once all disks return return asmdeactivationoutcome='Yes', you can proceed
with taking the griddisk offline in the next step.
Note: Taking the storage server offline when one or more grid disks return
asmdeactivationoutcome='No' will cause Oracle ASM to dismount the affected disk group,
causing the databases to shut down abruptly.
* Please note - This action could take 10 minutes or longer depending on activity. It is very
important to make sure you were able to offline all the disks successfully before shutting
down the cell services. Inactivating the grid disks will automatically OFFLINE the disks in
the ASM instance.
e. Confirm that the griddisks are now offline by performing the following actions:
i. Execute the command below and the output should show asmmodestatus=UNUSED
and asmdeactivationoutcome=Yes for all griddisks once the disks are offline in ASM.
Only then is it safe to proceed:
( there has also been a reported case of asmmodestatus= OFFLINE: Means Oracle
ASM has taken this grid disk offline. This status is also fine and can proceed with
remaining instructions)
****************************************************************************
LSI Corporation WarpDrive Management Utility
Version 107.00.00.05 (2012.09.07)
Copyright (c) 2011 LSI Corporation. All Rights Reserved.
****************************************************************************
Update the controller ID which has firmware 06.05.10.00 using the format "ddcli -c <ID>
-updatepkg <flash binary package file>". This command supports the upgrade of only the firmware
package. If the current firmware package version on the selected card is higher than the specified
firmware package version, the command returns an error.
In the example above, this is controller ID 1:
****************************************************************************
LSI Corporation WarpDrive Management Utility
Version 107.00.00.05 (2012.09.07)
Copyright (c) 2011 LSI Corporation. All Rights Reserved.
****************************************************************************
LSI WarpDrive Management Utility: Package Type: 0x7, PCI SubSystem ID: 0x581
LSI WarpDrive Management Utility: Please wait. Package update is in progress..
.
HBA Firmware Download...
Verifying Download...
Resetting Adapter...
If a firmware download fails on any of the card components, the process terminates.
If the firmware update fails, open a SR or reengage the SR owner if an SR is already opened.
. Repeat step 6 for each downrev Flash F40 card identified in step 5.
The firmware profile check should now pass during boot without warning, returning SUCCESS on
the console. This can be confirmed manually, when the system comes back online using the
following command:
Once the Exadata Storage Server comes back online the cell services will start up automatically,
however you will need to reactive the griddisks:
Verify all grid disks have been successfully put online using the following command:
Wait until asmmodestatus is ONLINE for all grid disks. Each disk will go to a 'SYNCING' state first
then 'ONLINE'. The following is an example of the output:
DATA_CD_00_dm01cel01 ONLINE
DATA_CD_01_dm01cel01 SYNCING
DATA_CD_02_dm01cel01 OFFLINE
DATA_CD_03_dm01cel01 OFFLINE
DATA_CD_04_dm01cel01 OFFLINE
DATA_CD_05_dm01cel01 OFFLINE
DATA_CD_06_dm01cel01 OFFLINE
DATA_CD_07_dm01cel01 OFFLINE
DATA_CD_08_dm01cel01 OFFLINE
DATA_CD_09_dm01cel01 OFFLINE
DATA_CD_10_dm01cel01 OFFLINE
DATA_CD_11_dm01cel01 OFFLINE
Oracle ASM synchronization is only complete when all grid disks show asmmodestatus=ONLINE.
(Please note: this operation uses Fast Mirror Resync operation - which does not trigger an ASM
rebalance. The Resync operation restores only the extents that would have been written while the
disk was offline.)
This firmware should only be installed as necessary. If another Exadata Storage Server requires it,
then before repeating the procedure and taking the next Exadata Storage Server offline, Oracle ASM
synchronization must complete on the restarted Oracle Exadata Storage Server. If synchronization is
not complete, then the check performed on another storage server will fail. The following is an
example of the output:
. The patch files can be removed from the Exadata Storage Server.
.
REFERENCES:
Sun Flash Accelerator F40 PCIe Card Documentation Library contains Service instructions for
firmware upgrade in the User's Guide - http://docs.oracle.com/cd/E29748_01/index.html