Oracle11g: Oracle Inventory On Shared Storage. Don’t Bother Trying To Install 11g RAC That Way.

A few days ago there was a thread on the oracle-l email list about adding nodes in an Oracle Database 10g Real Application Clusters environment. The original post showed a problem that Alex Gorbachev reports he’s only seen with shared Oracle Home installs. I found that odd because I’ve done dozens, upon dozens of RAC installs on shared Oracle Homes with both CFS and NFS and haven’t seen this error:

Remote 'UpdateNodeList' failed on node: 'af-xxx2'. Refer to
'/apps/oracle/oraInventory/logs/addNodeActions2007-07-08_09-36-12PM.log'
for details.
You can manually re-run the following command on the failed nodes after the
installation:
/apps/oracle/product/10.2/oui/bin/runInstaller -updateNodeList -noClusterEnabled
ORACLE_HOME=/apps/oracle/product/10.2 CLUSTER_NODES=af-xxx1,af-xxx2,af-xxx6
CRS=false "INVENTORY_LOCATION=/apps/oracle/oraInventory" LOCAL_NODE=
<node on which command is to be run>

I never have any problems with shared Oracle Home and I blog about the topic a lot as can be seen in in this list of posts. Nonetheless, Alex pointed out that the error has to do with the Oracle Inventory being on a shared filesystem. Another list participant followed up with the following comment about placing the inventory on a shared drive:

Sharing the oraInventory across nodes is not a good practice in my opinion. It runs counter to the whole concept of redundancy in an HA configuration and RAC was not written to support it.

Well, the Oracle Inventory is not a RAC concept, it is an Oracle Universal Installer concept, but I think I know what this poster was saying. However, the topic at hand is shared Oracle Home. When people use the term shared Oracle Home, they don’t mean shared ORACLE_BASE, they mean shared Oracle Home. Nonetheless, I have routinely shared the 10g inventory without problems, but then my software environments might not be as complex as those maintained by the poster of this comment.

Shared Inventory with Oracle Database 11g
No can do! Well, sort of. Today I was installing 11g RAC on one of my RHEL 4 x86 clusters. In the fine form of not practicing what I preach, I mistakenly pointed Oracle Universal Installer to a shared location (NFS) for the inventory when I was installing CRS. I got CRS installed just fine on 2 nodes and proceeded to install the database with the RAC option. It didn’t take long for OUI to complain as follows:

shared_home_11g.jpg

Ugh. This is just a test cluster that I need to set up quick and dirty. So I figured I’d just change the contents of /etc/oraInst.loc to point to some new non-shared location-aren’t I crafty. Well, that got me past the error, but without an inventory with CRS in it, Oracle11g OUI does not detect the cluster during the database install! No node selection screen, no RAC.

I proceeded to blow away all the CRS stuff (ORA_CRS_HOME, inittab entries, /etc/oracle/* and /etc/oraInst.loc) and reinstalled CRS using a non-shared locale for the inventory. The CRS install went fine and subsequently OUI detected the cluster when I went to install the database.

This is a significant change from 10g where the inventory content regarding CRS was not needed for anything. With 10g, the cluster is detected based on what /etc/oracle/ocr.loc tells OUI.

Summary
Shared Oracle Home is an option, shared Oracle Home means shared Oracle Home not shared Oracle Inventory. Oracle11g enforces this best practice nicely!

7 Responses to “Oracle11g: Oracle Inventory On Shared Storage. Don’t Bother Trying To Install 11g RAC That Way.”


  1. 1 Alex Gorbachev July 25, 2007 at 9:07 pm

    Hi Kevin,

    I should have said with shared locations for Oracle inventory. Thanks for correcting this.

    Btw, instead of cleaning all up you could have copied inventory to another location and change it in /etc/oraInst.loc as well as… I think one file in every Oracle home (can’t recall exactly which).

    Now back to catching up with your blog that has blown up while I haven’t paid attention for few days. :)

    Cheers,
    Alex

  2. 2 Jen Black March 10, 2008 at 2:36 pm

    What are your thoughts on using CDSLs to make the 11g directories node specific, but still live on a shared clustered file system.

    We are currently running a shared 9i on a Polyserve clustered file system, and are using CDSL directories for many of the our configuration files and logs. We are planning our migration to 11g and would like to consider restructuring our RAC cluster to use non-shared Oracle Homes, but wanted to get an expert opinion on using CDSLs to still take advantage of our shared disks.

    Thanks for your time,

    Jen

  3. 3 kevinclosson March 10, 2008 at 4:40 pm

    Hi Jen,

    CDSLs were important back in the 9i days. I see no need for them with 10g later. Shared Home and CDSL are not synonymous these days.

  4. 4 Jen Black March 11, 2008 at 3:17 pm

    We need a solution that allows us to continue using the shared storage that we currently have available through Polyserve. We have been told that getting local storage for our 50+ linux Oracle servers is not an option, and we need an alternative plan.

    So how would you suggest setting up these separate ORACLE HOME locations on shared disks? Our thought was to use the CDSLs and create a CDSL from the ORACLE_BASE directory with directories for each node under there, and each node directory would hold a “local” ORACLE_HOME, CRS_HOME, ASM_HOME, central inventory, etc, etc…

    In my head it seems like this will work, but there isn’t much chatter about this topic on the forums, so I wanted to see how others are setting their systems up.

    Thanks,

    Jen

  5. 5 kevinclosson March 11, 2008 at 4:34 pm

    Hi Jen,

    Readers sure would be interested in hearing just how big this setup is of yours. A significantly high percentage of Oracle Linux deployments are so small that readers must wonder why you don’t just back it up onto a few USB sticks and fidget away all the PolyServe stuff and restore from USB.

    :-)

    That aside, I see what you are saying. You want to use shared storage but make it so Oracle is not aware it is shared storage. I can see the value in that…definitely.

    You could have a OFA head like /u01/app/oracle and have all assets under there by CDSLs starting with the directories. That would be simple and would work…

    CDSLs are very helpful…I wish NFS supported them.

  6. 6 randy March 15, 2010 at 9:07 pm

    Well, I have an 11g RAC with one inventory and am in the process of creating another today. I simply click ‘local installation’ from the installer then move to the create database.

    It’s been a bit since I created the first setup so I may be forgetting something rather relevant. I’ll update if it’s different.

  7. 7 randy March 15, 2010 at 9:21 pm

    Woop, sorry, ignore my last – I did split the inventory. moderate it to the bin. Not sure what I was thinking but what I wrote is not accurate. I think I hit this point of fault (click local install) forgetting that it didn’t work. Sorry. \r


Leave a Reply

Please log in using one of these methods to post your comment:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s




EMC Employee Disclaimer

The opinions and interests expressed on EMC employee blogs are the employees' own and do not necessarily represent EMC's positions, strategies or views. EMC makes no representation or warranties about employee blogs or the accuracy or reliability of such blogs. When you access employee blogs, even though they may contain the EMC logo and content regarding EMC products and services, employee blogs are independent of EMC and EMC does not control their content or operation. In addition, a link to a blog does not mean that EMC endorses that blog or has responsibility for its content or use.

This disclaimer was put into place on March 23, 2011.

Enter your email address to follow this blog and receive notifications of new posts by email.

Join 1,719 other followers

Oracle ACE Program Status

Click It

website metrics

Fond Memories

Copyright

All content is © Kevin Closson and "Kevin Closson's Blog: Platforms, Databases, and Storage", 2006-2013. Unauthorized use and/or duplication of this material without express and written permission from this blog’s author and/or owner is strictly prohibited. Excerpts and links may be used, provided that full and clear credit is given to Kevin Closson and Kevin Closson's Blog: Platforms, Databases, and Storage with appropriate and specific direction to the original content.

Follow

Get every new post delivered to your Inbox.

Join 1,719 other followers

%d bloggers like this: