Comments on: Oracle on Opteron with Linux-The NUMA Angle (Part VI). Introducing Cyclops. http://kevinclosson.wordpress.com/2007/04/05/oracle-on-opteron-with-linux-the-numa-angle-part-vi-introducing-cyclops/ Oracle-related Platform, Storage and Clustering Topics (with the occasional rant) Mon, 13 Oct 2008 15:36:58 +0000 http://wordpress.org/?v=MU hourly 1 By: kevinclosson http://kevinclosson.wordpress.com/2007/04/05/oracle-on-opteron-with-linux-the-numa-angle-part-vi-introducing-cyclops/#comment-32651 kevinclosson Tue, 29 Apr 2008 16:44:17 +0000 http://kevinclosson.wordpress.com/2007/04/05/oracle-on-opteron-with-linux-the-numa-angle-part-vi-introducing-cyclops/#comment-32651 Hi Charles, It turns out that mention of a particular setting in a Validated Configuration doesn't imply a support requirement. That aside, I really can't comment on anything NUMA related in RHEL3 since I abandoned that long before I even got that DL585...which I no longer have since it is in an HP lab and I've moved on to Oracle... Hi Charles,

It turns out that mention of a particular setting in a Validated Configuration doesn’t imply a support requirement. That aside, I really can’t comment on anything NUMA related in RHEL3 since I abandoned that long before I even got that DL585…which I no longer have since it is in an HP lab and I’ve moved on to Oracle…

]]> By: kevinclosson http://kevinclosson.wordpress.com/2007/04/05/oracle-on-opteron-with-linux-the-numa-angle-part-vi-introducing-cyclops/#comment-32652 kevinclosson Tue, 29 Apr 2008 16:42:19 +0000 http://kevinclosson.wordpress.com/2007/04/05/oracle-on-opteron-with-linux-the-numa-angle-part-vi-introducing-cyclops/#comment-32652 Hi Charles, It turns out that mention of a particular setting in a Validated Configuration doesn't imply a support requirement. That aside, I really can't comment on anything NUMA related in RHEL3 since I abandoned that long before I even got that DL585...which I no longer have since it is in an HP lab and I've moved on to Oracle... Hi Charles,

It turns out that mention of a particular setting in a Validated Configuration doesn’t imply a support requirement. That aside, I really can’t comment on anything NUMA related in RHEL3 since I abandoned that long before I even got that DL585…which I no longer have since it is in an HP lab and I’ve moved on to Oracle…

]]> By: Charles C. Figueiredo http://kevinclosson.wordpress.com/2007/04/05/oracle-on-opteron-with-linux-the-numa-angle-part-vi-introducing-cyclops/#comment-32650 Charles C. Figueiredo Mon, 28 Apr 2008 21:44:21 +0000 http://kevinclosson.wordpress.com/2007/04/05/oracle-on-opteron-with-linux-the-numa-angle-part-vi-introducing-cyclops/#comment-32650 This is an excellent and very informative thread of blog, Kevin. Thanks! I'm curious, have you done any testing or have any insights into vm.numa_memory_allocator? There is little info available on this tunable that I've found so far, and before I start sifting through kernel source, I wanted to throw this out there. This was added to Update 7 of RHEL 3 (yes, I know, a bit dated at this point), but since Oracle recommends disabling NUMA on DL 585 gear, and I'm using that with RHEL 3 U8+, I'm curious about no NUMA vs. leaving it enabled in the BIOS and using this tunable. I'm wondering if it helps avoid the cyclops scenario... Thanks, -charles. This is an excellent and very informative thread of blog, Kevin. Thanks!

I’m curious, have you done any testing or have any insights into vm.numa_memory_allocator? There is little info available on this tunable that I’ve found so far, and before I start sifting through kernel source, I wanted to throw this out there. This was added to Update 7 of RHEL 3 (yes, I know, a bit dated at this point), but since Oracle recommends disabling NUMA on DL 585 gear, and I’m using that with RHEL 3 U8+, I’m curious about no NUMA vs. leaving it enabled in the BIOS and using this tunable. I’m wondering if it helps avoid the cyclops scenario…

Thanks,

-charles.

]]> By: Oracle11g Automatic Memory Management - Part III. A NUMA Issue. « Kevin Closson’s Oracle Blog: Platform, Storage & Clustering Topics Related to Oracle Databases http://kevinclosson.wordpress.com/2007/04/05/oracle-on-opteron-with-linux-the-numa-angle-part-vi-introducing-cyclops/#comment-22888 Oracle11g Automatic Memory Management - Part III. A NUMA Issue. « Kevin Closson’s Oracle Blog: Platform, Storage & Clustering Topics Related to Oracle Databases Fri, 24 Aug 2007 22:53:57 +0000 http://kevinclosson.wordpress.com/2007/04/05/oracle-on-opteron-with-linux-the-numa-angle-part-vi-introducing-cyclops/#comment-22888 [...] Now I’m glad I did that series about Oracle on Linux, The NUMA Angle.  In my post about the the difference between NUMA and SUMA and “Cyclops”, I shared a lot of information about the dynamics of Oracle running with all the SGA allocated from [...] [...] Now I’m glad I did that series about Oracle on Linux, The NUMA Angle.  In my post about the the difference between NUMA and SUMA and “Cyclops”, I shared a lot of information about the dynamics of Oracle running with all the SGA allocated from [...]

]]> By: Geoff http://kevinclosson.wordpress.com/2007/04/05/oracle-on-opteron-with-linux-the-numa-angle-part-vi-introducing-cyclops/#comment-7975 Geoff Mon, 23 Apr 2007 23:43:14 +0000 http://kevinclosson.wordpress.com/2007/04/05/oracle-on-opteron-with-linux-the-numa-angle-part-vi-introducing-cyclops/#comment-7975 Forgot to add: we're running Oracle 10.2.0.2 on SUSE Linux 9 SP2. Forgot to add: we’re running Oracle 10.2.0.2 on SUSE Linux 9 SP2.

]]> By: Geoff http://kevinclosson.wordpress.com/2007/04/05/oracle-on-opteron-with-linux-the-numa-angle-part-vi-introducing-cyclops/#comment-7974 Geoff Mon, 23 Apr 2007 23:38:55 +0000 http://kevinclosson.wordpress.com/2007/04/05/oracle-on-opteron-with-linux-the-numa-angle-part-vi-introducing-cyclops/#comment-7974 We have 3 DL585 machines each with 4 2.4 Ghz opterons and 32GB ram. We had no idea of the recommendation to turn off the NUMA setting at the bios level and have therefore been running in standard NUMA mode. Apparently using huge pages has saved us. The machine most heavily burdened is running along quite happily. Issues: One database instance on a separate machine will simply not use the huge pages. Another one was using it and stopped after reboot. We are not sure why. During research, we stumbled here (luckily). We have been looking at the "validated configuration" on Oracle's site and will not disable NUMA after reading your blog. Thanks! We have 3 DL585 machines each with 4 2.4 Ghz opterons and 32GB ram. We had no idea of the recommendation to turn off the NUMA setting at the bios level and have therefore been running in standard NUMA mode. Apparently using huge pages has saved us. The machine most heavily burdened is running along quite happily.

Issues: One database instance on a separate machine will simply not use the huge pages. Another one was using it and stopped after reboot. We are not sure why. During research, we stumbled here (luckily). We have been looking at the “validated configuration” on Oracle’s site and will not disable NUMA after reading your blog. Thanks!

]]> By: Herta http://kevinclosson.wordpress.com/2007/04/05/oracle-on-opteron-with-linux-the-numa-angle-part-vi-introducing-cyclops/#comment-7548 Herta Thu, 19 Apr 2007 13:03:21 +0000 http://kevinclosson.wordpress.com/2007/04/05/oracle-on-opteron-with-linux-the-numa-angle-part-vi-introducing-cyclops/#comment-7548 Good series of articles. Thanks. Looking forward to the next one. Any chance of adding RAC to the picture? I like the way in which you start up a single database, but don't see how I might apply that to a RAC environment running multiple databases. As to your concern about Oracle/HP recommending turning NUMA off: the page you refer to (http://www.oracle.com/technology/tech/linux/validated-configurations/html/vc_dl585_4node_fas9xx.html?rssid=valconfig) simply shows a "*validated* configuration". It's not a recommendation. Kind regards, Herta Good series of articles. Thanks. Looking forward to the next one. Any chance of adding RAC to the picture? I like the way in which you start up a single database, but don’t see how I might apply that to a RAC environment running multiple databases.

As to your concern about Oracle/HP recommending turning NUMA off: the page you refer to (http://www.oracle.com/technology/tech/linux/validated-configurations/html/vc_dl585_4node_fas9xx.html?rssid=valconfig) simply shows a “*validated* configuration”. It’s not a recommendation.

Kind regards,

Herta

]]>