Using Linux Processor Affinity To Beat The Oracle Licensing Police.

In my recent blog entry about LGWR processing I set up a proof case that included establishing hard processor affinity for LGWR and other processors in order to prove a point. I’ve received a few emails about what tool I used to affinity running processes to specific CPUs. The answer is not what you might presume. I did not use the taskset(1) command that ships with 2.6 kernel distributions because I’ve seen some irregularities with that tool in the past. I prefer to use a bit of my own code that uses sched_setaffinity(2). I’d provide that code as a reference but it is a mitt of a mess and does a lot of other stuff that doesn’t pertain to affinity.

The recent Oracle single socket Xeon “Cloverdale” TPC-C used hard affinity for various purposes. The FDR pointed to by that URL has a hard affinity program (affinity.c) listing that is very succinct. If you want to give hard affinity a play I recommend you cut and paste from that FDR, compile and give it a go. For instance, presume you have compiled affinity.c and wish to round up all the processes for a specific $ORACLE_SID and bind them to CPU 0 you could do something like the following. Note, unless the oracle user has CAP_SYS_NICE capability, you’ll have to make the affinity program setuid root in order to play with sched_setaffinity(2). Such as:

$ cc -o aff affinity.c
$ sudo chmod 6755 aff
$ TARGET_SID=PROD
$ ps -ef | grep ${TARGET_SID} | grep -v grep | awk '{ print $2 }' | while read PID
do
	./aff $PID 1
done

Oracle Licensing Again
I like nixCraft as a resource, but not when it comes to giving advice about Oracle licensing. What’s that got to do with hard affinity? I aimed to offer you readers more information about Linux hard affinity (note to self, blog sometime about the difference between hard and soft affinity if a reader ever asks) so I found this nixCraft article. It does offer more information about hard affinity scheduling, but oh so much more as well. In that page, the author is quoting an unspecified source as follows (quoted herein verbatim):

The scheduler attempts to keep processes on the same CPU as long as practical for performance reasons. Therefore, forcing a specific CPU affinity is useful only in certain applications. For example, application such as Oracle (ERP apps) use # of cpus per instance licensed. You can bound Oracle to specific CPU to avoid license problem. This is a really useful on large server having 4 or 8 CPUS

This quote is talking about how modern *nix schedulers implement soft-affinity (aka cache affinity) and thus handcrafting hard affinity is something better left to advanced situations. That’s fine and dandy, but pardon me for a moment because I’m choking on a hairball over that bit about ERP licensing vis a vis CPU count. That is a complete farce. Please do not try to negotiate a 2 CPU license on an 8 core system where you plan to pin Oracle E-Business Suite processes to the first 2 cores of the system using sched_setaffinity(2). I know I’ve provided a quote of a quote, but that idea is about as brain-dead as they come. It would work, but in a license audit it would not fly.

More About Hard Affinity
Be aware that if you run 100% of a given workload under hard affinity to CPU 0, for instance, that is not the same as running on a single CPU box. The other processors are still online and servicing device interrupts. For instance, if LGWR is running on CPU 0, the hardware interrupt for any given log file parallel write can be serviced by any of the online CPUs in the box. If, for instance, LGWR is pinned to CPU 0 and a LGWR flush I/O interrupt is handled by, say, CPU 3, you are getting both a blessing and a curse. The blessing is that LGWR’s CPU didn’t have to handle the noise of the interrupt. The curse is that your workload is getting “help” from CPUs you didn’t really want in the mix-that is if you are hoping for total isolation of the workload to CPU 0. This is not the same as sophisticated OS partitioning offerings out there that corral event he hardware interrupts to the CPUs in the partition. That is an entirely different topic.

Summary
No, binding a workload to CPUs with sched_setaffinity(2) is not the same as running on the same number of discrete processors. Whoever authored the words “You can bound Oracle to specific CPU to avoid license problem” is likely neither an English professor nor a contract lawyer. I’d ignore that “advice.” I wonder what “problem” the author was referring to? Is legal use of a software product a “problem?” All that aside, hard affinity can play an important role in affording ample processor bandwidth to key Oracle background processes. There were musings in the comment thread on my LGWR processing post about dedicating an entire CPU to LGWR although I have not followed up, that is certainly not the approach. I’ll need to blog about a possible best practice in this concept area soon.