Analysis and Workaround for the Solaris 10.2.0.3 Patchset Problem on VxFS Files

In the blog entry about the Solaris 10.2.0.3 patchset not functioning on VxFS, I reported that Metalink says the patchset does not work on VxFS. That is true. Since the Metalink notes have not been updated, I’ll blog a bit about what I’ve found out. Note, the Metalink note says not to use the patchset because of this bug. I am not here to fight Oracle support.

It turns out that what is happening is the Solaris porting group is now using an ioctl() that is not supported on VxFS files—but not calling the ioctl(2) directly. The bug results in an error stack a bit like this:

ORA-01501: CREATE DATABASE failed
ORA-00200: control file could not be created
ORA-00202: control file: ‘/some/path/control01.ctl’
ORA-27037: unable to obtain file status
SVR4 Error: 25: Inappropriate ioctl for device

The text in bug number 5747918 is nice enough to include the output of truss when the problem happens. The ioctl() is _ION. This is the ioctl(2) that is implemented within the directio(3C) library routine. No, don’t believe this developers.sun.com webpage when it refers to directio(3C) as a system call. It isn’t. However, they do provide an example of using the directio(3C) call in this small directio(3C) test program.

The Solaris directio(3C) call is used to push direct I/O onto a file. In the demonstration of the bug (5747918), the 10.2.0.3 patchset is trying to push direct I/O onto the file descriptor held on the control file stored in VxFS. That isn’t how you get direct I/O on VxFS. I wonder if this call to directio(3C) only happens if you have filesystemio_options=DirectIO|setall. That would make sense.

Workaround
If you use ODM on VxFS, this call to directio(3C) does not occur so you wont see the problem. Thanks to a reader comment on my blog and my age old friend still at Veritas (I mean Symantec) for verification that ODM works around the problem.

A Test Program

If you create a file in a VxFS mount called “foo”, like this:

$ dd if=/dev/zero of=foo bs=4096 count=16

And then compile and run the following small program, you will see the same problem Oracle 10.2.0.3 is exhibiting. The same program on UFS should work fine.

$ cat t.c
#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <stdio.h>
#include <sys/types.h>
#include <sys/fcntl.h>

main ()
{

int ret, handle;
handle = open (“./foo”, O_RDONLY);
if ((ret = directio (handle, DIRECTIO_ON)) < 0)
{
printf (“Failure : return code is: %d\n”, ret);
}
else
{
printf
(“The ioctl embedded in the directio(3C) call functions on the file.\n”);
}
} /* End */

Another Potential Workaround
If you want to test the rest of what 10.2.0.3 has to offer without ODM—even with a VxFS database—I think you should be able to explicitly set filesystemio_options=none and get around the problem. Be aware I have not tested that however. The worst thing that could happen is that setting filesystemio_options in this manner is indeed a workaround that would allow you to test the many other reasons you actually need 10.2.0.3!

If you find otherwise, please comment on the blog.