Re: Solaris 7 x86 lpset exploit.

[Casper.Dik () HOLLAND SUN COM (Casper Dik)]

> > >    set noexec_user_stack = 1
>
> > [...], there is a reason, why SUN does enable stack execution by
> > default, if i am correctly informed this is due to some fortran or
> > rare/old compiler issue, and might break some fortran or other alien
> > language code...
>
> It'll also break gcc's nested function support, since it's implemented
> with stack trampolines.  (It doesn't *have* to be; in principle
> function pointers could be widened to carry the same information.  But
> doing that would break function-pointer compatability with code
> compiled with other compilers...not to mention meaning that most
> function pointers would carry a bunch of unnecessary-for-them extra
> data around.  Another possible way around it would be to cause gcc to
> keep part of the stack in the data segment, out of what the kernel
> thinks of as the stack, and have it do its trampolines there.  This
> runs into big problems with setjmp and other nonlocal exits, and
> possibly with signal handlers as well.)

It's wrong to say it affects all nested functions; it only affects
nested functions passed as parameter.

In current gcc releases this has been fixed.  The trampoline code
now calls "mprotect()" after putting a trampoline on the stack.

Casper