Re: Announcing RSX - non exec stack/heap module

[Paul Starzetz <paul () starzetz de>]

Crispin Cowan wrote:

> > It is not very hard to mmap the libc code as non-executable are into
> > main memory. After the regular programm code jumps into some libc
> > function, we can check in the gp() handler if the gp fault resulted from
> > jumping into the libc area by a ret (the target address should still be
> > on the stack) or by a regular call/jmp instruction.
>
> That's an interesting idea, but the performance penalty will be substantial.
> You will pay for (at least) two system calls per library call.  In early
> StackGuard research, we experimented with hardware protection methods that
> imposed 2 syscalls per function call, and the overhead was between 500% and
> 10,000%, which just isn't realistic for prodution use.

Yes and no! I have written such a code for rsx. The overhead is more
precisely 1x page fault and 1x general protection fault + the emulation
code (jmp/call/ret), which is not equal to 2x syscall + emulation, but
indeed of similar magnitude. However it works.

Note that a simpler protection (but maybe not so effective) can be done
by means of ld.so. What does people mean if they talk about
ret-into-libc? I assume we speak about ret-into-plt, where libc is
linked to, because this is the only information an attacker can obtain
by analyzing the binary. Libc can be mmaped at some random location,
right?

So now assume we doesn't link the libc-plt to the real libc location -
instead we link it to a intermediate random glue code piece. The
protection arises from the fact that it is hard to guess the location of
this intermediate glue segment (and it is hard to guess the real libc
vma too). So the attacker neither easily jump into some offset (skipping
the ret checking code) in the glue code, nor directly jump into some
real libc function. The addresses of the glue code and libc should
change with every execve() and fork() (to prevent binary search...).

The glue code does now the similar thing that a pf() or gp() hook would
- look at the stack to switch between the cases 1) call from legal .text
code into plt or 2) ret from buffer overflow into plt.

This again does not protect against ret-into-text where some libc
function (via plt) is called. But maybe one can make this harder using
another trick. I think this case would also have a clear signature on
the stack. (hm what about jumping at libc-call-in-text - 4, 8, ...
offset?)

Paul.