Trusted process on an untrusted machine?

[frantzen () EXPERT CC PURDUE EDU (Mike Frantzen)]

[Warning:  long and fairly technical]

> You can't run a trusted client on an untrusted machine.
> A hostile user has complete access to the machine state, including
> memory, stack, and register values. He/she can log all communication
> between the client and the server and create a fake client that
> duplicates the "authentification" procedure of the real client, but
> without displaying ads.

A PhD friend of mine (Rick Kennell) and I spent _alot_ of time discussing
how to establish a trusted client on an untrusted machine.  His idea was a
customized kernel that would fetch a kernel module off the net at boot
time.  The kernel module's purpose would be to verify that the running
kernel was intact and had not been subverted (only the known good kernel
was running without any unknown modules).  The kernel module would
traverse memory and calculate a checksum (maybe of all the memory mapped
in the page tables).  The module would then send the checksum back to
the central authority who would grant credentials to the untrusted machine
if the checksum matched.  This worked off the assumption that the kernel
could be trusted to uphold the security of the system.
Keep in mind that it would make liberal use of digital signatures and
encrypted communication between the client and the central auth.

Some of ways an attacker could bypass this protection:
1)  Capture the checksum and just replay it.
    Solution:  Many different kernel modules that are each a slight variation
               off each other.  They each return a different checksum.
2)  Virtualized processor.  Use something like VMWARE to run the kernel in
    a virtual processor and take the checksum/key as soon as it is calculated.
    Solution:  There are operations that must be simulated in a virtualized
               processor.  Do alot of them and make the central authority
               require the checksum/key be returned in a short amount of time.
               (Example operation would be to add and remove TLB entries)
3)  Full software processor simulation.  Like the BACHS simulator.
    Solution:  same as that of a virtualized processor.
4)  Kernel wars!  A SMP machine that boots an untrusted kernel.  Have
    the APIC vector the attacking processor the timer interrupt then vector all
    other interrupts to the 'good' proc.  The attacking proc then destroys
    the MP configuration table so the 'good' proc doesnt know it is an MP
    system.  The attacking proc then tries to take over the system after X
    amount of time and steal the checksum/key.
    [It has been a few months since I've looked at x86 SMP]
    Solution:  There should be a LOCK pin on most processors that locks the
               memory bus.  The kernel module can lock the bus and proceed to
               zero out all memory not used by the good kernels page tables.
               [Thus trashing the minikernel of the attacking processor]
               Or it could hope the APIC is at the known memory address and try
               to detect it.
5)  Hardware bus snooping.  A PCI device on the memory bus to grab the
    checksum/key then give the key to another malicious machine.
    Solution:  ???

I've left out (and forgotten) many implementation details.  And this makes
the assumption that a kernel could be trusted to maintain security once
running.

The purpose for the idea was to take the several hundred machines on campus
that run linux and turn them into a compute cluster at night.  Unfortunately
they would need write permissions to do any useful simulation work and I'm
sure we all know how (in)secure linux boxes get when the student body can
park their collective asses in front of one.

later,
.mike

---
        "We reject kings, presidents, and voting;
        we believe in rough consensus and running code."
                        --  Dr. David D. Clark   1992