"If you can't see anything because you can't get the source code... well, who knows what they might be hiding?"
Actually, it's worse than that. Even if you can review the source code, you STILL won't know for sure. As Ken Thompson put it three decades ago, "no amount of source-level verification or scrutiny will protect you from using untrusted code."[1] This is because the tools we all use to process or transform source code (e.g., already-compiled-to-binary compilers, assemblers, loaders, etc.) may already contain secret back-doors. Ditto for even lower-level tools like hardware microcode.
While open-source software tools -- and open hardware too -- are much less likely to have secret back-doors embedded in them, there are no 100% guarantees. Ultimately we have little choice but to trust the people who create, package, and distribute all the software and hardware we use but didn't create ourselves from scratch.
--
EDITS: Added last paragraph. Made minor edits to second paragraph so it more accurately conveys my thoughts.
It's not quite as bad as it sounds: I don't think even the NSA has people smart enough to insert code into a compiler which will add backdoors to software which didn't yet exist when the compiler was written.
If you're paranoid, use software written in the past 5 years and compile it on a system running FreeBSD 4.1 off a CD which you've had in your closet for the past decade.
> I don't think even the NSA has people smart enough to insert code into a compiler which will add backdoors to software which didn't yet exist when the compiler was written.
I doubt such a thing is possible given the current state of the art in program analysis. Even if it were possible, it's hard to imagine it being undetectable (e.g. it would probably cause massive slowdowns in compilation).
> If you're paranoid, use software written in the past 5 years and compile it on a system running FreeBSD 4.1 off a CD which you've had in your closet for the past decade.
If you have a trusted compiler, you can bootstrap any non-trusted one [1]. That said, I suspect implementing diverse double-compiling to build any nontrivial system would take significant work.
I don't think even the NSA has people smart enough to insert code into a compiler which will add backdoors to software
Really? Why wouldn't they? That's one of the most valuable attack vectors, precisely because it's so difficult.
I wouldn't have guessed the NSA had people smart enough to break Windows Update's encryption using a brand-new cryptographic technique that also required several hundred thousand dollars of machine time to execute, but it happened nonetheless.
Deterministic builds do more than merely defend against the Trusting Trust attack (what you and cperciva are dismissing as a flight of fancy equivalent to worrying about megabit-scale quantum computers in 2013) --- they also prevent compromised open-source binaries, a much more serious and realistic attack vector.
So we need to start implementing deterministic builds into every major open source project if we're even pretending like we care about putting up resistance to what's going on. If Tor browser can do it in just a few weekends, then so can we for mainstream Firefox, and hopefully eventually Ubuntu.
> I wouldn't have guessed the NSA had people smart enough to break Windows Update's encryption using a brand-new cryptographic technique that also required several hundred thousand dollars of machine time to execute, but it happened nonetheless.
That's rather unsettling. Do you have any further information on this?
Why are you finding it unsettling? I think that's exactly what the NSA do: stay ahead of everyone else and take advantage of what they know. In this case, a different MD5 collision attack technique was invented by Marc Stevens at about the same time frame, so you couldn't even say that [whoever wrote Flame] was ahead by a lot.
I have found it more interesting that they knew about the Microsoft design errors that they exploited to break the update mechanism. And, of course, wondered if the design errors were not forced.
Is the idea behind deterministic builds that you build the software on your own trusted system and then compare a hash of the binary with the hash the original builders got for their build? Thus you know that the binary you have must not contain any backdoors (or both binaries do)?
I'm just trying to understand how it would work.
How are deterministic builds better than distributing binaries with hashes for verification? Just because I don't have to trust the original author's compiler?
How are deterministic builds better than distributing binaries with hashes for verification? Just because I don't have to trust the original author's compiler?
Indeed, that's precisely why it's such an important protection mechanism. But it's about more than just not having to trust the original author's compiler. The original author might maliciously copy-paste some additional source code into the build process just before compilation.
Deterministic builds are a way for any of us to download any source code, build it, and verify that all of us are using a binary derived from exactly that source code, and nothing else.
Ah, I get it. Instead of relying on one guy, I can see that lots of different people who don't trust each other all agree that the binary hash should be X. Lots of people unknown to each other are unable to conspire to backdoor a binary.
Cool. Thanks for the explanation.
My first thought is that the next attack will be analogous to fake Amazon reviews. "This binary with the hash e99a12d388afa2fa5fdde8ed3bcbe055 gets five stars! After using 51fc8eff10b2fccec9890fe5d1b0cfd9 for years, I've realized that e99a12 is far superior in speed and robustness. I recommend you upgrade today!"
Really? Why wouldn't they? That's one of the most valuable attack vectors, precisely because it's so difficult.
You're assuming that because the NSA would like to have such a capability (as would most programmers), that they actually do, because they have abundant money to throw around. Likewise, I'm sure the Air Force, Army etc. would all like to have antigravity generators, and they have abundant money to throw at the problem, so they must have them, right?
Of course not. 'A compiler that adds backdoors to software' implies a compiler that knows which routines are for security and which are not. How exactly is it to distinguish between
get_string(super_sekrit_password)
and
get_string(pretty_background_color)
for example? What if the background color is actually a way of revealing hidden messages, do you think it can see that coming? You're asking for a system which not only inserts unwanted code seamlessly into an application, but which can actually model the intention of the programmer and make decisions about how to compromise the code. From within the compiler running on a standard desktop or laptop, without noticeably extending compile times. I'm sorry, but given what a poor job humans do at turning specifications into code, the idea that there's a super-clever anti-security demon lurking inside every compiler is just laughable.
And that's before the other obvious objection that looking at compiled code in a debugger/disassembler is going to reveal lumps of code that were not put there by the original author. Invisible on a sufficiently large project? Sure, but an encryption/decryption program doesn't need to be very large to begin with - all it has to do is to reliably transform a block of data into scrambled form and back again. This is more amenable to proof than most computer programs (not least because it has no need to be interactive). Furthermore, we can easily imagine test cases that are very very short; ROT-13 is a lousy cipher, but it is a cipher, and one that can be implemented in ~20 lines of (non-obfuscated) code. Now suppose we make a variant that asks the user how many places to rotate by (eg 14) and that number functions as our 'secret' key. Still hopelessly insecure to anyone over the age of 10, but what of it? Wouldn't your hypothesized 'insecurity demon' need to put a backdoor in anyway, because it is an encryption tool, be it ever so primitive? And wouldn't that block of code show up in a debugger? If your answer is no, you're now positing hidden functionality that not only divines programmers' intentions and subverts those that are intended to add security, but also looks at the quality of the security algorithm and only sticks in a backdoor if it passes a certain threshold of cleverness.
I wish compilers had those smarts built in! Think how much tedious/craptacular code could be automated away by simply labeling things as 'sekrit!!' and having the NSA module inside the compiler generate lightweight, reliable code with no real penalty! It would take the pain out of unit testing for ever!
You're using a very specific definition of backdoor there. I can think of others that at least seem plausible to introduce at the compiler level.
Timing attacks for instance, could plausibly fall out of optimizations that terminate a loop when it's clear that the value being computed won't change (e.g. is false and is repeatedly getting anded with things). This would probably be even worse for power consumption or other side channels. For a potentially easier to measure side channel, you might try introducing some state-dependent delay (short and caused by something like that loop optimization) in a bit of code preceding a packet send.
Alternately, introducing (via incorrect optimizations) the right kinds of buffer overruns or race conditions that corrupt a pointer just right, could get you a nice stack smashing exploit with certain (very abnormal) inputs -- and remote access to the machine in question.
You're presenting your argument as if that's the only kind of malicious binary we have to worry about, and since it's (probably) impossible, then therefore we don't have to worry. But there are more ways to compromise a binary than via the compiler automatically backdooring them. Deterministic builds protect against compromised binaries of any kind, so we need to use it.
The attack to be worried about goes like this: You build "Firefox Setup 23.0.1.exe" and intentionally insert a backdoor into the setup process. You make sure the setup process appears to function exactly the same as the clean installer (not hard). You then replace Firefox Setup 23.0.1.exe on various distribution websites with your malicious version. Or you MITM the distribution websites in order to send your malicious version in place of the one the user expects to be downloading.
Deterministic builds defend against that attack vector, while also defending against any hypothetical compiler-backdoor-autoinjector. You get both defense layers for free, just by using deterministic builds. This is a necessary step for the future, not an optional security layer.
I responded to the claim you made. If you wanted to talk about something else in the first place, maybe you should have done that instead.
Deterministic builds protect against compromised binaries of any kind, so we need to use it.
Unless I'm a spook (or group of spooks) any my binary includes a backdoor by design from the outset. Or unless I have backdoors built into the chips (a far more likely possibility than magic compiler demons). ISTM you're yo-yoing between treating the NSA as omniscient/omnipotent one moment and then holding up things like this as silver bullets the next.
>don't think even the NSA has people smart enough to insert code into a compiler which will add backdoors to software which didn't yet exist when the compiler was written.
Oddly enough, that is almost exactly how the Flame malware worked.
It injected itself into a compiler, causing the compiler to silently compile malicious code into custom firmware being compiled for nuclear reactor control equipment.
So at least one state actor has done just that at least once.
GCC's optimiser does some crazy shit sometimes including removing necessary code in -O3. Perhaps some of that crazy could be related?
As for using FreeBSD 4.1 off a CD in your closet, I recently set up an NT4 machine with SP2 (requirement!) that was to be used entirely offline. Surprisngly it booted and works fine on a brand new Intel chipset H61 based machine with an SSD in it (graphics are stuck at 800x600x16 but that's a requirement too).
If I understand your proposal correctly, it would be found out almost immediately, as file checksums would be altered and quite a few places do check them.
My idea was that, if you have an object file open with write permissions, you can assume that it has been created/modified anyway, and people will expect the checksum to change.
Of course this does not work if the checksum is calculated before the file is closed by your backdoored compiler.
Be careful about magical thinking here. They aren't omniscient supermen. The materials on the Guardian implied that these exploits were "fragile".
Given that statement and the insinuations that the NSA had folks participating on standards discussions basically gumming up the works for things like IPSec, NIST standards, etc (effectively crowding the people who know what they are doing out of the room), it's likely that they're created circumstances where accepting defaults or not really understanding how to implement various technologies creates attack surfaces or weaknesses that the NSA has known means to exploit.
Like anything else, you need to think about the risks and controls for whatever your doing. If you're protecting the interests of a government or company likely to be spied upon, you probably want to factor in the ability of a nation-state actor to potentially intercept data into your risk calculus and operational strategy.
For the rest of us, you need to think about what the risks really are -- if you're a politician or other visible individual, you need to be very mindful of your communications privacy and practices. (Google "Anthony Weiner"). If you're posting on Hacker News and buying stuff from Amazon, the NSA decrypting that is a low-risk threat.
Even if you don't fully trust both compilers, the 'diverse-double compiling' is still useful:
"Finally, note that the “trusted” compiler(s) could be malicious and still work well well for DDC. We just need justified confidence that any triggers or payloads in a trusted compiler do not affect the DDC process when applied to the compiler-under-test. That is much, much easier to justify."
Open source software represents what is IMHO the best combination between ease of self-verification and total security. In other words if you have only a little bit of time/resources/etc. to put into verification so that you don't have to rely on trust, the software that you're running is probably the best place to put that time and energy.
It's probably possible to verify closed-source binaries by decompilation and a lot of effort, if you wish.
But on the other hand between compilers/hardware/OS/etc. the part of the computer that is most susceptible to being attacked (the application software) is the part that's easiest to audit with open source.
If your chip's built-in RNG is cracked you can simply refuse to use it.
But if your libopenssl.so or CRYPTAPI.DLL is cracked you're probably screwed. It's at least somewhat straightforward to build your own libopenssl.so (or at the very least, verify that your distro's signed package RPM builds the same sha1sum from the known-good source tarball + patches). There's nothing you can do to independently verify the source used for CRYPTAPI.DLL, and even the binary of that DLL requires trusting Microsoft (and figuring out which Service Pack, KB patchset, etc. are installed).
However in the end you do have to trust people no matter what you do, so if you're willing to trust FooCorp's binaries then just go into decision with eyes open. Even most open source users are effectively no different in that they simply trust what Fedora or Debian are pushing out there.
But even one alert user is enough to catch a hacked .deb or .rpm, so you don't need most of the users to be paranoid. It's kind of a reverse herd immunity, if you will.
I've seen security problems sit in open-source code for years because no one was interested enough in going through it.
If you are talking emacs, or the kernel, or gcc, there is probably enough interest there (and strong personalities in charge of the projects) to keep things safe.
I don't know where the cutoff is. At thousands of users you probably can't count on someone else having looked through it. At millions you can.
That's certainty true. I can tell you as an OSS dev that that are simply not enough eyeballs for all bugs to be shallow.
However you don't have even the possibility with closed-source products. That means you must instead rely on the process used to create the software. So something by MS might actually be pretty trustworthy from a "security issues not accidentally introduced" standpoint at this point after years of improvement on secure coding practices.
But they still operate by a profit motive, and there's still the possibility of deliberate introduction of security issues when it suits business purposes or for legal compliance.
Unfortunately there's no great way to tell that an OSS project from RANDOM_DEV is well-coded without looking at the code and it may be that a well-intentioned but junior dev introduces a relative swath of security bugs by accident.
I don't know where the cutoff is either. It may depend more on your threat model than anything else.
What I've always wondered is how hard it is to have both a production system running some software on some hardware both of which may or may not have a backdoor and have a sentry system running next to it on different hardware that is somehow watching the first system for traffic anomalies not aligned with how the first system should be functioning. On top of that the first production system also watches the sentry for signs of disabling or tampering. With two machines watching each other, I would imagine that it's much harder to use any backdoor without being detected and shut down.
In a similar vein, would it not be possible to spot hardware backdoors in PRNG by running many identical encryption tests across many different types of hardware looking for one that doesn't behave like the others?
More generally, what tools exist for the automatic detection and mitigation of backdoors?
It doesn't mitigate untrusted software, but an interesting approach to tamper resistance can be found in
Infineon SLE78 security controllers (for smart cards, etc). They run dual CPUs out-of-sync and compare results after each instruction.
Look into Gitian, which is used by the bitcoin developers to create deterministic, verifiable builds that can be compared by multiple developers on multiple machines before doing a binary release. In theory, the process could be extended to an entire *nix distribution (and there is interest in doing so).
Actually, it's worse than that. Even if you can review the source code, you STILL won't know for sure. As Ken Thompson put it three decades ago, "no amount of source-level verification or scrutiny will protect you from using untrusted code."[1] This is because the tools we all use to process or transform source code (e.g., already-compiled-to-binary compilers, assemblers, loaders, etc.) may already contain secret back-doors. Ditto for even lower-level tools like hardware microcode.
While open-source software tools -- and open hardware too -- are much less likely to have secret back-doors embedded in them, there are no 100% guarantees. Ultimately we have little choice but to trust the people who create, package, and distribute all the software and hardware we use but didn't create ourselves from scratch.
--
EDITS: Added last paragraph. Made minor edits to second paragraph so it more accurately conveys my thoughts.
--
[1] http://cm.bell-labs.com/who/ken/trust.html