The Intercept and its publisher, First Look Media, strongly believe in the benefits of free and open source software — in part because we rely on such software every day. To keep our journalists and sources safe, we use secure communication tools like the data-encryption system GnuPG, the Off-the-Record secure messaging protocol, the SecureDrop communications platform and the secure calling and texting app Signal. To publish on the web, we use the GNU/Linux operating system; the Apache web server; OpenSSL, a web encryption library; WordPress, the open-source blogging engine; and Piwik, which tracks web traffic. The list goes on.
We greatly appreciate the hard work of developers who give away their code to benefit the internet and the world. And today we’re excited to contribute back to the open source community by launching First Look Code, the home for our own open source projects related to privacy, security, data, and journalism. To begin with, First Look Code is the new home for document sanitization software PDF Redact Tools, and we’ve launched a brand new anti-gag order project called AutoCanary.
PDF Redact Tools
When The Intercept first launched, part of my job involved redacting documents from NSA whistleblower Edward Snowden before publishing them. Because we didn’t want to inadvertently publish sensitive information we’d intended to redact — as no less cautious an institution than The New York Times once did — I developed PDF Redact Tools, a simple command-line program for Mac OS X and Linux that helps with redacting, stripping metadata, and sanitizing PDFs in preparation for publishing.
Read more about PDF Redact Tools on its new website.
AutoCanary
A warrant canary is a regularly published statement that a company hasn’t received any legal orders that it’s not allowed to talk about, such as a national security letter.
Canaries can help prevent web publishers from misleading visitors and prevent tech companies from misleading users when they share data with the government and are prevented from talking about it. One such situation arose — without a canary in place — in 2013, when the U.S. government sent Lavabit, a provider of encrypted email services apparently used by Snowden, a legal request to access Snowden’s email, thwarting some of the very privacy protections Lavabit had promised users. This request included a gag order, so the company was legally prohibited from talking about it. Rather than becoming “complicit in crimes against the American people,” in his words, Lavabit founder Ladar Levison, chose to shut down the service.
Warrant canaries are designed to help companies in this kind of situation. You can see a list of companies that publish warrant canary statements at Canary Watch. As of today, First Look Media is among the companies that publish canaries.
We’re happy to announce the first version of AutoCanary, a desktop program for Windows, Mac OS X, and Linux that makes the process of generating machine-readable, digitally-signed warrant canary statements simpler.
Read more about AutoCanary on its new website.
I started writing an email to Micah Lee, but then thought better of it.
Here’s the beginning:
I went back and I could not find ‘First Look Publishes Open Source Code To Advance Privacy, Security and Journalism’.I’m no genius and I have a limited time to spend on each article.I found it in my ‘history’.
I know, from last year, NOTHING will ever happen to make it easier to read and access TI.
Except…you stopped publishing comments in-barely-visible-grey, after a year or so of writing.
For that alone, thank you, TI.
I don’t find the present design attractive, but not being able to find an article you have just read,
is inexcusable.
I never go to TI, except when prompted by some other site.
It was not always thus.
I am an ordinary reader, no special computer skills.
@Useful Idiots the peripherals may come with the malware pre-installed.One the things about pgp, off the record, anonymizing sw, red phone, etc., is we all should be using them. Why? Governments find it harder to detect such sw,
used by the one person whose life depends on it, if there are 10,000 or 100,000 computers/phones doing the same thing.Diversity is anonymity’s friend.
No government has the right to see what I am doing, at any moment. NSA used general warrants, held in secret, to survey
what the whole world is doing.NSA will store these data forever.Forever is a long time.
GPG and OTR I agree with. I don’t trust WhisperSystems’ model at all (it’s a centralising model that collects info and can easily act as a one-stop-shop watering hole because of how it works) — or any software that outright declares its intentions if you’re trying to ‘get around’ the whole metadata dragnet — if anything you draw more attention to yourself while opening yourself up to easier collection. I also don’t believe in the use of any software that attaches your phone number (and its associated information) to you — you sort of defeat the purpose, don’t you?
But you asked about hardware implants. None of what you suggested will do anything about those. Even a certified computer tech often cannot differentiate between an implanted device and a clean one, so it’s not like one can merely swap out a component. That said I wouldn’t ever use a wireless peripheral (wifi or bt, etc) (even if I bought it in person); that has its own bag of security problems that don’t require any fancy man-in-the-shipping-middle to go with it.
BTW, diversity can be anonymity’s friend but it can also be the smoking hot potato that gets ya if your fingerprint is “too unique”. Alternately, diversity is great for opsec methods.
But govs don’t find it hard at all to detect anything you mentioned. They just have a harder time finding out which needle you are in the ‘interesting haystack’ if more people use the tech. In the meanwhile, the way collection is done, all encrypted data is stored forever (as opposed to facing retention time limits (if you believe they really exist)). Do you care if it is? Probably, if your metadata is attached to it.
I don’t want anything held forever by anyone — but I don’t believe anything should be permitted to be collected in the first place without a REAL warrant for REAL probable cause, and within strict limitations. The way the world is going, ‘forever’ feels like it won’t be as long as we hope, though — and by hope, I don’t mean the collection, I mean the forever, so it’s not an optimistic statement — more a ‘look towards Rome for examples’. I am deeply concerned about the state of the world, and the future.
Recently, I was listening to a talk, given Jacob Appelbaum in the NL.During Q&A, I heard someone ask about their peripherals, i.e. kb, mouse, printer, etc. as a source of malware. There is the possibility that this malware might be put on in the factory, or somehow installed, before you even opened it.
How do these tools interact with putative malware in the peripherals?
thanks!
There’s nothing you can do about that but get rid of the hardware and get new stuff. There aren’t much by way of software mitigation and unless you want to live your life in a Faraday Cage and never leave or communicate with anybody or bring anything in or out of it…
Get new hardware, randomly, in person, with cash. You can’t fix hardware with software (at least not easily). If you’re using Linux, you can possibly make sure there’s no kernel modules/drivers available to parse what’s there, *maybe*. But that’s a crapshoot and you have no idea who’s doing what, how. Then again, you’d need to do something ‘interesting’ to be worthy of that level of hardware infiltration. Appelbaum may be that interesting. It’s up to the reader if they themselves are. The more “interesting” you may seem, the more careful you need to be buying/acquiring hardware — and how you use it.
I am with you but I am not that cynical. I don’t get why some folks out there who know technical sh!t keep perpetuating technical bs on other people (they should know better)
Yet, I don’t think that the NSA, USG exist in a different physical or logical reality or moral high ground. There is a lot that could be done but we have to redefine the game from the ground up if we want to get anywhere
Satyagraha,
RCL
He was asking specific questions about hardware. I was merely addressing those specific hardware-related questions, not suggesting overall technical solutions. Regardless of who you are though, you generally ARE better off buying your hardware in person with cash. That’s not technical. :)
I think most of us would prefer you exercising your technical skills on fixing the commenting system and securing your own systems and networks instead of branching out into the land of things that already exist (exiftool cleans things nicely already) or can easily be done with a two second script (some VPN providers just crontab a grab of a newssite’s rss feed, plunk the date with a “and as of yet we have not been”, and done with it. So you’re basically calling out maybe…. 5 minutes? of scripting as a remarkable thing.
You’re hiring more ‘journalists’ (some are good, but not the point) but you’re completely ignoring opsec of your users and you’re misappropriating what technical talent you have.
Hire more techies. For the love of all that is sacred.
I could’ve sworn I mentioned what one of those things would be: fix your commenting. Fix your site. Stop moving articles around based on popularity and stop making it impossible to keep track of articles after only a few days have passed. But especially the commenting. Let it all fly if you must, and add up/downvotes, but don’t censor unless things get viciously personal and ad-hominem. Hire an intern to do a quick wrist-flick if necessary (they’re free, or almost free and they’d get resume cachet). It’s not just a matter of convenience. You realise you’re angering people — people who get angry by the very things you report on then have nowhere to direct their comments. It’s either brilliantly twisted, or it’s just plain poor judgment.
How much does a web dev for a few weeks cost?
TI’s “get it down from us” censoring, controlling culture seems to be making some people (including yours truly) tired of it (which isn’t necessarily something bad)
Satyagraha,
RCL
Very exciting development!
Is there a First Look Github / other online source repository? How will you accept bug fixes / patches? What other types of are on the horizon?
Yup, our GitHub page is at https://github.com/firstlook. We accept patches through pull requests on GitHub, and you can submit issues on GitHub too if you find bugs or have feature requests. We also have a First Look Code development blog, and our first blog post explains a little bit of this.
Very nice, will keep a close eye on this. Look forward to more tools in the near future.
Can I expect some exploit detection tools in the future? Maybe something that can detect those pesky “man-on-the-side” attacks?
Indeed! But I would start by:
1) bringing networking in Linux (and IlumOS (and FreeBSD)) out of the kernel (into user land)
2) creating live versions of OSs (based on Debian live in order to take advantage of a huge user base) to ease the transition of regular folks out there out of Windows
3) mess a bit with GRUB and the boot up process in order to make it more NAS-proof
4) fork off reading capabilities from writing ones inside of the source code itself and (a la java security model) have processes access certain features only
…
Now, imagine first look media teams up with archive.org and wikipedia.org to come up with a search engine that is not google …
Satyagraha,
RCL
For the PDF Redact Tools, does the pdf need to be searchable? I’m guessing it’s fairly easy to use the tool in a terminal program to strip nasty metadata like tracking information, I was more wondering about redaction. Say you want to redact a name from a document, how would you do this in a terminal interface if the pdf was treated more as an image? It doesn’t sound like you’re crossing out info with black lines in Paint, I’m just curious how this would work if the pdf can’t easily be converted to plain text, like if the original was a scanned piece of paper instead of an electronically created document. Also if it works on Mac & Linux command line, does it work with BSD too?
PDF Redact Tools splits the PDF into a bunch of PNGs. The project page explains that you need to “Edit each page that needs redacting in graphics editing software like GIMP or Photoshop.” So yeah, basically you do cross out info with black lines in Paint.
And I’m sure you could get it running BSD, and likely Windows too. Basically, it relies on imagemagick and exiftool, so if those two pieces of software are available for your platform, PDF Redact Tools will probably run. But we’ve only tested it in Linux and Mac OS X.
Another use I can imagine for the PDF Redact Tools is to make an unsanitized PDF from a source safe for local viewing (in case the PDF is infected with malware, or location-leaking remote url requests, etc.).
Using it this way will reduce your attack surface greatly. Unless there is an exploitable zero-day in imagemagick or exiftool, an attack is thwarted. Don’t run PDF Redact Tools as root!
Be careful where you get your PDFs people! Aside from MS Office document formats, PDF is the most exploited file type, to my knowledge anyway.
This is exactly what pdf-redact-tools –sanitize is meant for. But you’re quite right, malicious PDFs could still target PDF Redact Tools itself (which probably really means targeting imagemagick or exiftool). If you want to use it like this, it’s safest to do this inside of a virtual machine as well.
Ok thanks for clearing that up. I wasn’t sure how you would redact via command line. Converting the pdfs to images makes more sense.