23 thoughts on “Persistence”

  5. Smartpatoot #2 here to say that, instead of “destroying it”, you should clarify with “destroying the flash drive”. “Destroying it” sounds like it refers to destroying the data.

  8. I bet a well timed tachyon pulse from a main deflector array could impart a quantum irregularity into the bit polarity flux thereby making all the data on the flash drive instantaneously convert into a lolcat pic.

  9. Ahh, good point, Josh.

    Firing up the Improbability Drive could potentially turn the flash drive ITSELF into a lolcat. 😉

  11. I always figured the best way to “erase” data was to replace it and fill it up with some large audio or video files copied into it over and over. Lou Reed’s “Metal Machine Music” seemed appropriate.

  12. Raoul-

    The short (and irritatingly split into two pages) article explains why, on SSD devices, such a method is unreliable, at best. An all-zero write, taking 58 hours in some cases, was reliable less than 50% of the time, and one drive retained 1% of its data after 20 passes.

    Physical destruction, as usual, is about the only reliable method of erasing the contents of a drive.

  14. If it’s a USB drive, wouldn’t a couple alligator clips hooked to 110 VAC do the trick, or else filling it with a bunch of Justin Bieber videos to repel any would-be snoops?

  16. From their attempts, it sounds like these guys from UCSD went into this with little knowledge of how modern SSDs work.

    Hopefully drive manufacturers (or more likely, it would be the flash controller manufacturers) implement ATA full drive encryption reasonably, that feature was added to the ATA specification at the request of the NSA, if i recall correctly.

    A number of high performance consumer drives use the Sandforce flash controller family, which does compression and deduplicaton–so you won’t be able to use a repeating pattern of any sort to overwrite (much) data. You would have to use randomly generated data to force blocks to be overwritten.

  17. I don’t get why filling it up won’t work. Say I’ve got a 4 Gb drive. I start downloading random Youtube videos to it until the OS squeaks “drive full!” Doesn’t take real long. Certainly not 58 hours.

    Now what? Is the drive not really full? How can any significant amount of my data be left? I’m not real concerned if one bit is left, somewhere. That’s not going to do anyone any more good than if I revealed to y’all, here and now, in defiance of the threat of identity theft, that my social security number contains at least one 9.

  18. One bit is one thing. One percent is another matter entirely. 1% of a 4 GB drive (assuming thumb drives) is still 40 MB worth of data; that’s a pretty significant chunk of data to leave behind on a drive.

    However, the paper referenced in the article tested the methods on both usb thumb drives (which can exceed 32 GB nowadays, meaning 320 MB worth of data left behind), as well as SSD hard drives, which have a typical capacity in the 128 GB – 256 GB range, meaning 1.3-2.6 GB worth of data left behind.

    This is an important issue for governments and other agencies that want to RE-USE the hard drive or flash drive, or that want to have a reliable method of data destruction in the event of an emergency (data dump and destruction in the event of a military incident or accident). As easy as it would be to just dump the drives into a furnace and call it a day, that’s not always possible.

    Also, filling up a drive doesn’t work because of the methods used to recover data from old drives. Each sector on a magnetic drive still contains a “ghost” image of the previous data written at that sector, so if you overwrite the entire drive with 20 hours’ worth of Rick-rolling, it’s a fairly trivial matter to take the original rick-roll video and use it as a mask/filter to “see through” the top layer of data to the underlying “ghost” data. Writing all 1’s, followed by all Zeroes, multiple times in a row, effectively erases the “ghost” data by flipping each individual bit a number of times until there’s no ghosting left and the entire drive is just one big pile of 1’s (or zeroes).

    The article itself isn’t all that informative, but the linked paper gives enough background and information to be able to understand what’s being done, and why it’s a big deal.

  19. Carl,
    1) SSDs do something called “overprovisioning”. The non-geek translation of that is: They flat lie about their capacity. Ordinary hard drives generally state a number (100 GB!) and then use some sliver of that up in organizational cruft leading to smaller usable sizes (87GB usable!).

    SSDs have a problem where individual blocks can only be written a limited number of times, and they want to be able to give warranties on their drives that aren’t exceeded during normal usage…. So they say one thing (100GB!) but they actually have (130GB) and spread the writing out so that a normal user will still see the stated size after three or five years of normal use. (And thus, not need to payout a warranty replacement.)

    So just making a file the size of the stated size (100GB file of crud) means there’s still a large and hidden piece of the disk. 30GB of files is a hefty chunk. A quarter of my SSN is useless to you. But a 1-in-4 chance of my entire SSN is not.

    2) If the file you’re writing isn’t explicitly random as opposed to a known quantity (like a known movie), I think the forensic tools can still determine “What was the bit before this last bit was written.” At least, they do this for regular hard drives. It isn’t easy with regular hard drives, and I’m not sure if it’s even conceptually possible with Flash memory. But it’s an engrained ‘best procedure’ sort of thing at the very least.

    A “random YouTube movie” doesn’t have the randomness in the right spot. Anything where someone can determine after the fact exactly what you wrote on the disk is no good.

  20. Carl: Sure, for a 4 gig USB drive, it won’t take 58 hours.

    But these guys are talking about 64-128 gig SSDs.

    On the other hand, for 99.99% of users and uses, this doesn’t matter. And that other .01% know how to use a hammer, and do that to their drives anyway, because that’s how real data security paranoids operate.

    There’s a lot of surety in having holes drilled through the platters and/or the drive smashed flat. You don’t have to worry that maybe someone hacked your erase program, or that the guy whose job it is to “securely erase the data” got lazy and didn’t bother, or missed that drive…

    (Hell, most people have far more issues with losing data they want to keep than the reverse…)

    Al: Show me a forensic tool that can do that on a magnetic platter, ’cause I don’t think it actually exists.

    (Might you be thinking of the notional “get a STEM and look at the platter physically” whitepaper from a few years back?

    Problem is nobody’s ever actually done it, and it’d take months or years to get any data off … and even then, the STEM can’t tell “the previous state” from “the previous state five iterations ago”.

    It’s not like layers of ink where previous states “stack” and you could notionally “peel them off” to get to a definite prior state. All we really ever have on a magnetic disk is “It’s 0/1 now”, and perhaps “it was 1 at some point in the past”.)

    All this “three passes of random data” and the like stuff we’ve had thrown at us for years as “necessary for security” is theater based on, essentially, nothing. A single overwrite is sufficient according to everything verifiable.

  21. Sigivald, I basically agree on the feasibility of recovering a damn thing. Even on an ordinary disk with just the content tree blasted recovering anything worthwhile is a jaunt through every fire in Hades. One write with random data is enough for me.

    But I could really see “one write with zeros” just not being enough. (Which is sort of where things were before the complete-overkill mania overtook things.)

  22. Thanks for the comments, gentlemen.

    Myself, I wonder if this is a leftover from the 70s, when disks were expensive. Nowadays they’re so cheap I don’t even bother to decruft my filesystems: when they fill up, I just buy another drive.

    In the same sense, if I wanted to erase a whole drive, I’d just trash it and buy a new one. It would have to be rather an expensive drive before it cost me more to replace it than my time is worth superduperpurging it.

    I suppose that wouldn’t the case if I wanted to erase a drive frequently. But then this starts to become weird: what kind of work am I doing where I am really paranoid about the security of my drive after I’ve erased it and these drives are in such constant use I need to be brainwiping them every 5 days?

Comments are closed.