Promising New Hard Disk Technology

It has been too long, two-and-a-half years, since the last of Tom Coughlin's Storage Valley Supper Club events. But he just organized one to coincide with the Flash Memory Summit. It featured an extremely interesting talk by Karim Kaddeche, CEO of L2 Drive, a company whose technology seems likely to have a big impact on the hard disk market. Follow me below the fold for the explanation. I didn't take notes, so what follows is from memory. I apologize for any errors.

SSD vs. HDD (Updated)

IDC & TrendForce data
via Aaron Rakers

Chris Mellor's How long before SSDs replace nearline disk drives? starts with a quote I think the good Dr. Pangloss would love:

Aaron Rakers, the Wells Fargo analyst, thinks enterprise storage buyers will start to prefer SSDs when prices fall to five times or less that of hard disk drives. They are cheaper to operate than disk drives, needing less power and cooling, and are much faster to access.

Below the fold, some skepticism.

HAMR-ing Home My Point

In Double-headed Seagate disk drives? Yes, on their way, Chris Mellor mentions that Seagate:

expects to intro 20TB+ HAMR-based nearline HDDs in calendar 2020.

Volume production of HAMR drives is still 1 year away. In 2009 Dave Anderson of Seagate presented this roadmap. It shows HAMR drives a year away in 2010. They have been a year away ever since. A decade of real-time slip.

Only the good Dr. Pangloss believes industry roadmaps.

Demand Is Even Less Insatiable Than It Used To Be

In Demand Is Far From Insatiable I looked at Chris Mellor's overview of the miserable Q2 numbers from Seagate, Nearline disk drive demand dip dropkicks Seagate: How deep is the trough, how deep is the trough?, and Western Digital,  Weak flash demand and disk sales leave Western Digital scrabbling to claw back $800m a year. This quarter was equally dismal. Below the fold, the gory details.

It Isn't Just Cryptocurrency Mining

Izabella Kaminska's Just because it's digital doesn't mean it's green reports on:

A new report by the carbon emission think-tank The Shift Project out this week highlights that not much has changed since [2014]. ICT still contributes to about 4 per cent of global greenhouse gas emissions, which is still twice that of civil aviation. What is worse, its contribution is growing more quickly than that of civil aviation.

Cryptocurrency mining is definitely a problem, but how big a part of the problem isn't clear. It could be quite big. Follow me below the fold for some surprising details.

Chia Network

Back in March I wrote Proofs of Space, analyzing Bram Cohen's fascinating EE380 talk. I've now learned more about Chia Network, the company that is implementing a network using his methods. Below the fold I look into their prospects.

Shorter talk at MSST2018

I was invited to give both a longer and a shorter talk at the 34^th International Conference on Massive Storage Systems and Technology at Santa Clara University. Below the fold is the text with links to the sources of the shorter talk, which was updated from and entitled DNA's Niche in the Storage Market .

Longer talk at MSST2018

I was invited to give both a longer and a shorter talk at the 34^th International Conference on Massive Storage Systems and Technology at Santa Clara University. Below the fold is the text with links to the sources of the longer talk, which was updated from and entitled The Medium-Term Prospects for Long-Term Storage Systems.

Flash vs. Disk (Again)

Gartner's graph

Chris Mellor's NAND chips are going to stay too pricey for flash to slit disk's throat... is based on analysis from Gartner. It continues the theme that I've been stressing for quite some time; flash will not displace hard disk from the bulk storage layer of the hierarchy in the medium term. Follow me below the fold for some commentary, and more graphs.

Updating Flash vs. Hard Disk

Chris Mellor at The Register has a useful update on the evolution of the storage market based on analysis from Aaron Rakers. Below the fold, I have some comments on it. In order to understand them you will need to have read my post The Medium-Term Prospects for Long-Term Storage Systems from a year ago.

Storage Failures In The Field

It's past time for another look at the invaluable hard drive data that Backblaze puts out quarterly. As Peter Bright notes at Ars Technica, despite being based on limited data, the current stats reveal two interesting observations:

• Backblaze is seeing reduced rates of infant mortality for the 10TB and 12TB drive generations:
  

  The initial data from the 10TB and 12TB disks, however, has not shown that pattern. While the data so far is very limited, with 1,240 disks and 14,220 aggregate drive days accumulated so far, none of these disks (both Seagate models) have failed.

• Backblaze is seeing no reliability advantage from enterprise as against consumer drives:
  

  the company has now accumulated 3.7 million drive days for the consumer disks and 1.4 million for the enterprise ones. Over this usage, the annualized failure rates are 1.1 percent for the consumer disks and 1.2 percent for the enterprise ones.

Below the fold, some commentary.

Will HAMR Happen?

For more than five years I've been skeptical of the storage industry's optimistic roadmaps in general, and the idea that HAMR (Heat Assisted Magnetic Recording) will replace the current PMR (Perpendicular Magnetic Recording) as the technology for hard disks any time soon. The first ship date for HAMR drives has been slipping in real time for nearly a decade, and last year Seagate slipped it again:

[Seagate] is targeting 2018 for HAMR drive deliveries, with a 16TB 3.5-inch drive planned, featuring 8 platters and 16 heads.

Now, Chris Mellor at The Register reports that:

WDC has given up on heat-assisted magnetic recording (HAMR) and is developing a microwave-assisted technique (MAMR) to push disk drive capacity up to 100TB by the 2030s.

It's able to do this with relatively incremental advances, avoiding the technological development barrier represented by HAMR. These developments include multi-stage head actuation and so-called Damascene head construction.

Below the fold, I assess this news.

The Medium-Term Prospects for Long-Term Storage Systems

Back in May I posted The Future of Storage, a brief talk written for a DARPA workshop of the same name. The participants were experts in one or another area of storage technology, so the talk left out a lot of background that a more general audience would have needed. Below the fold, I try to cover the same ground but with this background included, which makes for a long post.

This is an enhanced version of a journal article that has been accepted for publication in Library Hi Tech, with images that didn't meet the journal's criteria, and additional material reflecting developments since submission. Storage technology evolution can't be slowed down to the pace of peer review.

Where Did All Those Bits Go?

Lay people reading the press about storage, and even some "experts" writing in the press about storage, believe two things:

• per byte, storage media are getting cheaper very rapidly (Kryder's Law), and
• the demand for storage greatly exceeds the supply.

These two things cannot both be true. Follow me below the fold for an analysis of a typical example, Lauro Rizatti's article in EE Times entitled Digital Data Storage is Undergoing Mind-Boggling Growth.

Brief Talk at the Storage Architecture Meeting

I was asked to give a brief summary of the discussions at the "Future of Storage" workshop to the Library of Congress' Storage Architecture meeting. Below the fold, the text of the talk with links to the sources.

End of Moore's Law

Richard Chirgwin at The Register reports that the Semiconductor Industry Association has issued their roadmap for chip technology, the ITRS:

The group suggests that the industry is approaching a point where economics, rather than physics, becomes the Moore's Law roadblock. The further below 10 nanometres transistors go, the harder it is to make them economically. That will put a post-2020 premium on stacking transistors in three dimensions without gathering too much heat for them to survive.

This is about logic, such as CPUs, but it is related to the issues that have forced flash memories to use 3D.

Energy demand of computing

There are other problems than the difficulty of making transistors smaller:

The biggest is electricity. The world's computing infrastructure already uses a significant slice of the world's power, and the ITRS says the current trajectory is self-limiting: by 2040, ... computing will need more electricity than the world can produce.

So we're looking at limits both on the affordability of the amounts of data that can be stored and the computations that can be performed on it.

The ITRS points to the wide range of different applications that the computations will be needed for, and the resulting:

research areas a confab of industry, government and academia see as critical: cyber-physical systems; intelligent storage; realtime communication; multi-level and scalable security; manufacturing; “insight” computing; and the Internet of Things.

We can see the end of the era of data and computation abundance. Dealing with an era of constrained resources will be very different.In particular, enthusiasm for blockchain technology as A Solution To Everything will need to be tempered by its voracious demand for energy. An estimate of the 2020 energy demand of the bitcoin blockchain alone ranges from optimistically the output of a major power station to pessimistically the output of Denmark. Deploying technologies that, like blockchains, deliberately waste vast amounts of computation will no longer be economically feasible.

SSDs: Cheap as Chips?

Four and a half years ago Ian Adams, Ethan Miller and I proposed DAWN, a Durable Array of Wimpy Nodes, pointing out that a suitable system design could exploit the characteristics of solid-state storage to make system costs for archival storage competitive with hard disk despite greater media costs. Since then, the cost differential between flash and hard disks has decreased substantially. Below the fold, an update.

Discounting the far future

In 2011 Andrew Haldane and Richard Davies of the Bank of England (HD) presented research showing that, when making investment decisions, investors applied discount rates much higher than the prevailing interest rates, and that this gap was increasing through time. One way of looking at their results was as an increase in short-termism; investors were increasingly reluctant to make investments with a long-term payoff. This reluctance clearly has many implications, including making dealing with climate change even more difficult. Their work has influenced our efforts to build an economic model of long-term storage, another area where the benefits accrue over a long period of time.

Now, Stefano Giglio of the Booth School and Matteo Maggiori and Johannes Stroebel of the Stern School (GMS) have a post entitled Discounting the very distant future announcing a paper entitled Very Long-Run Discount Rates. Their work, at first glance, seems to contradict HD. Below the fold, I look into this apparent disagreement.

Bezos' Law

Greg O'Connor had a piece at Gigaom entitled Moore's Law Gives Way To Bezos' Law sparked by the recent price cuts by Google and Amazon in which he claimed that:

The latest cuts make it clear there’s a new business model driving cloud that is every bit as exponential in growth — with order of magnitude improvements to pricing — as Moore’s Law has been to computing.
If you need a refresher, Moore’s Law is “the observation that, over the history of computing hardware, the number of transistors on integrated circuits doubles approximately every two years.” I propose my own version, Bezos’s law. Named for Amazon CEO Jeff Bezos, I define it as the observation that, over the history of cloud, a unit of computing power price is reduced by 50 percent approximately every three years.

Both Moore's and Kryder's laws held for multiple decades. Below the fold I ask whether a putative Bezos' law could be equally long-lived?

Talk at Seagate

I gave a talk at Seagate entitled:

Storage Will Be
Much Less Free
Than It Used To Be

Below the fold is an edited text with links to the sources.