NEXT STORY
Thinking Machines – my second education
RELATED STORIES
NEXT STORY
Thinking Machines – my second education
RELATED STORIES
Views | Duration | ||
---|---|---|---|
101. Calculators and computers are the future | 126 | 01:33 | |
102. 'I think I want to do what he does': Alan Kay | 154 | 02:33 | |
103. Getting to know people before they were famous | 154 | 03:59 | |
104. Meeting Freeman Dyson | 338 | 02:48 | |
105. Richard Feynman and Thinking Machines | 440 | 02:17 | |
106. Richard Feynman – quartermaster for the stationery | 287 | 01:27 | |
107. Neurobiology and fast computing | 160 | 02:17 | |
108. How Thinking Machines started | 170 | 03:10 | |
109. Proving Amdahl's Law wrong | 316 | 02:24 | |
110. Thinking Machines – my second education | 142 | 03:22 |
As it turns out, it turned out to be good for a lot of other things other than neural networks. In fact, people did kind of use it for neural networks, but it was still, as it turned out, thousands of times too slow to really make it work. So it took another few decades before computers got fast enough. But that basic architecture that we were trying to do there is basically what makes things like Deep Mind work and so on. It's pretty much the same algorithms, it's just I was only off by a factor of a thousand, maybe a million.
But the idea was right. And it actually turned out, too, once we built them, that they were useful for other things. And so that was the customers wanted to use it for other things. And so things like looking for oil and designing airplanes and things like that. So it turns out actually that Gene Amdahl's proof was wrong; it had a flaw in it. Which only later did I understand it, which is: he had assumed that you were solving exactly the same problem on the bigger machine, and sure enough, if you have one problem, and the problem doesn't get bigger, if you make the machine bigger, it gets less inefficient... less efficient. But in reality, what happened was as you got bigger machines, you solved bigger and bigger problems, and so there was more data to work on. And so in fact you could get... you could keep operating it efficiently, because the problem just got bigger. And so all the real hard problems, like simulation or neural networks, they all had that characteristic that they became big because the data was big. In fact, it's very hard to find a problem that is an exception to that rule. So in practice, Amdahl's Law turned out not to be a law at all. Although for years and years, whenever I would describe my project, people would dismiss me by saying, 'Oh well, you've obviously never heard of Amdahl's Law, this will never work.' So... Fortunately, I just bombed on ahead without really understanding why Amdahl's Law was wrong.
W Daniel Hillis (b. 1956) is an American inventor, scientist, author and engineer. While doing his doctoral work at MIT under artificial intelligence pioneer, Marvin Minsky, he invented the concept of parallel computers, that is now the basis for most supercomputers. He also co-founded the famous parallel computing company, Thinking Machines, in 1983 which marked a new era in computing. In 1996, Hillis left MIT for California, where he spent time leading Disney’s Imagineers. He developed new technologies and business strategies for Disney's theme parks, television, motion pictures, Internet and consumer product businesses. More recently, Hillis co-founded an engineering and design company, Applied Minds, and several start-ups, among them Applied Proteomics in San Diego, MetaWeb Technologies (acquired by Google) in San Francisco, and his current passion, Applied Invention in Cambridge, MA, which 'partners with clients to create innovative products and services'. He holds over 100 US patents, covering parallel computers, disk arrays, forgery prevention methods, and various electronic and mechanical devices (including a 10,000-year mechanical clock), and has recently moved into working on problems in medicine. In recognition of his work Hillis has won many awards, including the Dan David Prize.
Title: Proving Amdahl's Law wrong
Listeners: Christopher Sykes George Dyson
Christopher Sykes is an independent documentary producer who has made a number of films about science and scientists for BBC TV, Channel Four, and PBS.
Tags: Gene Amdahl
Duration: 2 minutes, 24 seconds
Date story recorded: October 2016
Date story went live: 05 July 2017