a story lives forever
Sign in
Form submission failed!

Stay signed in

Recover your password?
Form submission failed!

Web of Stories Ltd would like to keep you informed about our products and services.

Please tick here if you would like us to keep you informed about our products and services.

I have read and accepted the Terms & Conditions.

Please note: Your email and any private information provided at registration will not be passed on to other individuals or organisations without your specific approval.

Video URL

You must be registered to use this feature. Sign in or register.


Thinking Machines – my second education


Proving Amdahl's Law wrong
W Daniel Hillis Scientist
Comments (0) Please sign in or register to add comments

As it turns out, it turned out to be good for a lot of other things other than neural networks. In fact, people did kind of use it for neural networks, but it was still, as it turned out, thousands of times too slow to really make it work. So it took another few decades before computers got fast enough. But that basic architecture that we were trying to do there is basically what makes things like Deep Mind work and so on. It's pretty much the same algorithms, it's just I was only off by a factor of a thousand, maybe a million.

But the idea was right. And it actually turned out, too, once we built them, that they were useful for other things. And so that was the customers wanted to use it for other things. And so things like looking for oil and designing airplanes and things like that. So it turns out actually that Gene Amdahl's proof was wrong; it had a flaw in it. Which only later did I understand it, which is: he had assumed that you were solving exactly the same problem on the bigger machine, and sure enough, if you have one problem, and the problem doesn't get bigger, if you make the machine bigger, it gets less inefficient... less efficient. But in reality, what happened was as you got bigger machines, you solved bigger and bigger problems, and so there was more data to work on. And so in fact you could get... you could keep operating it efficiently, because the problem just got bigger. And so all the real hard problems, like simulation or neural networks, they all had that characteristic that they became big because the data was big. In fact, it's very hard to find a problem that is an exception to that rule. So in practice, Amdahl's Law turned out not to be a law at all. Although for years and years, whenever I would describe my project, people would dismiss me by saying, 'Oh well, you've obviously never heard of Amdahl's Law, this will never work.' So... Fortunately, I just bombed on ahead without really understanding why Amdahl's Law was wrong.

W Daniel Hillis (b. 1956) is an American inventor, scientist, author and engineer. While doing his doctoral work at MIT under artificial intelligence pioneer, Marvin Minsky, he invented the concept of parallel computers, that is now the basis for most supercomputers. He also co-founded the famous parallel computing company, Thinking Machines, in 1983 which marked a new era in computing. In 1996, Hillis left MIT for California, where he spent time leading Disney’s Imagineers. He developed new technologies and business strategies for Disney's theme parks, television, motion pictures, Internet and consumer product businesses. More recently, Hillis co-founded an engineering and design company, Applied Minds, and several start-ups, among them Applied Proteomics in San Diego, MetaWeb Technologies (acquired by Google) in San Francisco, and his current passion, Applied Invention in Cambridge, MA, which 'partners with clients to create innovative products and services'. He holds over 100 US patents, covering parallel computers, disk arrays, forgery prevention methods, and various electronic and mechanical devices (including a 10,000-year mechanical clock), and has recently moved into working on problems in medicine. In recognition of his work Hillis has won many awards, including the Dan David Prize.

Listeners: Christopher Sykes George Dyson

Christopher Sykes is an independent documentary producer who has made a number of films about science and scientists for BBC TV, Channel Four, and PBS.

Tags: Gene Amdahl

Duration: 2 minutes, 24 seconds

Date story recorded: October 2016

Date story went live: 05 July 2017