For 60 years artificial intelligence work went on in dusty university computer labs, where nobody paid it much attention.
Then in 2006, a breakthrough: Britain’s very own Dr Geoffrey Hinton invented “deep learning”. Deep learning is a way of teaching computers to program themselves, based on the human brain. And the technology world perked up and took notice.
Before long Microsoft, Google and Facebook had signed up every PhD researcher with a passing interest in deep learning. Google even paid £378m for a startup called DeepMind, mainly to get its hands on a dozen top drawer AI experts.
So that’s been the state of play for the last while – the experts all work at the big companies. And the most exciting applications of deep learning, such as driverless cars, voice recognition, instant translation and medical diagnosis, have come from the big companies too.
Why AI needed scale
There are three main reasons why AI research has, up to now, been concentrated at big companies. The first, as I’ve explained, is to do with talent. Back in 2006 there were only a handful of deep learning experts on the planet. Big companies were able to outbid for their services.
The second reason, as cited by Marc Andreessen recently in a conversation with Tim B Lee of Vox, is the scale of the projects. Teaching a car to drive is a huge task, requiring huge resources. The Amazon Echo, a household helper which uses voice recognition, is the product of four years’ work by 1,500 engineers.
The third reason is data. Deep learning basically has two components: clever algorithms which show the computer how to learn from data; and lots of data. The big companies – Google and Facebook in particular – have more data than anyone. More data means better AI, all things being equal.
For those three reasons, many people assume AI is going to stay concentrated in giant technology companies. Where it’s hard for us to invest in it.
Well, happily, things are changing. The three forces which had kept AI inside big companies up to this point are starting to weaken. The industry is blooming.
Change is coming
According to Marc Andreessen, the venture capitalist, a few things are changing. The first is that there are just more AI experts around these days. Deep learning has been around for a decade, and lots of ambitious computer science types have realised that it’s a path to riches.
The second change is that the machine learning experts have cracked some big problems, such as teaching computers to drive and recognise objects in videos. With those breakthroughs “in the bank”, it’s getting easier to apply the techniques to new problems. So: you don’t need a small army of engineers to teach computers a new trick any more.
The third factor is that machine learning algorithms are getting more efficient. Where in 2011, an algorithm might have needed 12 million examples of a cat to figure out what a cat looks like; today it can do it with much fewer examples. That’s more or less the holy grail of machine learning – to teach a computer how to do something with relatively few examples. The human brain is good like that. Eventually it’s hoped computers will be able to pick things up quickly, too.