There is a popular theory that it takes 10,000 hours of training for someone to be ‘good’ at something and even then they are still more likely to make errors on occasion – hence the term human error. Whereas the success of Deep Learning comes from the speed and precision with which the network is trained, often taking much less than 10,000 hours. In fact, a Neural Network can be trained in as little as a couple of hours, to a couple of days.
Training & Network Types
Setting up a Neural Network
Neural networks are made up of three parts, an input, an output & inner layers in the middle. The inner layers of the network process the input data to determine the output, with three main types of output:
- Categorical – to discriminate between different classes only one class can be true
- Multiple Category – Example, network for detecting dogs and hats. Both classes can be true if a dog is wearing a hat.
- Regression – Predicting the location of a feature in an image, e.g. finding a car in a photo
Each inner connection has a weight associated with it that determines how much the input at this connection will influence the output. The value of the weights is what is learned during the training process. There are three ways Neural Networks can be trained; supervised learning, unsupervised learning and reinforcement learning. During the training phase, adjustments to the parameters of the network minimize the prediction error. Once your Neural Network is set up and trained, it can then be used to make predictions on new, previously unseen data.
Supervised learning is one way of training a Neural Network. This type of training requires a human (supervisor) to provide examples of input data and desired output (the label) so that the machine can learn how to process the input to get the desired output. An example of this type of training is giving the network a selection of images and telling it that the input is different cuts of meat, with hindquarter and ribs being the labels. The network then learns the features that allow it to distinguish between these two types of cuts. For instance, a possible feature of a rib cut could be the presence of a sequence of parallel lines (the ribs), for the hindquarter it could be the presence of a H-bone. But a well-trained Neural Network will use many other features to draw its conclusion because in many applications obvious or key features may be obscured in the image. Once sufficiently trained, the network will then be able to classify any new images input into cuts of ribs and cuts of hindquarter.
Another type of Neural Network training is unsupervised learning, where the network will use unlabelled data to train. It is normally used for classification tasks. For example, if you tell the network there are two distinct classes it needs to find, then the model will try to discriminate the data into two groups. Another example would be trying to classify visitors to an office into two categories. There is no way of knowing what features the network will use to discriminate into groups. There are a number of features it can take into account including; gender, skin colour, weight, height etc. But because it does not know what data belongs with what label it may separate data into groups by using any combination of features, and not necessarily the ones we are after - a very real problem in unsupervised learning.
The third type of training is reinforcement learning and works on the principle of feedback. As an example, if a machine was to learn how to race a car in a game it would be provided feedback automatically based on how far the car drove without crashing or how quickly it managed to complete the race. The machine will make thousands of copies of the network, each with small mutations to the parameters, and will then test each one of those networks. Those that survive the furthest have mutations made from them, while the unsuccessful tests drop off and so on. The cycle continues until we have a machine that is excellent at racing a car in a particular game. No humans are involved and the machine can sit there and learn on its own. The machine that beat humans at Go – the Chinese strategy game once considered too advanced for computers to beat, was won by a machine that had played itself millions of times, testing and continuing with strategies that worked and removing strategies that didn’t. It managed this all within 4 hours, much faster than the upwards of 10000 hours its human counterparts would have had to play to achieve a similar feat.
The ability to train a Neural Network to be more efficient, faster and able to process more data than a human, or specialist programme is one of the key reasons why Machine Learning is already such a game changer, particularly within automation & robotics industries.
Read part 3 of our Machine Learning series Neural Networks in Automation, or refresh your knowledge with part 1 Neural Networks, Machine Learning and how they work.