The 81st Installment
Machine Learning and Human Learning

By Atsushi Shibata,
Assistant Professor, Master Program of Information Systems Architecture

1. Introduction

Everyone likes to study, right? I personally hate studying by myself, however, the research I do is in the field of machine learning, which amounts to making machines study. Just as the name alludes to, this field of research involves equipping machines with learning capabilities similar to those humans possess, in order to have them solve all kinds of problems. While it sounds simple, there are of course many approaches to this, and many phenomena and challenges arise.

I am also engaged in education, which has led me to find that some of the various phenomena that occur in machine learning also occur in human learning.

Given this, as I introduce you to machine learning here, I would also like to reflect back on human learning as well.

2. Machine Learning and Human Learning

Machine learning can loosely be broken down in a few steps. First, you must prepare a model for your problem. Then, you must test some data to get a feel for the proper behavior and turn that into values. Based on these values, you then adjust the parameters within the model to allow for the proper behavior. The aim of machine learning is to create superior models by repeating this over and over.

To put this in human terms, the model is the theory, or can even be described as the human brain, the data-based behavior is the lesson/skill/practice, and the values are the test. Thus, by repeating this, we can develop better theories and smarter brains, much like in human learning.

However, in both human and machine learning, you cannot just repeatedly make minute adjustments to values and expect things to turn out the way you want them to. There are many diverse problems to be solved, and a wide variety of models out there, all of which are suitable for different things. For example, for the simple task of administering a majority vote, you would want to use a non-complex model, such as a linear separation model. If you wanted to identify objects in an image, you would need to use a model that can process substantial amounts of data, such as a neural network. In fact, it is probably best to pick apart the various steps in image processing and apply different models to them.

I think this all can be applied to humans as well. Simple problems do not require complex logic, while complex problems require you to think using logic that applies to that particular problem. The most effective method is to break down the problem and discriminate the various forms of logic needed.

3. Approaches to Problem Solving in Learning

Although it is usually just simply described as studying models, there are many problems and many approaches to solving them that have been proposed. Among these, I will discuss localized solutions and the curse of dimensionality.

Localized solutions are solutions that are not optimal in the broader sense, but rather, are only effective given specific circumstances. In learning, these cause plateaus in behavior. In machine learning, even if the initial learning progresses, stagnation may occur if the learning is repeated to a certain extent. This is what is referred to as a plateau. With models, we seek values that will provide the optimal output, no matter the data, while gradually, and at times greatly, changing the internal parameters. However, during this learning process, we find solutions that are only optimal under certain circumstances, or in other words, localized solutions. In these cases, stuck with the results that are only valid locally, the learning cannot progress. These are the plateaus in machine learning. In order to get over these, we must revise the learning with what we call momentum, and continually search for parameters even when we seem to have found something optimal. Even in human training, you make many initial gains, however, at some point you will experience stagnation, or a plateau. In sports, this can be caused by malnutrition, lack of rest, or lack of diversity in training methods. How about when it comes to studying? When we first start studying a new discipline, we first apply the knowledge and experience we already have in an attempt to comprehend the subject. At some point, many people experience being unable to understand the content, no matter how hard they try. There are many potential reasons for this, however, as with localized solutions in machine learning, people end up sticking with the amazing theories and experiences they already possess instead of searching for a new solution. In this case, it might be best to emulate machine learning by temporarily putting your existing ideas to the side and trying to see things from a fresh point of view.

The curse of dimensionality refers to the phenomenon where, as the number of dimensions increases, the amount of data that can be expressed increases exponentially, which can lead to insufficiencies in the model’s expressive power and in data volume. For example, you are thinking about the weather tomorrow, but end up judging it by the weather outside today. If it is clear out today, it will be clear out tomorrow, however, if it is cloudy out today, it will rain tomorrow. In this example, there are few patterns, so the learning is simple, but conversely, the prediction accuracy is low. Now, let’s try using the weather from two days prior to make a prediction. This time we input (the weather from the day prior) × (the weather from two days prior) as data patterns, or in other words, we increase the number of dimensions. In order to increase the prediction accuracy, what if we start to add more past weather data into the mix? How about adding atmospheric temperatures and humidity levels into it? Or, how about even seasonal and regional weather data? If you increase the number of dimensions in this way, the prediction accuracy seemingly increases, which gives you the feeling that you have created an optimal model. However, this model now encompasses a number of dimensions including all past weather, atmospheric temperature, humidity level, atmospheric pressure, wind direction, wind speed, and other kinds of regional data. This would be fine if we were talking about a model that covers all these patterns, however, for patterns that cover an extensive number of items, the data volume is relatively small. This is because, for test cases carried out to learn a single pattern, the number of matching situations decreases as the pattern data becomes more specific. Will there ever come a day when the weather matches every past weather condition ever observed by mankind? As the data applied to the model becomes more specific, the amount of correct datum in the pattern decreases. This is the problem in learning that is known as the curse of dimensionality.

In order to avoid the curse of dimensionality in machine learning, we must start with limited patterns using as much data as possible. When it comes to people, we unconsciously discern what information if necessary or not when making judgments. Most people would agree that we do not need to use information on today’s breakfast menu to forecast tomorrow’s weather. However, we also end up convincing ourselves that many factors need to be included when it comes to learning. Should we review arithmetic in order to do college-level mathematics? It is certainly important to understand the fundamentals. However, humans are limited by time. It might be best to attempt to learn things in small increments, for example, first try to do college-level math, and if you find it too difficult then move on to high-school level math first.

4. Conclusion

So, what do you think now? I am sure there are still many of you who are of the opinion that machine learning and human learning are completely different concepts. Of course, I realize that machine learning cannot reproduce all human learning processes. However, if you understand the core properties of learning in general, and reconsider your opinion on machine learning by starting to see it as a resource for your own learning, you might start to find the field more interesting.

I hope you challenge yourself by attempting various approaches to learning during your educational journey.