The Future of Artificial Intelligence and Reinforcement Learning

Tell us about yourself and the work you do.

I am a professor and also a scientist. I study a particular kind of machine learning called reinforcement learning. The heart of it is to understand how the mind works and how we can recreate the mind’s abilities in computers. I work at the University of Alberta, at Amii and in a satellite laboratory of DeepMind in Edmonton. 

What are machine learning and reinforcement learning?

To understand what artificial intelligence (AI) is, the best way is to think about people and the capabilities of people. People interact with the world, they perceive the state of the world, they model the world, they understand it and they make decisions. They choose actions that influence outcomes. Artificial intelligence is about recreating humans’ ability to choose actions that influence the world in computers and machines.

“In reinforcement learning, we are interested in agents that have a life of their own.”

There are several kinds of machine learning. The most well-known kind is supervised learning where computers learn from examples. It is a lot like pattern recognition. However, in reinforcement learning, we are interested in agents that have a life of their own. They are capable of making a sequence of decisions and experience a sequence of observations from the world, so it is more naturalistic. It is modelled after animals: how does an animal learn by trying things and either attaining rewards or getting punished? Punishments and rewards are not things that exclusively come from other people; they come from the world around us too. For instance, an animal going about its life may be rewarded by finding food and punished by injuries.

My wife likes to simplify reinforcement learning to: if it feels good, do it. That is pretty much what the idea is. If it feels good, you do it and remember it so you can do the same thing again next time. That is a really powerful principle.

Which areas of research in artificial intelligence and machine learning will have the biggest impact on the future economy?

We are working towards intelligent systems that learn on their own so they can learn from their interactions with the world rather than from prepared data. Right now, the greatest impact of machine learning in economic matters comes from learning from training sets where all the desired behaviour is provided by people having prepared carefully labelled training sets. That is great and it has been powerful but the real exciting steps will come when a machine can learn on its own. For example, if it is making decisions about how to place things on the internet to appeal to users, it could learn from placing things on the internet and then seeing if people react to them or ignore them.

“If we can automate all the decision-making in industry to a good extent and do it well, it will be enormously economically powerful.”

AI is essentially automated decision-making. It is applicable in every place where there are decisions to be made. Ideally, it is best used in environments where decisions have been made over and over again, and there is data and time to pick out the patterns. The internet is one place where there are many decisions being made all the time, such as what to show to people. An elevator might have to decide where to wait to most likely pick up new people. Conversely, faucets in a public bathroom should not be automated to turn on and off on their own. If you are a financial trading system, you have millions of decisions to be made. if you are a company and you have a logistics chain, you have to decide when to order things, when to ship things and when to start production. Those are all decisions that can be made using data about past successes and failures. If we can automate all the decision-making in industry to a good extent and do it well, it will be enormously economically powerful.

What do you see on the horizon for Alberta in artificial intelligence and what role does Amii play in that?

Albertans should be excited about artificial intelligence because it is happening right now and in the next few decades. There are immediate applications for it and all this is happening here in Alberta. Alberta has a large concentration of scientists working in the reinforcement learning approach to machine learning and also generally in artificial intelligence at the University of Alberta and at Amii. We have a world-leading group of 30 professors in AI and 10 or so in reinforcement learning. 

“Alberta has had a focus on artificial for a long time and it is starting to be obvious to the rest of the world that we are punching above our weight.”

Alberta has had a focus on artificial for a long time and it is starting to be obvious to the rest of the world that we are punching above our weight. We have a leading position in it and it is going to have a bigger impact on what we do in our economy.

Why did you choose to pursue your research in Alberta and Canada? 

It is a little bit hard to say how it has come about. The University of Alberta was always a leader in AI and I was drawn to that. Now, we are overwhelmingly the largest and most impactful group in my subspecialty, reinforcement learning. We have 10 principal investigators in our reinforcement learning and AI lab. I cannot think of any place that would be as exciting to work in as Alberta’s artificial intelligence ecosystem.”

Alberta is strangely sticky. People who come here like it. It is a cold place and we should be annoyed about this, but we like it because it is a good place to raise kids or maybe and the people are friendly. Alberta has the friendliness of the middle of the country and yet has the excitement of the coast because we still have lots of immigration and different kinds of people. Alberta is a very tolerant, open and accepting place to do good work. I do not want to go anyplace else.

Who and what would you pitch to make Canada a global leader in artificial intelligence?

I am a big believer in the power of individuals. It is our individual decisions that make the difference rather than what government tells us to do or tries to influence us to do. When it comes to AI, both in terms of the scientific and entrepreneurial aspects, it is really in the hands of our people. I urge people to see what they can do or what company they can start in the AI space. Think about the journey of learning you can embark on about the science and how you can contribute to what is known about how our minds work.