02 - Introduction to CS7637

01 - Preview

In this lesson, we’ll talk most specifically about what you should expect from CS7637. We’ll start by talking about the learning goals, the learning outcomes, and the learning strategies that we’ll use for this class.

Then we’ll discuss the class projects and assessments. That will lead us to talking about something called Computational Psychometrics, which is one of the multi-weighting principles behind the projects in this class.

Next we’ll talk about the Raven’s Progressive Matrices test of intelligence.

I think you’re going to find it fascinating. The Raven’s Progressive Matrices test of intelligence are the most commonly used tests of human intelligence. And that test was the target of the projects in this class. Very ambitious, you’re going to enjoy it. FInally we’ll discuss something commonly reoccuring principle in this class and you should be on the lookout for them.

02 - Class Goals

There are four major learning goals for this class. First, you’ll learn about the core methods of knowledge-based AI. These methods include schemes for structured knowledge representation, methods for memory organization, methods for reasoning, methods for learning, [UNKNOWN] architectures as well as methods for meta reasoning. Meta reasoning is reasoning about reasoning. Second, you learn about some of the common tasks addressed by knowledge-based AI, such as classification, understanding, planning, explanation, diagnosis, and design. Third, you will learn ways AI agents can use these methods to address these tasks. Fourth, you learn the relationship between the knowledge-based AI cognitive science. Using theories of human cognition to inspire their design of human level, human-like AI and using AI techniques to generate testable hypothesis about human cognition.

03 - Class Outcomes

What are the learning outcomes of this course? At the conclusion of this class, you will be able to do three primary things. First, you’ll be able to design, implement and evaluate and describe knowledge-based AIs. The design and description of knowledge-based AI agent is really the first learning goal.

In order to be able to design an agent, you need knowledge of the methods of knowledge-based AI. Second, you will also be able to use these strategies to address practical problems. This learning outcome addresses the second learning goal, where you will be able to [INAUDIBLE] the relationship between AI agents and real world problems. Third, you’ll also be able to use the design of knowledge-based AI agents to reflect on human cognition and vice versa. This addresses the fourth learning goal.

04 - Class Assignments

During this course, you’ll complete a variety of different kind of assessments.

These assessments play different roles.

First, they help you learn by demonstrating and testing what you know. Second, they help you reflect on what you’ve learned.

Third, they help us understand what material is being taught well, and what is not being taught well. The main assessment of the projects. You’ll complete a series of programming projects in designing AI agents that address a pretty complex task. We’ll talk a little bit more about it in a few minutes.

Second, written assignments. We’ll compete a number of written assignments that will tie the course material to the projects. Third, tests. There will be two tests in this class using the content of this class to introduce a broad variety of problems. Fourth, exercises. Throughout the lessons there’ll be a number of exercises to help you evaluate and manage your own learning. Fifth, interactions. We’ll be looking at the interactions of the forum and other places to get a feel for how everyone is doing and how can we help improve learning

05 - Class Strategies

In this class we’ll use five main learning strategies. First, Learning by

Example. Almost every lesson of this class starts with an example of the type of reasoning we want you to learn. The example that runs throughout the lesson, could demonstrate that reasoning. Second, Learning by Doing. In most lessons, the lesson will end with a multi-part exercise, where you are doing the exact reasoning that you learned in that lesson. There’s first you see an example, then you do a similar example yourself. Third, Project-Based Learning.

The class is largely structured around a series of challenging projects. And you will frequently be asked to relate each lesson you learn, to the projects in the class. Personalized learning. Personalization permeates throughout this course. You can watch the lessons in any order you choose, and at your own pace.

You can choose which concepts to focus on, and everything the assignments.

You’ll receive personal feedback on every exercise throughout the course.

Fifth, Learning by Reflection. At the conclusion of each lesson, you’ll be asked to reflect on what you learned in that particular lesson.

At the conclusion of each project, you’ll write a designed report that will reflect on the experiments that you did as part of the project. We’ll also use other learning strategies as needed, such as collaborative learning.

06 - Introduction to Computational Psychometrics

Let us talk about Computational Psychometrics a little bit.

Psychometrics itself is a study of human intelligence, of human aptitude, of human knowledge. Computational Psychometrics for our purposes, is the design of computational agents that can take the same kind of tests that humans do, when they are tested for intelligence or knowledge or aptitude.

Imagine that you design an AI agent that can take an intelligence test.

After designing it, you might want to analyze how well does it do compared to the humans on that test? You might also want to compare the errors it makes with the errors that humans make. If it does as well as humans do and if its behavior, its errors are the same as those of humans, you might conjecture then that perhaps its reasoning mirrors that of humans.

In this class, we are going to be designing AI agents that can take the Raven’s Test of Intelligence. In the process, we will want to use this agents to reflect on how humans might be addressing the same intelligence tests.

07 - Ravens Progressive Matrices

The class projects will be based on the Raven’s Progressive Matrices test of intelligence. This test was written in the 1930s to examine general human intelligence. It consists of 60 multiple-choice visual analogy problems.

The Raven’s test is unique among intelligence test in that all problems in the Raven’s test are strictly visual. No words. It is the most widespread, the most commonly used, the most reliable test of intelligence. The Raven’s test consists of two kinds of problems, two by two matrix problems and three by three matrix problems. We’ll also consider a special case of two by one matrix problems. They’re not part of the original test. But they will providing a baseline for starting to construct AI agents. Your project will be to implement AI agents that can solve problems like those that appear in the Raven’s Test of Intelligence. Let’s look at a few sample problems right now.

08 - 2x1 Matrices I

Let us consider an example. We are shown initially three images, A, B and

3. And you have to pick a candidate for the D image here on the top right.

And it can be one of these six candidates that would go here in the D image.

Given that A is to B, as C is to D, what would you pick among the six choices at the bottom to put into D?

09 - 2x1 Matrices I

>> Very good, that is in fact the correct answer for this problem. Now, of course, here’s a situation where a human being,

David, answered this problem. The big question for us would be, how to write a air agent that can solve this problem?

10 - 2x1 Matrices II

The previous problem was pretty simple. Let’s try a slightly harder problem.

Once again, we’re given A, B, C, and D. Given that A is to B, what would we pick between 1, 2, 3, 4, 5, and 6 to put into D?

11 - 2x1 Matrices II

>> This of course, raises another issue. How do we do it?

How do you solve the problem? Why was it so easy for you? Why is it so hard for

AI? You remember the question. When David was trying to solve this problem, he looked at the relationship between A and B and then marked it to C and some image here. But one could have gone about it the other way.

We could have picked any one of these images, put it in the D, and ask whether this would be a good fit. So in one case, one can start from the problem and propose a solution. In other case, one could take one of these solutions at a time and see if it matches. Two different strategies

12 - 2x1 Matrices III

Let’s try an even harder problem.

And as you solve this problem, think a little bit about how do you go about solving it.

13 - 2x1 Matrices III

What do you think is the correct answer, David?

>> So on the left, we have the same two frames we had in the first problem.

So first, I thought that the circle in the middle disappears, so the triangle should disappear.

But none of these options match that.

So then I went back and looked and said, the other way we can think about this is to say the circle on the outside disappeared but the circle on the inside grew, since their both circles we can’t really tell the difference between those, but once we know that the correct answer is not just the big square, we can say the only logical conclusion is to say that the square disappeared, and the triangle grew.

So the answer has to be three, the big triangle.

>> That’s a correct answer, David.

But notice something interesting here.

This is an example of generate and test.

You initially generated an answer from it and then tested it against the choices of a level.

Yet the test failed, so you rejected a solution.

And you generate another solution.

For that one, the test succeeded, and you accepted it.

14 - 2x1 Matrices IV

I like this problem.

This one is really interesting.

Everyone, try to solve this one.

15 - 2x1 Matrices IV

Okay.

What do you think is the right[br]answer to this one, David?

>> So what I said is that it looks like[br]there’s a 180 degree rotation going on.

So this frame is rotated 180[br]degrees to get this one.

So I’m going to take C,[br]rotate it 180 degrees to get number six.

>> That’s a fair answer, David.

Well done.

But notice there is another[br]possible answer here.

Two is also a possible answer.

Why is two a possible answer?

Because one can imagine that[br]B is really a reflection of

A across a vertical axis, and[br]that way if we think of a vertical axis on C, then two will be the[br]deflection/ of C on the vertical axis.

So both two and[br]six are good answers here.

And one question will then become,[br]which one do humans pick, do they pick six or do they pick two?

And second which one[br]should an AI program pick?

Six or two, and how would you make sure that the AI[br]program picks two or six, and if you are thinking I am going to give[br]you the answer, sorry to disappoint you.

I’m going to leave this as a puzzle for[br]you.

Your AI program will[br]address this problem

16 - 2x2 Ravens Progressive Matrix I

Okay, here are some two by two problems.

Two by two matrix problems.

The situation is somewhat similar,[br]but not exactly similar.

Once again, we’re given A, B, C, and D[br]is unknown, and we’re given six choices,

1, 2, 3, 4, 5, 6, and we are to pick[br]one of these choices and put it in D.

What is different here however is that,[br]this time it is not just that A is to B as C is to D but[br]also A is to C as at B is to D.

That’s why it’s a two by two matrix.

So it’s not just the horizontal[br]relationship that counts but it’s also the vertical[br]relationship that counts.

17 - 2x2 Ravens Progressive Matrix I

Okay David, are you ready for this one?

What do you think is the right answer?

>> So I said that 3 is the right answer.

Going left to right, the square became[br]clear, so the circle becomes clear.

Going top to bottom,[br]the square becomes a circle, so the square becomes a circle.

So the 3 preserve the relationships[br]both horizontally and vertically.

>> That’s the right answer!

But this was an easy problem.

Let’s see how you will do[br]on a harder problem David.

18 - 2x2 Ravens Progressive Matrix II

Okay, here is a slightly harder problem.

Why don’t we all try to solve it?

19 - 2x2 Ravens Progressive Matrix II

What do you think is the right[br]answer to this one, David?

>> So this one reminded me of[br]that third problem we did.

The first thing I thought was that[br]it looks like the entire figure is rotating.

So I”m going to say that the figure[br]will be this with the triangle pointing up and to the left, or[br]up and to the right.

However, looking over here, there are no[br]answers that have the triangle rotated.

So second thing I think is maybe[br]just the outside figure is rotating.

The square here rotated while[br]the circle stayed stationary.

So the circle here rotated,[br]while the triangle stayed stationary.

Because it’s a circle we can’t actually[br]tell a visible difference between the two, but it seems to be the one[br]that most preserves a relationship between A and B.

Similarly, between A and C, the square on the outside[br]becomes a circle on the outside.

It’s the same thing here, the square on the outside[br]becomes a circle on the outside.

>> That’s a good answer David.

Here is another point to note.

Supposing we put one here in D,[br]then C and

D become identical, but[br]A and B are not identical.

Is that a problem here?

Not really because we can imagine[br]that the outer image in A is rotating to become B.

And we can imagine that C in the outer[br]image and C sort adding to become D.

It just so happens that the resulting[br]image is identical to the inchman.

Note that this will be a challenge for[br]an AI program.

The AI program will have[br]to generate solutions.

It will have to evaluate solutions. [INAUDIBLE] programmed solutions[br]that it self generates.

What kind of knowledge representations[br]would it allow it to generate good solutions?

What reason strategies would allow[br]it to generate plausible answers?

20 - 2x2 Ravens Progressive Matrix III

Let us try one more problem[br]from this two by two set.

21 - 2x2 Ravens Progressive Matrix III

For this one, David,[br]what do you think is the right answer?

>> So I put that the right[br]answer was number 5.

Number 5 looks like it preserves[br]the reflection that’s going on across the horizontal axis.

>> That’s a good answer, David, but[br]note that 2 is also a plausible answer.

One can imagine that the image A is[br]rotating by 90 degrees to the image B.

And if we rotate the image in C by[br]90 degrees, we’ll get the answer 2.

So both 5 and[br]2 here are plausible answers.

Okay, and the question arises,[br]which answers do most humans choose?

Why do they choose the answer[br]that they do choose?

What in their cognition is telling them[br]to choose one answer over the other?

And then how can we write an AI program[br]that can choose one answer over the other?

>> An interesting thing to note[br]about this problem as well is that

I phrased this as reflections.

Ashok defined number 2 as rotations.

But it’s possible that we[br]could do this a third way.

Instead of looking at rotations or[br]reflections, which are kind of semantic ways of[br]describing the transformations, we could look at which image[br]completes the overall picture.

Here, number 5 would seem[br]to be the right answer, because it finishes creating[br]the square we see forming.

So that would be a strictly visual[br]way of doing this problem as well.

>> One more thing to note here.

So far we have been talking[br]about horizontal and vertical relationships and[br]not diagonal relationships.

So A is to B as C is to D,[br]and A is to C as B is to D.

What about diagonal relationships?

Should A to D be as B to C?

If we add that additional constraint, then the choice between 2 and[br]5 becomes clear.

5 is the right choice because that’s[br]the only way we’ll get A is to D as B is to C.

22 - 3x3 Ravens Progressive Matrix I

Now let us look at some[br]three by three problems.

This time the matrix has three rows and[br]three columns.

We are given not just A, B, and C.

We are given A, B,[br]C in the first row, D, E,

F in the second row,[br]G and H in the third row.

We do not know what[br]will go here under I.

Again, we want horizontal, vertical,[br]and diagonal relationships.

A is to B is to C, as D is to E is to F,[br]as G is to H is to what?

And similarly vertically.

As well as diagonally.

If we take all three of those[br]constraints, rows,columns and diagonals, which would be the correct choice among[br]one through six to put under the square?

23 - 3x3 Ravens Progressive Matrix I

What do you think is[br]the right answer David?

>> So, looking horizontally every row[br]except for the third row has a diamond.

Vertically every column except[br]the third column has a diamond.

And diagonally the shapes are preserved[br]if we imagine C’d be coming down here,

G coming up here.

So it seems like all signs[br]point to number one.

>> That is indeed the correct answer.

But, you said something very important[br]about this particular problem.

And that is that, we can imagine that[br]these rows are rotating, so that this C gets aligned with D, and D gets aligned[br]with H, as if they were on a diagonal.

One more point about this.

Once again David was able to solve[br]this problem within a few seconds.

What about an AI program?

How could the AI program[br]solve this problem?

What representations would it use?

What reason strategies would it use?

Would it induce something[br]from the first row?

Would it learn something from the first[br]row and apply it to the second row?

If so, how would it do that induction?

24 - 3x3 Ravens Progressive Matrix II

Okay, let’s try a harder one.

I can tell you that[br]this problem is hard.

Even I have difficulty[br]with this problem.

So let’s take a minute, think about it.

Again, this is a three[br]by three matrix problem.

You’re given the six[br]choices to the right.

25 - 3x3 Ravens Progressive Matrix II

Okay, David, are you ready for this one?

What answer did you come up with?

>> So after pondering it for far too long, I finally came to[br]the answer that the answer is five.

This problem is very different than[br]the one that we’ve seen in the past, because it’s the relationship between[br]the first two frames in each row and column that dictates the third frame.

The relationship is called exclusive or.

If the box appears in both[br]the first two frames in a row or column, it doesn’t[br]appear in the third one.

If it appears in neither[br]of the first two frames, it doesn’t appear in the third one,[br]but if it appears in exactly one of the frames,[br]it appears in the third one as well.

So here the top right square appears in[br]both A and B, so it doesn’t appear in C.

The top left and[br]bottom left squares appear in B only, so they do appear in C.

If you look across the rows and[br]down the columns, you’ll see that, that relationship holds true for[br]every row and column.

And in fact, both rows and[br]columns give us five as the answer.

So if the row, the bottom left and[br]bottom right appear each in only one of those frames,[br]while the top right appears in both.

So bottom left and[br]bottom right appear here.

For the right column, top left and[br]top left appear both times, while bottom left and[br]bottom right each only appear once.

So the answer here again is[br]bottom left and bottom right.

>> That was excellent.

Five looks like the right answer.

Now did you follow the same[br]strategies this time that you had followed the last time?

>> No, definitely not.

In the earlier problem, we saw that the[br]first row had relationships that carried through for every single row.

It didn’t really matter what[br]order the figures were in.

All that mattered was[br]the relationships between them.

Here, if we were to switch around some[br]of the figures it would change what the other figures would have to be.

And change the nature of the[br]relationship inside each row and column.

So we’ve used a fundamentally[br]different kind of reasoning process.

And that’s part of what[br]makes this problem so difficult is that it’s unlike[br]the ones we’ve seen in the past.

>> That’s very interesting,[br]couple of other things to note as well.

I wonder whether this time[br]you actually pick one, put it under here, and then solve[br]with the completed pattern or not.

And if they did not succeed, you pick[br]the second one and put it here and solve the patterns succeeded or[br]not and went through it systematically.

And finally came up with five because[br]that fit the pattern the best.

If that were the case, then this would[br]be a different strategy from looking at the first row and the second row and[br]the first column and the second column, inducing some rule[br]and applying it with the third row.

One other thing to note, something very[br]interesting about knowledge based AI.

We can ask ourselves, how is David solving these problems,[br]if we can figure out, if we can generate hypothesis about how[br]David is solving these problems, then that will inform how we can build an AI[br]agent that can solve these problems.

There is a side where we’re going[br]from human cognition to AI.

Alternatively, we can write an AI[br]program that can solve these problems and by looking at how we are programmed[br]to solve these problems we can generate hypotheses about how David[br]might be solving problems.

That’s going from the AI side[br]to the cognitive science side.

26 - Exercise What is intelligence

If you are designing a AI agent[br]that can take an intelligence test.

That raises the question,[br]if we succeed is the AI intelligent.

What do you think, David?

>> So, I would say no, even if the[br]agents that we design successfully solve the intelligence test,[br]they aren’t themselves intelligent, they are just processing signals and[br]inputs in the correct way.

>> What do you think?

27 - Exercise What is intelligence

The problem with David’s answer,[br]in my opinion is that at a certain level humans too are just processing[br]signals and inputs in the right way.

What then makes us intelligent?

Intelligence is hard to define.

In the life sciences,[br]scientists study life, but don’t always agree on[br]a definition of life.

Similarly, in the cognitive sciences,[br]we study intelligence, but don’t necessarily define it.

And knowledge based AI, will take the view that knowledge is[br]central to human level intelligence.

28 - Principles of CS7637

Our discussion of knowledge-based[br]agents in this CS7637 class is organized around seven principals.

Be on the lookout for the seven[br]principals, they’ll occur again and again throughout the course.

Principal number one, agents use[br]knowledge to guide reasoning and they represent and organize this[br]knowledge into knowledge structures.

Principle number two,[br]learning is often incremental.

This connects back to one of[br]the characteristics of problems, where data and[br]experience was coming incrementally.

Number three, reasoning is[br]top down not just bottom up.

We just don’t reason from data, we also use data to pull[br]out knowledge from memory, then we use this knowledge to generate[br]expectations to make sense of the world.

Principle number four, knowledge-based[br]AI agents match methods to tasks.

We’ll discuss a number[br]of different methods.

We’ll also discuss several tasks.

Later, we’ll discuss how AI agents can select specific methods to[br]address particular tasks and even integrate different methods[br]to address complex tasks.

Principle number five, agents use heuristics to find[br]solutions that are good enough.

They do not necessarily[br]find optimal solutions.

This is because of a trade off[br]between computer efficiency and the optimality of solutions.

Our focus will be on using[br]bounded rationality and yet, giving near real-time time performance[br]on [INAUDIBLE] intractable problems.

This happens, because agents use heuristics to find[br]solutions that are just good enough.

Principle number six, agents make use of recurring[br]patterns of problems in the world.

Of course,[br]there are a number of problems.

But even this number of problems are[br]characterized the patterns that occur again and again.

Number seven, reasoning, learning and[br]memory constrain and support each other.

We’ll build theories that are not[br]just theories of reasoning or theories of learning or theories of[br]memory, but we’ll build theories that unify the three of them into[br]one single cognitive system.

These principles will come again and[br]again in this particular class.

So we highly recommend that you[br]take a moment, pause the video and read over the principles once again.

29 - Readings

Throughout this course, we will be[br]using materials drawn from a number of different textbooks and papers.

You’ll find specific references[br]to all these sources for this examples in the Class Notes.

Generally, however, we will use a handful of books from[br]which we will draw a lot of material.

Artificial Intelligence[br]by Patrick Winston.

Knowledge Systems by Mark Stefik,[br]Artificial Intelligence by Elaine Rich and Kevin Knight and Artificial[br]Intelligence: A Modern Approach by Stuart Russell and Peter Norvig.

You don’t have to buy[br]any of these books.

But if you want to increase your[br]knowledge beyond what we’ll discuss in this class, you may want to[br]consider both these text books and the specific references at[br]the end of each lesson.

30 - Wrap Up

Today we started off by discussing[br]the goals, outcomes, and learning strategies for this class.

That led us into discussing the project,[br]the main assessment in this class.

The project builds on Raven’s[br]Progressive Matrices, an apocryphal human intelligence test, an idea[br]called computational psychometrics, the application of computer models[br]to understanding human cognition.

We then discuss the seven main[br]principles of CS7637 that are going to come up again and again in this course.

We recommend you keep and eye out for them because they’ll come[br]up in every single lesson.

This is the end of the introduction[br]to this course, so we hope you’re excited to get started.

We’re going to start off with[br]the fundamentals of knowledge based AI, including one of the most fundamental[br]knowledge structures called semantic networks.

31 - The Cognitive Connection

Let us look at a connection between[br]your class projects and human cognition.

Beginning with psychometrics,[br]psychometrics is the science of measuring human intelligence,[br]aptitude, and knowledge.

Computational psychometrics is[br]the science of building agents that can take the same tests of[br]intelligence that humans take.

All the [INAUDIBLE] building[br]AI agents that can address the Ravens test of intelligence,[br]it will provide opportunities for thinking about human cognition.

While we will be looking only for how well your agents perform on the[br]Ravens test, in principle [INAUDIBLE] psycho- metrics will also look at[br]the kinds of errors that AI agents make.

If the errors the AI agents make[br]are similar to those that humans make, then that may provide[br]a source of hypothesis about human thinking on this[br]Raven’s Test of Intelligence.

It is also interesting to note that[br]people with autism perform about as well on the Raven’s Test of Intelligence[br]as neurotypical people.

This is a little surprising, because[br]in general, people with autism do not perform as well on other tests of[br]intelligence as the neurotypical people.

Note that Raven’s test of[br]intelligence is the only test that consists only of visual[br]analogy problems.

All other tests of[br]intelligence also include a large number of verbal problems.

Might I suggest that some of[br]the thinking strategies that people with autism are better aligned[br]with visual reasoning.

32 - Final Quiz

And now to the quiz at[br]the end of this lesson.

Will you please fill out what[br]you learned in this lesson?

33 - Final Quiz

Great. Thank you so much for your feedback.