Seqsee Diary

This is a short entry just to publicize the existence of a Wiki for Seqsee. It is world readable but not world editable. I am also working on a web site where I would put longer articles that would not easily fit into a wiki format or a blog format, and on which I can spend more time polishing.

Relations in Seqsee play a role similar to that played by bonds and bridges in Copycat and Metacat. As mentioned in the previous entry relations in Seqsee need to be generalized to take care of what I called weak relations. Because of this transition, this entry is tentative but not very much so.

 
 

Let us stay concrete with an example. Let group A be [2 3 4] and group B be [3 4 5], with neither having yet been seen as ascending groups. Then the function “find relation” returns empty-handed: in Seqsee, all relations are based on categories shared by the two ends. Without a common category, no relation is discovered.

 
 

Let us proceed to a subsequent time when both groups have been seen as ascending. Now the function “find relation” does succeed and discovers the relation based on this shared category: in going from group A to group B both the start and the end have gone up by one. ‘Start‘ and ‘end‘ are words that are applicable to things that are ascending groups. For mountains ([3 4 5 6 5 4 3], for example), applicable words include ‘peak‘, a word that would scant make sense for ascending groups.

 
 

This restriction of sharing a category in order to be related might seem restrictive. However, several relations are of this type. Those that are not include the relation between George W. Bush and the United States. One is a country and the other is not. At least in the domain of sequences, though, this restriction does not seem crushing. There are also good reasons why similarity must play a role in describing differences. A quote from Michael Agar’s “language shock” has vividly stuck in my mind. He was describing the classic experiment of Berlin and Kay where they tried to make sense of differences in color terms in different languages. Using the spectrum, they asked natives to pinpoint what the central example of the color red was. I quote from page 75:

 
 

Berlin and Kay made sense of differences between languages in terms of similarities that humans share. Before their work, linguists used the spectrum to highlight the differences. What Berlin and Kay did was to show how the differences showed up against the background of human universals of color perception, abilities wired into the eye and brain.

 
 

Categories in Seqsee provide just such a backdrop of similarity with which to compare. To drive the point home—at the risk of repeating myself or of stating something obvious—let me point out that a question like “how does evolution differ from gravity?” is silly, unless you make both the things being compared belong to the same category, say, the category “scientific theory”. Now the two can be compared in terms of how much people believe them, how mathematically quantifiable they are and so forth.

 
 

So far so good. Let us go into deeper waters. Consider the groups [ [2 2] 3 4] and [3 [4 4] 5] where the ‘2 2′ has been seen to be just a funny 2, and the two groups have been seen as blemished versions of [2 3 4] and [3 4 5] respectively. How do we now describe what changed? Another quote from Michael Agar from the same chapter is relevant (page 82):

 
 

[...] similarities are always too simple and differences are always too complicated. The academic tendency is to put together a framework of similarities and then go forth and understand the world in terms of it. With a good framework, you can understand something of each piece of the world, no doubt about it. But each piece of the world will not just consist of that similarity; it will consist of that similarity plus plenty of other things working within the system where the similarity is found.

 
 

The author was not talking about a program understanding sequences, to be sure, but rather about people trying to understand other cultures. That does not make the quote any less true here.

 
 

Relations in Seqsee are programmed to note several other characteristics like the blemishes present and their positions. This does give it some framework in which to describe differences. This framework needs to be at the same time stronger and more flexible. Relations like that between [3 4 5] and [8 9 10] are not supported yet. I did not reach the stage of talking about weak relations as I had expected to, but I will get there in the near future.

This entry will try to pinpoint what is still not done in Seqsee. Here is the programme: I will trot out a sequence at a time and point out whether Seqsee would get that, and if the answer is no talk about what the missing pieces are. Often, it is not so much that pieces are missing as that they have not been fit into the puzzle.

 

1 2 3 4 5…

 

Yes, Seqsee gets this. I wanted to put in at least one thing that works.

 

1…

 

Nope, not just yet! This requires a piece that sits half-built in my workshop: long-term memory. By long-term I mean a memory capable of remembering things between runs. One of the next few entries will spell out what I am thinking.

 

1 1 2 3 1 2 2 3 4 1…

 

Yes, kind of. Seqsee sometimes gets this, but has a way of unseeingly staring at the solution. Rules will help it see this sort of thing much faster, and rules shall be the topic of an entry in the next couple of days.

 

2 15 16 1 2 3 8 9 10 11 5 6 7 8 9 11 12 13 14 15 16…

 

No. For most sequences that Seqsee shall deal with, the next term after seeing several terms is more or less well determined. Not in this case, though. The next seven terms will form an ascending group but it could be any such group. What is missing from Seqsee that prevents it from seeing such sequences is the notion of a weak relation. A weak relation is exactly of the sort present in the sequence: it under-determines the right hand side given the left-hand side. In this particular sequence, the relation is that the right hand side is also an ascending group but longer by one.

 

What is also missing is the ability to create random groups meeting a specification. In this particular case, the subsequent group has the specification “ascending group of length seven”. There are other less spelled out constraints like “the group shall start with a smaller number (and not 1 million)”.

 

A third bit that is missing is that I do not have a mechanism yet to look for a group meeting a specification already present in the workspace. Currently, since sequences that Seqsee deals with are deterministic, it is enough to check whether a particular thing is in the workspace at a particular point. I would need to extend that to be able to check whether that particular type of thing is in the workspace at a particular point.

 

1 1 2 3 1 2 2 3 1 2 3 3 1 1 2 3…

 

No! After the 12th term, Seqsee hits the big snag, and it has not yet learned to recover. Rules, again, are implicated in this.

 

All the foregoing might suggest that there is much that Seqsee cannot do. This is true: even when Seqsee solves all the sequences on this page it will be no less true (just as it is not true that people living in high-rise buildings are somehow closer to God). I do hope that by the time I am done with my Ph.D. Seqsee would at least get all these sequences.

In my last post I mentioned how I think Metacat’s temporal trace did not get it exactly right when it allowed important but ancient events to be accessible without also allowing recent events just below the importance threshold to be accessible. I just realized that the stream of thought in Seqsee makes the exact opposite error: it remembers the most recent thoughts but not important thoughts that are somewhat old.

 

Let me review briefly how the stream of thought currently works. After that I will be able to explain what I am thinking would be required to merge both views.

 

Seqsee contains “thoughts”, which could be looked at as codelets that are handled in a special way. When a thought is “the current thought”, its fringe is calculated. The fringe contains things related to the current thought. More on this later at some point but suffice it to say for now that if the fringes of two thoughts overlap then they might be similar in some way. A concrete example would help: the groups [2 3 4] and [7 8 9 10] may both have “ascending” in their fringe. Overlapping fringes is the primary method by which similarity discovery happens in Seqsee.

 

Another thing that the “current thought” does is to suggest potential actions to be taken. For instance, if the current thought is about a group [2 3 4] then a possible suggested action could be “try to make the group longer”.

 

The stream remembers the last few thoughts and their fringes. The most recent thought’s fringe is present to the greatest extent. Currently, the stream only remembers the last 10 thoughts. The stream then checks the fringe of the current thought for overlap with the stored fringes. If a significant overlap is discovered with a prior thought then a codelet is launched to explore whether the two are related (thus possibly precipitating a bond formation).

 

In Seqsee, this overlap was looked at A Good Thing. After all, uniformity is a good thing. Finally, we want to discover a rule that explains the sequence harmoniously and is itself simple. Therefore, Seqsee currently tries to make the most of the good news that overlap happened by responding with attempted bond formation. However, fringe overlap can also signals problems. If you are having the same thought over and over and over again, the fringes will, of course, overlap.

 

The temporal trace is sensitive to the repetition of certain types of events. There are codelets that try to find patterns amongst snags that the program hit. If snags were one type of thought in Seqsee then a new snag— if it was similar to another recent snag— would awaken Seqsee to this snag repetition. Unlike groups with overlapping fringes, snags with overlapping fringes are bad news and Seqsee can take corrective action.

 

This does not yet I mean that the stream of thought can take over all the work that the temporal trace did. The trace also remembered the order of events and that has not yet been integrated into the stream. However, whether (and how much) we remember the order of events is not clear to me. I remember several conversations with two friends at different times. While I have some sense of the ordering of conversations with the first friend (i.e., I have some sense of which conversation which conversation happened before which), and also some sense of ordering of the conversations with the second friend, I cannot at all re-create temporally the set containing both sets of conversations: I cannot say if this conversation with the first friend happened before or after that conversation with the second friend. I suspect that the ordering I could remember within each individual set was something that I calculated (figured out) instead of really remembering. I remember Roger Schank talking of something similar in his book “dynamic memory”.

 

The previous paragraph notwithstanding, we do remember some amount of ordering information, I think, and the stream needs to be able to do that at least a little bit. I also need a way to remember important but old thoughts.

 

I think I am happy with the way things are going with this “integration”. I am curious to see how the story unfolds!

I have finally come to a stage where I can realistically think about how I will implement self-watching in Seqsee. The most prominent FARG program that watches itself is Metacat: after all, it is subtitled “a self watching cognitive architecture for analogy making and high level perception”. As I was reading Jim Marshall’s dissertation I could not help but feel that something very similar would do the trick for Seqsee.

 

Here is a brief description of Metacat’s self-perception. The crucial components include the temporal trace and two new codelets, the progress watcher and the jootser. The temporal trace can be, in Jim’s words, “thought of as a short term memory for storing the recent past events during a single run”. Codelets can examine these structures, and Jim Marshall goes on to say on page 163 of his dissertation:

 

From a theoretical standpoint … the structures stored in the trace are no different from other types of perceptual structures stored in the workspace, since they are all subject to processing by codelets, so in an important sense, the temporal trace and the workspace can be identified with each other— implying a kind of “level collapse” between Metacat’s cognitive and sub cognitive levels (see [GEB, chapter 20] …). Thus, conceptually, the same set of general processing mechanisms is responsible for perception, for self-perception, and for self-control in Metacat.

 

I agree absolutely with the sentiments mentioned in the quote above. I feel that this is how things should be. What I am not sure about— given my possibly incorrect understanding of Metacat— is how much of a level collapse actually happens in Metacat, as it currently exists. Please allow me to elaborate.

 

The way both Copycat and Metacat perceive letter-strings involves adding descriptions of various objects, noticing relations and creating groups. All this activity is crucial. Almost nothing like this happens at the level of the temporal trace: there are no bonds formed between the events stored, nor are descriptions added to events to allow similarity discovery (though some description is added, namely the important themes present when the event occurred). There are certain types of events whose repetition Metacat spots: repeated snags and repeated clamping, and is able to take evasive action. Other repetitions—like attempting to create the same relation between the same two objects and repeatedly destroying it—goes unnoticed (I think). Thus, while the trace does go somewhat towards doing “the same sort of processing” for the strings and the trace, it could go further still.

 

I must hurry and point out that the two types of processing cannot be identical: the strings are “permanently” present, while the things in the trace are ephemeral in the sense that very old parts of the trace (things that happened 50000 codelets ago, say) should be pretty nigh inaccessible. Mike Gasser would say that the strings are being read (and we can dart our eyes back and forth over them), where as the trace is being “listened to”, sort of.

 

Another slightly problematic thing with Metacat’s trace, I felt, was that important old events are remembered, where as relatively unimportant stuff that just happened is not.

 

My next entry would probably be about some half-baked ideas about how I could use the stream in Seqsee to also act as the trace, sort of, and get more of the “same processes” to bear on the event history.

This post would never end in my dissertation; but it is a recent day of hacking I enjoyed, and a lot of good can come out of it.

What I did was simple: until now, I had coded the UI imperatively, and I merely shifted to a declarative style. Thus, I went from something like

 
 

$button_frame = $MW->Frame()->pack(-side => ‘top’);

$button_frame->Button(-text => ‘quit’, -command => sub { exit })

->pack(-side => ‘left’);

 
 

to something like:

 
 

[widgets]

button_frame MW Frame top

…

 
 

[buttons]

quit: exit

 
 

The main reason to do this was to make the UI more configurable and to get rid of lots of duplicate code inherent in an imperatively coded UI. However, a far nicer side effect has emerged. In the process of going to declarative style, I had to handle the special code I had cut to handle widgets I had specifically crafted for Seqsee. This made me start thinking about what these widgets were, and I realized that these are just views into the innards of Seqsee, and I could have arbitrarily many of them, even with overlapping functionality. Until now, I just had viewers for individual components of the program. I used one to visualize the stream, for instance. In this mindset, it did not make sense to have multiple viewers for the same component. Now under the new mindset “it’s just a view” that restriction is lifted. I am free to have as many different widgets as I fancy.

Here are just some potential widgets that would help me understand what the program is doing much much better.

• A strength visualizer that would let me compare how strong various components are. This would be crucial as the relative probability of choosing a particular object is strongly modulated by its strength. So having a good way to calculate strength is crucial, and checking if I indeed have a decent way to calculate requires a visualizer.
• A workspace activity visualizer, something that tells me whether the program is thinking that things are going well. Think of it as a temperature visualizer: it shall show me how the temperature has been changing over time. Several activities like destroying objects depend on this number.
• Any other as required when I get stuck and need to see what Seqsee is thinking.

There is also code duplication amongst the various widgets and I should eliminate these if I begin producing several widgets.

All I have been saying, briefly, is that the MVC pattern is often a very good idea, and it allows you the freedom to dream up strange V’s.

Being able to look at what is going on from different perspectives is crucial. Hey, for all I know, I might be able to work this into my dissertation!

Original post: Nov 21, 2006.

   
 

For the last couple of days I have been thinking about how to deal with conflicting groups. Here is the issue: suppose that we are trying to solve the sequence “2 1 2 2 2 2 2 3 2″ then the right way to see the sequence is as “[2 1 2] [2 2 2] [2 3 2]“. However, can the following two groups coexist: the correct [2 2 2] just mentioned and the [2 2 2 2 2] group that jumps out at you?

Here are some relevant comments from Doug in FCCA, while talking about his program Jumbo (page 112):

   
 

A certain kind of parallelism is thus nonexistent in Jumbo— namely, the parallelism that would correspond to consciously entertaining two rival and very different full-fledged thoughts at once, or consciously perceiving something in two contradictory ways at the same instant. At any time, Jumbo has but one overall interpretation (the current contents of its cytoplasm). On the other hand, parallel dipping into or scouting out of alternative realities (i.e., slightly different alternative states of the cytoplasm) is permissible in Jumbo; in fact, it is of the essence. Jumbo is free to dip into, or muse about, many possible counterfactual worlds, as long as they are “close” to the current world, not radical variants thereof. In and of themselves, such musings do not affect the cytoplasm, which is why we call them “musings”— they are like inconsequential little daydreams.

Doug mentions though that the HEARSAY-II program that inspired Jumbo (and indeed all FARG architectures) does support contradictory views, but that he finds this not very cognitively plausible. However, consider now the following situation where perhaps we can and do notice parallel interpretations and the understanding of the situation crucially involves being aware of the multiple interpretations. Understanding physics involves knowing that both the particle and the wave interpretation can coexist. Alternatively, in the case of the sequence “1 2 2 3 3 4 4″ the two interpretations “1 [2 2] [3 3]” and “[1 2] [2 3] [3 4]” are possible.

   
 

In trying to choose the strategy to deal with conflicting groups, one must also decide what to do when the program finds conflicting groups. One way is to prevent conflicting groups from ever existing (by demolishing one of the conflicting groups as soon as the conflict happens), and the other way is to keep both groups around but keep generating codelets that try to resolve the conflict. This latter way is like a nagging doubt that something is wrong.

   
 

What I am currently considering is the first alternative, of banning conflicting groups. My interpretation of when groups are in conflict is narrower, and the groups [1 1], [2 2] and [1 2] can peacefully coexist. I shall now try to define exactly when groups do not like each other, for I shall need a clear definition in order to program.

   
 

1. If the two groups span exactly the same part of the sequence, they conflict. In other words, no part of the sequence can have two distinct interpretations.
2. If neither group completely covers the other, they are not in conflict.
3. If one group is just a single element of the other, they are not in conflict. This is an obvious rule because if a larger group were composed of other smaller groups then without this rule the larger group would conflict with its own parts.
4. If one group consists of a few elements of the other, they are in conflict. Thus, [2 2 2] and [2 2 2 2 2] would conflict.

I need to think a little more about all this.

 

Original post date: Nov 20, 2006

It has been a long time since I have posted. Perhaps it was just that I had not got into the habit of writing regularly. In the intervening days, several things have happened, including an internship at Google and finishing my research proposal. Since yesterday, I have started using index cards to organize my thoughts. So far, it is proving to be very useful! For several things that I have read in the recent past, I intend to go back and create notes for all over again. It is nice to have stuff written on cards that I can shuffle around and reorganize. An important criterion that seems to make this endeavor successful and useful is having a decent set of bins into which to put cards. Recently I was working on the potential structure of my final dissertation, and the chapters are what I am using, more or less, for the bins I am feeling a little disoriented while writing this and I am really writing only to get back into the flow. A more sensible, better-planned entry will be shortly forthcoming.