Virtual Mind
Last updated 1999.
These are notes outlining a "virtual mind" project I sketched
together many years ago. It stores data about its environment and its internal
states and associations between them, and selects behaviors based on the
current state of activation of the associative web.
The original motivation was to build a "virtual animal" that could wander
through MUDs and similar systems, exploring the world and responding to
it in sufficiently complex-but-motivated ways to inspire anthropomorphism.
Sadly, since objects in a MUD universe don't really have consistently explorable
properties (at least, not without *very* sophisticated language interpretation),
I dropped that idea in favor of creating a virtual world (not intended
to represent the the real world in any detail, just to create an environment
for a mind to explore and interact with)... and dropped the whole project
shortly thereafter.
Still, the idea intrigued me and I've come back to it from time to time.
The general idea is a program that explores a virtual physical universe
and builds up a set of data structures that relate to that universe in
ways that inform its subsequent exploration, building up a pattern of motivated
behaviors.
I describe it here as follows:
-
A high-level overview of the system as a whole
-
Definitions of parts of the system, indicating how they interact
Lots of stuff here is inspired, derived, or shamelessly stolen from a variety
of sources -- primarily books on cognition and AI, though various other
places as well. (In fact, the initial idea was sparked by Persig's "Lila",
a book unrelated to it in any obvious way.) I won't try to credit them
all, but Lakoff's "women, fire, and dangerous things" and several of Hofstaedter's
books played a significant role.
Overview
-
The physical world consists of Objects and Events.
-
Objects include Receptors and Effectors.
Receptors change the Object state in response to Events, Effectors
produce Events based on the Object state.
Certain Object attributes are conventional and relied upon by Effectors
and Receptors.
-
All Objects exist in a containment hierarchy, though not all Objects are
containers. All Objects are ultimately contained by the Universe.
-
Certain Objects ("bodies") serve as signal transducers for virtual minds
Certain Effectors on a body ("behaviors") respond to intentions in
addition to Object state.
Certain Receptors on a body ("senses") produce perceptions in addition
to state changes.
-
A mind is a set of linked Nodes
Some Nodes (intrinsic Nodes) are built in to the mind; others are created
based on experience.
Active Nodes can execute functions, such as linking themselves to other
Nodes or creating new Nodes.
Glossary
Capitalized terms refer to instantiated constructs (data types,
programs, and so forth); lower-case terms are terms of convenience and
refer to more abstract collections, subsets, or attributes of those constructs.
Many sets of terms are defined circularly -- for example, body, behavior,
intension, and Self -- because the system as a whole is a closed, tangled
loop. This is very much the whole point, but it can take some getting used
to.
activation level
An abstract value associated with a given Node which directly affects the
effect that Node has on other Nodes.
The activation level of a Node is not necessarily stored explicitly.
When a Node's activation level increases, the activation level of Nodes
it excites increases as well, and the activation level of Nodes it inhibits
decreases. Similar behavior occurs when a link is first created - that
is, if Node A is active and gains a +link to Node B, node B's activation
increases. The degree to which a Node's activation level changes based
on an activation-change in a Node linked to it depends on the strength
of the link, its type, and (potentially) on an "availability" constant
associated with the Node itself.
The activation level of all Nodes decay naturally over time. When a Node's
activation level decreases, it does not affect the activation of Nodes
it excites or inhibits.
active Node
A Node that can currently perform functions. See scheduler.
associator
An interpreter primarily intended to induces existing symbols to form links
to one another.
associator function
An attribute of a Node that creates links to other Nodes. An associator
function can also merge two sufficiently similar Nodes, consolidating their
attributes.
For example, a Node could have associator functions that:
creates +links to Nodes to which =links exist
creates =links to similar Nodes
creates +links to Nodes that have +links to it
creates +links to any sufficiently active Node
attention
The strength with which Now excites a Node. This tends to decay over time.
behavior
An Effector that generates Events based on intentions Self excites.
body
An Object whose Receptors (also called senses) produce perceptions and
whose Effectors (also called behaviors) respond to intentions
boredom
A Mood component inversely related to the number of active Nodes. High
boredom lowers happiness. Interpreters generally respond to high boredom
by increasing their activity.
category
An index Node that relates similar symbols, creating symbolic hierarchies.
Nodes that are both categories and symbols can be thought of as prototypes
of baseline categories, as distinct from superordinate categories like
"red things".
conceptualizer
An interpreter that creates new symbols and quasisymbols, based on existing
symbols.
The simplest conceptualizers create categories based on shared attributes,
but conceptualizers can also create variations on a theme, hypothetical/fictional
symbols, or even Nodes completely unlike anything else, depending on how
they are designed.
confusion
A Mood component inversely related to the degree of interlinkage among
active Nodes. If the majority of active Nodes excite one another, the mind
has low confusion; if many active Nodes inhibit one another or the active
Nodes form distinct clusters, the mind has high confusion.
Interpreters generally respond to confusion by creating fewer intensions
and more symbols. (This can have the effect of increasing confusion, unfortunately.)
Some minds may have several different confusion attributes related to different
types of consistency. For example, if Self excites two (sets of) Nodes,
A and B, where A inhibits B (or vice-versa, or both), this can be a distinct
confusion attribute ("confusion about Self") tracked separately from the
global confusion. In this case, interpreters may target intensions/symbols
related to the specific confusion.
Some low-level confusions may have to do with inconsistencies in perceptions
-- events performed by absent objects, arriving objects that were already
there, stuff like that.
container
An Object with the following attributes:
An internal extent (describing the space inside it)
Zero or more contained Objects
One or more portals
danger
A symbol that excites fear.
In general, punishments and symbols that excite punishments tend to become
danger symbols.
Effector
An attribute of an Object, which generates Events and/or changes the Object's
state.
An Effector specifies the state(s) of Event(s) it produces, and the parameters
passed to that Event, given particular Object state(s). An Effector
can also respond to more global state information, such as the time.
In general, Effectors produce Events targetted for specific types of Receptors.
(An Object that produces no Events, or that produces Events for which no
Receptors exist, is an epiphenomenon and need not be represented in the
world.)
Some examples:
An Effector that generates Events intended to activate
"move me" Receptors for Objects whose extents overlap it, depending on
stacking order
An "appearance" Effector that generates Events intended
to activate senses
episode
An index Node that groups symbols of sequential related events.
excite
A type of link (sometimes called a +link) that causes the target Node to
become more active whenever the source Node becomes so.
If A has a +link to B, A is said to excite B (even if neither A nor B is
currently active).
extent
An attribute of an Object, which specifies the physical space it takes
up.
The nature of this attribute depends on the dimensionality of the universe
-- for example, in a 2-d hex-grid universe, it can be expressed as a number
of hexes, each with a position relative to the position hex.
Event
A (usually) transient machine that can trigger state changes in an Object.
All Events have at least the following attributes, and can have others:
An intensity function specifying the intensity at various
places and times
A reference to the Object that initiated the Event
An (optional) end-timeslice (after which the Event disappears
altogether)
fear
A Mood component directly related to the strengths of currently active
dangers. Interpreters respond to high fear by activating Nodes that inhibit
dangers.
happiness
A Mood component directly related to the strengths of currently active
rewards, and inversely related to the strengths of currently active punishments.
Interpreters respond to low happiness by activating rewards and Nodes that
inhibit punishments.
Here
An intrinsic Node relating to the mind's awareness of position
inactive Node
A stored Node that cannot currently perform functions. See scheduler.
index
A Node primarily intended to organize other Nodes. See category and episode.
inhibit
A type of link (sometimes called a -link) that causes the target Node to
become less active whenever the source Node becomes more active.
If A has a -link to B, A is said to inhibit B (even if neither A nor B
is currently active).
instinct
An interpreter that generates intentions based on perceptions.
Generally, instincts result in faster, more mechanical responses than other
interpreters.
intensity function
A function associated with an Event that determines the effective intensity
of the event at a given point in spacetime. It is generally dependent on
the position of the originating object and at least one initial parameter
(which can be thought of as the instantaneous intensity at the Object's
center), if not more. It can also be dependent on other properties of the
universe (for example, intervening Objects).
intensity
The result of the intensity function of an Event at a given point in spacetime.
intention
A transient symbol of an Event that a behavior associated with the mind's
body is potentially able to produce.
Intensions' association functions tend to be optimized for eliciting +links
from Self.
interpreter
A Node primarily designed to generate/modify symbols. See recognizer, conceptualizer,
associator, instinct, and motivator.
Interpreters look for specific patterns in nearby Nodes and produce certain
kinds of symbols; thus, many different interpreters can process the same
Nodes in parallel and produce radically different symbols. Some interpreters
are single-minded and produce their target symbols on the slightest provocation,
others mix top-down, bottom-up, and time-dependent processing to encourage
overall consistency.
Many (but not all) interpreters are intrinsic Nodes.
Some interpreters (sometimes called "recognizers") primarily process perceptions;
others (sometimes called "conceptualizers") primarily process symbols and
categories. Thus, interpreters can be arranged in networks, processing
each others' output.
Some examples:
An "X recognizer" that creates a symbol =+linked to
symbol X from perceptions similar to it.
A "good stuff recognizer" that generates rewards from
perceptions similar to existing rewards
An "avoidance instinct" (see instinct) that generates
avoidance intentions in response to perceptions similar to punishments
intrinsic Node
A persistent Node that exists in a mind initially, rather than being derived
from later activity.
Intrinsic Nodes can always be referenced by other Nodes, even if
no link exists between them.
See Here, Now, Self, Mood, interpreter, and Pain.
link
An attribute of a Node that relates it to another Node.
Links are unidirectional -- that is, if Node A has a link to Node B, the
reverse is not necessarily true. In this case, A is called the source Node
and B the target Node.
Links have different strengths, which determine the degree of relation.
The strength of a link can vary over time (usually slowly).
Links can be of different types (see excite, inhibit, similar), which determines
the kind of relation. The type of a link is fixed.
memory
A stored, persistent symbol with an associated timestamp (as distinct from
a recently generated symbol not yet stored).
mind
A collection of associated Nodes
Mood
An intrinsic Node whose attributes represent global conditions of the mind.
For example, a mind's Mood can include states of happiness, confusion,
boredom, fear, and so forth. In general, these attributes modify the behavior
of Node-activating routines and depend on the currently active Nodes, resulting
in an ongoing feedback loop.
motivator
An interpreter that creates new intentions based on existing symbols and
induces Self to form +links or -links to them.
Node
The unit of which minds are built.
Nodes possess attributes, which define their state, and can perform functions
when active. For example, all Nodes have one or more links, and one or
more associator functions which can create/modify/delete links. Some Nodes
can create other Nodes.
One can think of Nodes as falling into different categories -- for example
perceptions, intentions, symbols, categories, memories, and interpreters
-- but these are categories of convenience that may not describe any given
Node particularly well.
Now
An intrinsic Node with a timestamp that generally increases over time.
Nodes with timestamps can compare themselves to Now to determine how "long
ago" they occurred, which in turn can affect their processing by other
Nodes. Note that Now may not necessarily reflect the current timeslice...
the mind may be "unaware of the time" or even "living in the past", as
it were.
Object
A persistent machine that generates Events.
All Objects have at least the following attributes, and can have others:
A position
An extent
A stacking order
Zero or more Receptors, which change the state of the
Object when activated
One or more Effectors, which generate Events given particular
Object states
A container (all Objects form a containment hierarchy
that terminates in the Universe)
A start timeslice and (optional) end timeslice
(after which the Object disappears altogether)
Pain
An intrinsic Node relating to the mind's happiness and, ultimately, its
motivation.
Senses can directly activate Pain.
perception (or percept)
A transient Node that serves as an intermediary between an Event and a
symbol
Perceptions can be linked to intrinsic Nodes but not to other Nodes.
persistent Node
A Node that gets stored when inactive.
portal
An Object, associated with a container, whose Effectors and Receptors control
how Objects become contained by that container.
position
An attribute of an Object, which specifies the physical distance of an
Object from some other Object. If two Objects are physically linked, the
position of one is relative to that of the other.
The nature of this attribute depends on the dimensionality of the universe
-- for example, in a 2-d hex-grid universe, it can be expressed as a number
of hexes and an angle from 1 to 6.
punishment
A perception +linked to Pain
Receptor
An attribute of an Object, which changes the state of the Object when activated.
A Receptor specifies the change(s) it makes to the Object's other attributes
given one or more Event(s) of particular state(s) and intensity(ies)
*** Note: Actually, Receptors probably ought to have access to states of
"intervening" Objects to determine opacity.
Some examples:
A "move me" Receptor that changes the Object's position
A "big red button" Receptor that toggles a "panic" attribute
recognizer
An interpreter that creates symbols based on perceptions and existing symbols
that meets the recognizer's standards for similarity to those perceptions,
with +=links between the new symbol and its existing "template".
reward
A perception -linked to Pain
scheduler
Doesn't represent anything in the world, just a system that tracks the
current timeslice and controls which Objects and Nodes can run in that
timeslice.
This isn't timesharing in the operating-system sense... an arbitrarily
large number of routines may run simultaneously in a given timeslice, even
if the operating system is swapping them sequentially in and out of memory.
However, it's easiest to think of "active" structures as memory-resident
and "inactive" ones as in persistent storage.
Each area has a local scheduler which allocates runtime preferentially
to Objects likely to interact with minds, directly or indirectly. So if
there's nobody to see the tree fall, it probably didn't, although it may
suddenly "have fallen a while ago" the minute someone becomes aware of
it. This isn't metaphysically important; it's just to allocate CPUs to
the most interesting places.
Each mind has a local scheduler which which allocates runtime preferentially
to Nodes with high activation levels, creating a spreading network of activation
and deactivation that (hopefully) results in an internally consistent but
chaotic set of active Nodes at any one time. It also allocates runtime
preferentially to Nodes excited by Now ("attended-to" Nodes), and to interpreters
and intention when boredom is high.
These "local schedulers" needn't (and probably oughtn't) be separate programs;
it's just a convenient way of thinking about scheduling algorythms.
Self
An intrinsic Node relating to the mind's body
sense
A Receptor that generates percepts, based on Events and guided by relatively
permanent, built-in mapping algorythms.
Senses have access to intrinsic Nodes but not other Nodes.
Most senses keep local caches to avoid generating lots of copies of the
same perception, generating new perceptions only when the state changes
somehow.
Some examples:
A "kinesthetic" sense that responds only to Events generated
by the mind's own body.
A "visual" sense that responds to Events with a "visible"
attribute set.
A "tactile" sense that responds to Events whose initiating
Object is adjacent to the mind's own body.
similar
A type of link (sometimes called a =link) indicating that some set of attributes
(S) exists in both the source (A) and target (B) Nodes, where:
S is salient to A
some association function in A determines that the pattern
of values of S in A is similar to the pattern of values of S in B.
If A has an =link to B, A is said to be similar to B (though the reverse
need not be true).
A is also said, more loosely, to be similar to B if the conditions described
above exist
stacking order
An attribute of an Object, which specifies what happens when different
Objects' extents overlap. Objects might be interpenetrable, stackable,
mutually exclusive, etc...
symbol
A Node isomorphic to an (actual or potential) Object or Event in the world.
A perfect, concretely symbolic Node contains perfect correspondances to
all attributes of a single Object or Event; imperfectly symbolic ("subsymbolic"
or "supersymbolic") Nodes are more common. For example, a symbol may "represent"
a class of Events, in which case some attributes of the symbol may be universal
(shared by all Events in the class) while others are simply common (shared
by most Events), or may represent an Object in an abstract way (missing
some attributes and including others).
timeslice
A "clock tick" or time quantum, used to keep everything synchronized.
Everything in the system responds to its state in the previous timeslice
and modifies its state in the current timeslice. Nothing in the system
has access to earlier or later timeslices (although they may be stored
for external analysis or backup or whatever).
All ages and durations in the system are given in terms of timeslices.
The scheduler keeps track of the current timeslice and updates it when
all appropriate events have been completed. Thus, the "real-world" time
associated with a timeslice is variable.
timestamp
Reference to a particular timeslice. For example, an alarm clock may store
a "when-to-ring" timestamp.
Timestamps are not necessarily accurate -- it depends on the attribute.
For example, a memory Node might be created at timeslice 250 in response
to an Event, but timestamped as 32, or NULL.You could (loosely speaking)
describe this state as "the mind thinks Event happened at 32" or
"the mind doesn't remember when Event happened" or similar states.
transient Node
A Node deleted when it becomes inactive. See scheduler.
world
The collection of all Objects and Events.
Universe
A unique container Object that contains all other Objects.
The Universe's position is always 0... it is by definition the origin point
for other Objects' positions.
Notes
Think about this notion of "nearby" active Nodes and "encountering" a Node...
what does that mean? Are active Nodes embedded in some kind of space? Is
it simply a factor of shortest-distance across linkages? etc.
Think about recall -- what initiates it, what attributes and patterns
of attributes are legitimate "matching fields", difference between recall
and recognition, and how to retrieve Nodes quickly. Think about "quick-retrieve"
vs. "full-retrieve" vs. "best-retrieve" vs "general sense of familiarity"
vs. "deja-vu". Think about crisp memories of single events vs fuzzier classes-of-things
memories. Think about confabulation.
Think more clearly about recognition and linking to inactive Nodes.
The "total similarity" between two Nodes is really an emergent property
of the full set of attributes - it's a vector of individual similarities
-- but also asymmetric. Think about "out of the corner of my eye" events
and sensory illusions.
Think about Senses and recognizers combining and splitting apart the
attribute-lists from Events... realize that identifying an "event" or "object"
is not necessarily straightforward or done the same way by all minds. It
might be better to have a Sense bundle all simultaneous Events into a single
percept, or a set of percepts that aren't necessarily mapped to the individual
Events. Think about sensory acuity and sensory illusions, as well as intermodal
discrepancies and priorities -- important? It might be more correct to
build a "perceptual space" out of raw percepts projected to positions relative
to Here, and then have interpreters process the perceptual space
in localized ways. (You still want Sense-linked interpreters for instincts,
but not necessarily for recognizers.)
Think about "inferred" properties of Events/Objects -- that is, the
"white on this side" problem. Are these stored directly in a symbol? Is
there some sense in which a symbol with inferred properties is less symbol-like,
or less reliable? Should they be stored in a separate Node linked to the
symbol somehow?
Think more clearly about what an intention actually is.. this probably
requires a clearer understanding of Events. Direct and indirect objects,
and the ability to "learn new skills", and some notion of what the likely
consequence of an Event is are all important. Think about "intentions"
tthat don't necessarily map to behaviors -- study, for example. Think about
"expectation of state change" that can prime the pump for later intentions,
and unfocused intentions to have things change ("wishes", which really
aren't intentions at all). Think about how to recognize an intention as
distinct from a symbol of a behavior the system isn't actually capable
of, both when produced by motivators and by conceptualizers... that is,
think about the "hey, that's a symbol for something I can do!" and "hey,
I can't do that!" experience. Think about recognizing an intention as similar
to an intention or behavior produced in the past, and reacting accordingly.
Think about other types of links:
contiguity/simultaneity
-- Source and target were perceived in the same location/time.
performs/performedBy
-- Source represents an Object/Event that initiated/was initiated by an
Event/Object represented by target.
instanceOf/classOf--
Source is a symbol/category; target a related category/symbol.
precedes/follows
-- Source/Target represents an Object or Event that preceded the Object
or Event represented by target/source.
Consider responding to sufficiently major, chronic confusion by creating
two minds (that is, replicating the unique/internal Nodes).
Think about interpreters (or senses) recognizing a single thing changing
over time.
Think about "getting from point A to point B" -- path knowledge, or
map knowledge, or both.
Think about panic.
Think about prioritizing intentions, especially based on interactions
among the intentions currently queued on Self. However, not all intentions
get done in priority order... that'd be boring, it's more stochastic. But
really important things get done to the exclusion of less important things.
Think about mutually exclusive intentions (approach and avoid) and mutually
reinforcing intentions and intentions with an implicit sequence (pick up
bottle and drink, not the other way around). Think about intentions
to perform sequences of behaviors -- implicit or explicit? Ideally should
be implicit, there should be no explicit "search for" behavior... but there
may need to be, I'm not really sure. The body needs to somehow deal with
conflicting behaviors that nevertheless gets triggered -- that is, it's
one thing to avoid intending approach and avoidance at once, but it's a
different thing to implement what happens if you fail to avoid it. Think
about aborting behaviors partway through, and the general status of a behavior
("how well is this going?"). Think about "hesitation behaviors" and "background
behaviors". Think about the priority of a behavior -- it's attended-to-ness
-- as affecting its abortability and the difficulty of maintaining focus
on a task.
Think more carefully about Mood... socialness, aggression, hunger,
fatigue? Think about play and competition in this context. Also, sudden
Mood-shifts should result in some kind of link between active Nodes and
"the mood shift", somehow. This is particularly crucial for active Objects
(eg, oneself and other people) -- actions preceding a Mood shift should
be tagged somehow.
Think about words... in a MUD-like environment, this is key. Even without
solving the NLP problem, one could store, organize, and reproduce speech
acts and poses in ways that are associated with active symbols... the system
wouldn't necessarily "have language" in any interesting sense, but it might
still be able to trade words usefully with other systems. It's important
here to note the difference between an Eliza-like playing with language
and a system that forms links between words/phrases and representations
of the world... with the latter, the system may not be saying anything
coherent, but it can be said to know what it's talking about. Parsing pronouns,
proper nouns, and other references to objects and events are particularly
important. Think about babbling, both directed and undirected.
Think about logical propositions... is there any bottom-up way to generate
Nodes that represent beliefs about the world, as distinct from representing
Events and Objects (either directly or indirectly) the way symbols do?
Does this even make sense? Could an inference engine operate on symbols?
Think about intrinsic behaviors:
position/"ownership" changes: take, give, drop,
throw, push, put in, take out, protect
locomotion: walk, run, follow, avoid, flee, etc.
These may also serve as a communication mode -- scamper vs. trudge.
manipulation: modify Object attributes in generic
ways -- of course it only works if the Object has those attributes (or
similar ones)
communication: conventional behaviors for expressing
various mood components (smile, frown, cower, laughter, etc.), conventional
responses to same, mimicry
sleep (dream?), wake up, drowse
play? - games as intentions; rules as symbols; singing,
dancing, frolicking; what the hell is play all about, anyway?