The Behavior Framework, Embodied AGI
An Embodied Artificial General Intelligence Architecture Integrating Advanced Biological and Computational Models
I’ve been revisiting topics (and old blog posts) from over 20 years ago, back when I used to pontificate, or wax philosophical on different topics. As in the first part of this series, The Anti-Anthropic Principle and Assembly Theory, I dumped most of my old notes, diagrams and writings into Claude and ChatGPT to help me summarize and organize the content.
For the second part of this series, I wanted to cover an embodied AGI framework I’ve noodled with for over 20 years. A subset of these subsystems can be fully replaced by modern LLMs and neural networks. But just like any autonomous system, a large software framework is still required to integrate these different neural networks (i.e. subsystems), outputs, instrumentation, alignment to safety systems, or ‘rules’, etc. Most of these concepts will still apply today.
I started exploring AGI/AI cognitive architectures in high school but it was accelerated and fed by a thirst for knowledge at the intersection of the following trifecta of topics.
Computing & Complexity
Biology & Evolution
Artificial Life & Artificial Intelligence (back when it was boring)
Heavily inspired by books, publications and research by John von Neumann (computing, software), Minsky (AI), Freeman Dyson (physics, future of intelligence), Jeff Hawkins (HTMs, cortical columns), Charles Darwin (natural selection / evolution), Stephen J Gould (biology, evolution), Jaak Panksepp (Affective Neuroscience), Carl Sagan & Ann Druyan (cosmos, origin of life), Ben Goertzel (speculative AGI architectures), Melanie Mitchell (genetic algos), John Holland (Complex Adaptive Systems), Karl Sims (virtual creatures), Yann LeCun (AI, CNNs) and Ian Goodfellow (AI, GANs).
The Behavior Framework: A Comprehensive Embodied Artificial General Intelligence Architecture Inspired by Biological Neural Systems
Abstract:
The Behavior Framework presents a comprehensive approach to embodied artificial general intelligence (AGI), integrating biologically inspired neural structures with advanced computational systems in a overarching software-based cognitive architecture. This architecture draws inspiration from various levels of the animal kingdom, integrating neural networks, reflexes, emotion systems, and advanced cortical models to create a cohesive system that drives autonomous behavior in both physical and virtual agents while ensuring ethical and safe operation of AGI agents by ensuring compliance with safety protocols, and facilitating complex social interactions. We conclude by exploring promising areas for continued research and prototyping, particularly in AI safety, social intelligence, and ethical considerations.
Introduction:
The pursuit of AGI that can function autonomously in complex environments requires a framework that can emulate the adaptability and intelligence found in biological entities while integrating advanced AI systems to regulate and refine AGI behaviors. The Behavior Framework offers a solution by replicating the structure and functionality of biological neural systems, from basic reflexive responses to advanced cognitive functions and robust AI Safety and Reward and Pain-based networks, and an overall integrated Social AGI Layer.
These systems work together to create AGI agents that are not only intelligent and adaptable but also aligned with safety goals and ethical guidelines. This paper presents an in-depth analysis of the Behavior Framework, detailing each layer's architecture and capabilities. By understanding the biological foundations of intelligence, we aim to create a framework that can be applied across various domains, including robotics, commercial and consumer AI agents, gaming, and virtual assistants.
System Architecture and Subsystems:
Substrate Layer
The foundation of the Behavior Framework is the substrate layer, which represents the agent's physical or virtual embodiment. This layer provides the crucial interface between the agent and its environment. The inclusion of virtual pain sensors is particularly innovative, allowing for nuanced feedback about potential harm or environmental exceptions.
Key components:
Sensors: the most abstract and basic form of an environmental sensor, generally modeled to capture environmental property values and to ‘fire’ based on threshold values.
Motors/Actuators/Effectors: an encapsulating class that wraps the low-level control and status of a real (or virtual) actuator. This allows interactions with the environment
Embodied surface: Defining the boundary between internal and external systems. A robotic (or animal/human skin) can be thought of as an embodied surface. Embodied surfaces are generally littered with different sensors to provide fine-grained status at this boundary.
substrate & reflex layers - static model and notes
Neural Reflexive Agent - Reflex Layer:
The Reflex Layer of the Behavior Framework is inspired by the spinal cord of vertebrates, particularly the development of interneurons. The integration of sensors and effectors (i.e. muscles or actuators/servos). This layer handles immediate, rapid reflexive reactions essential for survival in dynamic environments. It provides the base-level behaviors upon which more complex systems are built.
The basis of the reflex layer is the Reflex (or InterneuronReflex in diagram), which is simply modeled as the integration of a Sense unit and a Motor unit via an Interneuron. Low latency behaviors are essential for ensuring survival in dynamic environments. The reflex connects sensory inputs directly to motor outputs, bypassing higher cognitive processes for efficiency.
Neural Autonomic Agent - Autonomic Layer:
Building on the Reflex Layer, the Autonomic Layer is inspired by the homeostatic circuits and self-regulating mechanisms found in the Myelencephalon and Mesencephalon of the hindbrain and midbrain. This layer manages the internal balance of the agent by regulating autonomic functions such as body/blood temperature, energy levels, and other virtual and/or physiological states. The Autonomic Layer ensures that the agent can maintain homeostasis in changing environments, allowing for long-term survival and stability. It also incorporates feedback loops that adjust internal states based on sensory inputs, creating a self-sustaining system that requires minimal external intervention.
The most basic version is the AutonomicReflexCircuit, a specialized set of two reflexes that are wired to respond in opposing directions. They’re both tied to the same interneuron (i.e. sensor) and serve as the components for a self-balancing system, this is basically how a cooling/heating system works. Prior concepts regarding ‘difference engines’ explored by Minsky and others, perform similar self-regulating mechanisms.
Neural Planning Agent - Limbic Layer and Emotive Systems:
The Limbic Layer is inspired by the Reptilian brain, specifically the Diencephalon of the Forebrain, including the Basal Ganglia, Hypothalamus, and sensory integration centers. This layer introduces emotion processing and basic goal-directed behaviors. The Emotion System within the Limbic Layer manage lower-level emotional states, such as fear, stress, pleasure, and aggression. These systems drive the agent's behavior by responding to internal and external stimuli, ensuring that basic needs like hunger and thirst are met. The Limbic Layer also incorporates seeking systems that help the agent explore its environment and reconcile dysregulated internal states.
Emotion Subsystem
An emotion subsystem within this layer helps provide extensive internal and external sensory context to behavior and planning systems. In the diagram, the SenseClusterLimbic class integrates a host of lower level sensory units enabling the rich internal/external sensory context required to work effectively. The emotions are also related in a hierarchy from simplest (bottom) to most integrated (top), representing mammalian evolutionary history.
Reflexes, the simplest form of behaviors, are wired directly to the sensor via an interneuron, which fires when a threshold or exceptional condition is met. An example would be a breach of a particular location in the embodied boundary (i.e. skin) should trigger a real (or virtual) pain sensor and an immediate taxis based behavior away from the location/position of the breach. This immediate pull away action, can be viewed as the simplest form of a short-term goal — ‘Minimize pain’ or ‘Move away from painful stimulus’.
Most of the major emotive states are associated to simplified goals that regulate that emotive states of the agent. For example, a long time ago, water-based biological embodied agents (single and multi-celled organisms) evolved sensors to provide signals for thirst and hunger when certain metabolic, digestive or chemical presence thresholds were exceeded. If an animal is thirsty, then the regulatory behavior required is to search for resources to minimize thirst, a gradient based search, or a seeking system invocation or a memory-based path planning activity - depending on the complexity of the agent. The ‘minimization of thirst’, ‘minimization of hunger’, ‘conservation of energy’ or ‘minimization of energy use’ can be viewed as the simplest forms of goals modeled in the system, they drive the most intense short-term behaviors in the agent (outside of reflexes, which are invoked independently)
More complex emotive states integrate the states of the simpler (lower) emotions in the emotive hierarchy defined above, thus forming the architecture of a fully functional emotion system, providing ‘status’ information on the internal states of the embodied agents across relevant dimensions (i.e. hunger, thirst, pain, stress, etc). Other complex sensory units also connect to the higher level emotive states, such as fear, aggression and stress, such as the visual, auditory sensory systems. A predator or mate can be within sensory range of these systems, triggering a signal that’s integrated into the emotion system, driving these emotive state values beyond certain thresholds. The emotive states are tied to an emotion behavior planning system that can drive lower level autonomic and short-term regulatory goal-based behaviors. The agent’s goal is to drive equilibrium in these emotive states or to minimize and/or maximize certain values.
Pain Network:
The Pain Network serves multiple functions beyond simple reflex and emotional responses. By embedding pain sensors virtually or physically, the system can introduce various forms of pain, such as emotional pain, sympathetic pain, and empathetic pain. Most importantly, the Pain Network acts as a safety mechanism. When AGI safety goals or guidelines are breached, the Pain Network introduces pain signals to discourage harmful behaviors. This system ensures that the agent remains aligned with its safety protocols, effectively penalizing actions that deviate from established guidelines.
Reward Network:
Complementing the Pain Network, the Reward Network is modeled on the dopamine-based reward circuits found in humans. This subsystem provides the AGI agent with an affirmation signal when a behavior or outcome aligns with the agent’s goals. The Reward Network reinforces positive behaviors by generating pleasure signals, which encourage the agent to repeat successful actions. By balancing the Pain and Reward Networks, the agent can effectively navigate complex environments, seeking out actions that align with its goals while avoiding those that breach safety protocols.
The Reward Network also plays a crucial role in long-term learning and adaptation. By associating pleasure signals with specific behaviors, the agent can develop a more nuanced understanding of which actions lead to desirable outcomes. This system ensures that the agent not only survives but thrives by consistently aligning its behaviors with overarching goals and prime directives.
Cortical Agents - Neocortex Layer:
The Neocortex Layer represents the massive integration of cognitive processing within the Behavior Framework, drawing inspiration from higher mammals such as cats, dogs, apes, and humans. This layer is responsible for advanced learning, pattern recognition, modulatory and inhibitory control systems and predictive planning.
The modulation of lower-level behaviors and motor functions, emotions, and autonomic subsystems are arguably the most important aspects of this layer, in addition to the predictive cognitive capabilities enabling the agent to engage in complex decision-making processes. The Neocortex Layer also includes a simulation subsystem that allows the agent to replay memories, run predictive simulations based on past experiences, and bias (or prime) future decision-making based on anticipated outcomes. This predictive capability is crucial for navigating complex environments and achieving long-term goals.
This layer can be viewed as the ultimate integration layer, where most, if not all, mid-level functions, signals, emotions and subsystems are either integrated or can be modulated, inhibited, activated and/or manipulated by cortical subsystems.
Memory System:
Memory plays a central role in the Behavior Framework, providing a repository of sensory and motor contexts, behaviors, and outcomes. Lower level ‘spatial-centric’ memory can be used to store motor plans, goals. Temporal-centric memory can be used to store workflows, action plans and complex long-running behaviors. The Memory System can record an agent's experiences and use this data to inform future decisions. By associating outcome scores with specific behaviors and environmental/sensory contexts, the Memory System allows the agent to refine its actions over time, improving its adaptability. This system supports both short-term and long-term learning, ensuring that the agent can respond effectively to new challenges and environments.
AI Safety Subsystem:
The AI Safety Subsystem is responsible for enforcing safety protocols across the AGI agent’s behavior. This subsystem monitors the agent’s actions, ensuring that they align with predefined safety guidelines. If the agent’s behavior deviates from these guidelines, the AI Safety Subsystem can trigger the Pain Network to penalize the agent. Additionally, the AI Safety Subsystem has the authority to ration and throttle the agent’s access to power, energy, or compute capacity as a further deterrent against unsafe behaviors. This subsystem acts as the final arbiter in decision-making, ensuring that all actions comply with safety standards.
Social AGI Layer:
The Social AGI Layer represents the highest level of cognitive and social processing within the Behavior Framework. This layer integrates inputs from several subsystems, including the AI Safety Subsystem, Personality Subsystem, Social Planner, Predictive Simulator, and Advanced Predictive Cortex. The Social AGI Layer is responsible for managing the agent’s social interactions, ensuring that its behavior is not only intelligent but also socially appropriate and aligned with its personality.
Personality Subsystem: Defines the AGI agent’s unique personality traits, influencing its social behavior and decision-making processes. The Personality Subsystem works in conjunction with the Social Planner to ensure that interactions align with the agent’s personality.
Social Planner Subsystem: Manages the agent’s social goals and behaviors, ensuring that they are appropriate for the context. The Social Planner interacts with the Predictive Simulator and Advanced Predictive Cortex to anticipate the outcomes of social interactions and adjust behavior accordingly.
Predictive Simulator and Advanced Predictive Cortex: These systems allow the agent to simulate potential social scenarios, predicting the outcomes of various actions. By comparing current contexts with historical data, the agent can make informed decisions about how to behave in social situations.
Primer Subsystem: The Primer Subsystem is responsible for defining the agent’s long-term goals, known as Prime Directives. These directives are workflows of lower-level goals that must meet certain criteria to accomplish higher-level objectives. The Primer Subsystem is initialized by a single "Owner" of the AGI agent and is tightly integrated with the AI Safety Subsystem. Any conflicting goals or behaviors must be reconciled with the AI Safety Subsystem; otherwise, the Pain Network penalizes the agent.
Hive Intelligence & Training Integration Layer - ai generated - flux schell
Hive Intelligence & Training Integration Layer:
The Hive Intelligence & Training Integration Layer. This platform allows embodied AGI agents to connect and share knowledge across their neocortical layers. By accessing the memories and learned outcomes of other similarly architected agents, each AGI can rapidly adapt to new environments and associated sensory contexts, behaviors, dangers, rewards, and more. This collaborative learning environment enables agents to pool their experiences, creating a distributed intelligence network that enhances the efficiency of learning and decision-making. The Hive Intelligence & Training Integration Layer mimics social learning in biological systems, providing a robust mechanism for collective problem-solving.
Capabilities and Applications:
The Behavior Framework's multi-layered architecture enables it to support a wide range of applications across various domains. In robotics, the framework can be used to develop autonomous systems capable of navigating complex environments and interacting with humans. In gaming, the framework can create dynamic, emotionally responsive AI characters that enhance player engagement. In virtual assistants and metaverse applications, the Behavior Framework allows for the development of agents that can learn from interactions, adapt to user preferences, and provide personalized experiences.
Promising Avenues for Continued Research and Prototyping:
Optimization of AI Safety Subsystem: Research should focus on refining the AI Safety Subsystem’s ability to enforce guidelines without stifling the agent’s creativity or adaptability. Exploring adaptive safety protocols that evolve with the agent’s learning could provide a more flexible approach to maintaining safety.
Enhancement of Pain Network: The Pain Network’s integration with various subsystems should be further explored, particularly in how it can be used to simulate complex forms of pain and their effects on decision-making. Understanding how different forms of pain influence behavior will be crucial for refining this subsystem.
Development of Social AGI Layer: Further work is needed to refine the Social AGI Layer, particularly in its ability to handle complex social interactions and reconcile conflicting goals. Research should explore how personality traits can be dynamically adjusted based on experience and how social planning can be improved through real-time simulations.
Scalability and Computational Efficiency: As with the existing layers, scaling the new subsystems across large populations of agents will pose computational challenges. Optimization techniques, such as hardware acceleration and parallel processing, should be prioritized.
Ethical Considerations and Empirical Validation: As AGI agents become more human-like, ethical considerations around autonomy, social interaction, and decision-making authority must be addressed. Rigorous empirical testing will be essential to validate the framework’s effectiveness and ensure its safe deployment.
Quantum Computing: This framework may need foundational changes to be applicable in a quantum computing world.
Conclusion:
The Behavior Framework represents a significant advancement in the development of embodied AGI by integrating biologically inspired systems with advanced safety mechanisms and social intelligence. The introduction of the AI Safety Subsystem, Pain & Reward Network, and Social AGI Layer enhances the framework’s ability to operate safely and ethically while engaging in complex social interactions. These additions provide a robust foundation for creating AGI agents that are not only intelligent and adaptable but also aligned with human values and safety protocols.
