Papert's Principle proposes that structuring the administration of our knowledge and skills is a crucial part of learning and being able to use what we've learned. Middle-level managers perform this administration by organizing agencies to keep them from getting in each other's way. They separate unrelated agencies to prevent unnecessary conflicts, and they group related agencies to take advantage of non-compromise. How do our minds create these managers and decide what to group together under which agencies? It seems unlikely that we explicitly reason about the relationships between low-level agencies and resultingly organize them into hierarchies. One possibility is that the managerial structure evolves to optimize compromise and non-compromise between lower-level agencies in order to improve problem solving ability.
In the "Society of More" example from Chapter 10 of Marvin Minsky's The Society of Mind, we have a number of agencies working together to form a larger society. In a simple case, the Tall, Thin, Confined agencies all give input as to whether there is "more" of something. By reasoning about it, Tall and Thin could be grouped under an Appearance agency while Confined could be grouped under a History agency (see Section 10.4 of SOM). However, how would we group these agents without placing labels on them or reasoning about the contents of the agencies? If Tall and Thin disagree, the whole Appearance agency will be weakened by the Principle of Non-Compromise, deferring to the Appearance agency and giving the desired result. Conversely, if Tall and Thin agree, the Appearance agency will not be weakened and will contribute towards the state of More society as a whole, again giving the desired result. However, if Tall and Confined were grouped together under an agency, non-compromise between them would not be useful as it would only defer to the Thin agency. If Tall, Thin, and Confined are not organized under middle managers at all, conflicts between them will just weaken the whole society without contributing to a useful result.
This leads us to a mechanism for creating managers and agency hierarchies. Whenever a set of low-level agencies under a higher-level agency often conflict with bad results (such as yielding an incorrect answer), a B-Brain would create middle-level managers between the lower-level and high-level agencies. These middle-level managers might be created on an adjacent layer, and the original connections would be left intact. Rather than trying to make any distinctions at this point, a number of different possible managers would be created. In the extreme case, managers might be created for all possible groupings of lower-level agencies (although it seems more likely that a smaller number of managers are created with arbitrary groupings of lower-level agencies under them). At this stage, the results of the new managers might have very little influence in the overall society, although their importance will increase or decrease depending on how they perform on future problems.
As future problems are encountered, middle-level agencies and the links between agencies are strengthened and weakened depending on what results they have on the overall result. The overall importance of managers are weakened if conflict or a lack of conflict under them has bad results or lead to wrong answers. If non-compromise or compromise under a manager continues to yield good results, the manager would become stronger. A similar strategy might apply to the links between agencies. With experience, inappropriate middle-level managers would die off while managers which organized appropriate lower-level agencies would become fully integrated into the society. Many of the original links between the lower-level agencies and the higher-level agency would also die off, as they introduce unnecessary conflict.
The final result is a hierarchy of low-level agencies organized under middle-level managers. If this process of creating and evolving layers of managers is repeated, large societies can be organized into a multilevel hierarchy. Because of the way in which they are formed, the resulting structures are not strict hierarchies, but instead may have low-level agencies connected to multiple middle-level managers. This corresponds to the real world where not everything can be divided into tidy categories.
To summarize, the goal is to create an administrative structure which takes advantage of the Principle of Non-Compromise. One way to do this is by introducing multiple middle level managers whenever conflict continuously leads to bad results. Over time, managers which are poor divisions of low-level agencies die out while managers which successfully organize agencies to take advantage of both compromise and non-compromise become an integral part of the overall hierarchy.