With each passing year, software becomes more complex. This has been the overall trend for software in general, but much the same thing has happened with games, albeit in a compressed time scale. In the early days, computer games were simple, with maximum performance expected from minimum hardware. Accordingly, games were executed in assembly language. But by the mid-1980s, games were becoming more complex, and the transition to procedural languages began. By 1990, C was used for most programming. Nowadays, many developers are making the transition to object-oriented C++. It would seem, then, that the rising complexity of games has forced developers to move to higher levels of programming.
Where will this process take us? I have some interesting conjectures that I’d like to present in this essay. First, however, a disclaimer: I’m not up on recent developments in programming language theory. I have no real understanding of agents or daemons, and although I’ve read a bit about object-oriented programming, I’ve never done it. So my observations will be tainted by some naivete. On the other hand, perhaps I will be able to approach old truths from a different and revealing new angle.
It seems to me that the central problem in the advance of software is the management of complexity. Rather than invent all of our software management systems from scratch, perhaps we could learn a few tricks from other complex systems that require management. The first obvious candidate for such contemplation are the biological systems. However, such systems aren’t highly useful for our problem, because their basic control system survival of the fittest is a slow-moving and conservative control system. It always works, but it goes through endless prototypes before it reaches beta. Not fast enough for our needs, although research in genetic programming creeps forward every year.
The control system that I think offers many useful lessons is society. Social organization is nothing more than a control system for a large number of human beings. There can be no doubt that social control systems are capable of handling vast complexity; after all, they are coordinating the efforts of some six billion human beings as you read this essay. We have accumulated an enormous store of information regarding the ideal means of organizing large numbers of people. Political systems, economic systems, belief systems, educational systems, interpersonal systems, business systems -- we know a great deal about how to organize people for optimal performance.
At this point, I’d like to digress to a Calvin and Hobbes comic. It starts off in some sort of industrial control room. A dozen little uniformed Calvins man a variety of controls, shouting commands and information to each other. It devolves that these little Calvins are agents inside the brain of the big Calvin, desperately trying to avoid a fall after a misstep on the stairs. They fail, and Calvin falls on his face.
Why can’t we use such a model in our programming? Again, I can make no claims of originality here; the concepts of agents, daemons, and objects all anticipate this idea. But I’d like to play the concept for all it’s worth. Let’s think about programming structures as if they were teams of agents.
The first big change is that we stop thinking about "the program" as a monolithic block. Instead, we start to think of "the team". We are going to create a team of players, each of whom will be responsible for some aspect of the overall function. Each of these players is a subprogram, responsible for all of its function. At this point, we come to something very much like the concept of encapsulation: each player must have all the tools and data it needs to do its job. In other words, if we have an "explosion drawing agent", that agent must know everything there is to know about explosions, and have all the tools he needs to do his job. We don’t go off and declare some array of pixels that comprise an explosion image, and then later on in the program tell the explosion-drawing routine about them. No, we should treat the explosion-drawing routine like a human being and give it the data it needs to do its job right there in its own office. After all, it might be cheaper to have one dictionary for all the workers in your office, but if you have to walk down to the central office every time you want to use a dictionary, you’re not going to get much use out of it, are you? Give each worker his own dictionary. Thus, the first precept we have is: give each routine all the information it needs to do its job.
Another important idea in societal organization is clear assignment of responsibilities. Our society asks us to be a butcher, a baker, or a candlestick maker. Now, here we come upon an interesting observation: in primitive societies, specialization is limited and each person is expected to perform a wide variety of functions. However, in more complex societies, we see greater specialization of function. Much the same thing should be happening with software. Primitive programs use lots of global data that is readily shared by a variety of routines. Clear separation of function is less common in simple programs, because it’s too easy to perform certain tasks as the opportunity arises rather than according to some defined structure. But complex programs need to be run more like a big organization. This job is done by the janitor; that job is done by the secretary; a third job is done by the programmer. Sure, the secretary could clean up the programmer’s mess, and the programmer could type a letter, but things get snarled up when it’s done that way. "Who left the coffeemaker on?" "Where did my broom go?" "Why hasn’t this room been cleaned yet?" Things work out better when everybody has their own little territory. And the same thing goes for software. Each little agent needs his own little office, and his own little tools, and his own little data, and a clear set of responsibilities so that he can do his job happily.
Here’s another lesson we can learn from societal organization: parameter passing. If you want to get the county planning department to approve your house plans, you have to submit your plans in the prescribed format. These people are bureaucrats, and they expect you to fill out the form completely and correctly. We all love to bitch about petty-minded bureaucrats, but what can be more petty-minded than a computer program? The bureaucrat’s form to be filled out is nothing more than a parameter list. You fill out the parameter list, submit it to the bureaucrat, and he processes the information, outputting some other form or paper that you need to proceed. Same thing goes with a loan from a bank, or tax forms for the IRS, or an employment application... we’ve got this system down pat, don’t we?
Another issue that crops up in parameter-passing is the use of pointers or identifiers. When I go into a store and hand the clerk my credit card, I’m giving them a pointer to an account on a computer. They swipe my card through the machine and the machine then interrogates the main computer. Is this guy for real? Does he have enough money to pay for this? All the information that the local bureaucrat-computer needs is derived from the pointer and a consultation with the global computer. We see the same thing with medical records. We don’t expect the individual to keep track of his medical records, so the bureau in charge (the doctor’s office) keeps our medical records and we merely pass a simple identifier parameter to the doctor’s office. They then pull our file and we’re ready to go. But note: this is a good example of encapsulation. Why would anybody else need my medical records? The doctor’s office is the only place where they’re needed, and so they’re kept there, even though they’re MY records. I don’t walk into the doctor’s office with my medical records and pass them as parameters to the nurse.
Unfortunately, we do run into the occasional need for global variables. The most common of these global variables are financial. I can use my credit card all over the world; this is truly a "global variable". But it requires global variable handling. There has to be a central computer that maintains the records on my account, and it must be accessible at all times.
The societal model also suggests intriguing questions as to the proper organization of code. Of the many different ways of organizing society, two fundamental strategies arise: a hierarchical model and a legalistic model ("a government of laws and not of men"). The hierarchical model is best exemplified in feudal social structures: each member of society has an overlord and vassals. He takes orders from the overlord and gives orders to vassals. He must insure the safety of his vassals and looks to his overlord for protection. Thus, obedience flows upward and responsibility flows downward. In the legal system, all men are created equal in the eyes of the law; a body of rules determines their relationships.
Although our prejudices run toward the legal model, let me point out that the strength of the legal model lies in its provision for individual freedom. Legal systems are primarily constraintive in style; they declare what you can’t do, but seldom specify what you must do (other than pay taxes). Hierarchical systems, on the other hand, tend to work just as well with the proscriptive as with the prescriptive.
Another distinction: in hierarchical systems, society is defined by the relationships between individuals, and policy is a consequence of those relationships, but a legalistic society defines itself by its policies; the relationships between individuals are the consequences of the policies. In a hierarchical program, program flow proceeds from higher-level routines to lower-level routines under the control of the higher-level routines. A legalistic programming model requires multithreaded processing, with a group of autonomous modules (citizens) operating under the overall constraints of an operating system (body of law).
I think we can also agree that hierarchical systems represent a more primitive level of social organization than legal systems. Inasmuch as software complexity is at an early stage of social evolution, we probably don’t want to start off with the most advanced social organizations. It would be wiser to start off with the more primitive systems.
Indeed, if we think in terms of social evolution, we can see some fascinating parallels. At first, programming was in the hunter/gatherer stage where population densities were so low that people really didn’t need to worry much about their interactions with each other. Programs were little more than small family groups wandering about in an environment so big that the relationships between individuals (code segments) were simple and straightforward. Life was hard, survival was all that mattered, and so performance dominated all considerations. Hence the simple, free life of the savage assembly language program. But program sizes grew, agriculture set in, and soon there were larger groups of people and subroutines interacting in more complex ways, and some system of order had to be imposed. Tribal structures (structured programming) evolved. Individuals resented the loss of freedom, just as programmers resented the imposition of structural formalisms. Revolts were frequent, and warfare became commonplace; and today young programmers must often be gently guided into the discipline of good programming technique. ("I cut off the limbs of the officers who had rebelled. I took Ahiababa to Nineveh; I flayed him, and I spread his skin upon the wall of Nineveh." --Assurnasirpal II)
It seems to me that the programming concepts of object-oriented programming are vaguely analogous to the loosely hierarchical social structures of the peoples of northern Europe in classical times (the Celts and the Germanic peoples). People were organized by families, tribes, and nations, each with its own level of leader. Ireland had its sub-kings, kings, and high-kings; the Germanic peoples had corresponding offices. But social bonding was strongest at the lowest levels and quite weak at the highest levels. Each person (object) in the society (program) interacts strongly with other persons (objects) that are closely related in the hierarchy, but has difficulty interacting with people (objects) that are remote.
As human population densities increased, the need for tighter integration of social interaction rose with it, and the loose hierarchies of the Celts and Germans gave way to the stronger hierarchies of the feudal period. This suggests that the next step in the evolution of programming structures will follow the next step in social evolution: the creation of strong hierarchies where the highest levels of the program exert strong control over the lower levels in the same way that kings eventually established strong control over their nations.
We should remember, though, that the rule of law also evolved even as the kings consolidated their power. Runnymede preceded the peak of royal power by hundreds of years. Someday our programs will be sophisticated enough to require a strong rule of law, and even as we establish stronger hierarchical structures, we will need to begin work on the legalistic approach. Let us hope that our transition from a hierarchical to a legalistic program organization proceeds with less tumult than the analogous transition in human social organization.