Gödel, Escher, Bach: An Eternal Golden Braid

I got a good chunk of reading yesterday because I went on the shortest ride of my life. It was about 60 degrees yesterday in Boston, perfect riding weather. I drove to Blue Hills, my favorite place to ride, had Blink on the CD player, and was all strapped in… bunny-hopped onto the trail-head and got a pinch flat when my back tire hit the curb. I didn’t have any extra tubes because I gave my last one to a guy who had a flat on my last ride at Blue Hills. Then I stepped in poo walking my bike back to the car. Sheesh…

So anyways.. I’m currently reading “Gödel, Escher, Bach: An Eternal Golden Braid” by Douglas Hofstadter. This book is over 700 pages and is described by one guy on the back of the book as “…an entire humanistic education..” so a) I’m not even going to try and give a description of what it’s about and b) I’m going to blog it as I go, rather than blog it after completing the book.

On Kurt Gödel’s discovery of the Epimenides paradox: “Epimenides was a Cretan who made one immortal statement: ‘All Cretans are liars.’ A sharper version of the statement is simply ‘I am lying’; or, ‘This statement is false’. It is that last version which I usually mean when I speak of the Epimenides paradox. It is a statement which rudely violates the usually assumed dichotomy of statements into true and false, because if you tentatively think it is true, then it immediately backfires on you and makes you think it is false. But once you’ve decided it is false, a similar backfiring returns you to the idea that it must be true…. The Epimenides paradox is a one-step Strange Loop, like Escher’s Print Gallery. But how does it have to do with mathematics? That is what Gödel discovered. His idea was to use mathematical reasoning in exploring mathematical reasoning itself. This notion of making mathematics ‘introspective‘ proved to be enormously powerful, and perhaps its richest implication was the one Gödel found: Gödel’s Incompleteness Theorem.” (pg 17)
— It’s interesting to see that introspection (in software development) probably has its roots in this mathematical introspection that Gödel introduced above…. what was the first language that used introspection?

On intelligent behavior: “No one knows where the borderline between non-intelligent behavior and intelligent behavior lies; in fact, to suggest that a sharp borderline exists is probably silly. But essential abilities for intelligence are certainly:
— to respond to situations very flexibly;
— to take advantage of fortuitous circumstances;
— to make sense out of ambiguous or contradictory messages;
— to recognize the relative importance of different elements of a situation;
— to find similarities between situations despite differences which may separate them;
— to draw distinctions between situations despite similarities which may link them;
— to synthesize new concepts by taking old concepts and putting them together in new ways;
— to come up with ideas which are novel.” (pg 26)
— Makes these ‘intelligent agents‘ not seem so smart doesn’t it? Next time you see a claim that this or that software is intelligent, apply the above tests to see if it passes. I’d love to know about software that does…

On being inside and outside a ‘System’: “… Now let me be very explicit about what I meant by saying this shows a difference between people and machines. I meant that it is possible to program a machine to do a routine task in such a way that the machine will never notice even the most obvious facts about what it is doing; but it is inherent in human consciousness to notice some facts about the things one is doing. But you knew this all along. If you punch ‘1’ into an adding machine, and then add 1 to it, and then add 1 again, and again, and again, and continue doing so for hours and hours, the machine will never learn to anticipate you, and do it itself, although any person would pick up the repetitive behavior very quickly. Or, to take a silly example, a car will never pick up the idea, no matter how much or how well it is driven, that is supposed to avoid other cars and obstacles on the road; and it will never learn even the most frequently traveled routes of its owner…. The difference, then, is that it is possible for a machine to act un-observably; it is impossible for a human to act unobservant.” (pg 36-37)
— The newest versions of Windows *are* in fact observant… the “Programs” menu ‘learns’ what programs you use most frequently and shows only those programs. How else could software applications become ‘observant’? Might your Palm handheld ‘notice’ that it gets synced every night before you go to bed and ‘suggest’ to you that it schedule itself to sync at 9pm? Wouldn’t it be nice if your cell phone saw that you called take-out twice in the last week and suggest that you add that phone number to your list of contacts?

On jumping out of systems: “It is an inherent property of intelligence that it can jump out of the task which it is performing, and survey what it has done; it is always looking for, and often finding, patterns.” (pg 37)
— Substitute the word ‘intelligence’ for ‘software developer’ or ‘business manager’. Revealing isn’t it?

War in the Age of Intelligent Machines

Took me a couple days but I finished “War in the Age of Intelligent Machines” by Manuel De Landa today. I’m fascinated by AI research and I love reading about all the new autonomous machines our military machine comes up with so I thought this book would be right up my alley. It was, but not for those reasons. The book spent quite a few pages explaining the history of warfare, tactics, strategies and weapons and how those entities changed society and then scattered like a cluster bomb to cover such topics as RAND Corporation, organized chaos, collective rationality and singularities. Because I’m not a great book reviewer and because I write this blog for myself as much as for anyone else, here’s a great summary of the book by Wired and then some quotes from the book that sparked my interest.

From Wired 1.04: “De Landa’s argument turns on the notion that singularities – the “transition points…where order spontaneously emerges out of chaos” – catalyze curiously lifelike behavior in nonorganic matter. By extension, he argues, the military apparatus might be seen as a self-organizing process in which human agents function as “industrious insects pollinating an independent species of machine-flower that simply [does] not possess its own reproductive organs.” More auspiciously, he speculates on the use of the “machinic phylum” – a superset containing all singularities – to facilitate “a symbiosis in which the evolutionary paths of humans and machines interact for their mutual benefit.” [source]

“Mission-oriented tactics, in which only the outlines and overall goal of an operation are laid down, leaving the execution details up to field officers and soldiers, decrease the overall flow of information and therefore decreases the global effects of noise and friction. When armies adopt such decentralized tactical schemes during battle, they begin to resemble the self-organizing dissipative structures mentioned above, islands of stability amid the turmoil of war. Indeed, like the system of eddies and vortices in a self-organizing turbulent flow, decentralized modern armies (like the Israeli army in 1956) have sometimes been viewed as a form of ‘organized chaos.'” (pg 79)

In regard to an invitation that political scientist Robert Axelrod sent out to professional game theorists for strategies in a round-robin Prisoner’s Dilemma tournament: “Many different programs were submitted and then pitted against one another. A majority of programs simulated ‘traders’ who were out to explit other traders (reflecting the traditional pro-conflict bias), while other programs simulated traders who were willing to cooperate. Surprisingly the ‘winners’ of this competition were programs that emphasized cooperation. ‘Winning’ was not defined as defeating rivals in single encounters (in which case betrayers would have won), but in maximizing the benefits of trade. In this situation, programs that tended to betray quickly ran out of partners with which to trade, since one betrayal would start a vicious circle of counter betrayals and mistrust. In the long run, the winning programs were the ones that had the following characteristecs: they were not out to exploit other programs (in Axelrod’s terminology they were ‘nice’, because they did not betray first); they retaliated in kind after being betrayed; and they were willing to reestablish a relationship after retaliating (they were ‘forgiving’). Even in a second tournament, when the human programmers knew that ‘nice, retaliatory and forgiving’ programs had won(and were therefore able to writing betraying programs that took advantage of this knowledge), the same kind of programs won again.” (pg 86)

“A new form of computation is emerging. Propelled by advances in software design and increasing connectivity, distributed computational systems are acquiring characteristecs reminiscient of social and biological organizations. These open systems, self-regulating entities which in their overall behaviour are quite different from conventional computers, engage in asynchronous [that is, parallel] computation of very complex tasks, which their agents spawn processes in other machines whose total specification is unknown to them. These agents also make local decisions based on both imperfect knowledge and on information which at times is inconsisten and delayed. They thus become a community of concurrent processes which, in their interations, strategies, and competition for resources, behave like whole ecologies.” (pg 107)

On predatory machines: “Autonomous weapons are a revolution in warfare in that they will be the first machines given the responsiblity for killing human beings without direction or supervision. To make this more accurate, these weapons will be the first killing machines that are actually predatory, that are designed to hunt human beings and destroy them.” (pg 128).

On the transition from the clockwork to the motor as the dominant assembly paradigm: “An example of a clockwork logical system is the Aristotelian syllogism. Such a syllogism is a formalization of a small portion of deductive reasoning, yet it dominated logical thought for two millenia. Aristotle gave us a recipe for the mechanical transmission of truth from premises to conclusions, a recipe to go from “All x’s are y’s” and “All z’s are x’s” to All z’s are y’s”. If this doesn’t seem very exciting, that’s because it isn’t: the syllogism is a rather trivial mechanism that can correctly transmit data along a given path, but cannot produce new knowledge….. We may view logical notations as little machines, as conveyor belts for transporting truth from one sentence to another sentence. While dedeductive systems transport truth from a general principle (“All men are mortal”) to a particular statement (“I am mortal”), inductive systems operate in the opposite direction. They transport truth from a particular piece of evidence (“This emerald is green” to a statement applying a general category of things “All emeralds are green”). While deductive conveyor belts are prevalent in mathematics, inductive ones are the basis of the natural sciences. Only deductive logic has been mechanized, either as clockwork (the syllogism) or as a motor (Boolean logic). Inductive logic, on the other hand, cannot be mechanized so easily. Indeed, a mechanical version of inductive conveyor belts is equivalent to building a machine that can learn from experience.” (pg 144)

On intercept-antenna farms: “… One such area is Pine Gap, in the heart of the Australian wasteland, an endless expanse of wind-swept earth, reddened like a Martian desert by whirling dust storms of iron-oxide sand … But the conditions that were so disastrous to the local population were precisely those considered ideal by the NSA. Less rain meant less chance of a signal being washed out and less possibility of interference from an electrical storm. The isolation of the area brought with it the advantage of freedom from interference from spurious signals and lowered the chance of being detected…. Today Pine Gap looks like an advance moon colony in the Sea of Tranquility. Hidden in the valley is a secret community of 454 people, eighteen single story buildings…and most startling, a futuristic array of six silver-white igloo-like radomes containing dish antennas ranging in size from less than 20 feet to about 105 feet.”

Loosely Coupled

So when I first started this weblog, I called it “Loosely Coupled”, which probably means nothing to people outside the software industry. Since then I’ve vamped the title, but I the term still interests me. I forget how I found this article on Friday, but it’s a wonderul introduction to what the term means and how it applies to hardware, software and business in general. “Loosely Coupled: A Term Worth Understanding” defines loosely coupled as

“… loosely coupled is an attribute of systems, referring to an approach to designing interfaces across modules to reduce the interdependencies across modules or components – in particular, reducing the risk that changes within one module will create unanticipated changes within other modules. This approach specifically seeks to increase flexibility in adding modules, replacing modules and changing operations within individual modules.”

John focuses mainly on business practices in his article, while I find myself drawn more toward exploring loose coupling within software applications. Much like a writer of books or songs spends the majority of his or her time searching for that perfect word or rhyme, software developers spend inordinate amounts of time trying to reduce complex business functions into abstract and manageable (but useful) software functions or objects… and then spend the rest of their time trying to tie them all back together into a useful application. I think that loose coupling is the idea that when you’re designing your abstract and manageable functions & objects, you design them in such a way that getting them to work together is as easy as putting together a couple of Lego pieces. Implicitly then, you should be able to swap out a Lego piece for a different colored or shaped piece, as long both pieces share a common interface. Here’s a real world example:

I’m finishing up work on a ASP/.NET project that included commerce capabilities. The credit card charge and credit functions would be taken care of by a product that could only be interfaced using the HTTP POST mechanism and which returned it’s results not as XML (why not!!!?) but as HTML with hidden form elements (who thought this would be a good idea by the way?). A couple years ago (because of my knowledge, although the technology available was a hinderance as well) I would have hardcoded the ASP app to make an HTTP POST call to this app and then would have used a regular expression to parse and display the results. This results in a very tightly coupled design. If for some reason the credit card merchant came out with new software that returned an XML stream or if we decided on using a new credit card merchant, we would have to tinker directly with the ASP code, re-test that code, and deploy that new code to the n servers this project required. Note that this may have been (and still might be) the *fastest* (as far as execution time) method of fulfilling the requirements. A loosely coupled implementation (which I chose to use) would build a layer on top of the credit card merchant software, abstracting the web application built in ASP from having to know the details of the hidden form fields returned in the HTML generated by an HTTP post. I wrote two modules: a) a module that interfaced directly with the credit card merchant software, posting and retrieving information from the software, and b) a web service with an immutable interface (ie: the ASP web application with always be able to depend on the interface being the same, no matter what credit card merchant we use) that uses the aforementioned module to instantiate the credit card merchant application. While this might seem more complex (and it is), it’s also extensible (ie: wanna add some logging to the credit card charge/credit process? wanna cache results? wanna cluster the application? great! As long as the interface stays the same, you can could swap out the C# web service for a Java web service (or Perl… or ColdFusion). You could begin logging the transactions to a database. You could write some type of caching mechanism to remember credit card details, *all without modifying your core ASP web application code*.

I mentioned in my example that doing a direct HTTP post to the merchant software *might* be the fastest in terms of execution time. Going back to John’s article, it’s interesting to note that a loosely coupled system

“…creates a safe harbor for experimentation within the module. Greater ability to experiment enhances the ability to learn. New approaches can be tried more frequently and participants can learn which approaches yield higher performance within each module. As these successful experiments yield better results, the performance of individual modules tends to improve at a faster rate. The cumulative effect of this performance improvement across an entire loosely coupled system can be significant.”

Couple books mentioned in the article that might be worth checking out:

Design Rules – Vol. 1 : The Power of Modularity by Carliss Y. Baldwin, Kim B. Clark

Out of the Box: Strategies for Achieving Profits Today & Growth Tomorrow Through Web Services by John Hagel

The Component-Based Business: Plug and Play by Richard Veryard


Get a SSL cert for $119

From the Netcraft newsletter:

“Most of the Certificate Authorities now understand that the market is a naturally price sensitive one, with a large number of sites simply going with the cheapest certificate on the premise that they all approximately do the same thing.

Over the last three years [1]Geotrust have, primarily through competitive pricing, garnered 10% of the market, and have become the role model competitor to Verisign in the way that Thawte had been before Verisign bought it.”

Get a SSL cert for $119 or check out these other low cost alternatives: