The Declarative and Imperative Views

Appeared in Volume 9/3, August 1996

dennis@amzi.com
Dennis Merritt
20th May 1996

Fernando Pereira wrote:

print_people_report :-
   people_report(Report),
   print_report(Report).

people_report(Report) :-
   people(People),
   format_list(People, Report).

people(People) :-
   findall(Person, person(Person), People).

I'd like to present some ideas on why programmers don't, in general, gravitate to higher-level languages, based on having wrestled with these issues from the vendor side of the business for a number of years. It has been my experience that users often get confused and frustrated with products that seem intuitive and easy-to-use to individuals working for the vendor.

I originally started working with database vendors selling, among other things, report writers. Report writers, like Prolog, were sold by hyping the advantages of declarative specification of reports, as opposed to the old way of writing COBOL programs. They looked great for the various sample reports, but they could not be expanded to handle more complex reports.

As the report specifications became more complex, with various running totals and subtotals and formatting requirements and the like, the declarative syntax begins to get in the way. It is only the vendor's experienced field support people, who know exactly how the tool works, that can come up with the magic sequence of statements to make a complex report appear.

From the user's point of view, it is not at all clear where the line is that the declarative report writer tool starts to become difficult to use. In other words, the user becomes frustrated as he/she tries to push the envelop of the tool. At that point, it is often easier to go back to COBOL than try to figure out the tool. And when the next report comes in, well maybe its just safer to use COBOL again anyway.

The same is true for spreadsheets. At some time, a sophisticated user of a spreadsheet is going to have to understand the very procedural way in which the cells get calculated on the spreadsheet. Armed with that knowledge, the spreadsheet user becomes a power user.

I worked for a time for Dr. Larry Harris at what was once Artificial Intelligence Corp and later AICorp (and now part of Trinzic). We sold Intellect, a natural language query language for mainframe databases.

Again, many examples showed the power and expressiveness of Intellect, but when the database queries started to become complex, it became difficult to figure out how to express the queries, because underneath there was always the same old CPU running one instruction at a time, and unless you understood how Intellect worked, you couldn't get the queries to come out like you wanted.

But, Larry Harris could quickly write the most complex queries and get them right the first time. He didn't see a problem with generating complex elegant declarative natural language queries. But he knew exactly how the thing worked, although I don't believe he consciously used that information in generating his queries.

I propose that the same is true amongst those in the Prolog community. After sometime we begin to see the language declaratively and become more comfortable using it as Fernando Pereira does. But I suspect his skill level with the language is due in large part to what is at this point for him, a second nature, deep understanding, of exactly how Prolog works. (For example, it is not even worth showing the code to generate a report from a list, but it is exactly those kinds of recursive tasks that often drive new users nuts.)

My point again, after this rambling, is that I believe experienced users of any tool are no longer conscious of the depth of knowledge they have of the tool.

I recently re-read my "Adventure in Prolog" book (which we've just republished), and was surprised at how painstakingly slow it goes, and the time spent on topics which seem obvious. I would not write the book that way today, but it was written shortly after I learned Prolog and covers the language from the point of view of an experienced procedural programmer who found it difficult to get started with the language.

pereira@research.att.com
Fernando Pereira
21st May 1996

I particularly liked the NL DB interface example, since I worked in the area (not commercially) and also noticed the mismatch between the original claims ("NL makes it easier") and reality. But with hindsight this should not have been surprising. Crafting a complex specification in English, or a complex program in a declarative language, is a difficult, skilled task very different from day-to-day programming. It has been often observed that good English majors make better specification writers than good engineering majors. Only a minority who have the inclination, training and time are able to bridge the gap comfortably between specifications and execution, whether for programming in a declarative language or for designing an imperative program from a declarative specification.

Traditionally, programming was carried out by people capable of mentally orchestrating complex sequences of events and their interactions, not by specification crafters (those typically went to law school ;-)). Thus, languages that favor the latter will find poor soil in the former, who are the majority of the programming profession. The two ways of thinking come together in mathematics, although the marriage has never been perfect, viz the fights between "dynamic" and "static" views of infinity and limits, that go back to the conflicting Newtonian and Leibnizian views of the calculus (and perhaps as far as the classical world).

Thus some mathematically-inclined people find it relatively easy to jump between declarative and imperative views of a problem. However, it is presumptuous to suppose that this what "real world" programming is like.

In addition to the examples we have been discussing, I encourage anyone to look at the Microsoft Active VRML language for another instance. It is in my opinion a very elegant functional language based on ideas from Lucid, Concurrent ML and others to specify combinations of complex behaviors (space-time extents in virtual space) without unnecessary explicit mentions of time or synchronization.

I suspect that it will have a hard time being adopted compared with more conventional VR scripting languages because of the distant relationship between an algebraic specification of a dynamical system and the synchronizations and numerical integrations that implement it. This is not a criticism of practitioners, by the way, but rather a reflection on what skills and inclinations have been selected in the development of the programming profession (hence the fights between "systems engineers" and "coders," too).