Information Technology

No one knows how many computer software projects go awry, but my observations as a software consultant in many corporations suggest that easily a majority are colossal failures, though such blunt labels offend sensibilities and are typically eschewed. Causes generally precede effects, and indeed one is hard put to find anything being done right in the Information Technology (IT) domain. The challenge consists not in identifying flawed practices and misguided approaches, which fairly spring out upon inquiry, but in explaining why such costly inefficiency is tolerated, and how an almost risible level of ineptitude has generated so little reaction either within or outside the corporation.

To be sure, bad news softens as it rises, and one may suppose that by the time news of a bloody massacre reaches a chief executive it sounds more like a fender bender than something calling for serious concern. Software project deliverables are so ill-defined it is often possible to declare victory even as enemy troops patrol the capital. And while enthusiasm is rare when clients receive delivery of a software application, they are so utterly ignorant of what to demand of a decent system that outright rejection, not to say fury, does not occur nearly as often as it ought.

Still, even allowing for the way ambiguity and confusion thicken with increasing altitude, there must exist people with clout who suspect something is amiss. The systems, after all, are not working. As one scans the field of failed or abandoned information systems, the wonder is that bloody massacres are not more frequently visited upon IT producers themselves.

Is there a cultural explanation for the forbearance of aggrieved corporate leaders, turning the other cheek again and again and again? Certainly there is no shame in being thoroughly ignorant of all things technical. This much is evident from the readiness with which people proclaim their own “technical illiteracy.” Perhaps the view from the top is that, after all, IT is IT: a bastard stepchild of the engineering profession, distinct in having neither standards nor licensing authorities, peopled by practitioners who typically lack specialized education. Like generals fighting a battle with defective equipment, leaders may feel no need to blame themselves unduly for losses suffered when missiles frequently explode prior to launch.

That the disasters are essentially secret does nothing to complicate leaders’ tendencies to exculpate themselves. All that is seen from outside the organization is a normal (high) level of IT overhead (either in-house staff or consultants), a fact of life that is hardly anomalous. The inefficiency resulting from inadequate systems is not readily measured, and even undeniable failures need not be known to outsiders when they are scarcely mentioned aloud on the inside.

While one can explain the tolerance of executives for problems they are not personally blamed for, it is harder to account for the silence of commentators, pundits, consultants, the press, in the face of this important industry’s stunningly poor performance.

Impressive fiascoes enter the public domain at regular intervals. At the 1996 Summer Olympics in Atlanta, the failure of an $80 million dollar system led organizers to emulate their forebears in ancient Greece by resorting to human runners to deliver contest results. In 1997, an assistant commissioner of the IRS testified before a congressional panel that the agency had spent $4 billion developing new computer systems that “do not work in the real world.” What has prevented some enterprising reporter from asking whether such failures represent an abnormality? --And from discovering, and breaking the story, that they are the norm?

Within the industry, there is nothing in the panting self-promotion of software product vendors, the giddy pitches of consulting firms, the bold headlines of the computer press, or the polished certainty of renowned gurus, to suggest that anyone paying money for a computer-related good or service ever suffered a setback, anywhere, in any way -- unless a new and improved solution is being promoted that forever extinguishes the possibility of further setbacks.

This may partly explain why the truth about IT is never heard. In the face of such widespread exuberance, a naysayer simply lacks credibility. Even among many of the people well placed to know the truth -- the developers themselves, their managers, the long-suffering users -- there is an inability to extrapolate their experiences, to believe that what they are seeing is representative of the computer industry as a whole. What they see with their own eyes is seldom articulated in the first place. That can only bring trouble when organizations put a premium on positive thinking and teamwork.

From a larger perspective, there is cause for concern about the culture within which the corporation operates: workers are disquietingly susceptible to the big lie. They acquiesce in palpable falsehoods. And they are personally disinclined to think along lines that may lead them to uncomfortable discoveries; to begin with, that while they may be complying with their immediate superiors’ directives, the work in which they are engaged is futile in terms of their employers’ or clients’ rational objectives.

Other cultural forces have also acted to prevent recognition of the problem. Despite the general sense that senior management cannot be blamed for IT problems, what I see everywhere are violations of basic management principles. This is another secret: the root causes are not technical. The failures are management failures. The direct corollary may be what has so far made America impervious to the truth: there is a widespread problem of competence in the leadership of corporate institutions.

Americans are cynical about their political leaders. They have become cynical about most public institutions. This has been dwelt upon by pundits beyond the point of triteness. The corporate world, however, is another category altogether. It can be reviled as heartless, ruthless, “lean and mean.” But these are after all cheerful qualities, lifting the hearts of otherwise defeated patriots, reminding us why we’re #1, making it clear that beating Communism was no accident. While we are receptive to tales of “slashing” and “aggressive tactics,” it is impermissible to ridicule the corporate world on the whole as bungling and inept. This goes to the heart of American pride. Business management is one thing we’re supposed to be really good at.

Consider some utterly basic principles of good management: hire and promote good people; reward quality; establish incentives consistent with the company’s success. One might expect such precepts to be instinctive even without an MBA. All are flagrantly violated in the corporate IT realm.

In hiring and promotion, the qualities that make for success in IT are exactly the same as in every other domain of the corporation: enthusiasm, team spirit, quick-wittedness, a knack for compromise, “communication skills” (meaning fluidity and self-assurance, preferably with an American accent), and salesmanship. Has it occurred to anyone that these are not the primary qualities needed in software development?

The unremarked fact is that software development is a uniquely intellectual pursuit. Quality software demands, first of all, unusual -- I mean, unusual -- intelligence. We are not talking here about “emotional IQ,” discretion, life smarts, investing wisely, keeping one’s cards close to one’s chest, smiling while thinking villainous thoughts. We are referring to straightforward, raw mathematical intelligence, reasoning ability, the kind of thing standard intelligence tests aim to measure.

This undemocratic fact of nature immediately disqualifies the majority of IT staffers today. This should surprise no one who has glanced at IT resumes, which typically indicate university education in subjects like commerce or psychology, rather than computer science, math or physics.

Quality software demands deep, sustained concentration; a gift for abstracting business reality into mathematical models; an ability to build a structure of rules that correctly anticipate any contingency. An instinctive, aesthetic appreciation of simplicity is important, because the developer must fight the tendency of systems to become incomprehensible messes riddled with exceptions and special cases.

In the upper percentiles of mathematical intelligence, individuals are liable to be unconventional. They may flunk obvious questions like, “Do you value teamwork,” or “Would a fast-paced environment thrill you?” They may be more blunt than is appreciated in your corporate culture. Or quite possibly they do not seem “smart” to your HR people. Any of these possibilities may account for why they are not working at your firm -- and why you don’t have systems that work.

Your organization may also be rejecting prime candidates because it demands that applicants be experienced in a specific industry or application area, like banking. When searching for outstanding software talent, it makes little sense to exclude candidates who are not familiar with a few facts they could pick up in a week. Programming is programming, and exceptional developers perform exceptionally irrespective of the application area. Nevertheless, such irrelevant criteria are in widespread use.

Mediocrity breeds mediocrity, in IT as elsewhere. In building software, developers themselves use software products built by others, specifically, programming languages. The level of quality of modern programming languages is abysmal. What this means is that a developer testing a program he or she has written will generally find that it does not work -- even if the program is perfectly correct. The problem is that the developer’s instructions are not being carried out properly. Travelers approaching the 21st century along the “superhighway” cannot count on 2+2 yielding 4.

Are languages such complicated beasts that this state of affairs is inevitable? Of course not. Two decades ago, while using a language called APL over the course of several years, I never once encountered a bug. Today, using Sybase’s widely used PowerBuilder language, one can hardly write a dozen lines of code without two or three bugs being exposed. Tracking these down, and altering programs so as to circumvent them, is a decidedly time-consuming process. This decline in the quality of software tools goes a long way towards explaining why programs that were never supposed to remain in use as long as the year 2000 have still not been replaced.

The reason developers are plagued by the vast inefficiency of such defective tools is the unmistakable market signal received by language vendors: quality is for suckers. Feverish advertising is what the gullible IT sector responds to, as well as cultish propaganda promoting supposedly revolutionary approaches like “object-oriented programming” (OOP), “three-tier computing,” and “thin clients,” which managers who have never programmed find irresistible. This leads vendors to spend disproportionate resources on sales and marketing, rather than much-needed product development.

As for the alignment of incentives with corporate objectives, it may be only in the IT universe that the phenomenon of negatively contributing workers is commonplace. Since much IT work is done by consulting companies whose fees are based on billed hours, a high body count is an economic imperative. Teams of 10, 20 or even more developers are not unusual. In their drive to warm seats, consulting firms are prepared to sign up staffers who, at best, will do nothing to affect development schedules.

The indifference to productivity is readily perceivable by anyone who is technically informed. As recently as mid-1998, many developers were still working with the 1991 version of Microsoft Windows (v. 3.1) on tasks for which it is hopelessly ill-suited by comparison to Windows 95, which millions of non-technical users worldwide have used since its release.

How might corporations’ workings become more compatible with the conventional, innocent vision of capitalism? First, incentives promoting efficiency would come to be. The practice of making decisions based on uninformed executive groupthink would be curbed, and input from knowledgeable people sought. Mistakes would be acknowledged when they came to light, since errors not faced are likely to be repeated.

In IT, it would be discovered that developers are not fungible. Unlike tasks like moving rocks or clearing brush, in software development some people can do the work of ten. Compensation levels would recognize this, and more capable people would be attracted to the field.

By definition, we cannot know how many delusions we labor under, but we have learned that some of them don’t last forever. Our complacent belief in the business acumen operating in our corporate institutions is contradicted by daily reality, by obvious mistakes, by deplorable results and wasted human resources. The problem is broader, of course, than the computer industry. In striving for the “excellence” that is everywhere trumpeted, one is in large measure opposing the very sweep of great historical trends carrying our society towards ever more superficiality, mindlessness and duplicity. But oppose them we must, not just for wealth, but to help defend our society’s most deeply cherished values.