Friday, July 29, 2005

[Cool] My Software Engineering Rant Seven Years Ago

Great Caesar's Ghost! Someone just sent me an email commenting on a usenet post I made seven years ago on software engineering! I actually like what I wrote too, it still seems highly relevant in this day and age. Well, except maybe the part about Enterprise Java Beans... :)

I've edited the line breaks slightly to make it more readable in a web browser.

From: jasonh@xxx.yyy.EDU (Jason Hong)
Subject: [NEWS] Re: Why is programming so hard?
Date: 11 Dec 1998 23:02:30 GMT

: I'd like to hear more. Perhaps it would bring theory,
: philosophy, and poetry to engineering.

Here's a few I'd also add:

o No natural visualizations
There's no simple way to "see" or "draw" software, often
making its abstract representation the only one we have.
I'm also unconvinced that all of the various object
modeling tools and techniques (UML, Fusion, etc) really
augment our abilities to think and do design.

o Scale
It's difficult to think and design in several orders of
magnitude, from bits to gigabytes. Also, the size of state
grows exponentially with each new variable. We've had to
invent new ways of managing and abstracting this scale,
but we still need to do more.

o Logical thinking
Let's face it, humans are not innately good at it. Our
brains are underpowered for this kind of thinking without
lots of training. And logic is the foundation computer
systems are built upon.

o Lack of good tools
How many people still do debugging via print statements?
How many people still use editors that have little or no
knowledge of the programming language being used?
How easy is it to profile your application?
How easy is it to get good visualizations of what's going
on in the system?
Why is using a compiler more like talking to a wall than
a dialog between person and machine?
How many errors are still due to pointer arithmetic and
buffer overflows?
How long does it take to implement a _good_ user interface?
How many times have teams overwritten each other's code?

Let's face it, most of our tools are like paleolithic axes:
somewhat useful but very crude.

o Few metrics
There are few metrics that indicate we as a discipline
are improving. Faster processors, cheaper memory, and
larger disks only go so far, but what is their relationship
to programmer productivity?

Also, what's good code? How do you measure it?
What's good design? How do you measure that?

o Lack of orthogonal properties
If you want scalability, high availability, fault-tolerance,
and security, often you have to build it directly into the
software so that it permeates the very essence of the software.

This mixes the functionality and application logic of
the software with the properties of the software, making
maintenance, debugging, and testing more complex.

What we need is a way of separating desired system properties
from logic, making it easier to get what you want without
having to understand every underlying detail. I do believe
Enterprise Java Beans are one small step in this direction.

o Lack of widespread experience
People still aren't taught good design skills at universities.
People still aren't taught good testing skills at universities.
People still tend to reinvent the wheel because it's too hard
to find something that fits what you need.
People still aren't taught good systems engineering skills.
People are still making many of the same design and
implementation mistakes others made decades ago. It's as
if we haven't learned.

o Lack of good methods
What's the process for creating good software? I really don't
believe it's anything methodologies like OMT, Booch, Fusion,
and others describe. They're too prescriptive and completely
ignore wide areas of real software development.

o Mercilessly precise need for details
Computers are stupid, and have to be told every single detail
in order to work. Most people don't think in this manner at
all, but are forced to adhere to the absolute precision needed
by the computer in order to get anything done.

This also makes programming real applications more difficult,
since people think and do things differently, and your
application will have to handle not only the right case but
all of the exception cases.

o Poor understanding
A lot of people still don't understand that software
engineering is more than just programming. It's about
prototyping, designing architectures, re-designing
architectures, interacting with customers, working within
a team, communicating with others, testing the system, and
running usability studies, just to state a few.

: But I'm interested to hear some other people's views on why
: programming is hard by nature. Or if you don't have time for
: that, maybe some stimulating articles or books you can refer
: me to?

Here are a few. Check out:

o The Mythical Man Month, Fred Brooks
(especially "No Silver Bullet")
o Rapid Development, Steve Maguire
o Code Complete, Steve McConnell
o Hints for Computer System Design, Butler Lampson

Friday, July 22, 2005

Fortune: Can Americans Compete?

From Fortune Magazine, via the FoRK Mailing List

The No. 1 policy prescription, almost regardless of whom you ask, comes down to one word: education. In an economy where technology leadership determines the winners, education trumps everything. That's a problem for America. Our fourth-graders are among the world's best in math and science, but by ninth grade they've fallen way behind (see table). As Bill Gates says, "This isn't an accident or a flaw in the system; it is the system."

The good news is that we've overhauled the system before. A century ago, as America changed from an agricultural to an industrial economy, something called the high school movement swept the country...

We responded to a changing world again in 1958, after the USSR orbited Sputnik while our rockets kept blowing up on the launch pad... We went to the moon, science and engineering became cool, even glamorous, and we gained a wide technology lead.


A prescription urged just as widely is immigration reform. A critical element of America's economic dominance has been its attraction for the world's brightest, most ambitious people, but today's immigration laws favor family reunification far above talent, intelligence, or credentials.


John Doerr, the legendary Silicon Valley venture capitalist, recommends that every foreign student who gets a Ph.D. at a U.S. university should also get a green card (granting permanent residency) stapled to his or her diploma.


But the greatest challenge will be changing a culture that neither values education nor sacrifices the present for the future as much as it used to--or as much as our competitors do. And you'd better believe that American business has a role to play--after years of dot-com-bust- and scandal-driven reticence, more corporate leaders need to summon the courage to lead.

[Funny] Craigslist: How I got the best of this Nigerian scammer

One approach to fending off email scams is to flood the scammers with lots of bad data (an anti-spam amplifer, in the words of UWashington professor Oren Etzioni).

This person on craigslist has a rather humorous way of wasting the scammers' time...

Dear Sir,

I don’t know who this other person you were writing to is , but, my name is Bo Luke. I operate a moonshine business with my brother, Luke. You might say that we are in the “distribution” business. I am interested in your proposal. I know a little about steel – as I have that album “british steel” by Judas Priest. Please tell me more.


Bo Luke
Hazzard County, USA





It gets even better afterwards...

Wednesday, July 20, 2005

[HCI] Lincoln Lab TX-2 Group

Bill Buxton has a page up on the CHI2005 panel about the early days at the MIT Lincoln Labs. Cool history about some of the early pioneers in interactive computing.

Monday, July 18, 2005

On Failing To Think Long-Term

At first it was going to be a book of 18 chapters chronicling 18 collapses of once-powerful societies--- the Mayans with the most advanced culture in the Americas, the Anasazi who built six-story skyscrapers at Chaco, the Norse who occupied Greenland for 500 years. But he wanted to contrast those with success stories like Tokugawa-era Japan, which wholly reversed its lethal deforestation, and Iceland, which learned to finesse a highly fragile and subtle environment.


Diamond reported that his students at UCLA tried to imagine how the guy who cut down the LAST tree [in Easter Island - JIH] in 1680 justified his actions. What did he say? Their candidate quotes: "Fear not. Our advancing technology will solve this problem." "This is MY tree, MY property! I can do what I want with it." "Your environmentalist concerns are exaggerated. We need more research." "Just have faith. God will provide."


One common one is that elites become insulated from the consequences of their actions. Thus the Mayan kings could ignore the soil erosion that was destroying their crops. Thus the American wealthy these days can enjoy private security, private education, and private retirement money. Thus America itself can act like a gated community in relation to the rest of the world, imagining that events in remote Somalia or Afghanistan have nothing to do with us. Isolation, Diamond declared, is never a solution to long-term problems.

[Tech] IBM's New Paradigms Using Computers

I think this is the first time in seven years I haven't gone to this conference. Fortunately, JD Lasica did...

Ian Smith, Intel researcher, argues for the future of mobile phones as a development platform:

Smith holds up his mobile phone: "Most people don't even think of this device as able to run new software. They don't think of that for mobile phones, and that's really really weird. … You can retire to Jamaica if you figure out (how to create a business model in that space)." Ringtones and screen backgrounds for your cellphone aren't really software applications. "I'd argue there's no piece of software that's driving this platform today."


An audience member asked about proprietary systems that allow only a limited amount of applications and experiments with content. Smith says he thinks we'll see a "hungry hippo in this space," perhaps the No. 2 or 3 vendor, which will make a business decision to open up the box and allow this thing to have lots and lots of applications on it and let 1,000 flowers bloom.

More info, and an interview with IBM Researcher Dan Russell:

[Privacy] NYTimes: A Pass on Privacy?

Sort of meanders around, but the basic point is about privacy vs convenience.

The E-ZPass system, as it is called on the East Coast, seemed like idle gadgetry when it was introduced a decade ago. Drivers who acquired the passes had to nose their way across traffic to reach specially equipped tollbooths -- and slow to a crawl while the machinery worked its magic. But now the sensors are sophisticated enough for you to whiz past them. As more lanes are dedicated to E-ZPass, lines lengthen for the saps [JIH - ie, people like me] paying cash.

E-ZPass is one of many innovations that give you the option of trading a bit of privacy for a load of convenience. You can get deep discounts by ordering your books from or joining a supermarket ''club.'' In return, you surrender information about your purchasing habits. Some people see a bait-and-switch here. Over time, the data you are required to hand over become more and more personal, and such handovers cease to be optional.


The potential applications multiply: what if state policemen in the United States rigged E-ZPass machines to calculate average highway speeds between toll plazas -- something easily doable with today's machinery -- and to automatically ticket cars that exceed 65 m.p.h.?


People waver on whether to trade privacy for convenience, but they're pretty untroubled about trading privacy for security. On occasion, E-ZPass records have been used to track down criminal suspects.

Tuesday, July 12, 2005

[HCI-Sec] SOUPS - Symposium on Usable Privacy and Security

Ping Yee has summaries of many of the sessions at SOUPS2005, recently held at Carnegie Mellon. Some highlights:


SFGate has an article about how online resumes can be used by identity thieves and other kinds of criminals. It seems like we're getting to the point where "creative" criminals are leveraging any kind of personal information for gain.

"We're hearing from a couple of people each week who are having their resume accessed by criminals," said Pam Dixon, executive director of the nonprofit San Diego research organization World Privacy Forum.

Often, the offer is for a so-called remailing job. In one such case reported by the nonprofit Privacy Rights Clearinghouse in San Diego, a woman found a job through the Web site

The woman, whose name was not disclosed, was to receive packages in the mail and resend them to an address in Belarus. She was promised a $2,000 monthly salary for the relatively cushy job. When she never got paid -- or even reimbursed for her shipping costs -- she started to investigate and found out that the company didn't exist.

Wednesday, July 06, 2005

[Cool] NYTimes: Creative thinking foils car thieves

The LoJack security system has hurt the business model of auto theft, forcing thieves to go into new lines of work -- and that should inspire us to think more creatively and systematically about how to reduce crime.

Sold for $695, the LoJack is a radio transmitter that is hidden on a vehicle and then activated if the car is stolen. The transmitter then silently summons the police -- and it is ruining the economics of auto theft.

Car theft, it turns out, is a volume business. And so if even a small percentage of vehicles have LoJack, the professional thief will eventually steal a car with one and get caught.

The thief's challenge is that it's impossible to determine which vehicle has a LoJack (there's no decal). So stealing any car becomes significantly more risky, and one academic study found that the introduction of LoJack in Boston reduced car theft there by 50 percent.

Two Yale professors, Barry Nalebuff and Ian Ayres, note that this means that the LoJack benefits everyone, not only those who install the system. Ayres and another scholar, Steven Levitt, found that every $1 invested in LoJack saves other car owners $10.

Nalebuff and Ayres note that other antitheft devices, such as the Club, a polelike device that locks the steering wheel, help protect that car, but only at the expense of the next vehicle.


An article in The Milken Institute Review proposes other ideas for reducing crime. The author, John Donohue of Yale, notes that both building prisons and adding police officers reduce crime rates. But he argues that we get much more bang for the buck by hiring police.

We have about 300,000 more prisoners than is cost-effective, Donohue calculates. In other words, every extra $100 spent on incarceration reduces crime losses by some smaller amount, say $50. But he also finds that we could add up to 500,000 police officers, and they would pay for themselves in crime savings.

Some social programs also pay for themselves in reduced crime. Donohue argues that a good bet is the Perry Preschool program (which involves weekly home visits). Its graduates end up 40 percent less likely to be arrested than those in a control group. The Job Corps for at-risk teenagers has also been shown to be very effective.

Tuesday, July 05, 2005

[Pittsburgh] NYTimes Op-Ed on Pittsburgh

Three posts in a row, with more to come. This is what happens when you're jet-lagged and have too much time at night.

Anyway, John Tierney has an op-ed in the New York Times about the recent Supreme Court decision on eminent domain, using Pittsburgh as an example of eminent domain gown awry. I haven't lived long enough here in Pittsburgh to visit all the places he mentions, but I've been in downtown enough times to notice that no one really lives there, and while it's not unpleasant, it's not vibrant either.

Pittsburgh has been the great pioneer in eminent domain ever since its leaders razed 80 buildings in the 1950's near the riverfront park downtown. They replaced a bustling business district with Gateway Center, an array of bland corporate towers surrounded by the sort of empty plazas that are now considered hopelessly retrograde by urban planners trying to create street life.

At the time, though, the towers and plazas seemed wonderfully modern. Viewed from across the river, the new skyline was a panoramic advertisement for the Pittsburgh Renaissance, which became a national model and inspired Pittsburgh's leaders to go on finding better uses for private land, especially land occupied by blacks.

Bulldozers razed the Lower Hill District, the black neighborhood next to downtown that was famous for its jazz scene (and now famous mostly as a memory in August Wilson's plays). The city built a domed arena that was supposed to be part of a cultural "acropolis," but the rest of the project died. Today, having belatedly realized that downtown would benefit from people living nearby, the city is trying to entice them back to the Hill by building homes there.

In the 1960's, the bulldozers moved into East Liberty, until then the busiest shopping district outside downtown. Some of the leading businessmen there wanted to upgrade the neighborhood, so hundreds of small businesses and thousands of people were moved to make room for upscale apartment buildings, parking lots, housing projects, roads and a pedestrian mall.

I was working there in a drugstore whose owners cursed the project, and at first I thought they were just behind the times. But their worst fears were confirmed. The shopping district was destroyed. The drugstore closed, along with the department stores, movie theaters, office buildings and most other businesses.

You'd think a fiasco like that would have humbled Pittsburgh's planners, but they just went on. They kicked out a small company to give H. J. Heinz more room. Mayor Tom Murphy has attracted national attention for his grand designs - and fights - to replace thriving small businesses downtown and on the North Side with more upscale tenants.

The city managed to clear out shops and an office building to make room for a new Lazarus department store, built with $50 million in public funds, but Lazarus did not live up to its name. It has shut down and left a vacant building. Meanwhile, the city's finances are in ruins, and businesses and residents have been fleeing the high taxes required to pay off decades of urban renewal projects and corporate subsidies.

Yet the mayor still yearns for more acquisitions. He welcomed the Supreme Court decision, telling The Pittsburgh Post-Gazette that eminent domain "is a great equalizer when you're having a conversation with people." Well, that's one way to describe the power to take people's property.

But I think a future Supreme Court justice would have a different view of eminent domain after touring Pittsburgh's neighborhoods, especially those that escaped urban renewal: the old-fashioned business districts with crowded sidewalks and the newly gentrified neighborhoods with renovated homes and converted warehouses. The future justice would quickly see what sets the success stories apart from Gateway Center and East Liberty. No politicians ever seized those homes and businesses for a "better use."

[Cool] Science: 125 Questions on What Don't We Know?

A good overview of some frontiers of science.

[Research] IEEE Spectrum: The End of AT&T

At the time, Bell Labs managers generally regarded their company as a quasi-public institution contributing to the national welfare by enriching the country's science and technology. Seen in that light, AT&T's vigorous promotion of semiconductor technology made good sense—especially during a time the company was churning out profits and didn't feel any competition breathing down its neck.

But such generosity may have been one of the crucial forces behind its eventual downfall, as smaller, nimbler, and more legally unfettered firms seized the opportunity to develop and deploy innovations that would help undermine AT&T's dominance of U.S. telecommunications. "After its forced breakup in 1984," The Wall Street Journal's Rhoads wrote, "it was slowly crushed by technologies that drove down the price of a long-distance call, and more recently by wireless calling and Internet phoning."

At the same time Bell Labs and Western Electric were working on their many innovations, there was a resistance to rapid change rooted deep within the parent company's culture. According to Sheldon Hochheiser, former AT&T corporate historian, "a service ethos and the absence of the countervailing pressures of competition produced a corporate culture dominated to a great degree by an engineering mentality." That culture, he adds, "encouraged a value system where managers tended to take the time to get innovations right, as an engineer would define right."

Thus, AT&T engineers usually emphasized reliability and robustness of the network over the rapid introduction of advanced technologies. Often a decade or more passed before new features, such as long-distance direct dialing and touch-tone phones, would finally percolate throughout the system. And cellular telephony, first described in detail by Bell Labs engineers in 1947, never gained widespread commercial operation as part of the Bell System.

IN RETROSPECT, IT SEEMS UNREASONABLE to expect that a publicly held corporation can devote so much money to long-term research when facing the ruthless forces of the marketplace. AT&T added tremendous value to society, but as a condition of its regulated monopoly status, the company was not allowed to commercialize new technology that was not directly related to telephony.

Nor could AT&T charge customers for the technology except through its fees for telephone equipment and services. When it was a regulated monopoly, the company could build into those charges a pittance devoted to risky future-oriented research, such as setting up a solid-state physics department in the postwar years. But as ordinary corporations competing for customer dollars after the breakup and later spin-off, AT&T and Lucent could afford no such luxury.

We the customers are the ultimate losers. A vigorous, forward-looking society needs mechanisms like this to set aside funds for its long-term technological future. Letting governments serve the purpose is an imperfect alternative at best, fraught with the difficulty of making wise choices. The peer-review process widely used to select projects may be able to direct public funds to worthwhile research, but it usually favors established scientists and often overlooks bright young researchers—such as Chu—with bold but risky ideas.

AT&T, Bell Labs, and Western Electric effectively diverted a tiny fraction of our everyday expenses—and from all corners of the U.S. economy—into long-term R&D projects in an industrial setting that could, and often did, make major improvements in our lives. Today we are eating up the technological capital they built during those amazingly productive years. Are we doing anything to replace it?

[Cool] If Feynman interviewed at Microsoft

Funny, other people I'd like to see parodied:

  • RMS
  • Steve Jobs
  • Larry Ellison
  • Bill Gates

If Richard Feynman applied for a job at Microsoft

Interviewer: Now comes the part of the interview where we ask a question to test your creative thinking ability. Don't think too hard about it, just apply everyday common sense, and describe your reasoning process.

Here's the question: Why are manhole covers round?

Feynman: They're not. Some manhole covers are square. It's true that there are SOME round ones, but I've seen square ones, and rectangular ones.

Interviewer: But just considering the round ones, why are they round?

Feynman: If we are just considering the round ones, then they are round by definition. That statement is a tautology.

Interviewer: I mean, why are there round ones at all? Is there some particular value to having round ones?

Feynman: Yes. Round covers are used when the hole they are covering up is also round. It's simplest to cover a round hole with a round cover.

Interviewer: Can you think of a property of round covers that gives them an advantage over square ones?

Feynman: We have to look at what is under the cover to answer that question. The hole below the cover is round because a cylinder is the strongest shape against the compression of the earth around it. Also, the term "manhole" implies a passage big enough for a man, and a human being climbing down a ladder is roughly circular in cross-section. So a cylindrical pipe is the natural shape for manholes. The covers are simply the shape needed to cover up a cylinder.

Interviewer: Do you believe there is a safety issue? I mean, couldn't square covers fall into the hole and hurt someone?

Feynman: Not likely. Square covers are sometimes used on prefabricated vaults where the access passage is also square. The cover is larger than the passage, and sits on a ledge that supports it along the entire perimeter. The covers are usually made of solid metal and are very heavy. Let's assume a two-foot square opening and a ledge width of 1-1/2 inches. In order to get it to fall in, you would have to lift one side of the cover, then rotate it 30 degrees so that the cover would clear the ledge, and then tilt the cover up nearly 45 degrees from horizontal before the center of gravity would shift enough for it to fall in. Yes, it's possible, but very unlikely. The people authorized to open manhole covers could easily be trained to do it safely. Applying common engineering sense, the shape of a manhole cover is entirely determined by the shape of the opening it is intended to cover.

Interviewer (troubled): Excuse me a moment; I have to discuss something with my management team. (Leaves room.)

(Interviewer returns after 10 minutes)

Interviewer: We are going to recommend you for immediate hiring into the marketing department.

Keith Michaels