Tangible Illusions

How the ideas behind the graphical user interface also created whole new visual media.

2020-10-25 Previous Home Next

Apple's Macintosh burst onto the scene in 1984 as a low-cost computer for everyone. It had something that IBM's PC didn't: a graphical user interface.

Mac Plus Screen

The Macintosh Plus screen shows familiar things: a mouse-driven pointer, windows, and buttons. Any teenager today would have no difficulty using it.

The Graphical User Interface or GUI (pronounced gooey) has evolved somewhat, but its basic concepts have remained unchanged through the decades. You open windows, drag icons, and click buttons. It all feels quite natural.

But it wasn't always obvious that manipulating things on a screen would be the most intuitive way to use a computer.

Building objects

Think about a document window on your screen. If you press on its title bar, you can make it expand or collapse. If you drag on its sides or corners, you can stretch it this way and that. If you click inside it, you can edit or select words.

This window is an object that the computer conjures up for you— something concrete you can touch, to help you to interact with the document. The computer doesn't need it; in fact, it has to go to extraordinary lengths to maintain it.

Computers can really only draw dots on the screen and sense your mouse. To build and maintain tangible-seeming objects on top of such basic capabilities takes a lot of software. The GUI was forged over decades of sweat and toil, starting from a few good ideas.

Ivan Sutherland's Sketchpad

In 1963, MIT Ph.D. student Ivan Sutherland submitted his thesis, "Sketchpad: A Man-machine Graphical Communication System." A key idea: if a computer were to represent shapes as objects on a screen, people could use the computer without programming.

Well, ideas are a dime a dozen, and talk is cheap. Engineers don't get points for those, only for something that actually works. Sutherland had written a complete program called Sketchpad to demonstrate his thesis.

Sketchpad single-handedly changed the thinking about graphics forever. It let you create engineering drawings made up of lines and circles. You could use a "light pen" (sort of like a mouse) to quickly draw these shapes on the screen, copy, inter-connect, zoom, and modify them.

Sutherland explained in detail how he represented these objects inside the computer to make graphical interaction possible.

Sketchpad thesis diagram

Picture from thesis shows a horizontal line, an arc, and points (solid) surrounded by areas (dotted lines) for selection.

For example, Sutherland explained how you could select any object by pointing at it, a basic capability for any GUI.

The program defined areas around the objects so that if your pen pointed anywhere within, it could assume you meant to select the corresponding object.

Using this and other ideas, many other researchers created graphical user interfaces in the sixties and seventies. A research lab owned by Xerox in California created the Star computer which, while not commercially successful, combined graphical user interfaces and many other novel ideas that were used in the Apple Macintosh.

New visual arts

Ivan Sutherland was just getting started. He headed a Department of Defense research funding lab, taught computer science, and started consulting companies. His ideas and inventions have permeated the research community. He won the A.M. Turing award, computer science's Nobel prize.

As computers have gotten faster and more capable, entire new vistas have opened up for graphics. Some of Sutherland's students, and other engineers he influenced, went on to build their own software and hardware companies.

For example, Adobe Systems, the company that created Photoshop and the PDF file format, was co-founded by an engineer he mentored. Another example: a student of his co-founded Pixar Animation Studios (and also won a Turing this year).

Computers are now our partners in creating tangible illusions, and their creations get more elaborate every year.