You haven't seen anything yet - Augmented Reality (AR) is on the way

Programming the ENIAC. US Army.

Computers have been around for a very long time. ENIAC, the first general purpose electronic computer, was constructed over seventy years ago. And we have always struggled with how to communicate with them.

Your smartphone screen is littered with colorful graphic images representing applications (apps). The screen is touch sensitive, and you call up an app by simply touching an icon. Once in the app (for instance,  Facebook), you can view your feed or notifications by touching the appropriate widgets. Photos can be zoomed by a reverse pinch on the screen. When you want to type, a virtual keyboard appears as if by magic. If you’d like, you can specify an Italian or German keyboard. All this by touching and swiping and pinching on the smooth glass, illuminated with virtual tokens.

In 1946, that would have been beyond the imagination of all but the most brilliant visionaries. The ENIAC was difficult to interact with, requiring thousands of cables and switches to be plugged and set. Setting up even a simple request was a lengthy process and could take days or weeks.

And it took a long time for things to slowly, gradually improve.

For many years we typed our commands into the computer and saw the results printed back to us, first on paper and later on green, glimmering cathode ray tube (CRT) displays. There were no pictures, graphics, or diagrams, only letters and numbers. (Although truly talented and bored programmers could print out convincingly recognizable images of Snoopy or Marilyn Monroe using only row after row of carefully selected characters – at least when viewed from a distance).

Then along came Apple Computer, who freed us from the tyranny of the keyboard by presenting us with the mouse and a graphic (as opposed to character) display. This was the graphical user interface (GUI) which presented a virtual desktop, containing figurative icons representing stuff you might want to do – email, word processing, drawing, and so on. By pointing and clicking with the mouse, you could fire up the email application without typing an arcane command. The human-machine interface was beginning to grow up.

But the early Macintosh had its problems. The power of personal computers in those days was very limited, and the GUI interface demanded a lot of processor and memory performance. As a result, the early GUI machines tended to be sluggish. This problem was eventually solved as manufacturers delivered increasingly more capable components. By the mid-1990s, the GUI was king.

It is hard to comprehend how things have progressed. In the 1990s, huge, multi-ton supercomputers calculated weather models and were able to predict, possibly, whether tomorrow called for sunglasses or a raincoat. Now, the smartphone in your pocket has far more power than those goliaths.

Which presents us with an opportunity.

The computer in your pocket has a million times the processing power of the old mainframes. This is mind boggling. And it gets even better – our smartphone computers are doubling in power every couple of years. What to do with all of this power?

There is a new human-machine interface on the horizon – augmented reality (AR).  Impossible even a few years ago, AR demands huge amounts of processing power. But its time has come.

According to Christopher Mims, technology writer for the Wall Street Journal, AR is “the story of the most exciting technology you’re ever likely to encounter, which could transform how we interact with computers in the 21st century.”

The key idea is that computer displays will become uncoupled from the physical desktop monitor or smartphone screen that you are used to.

Mims expounds further. “To understand AR, imagine a display that sits, not on your desk or in your hand, but in front of your eyes. Today, these displays are unwieldy, ranging from bulkier versions of safety glasses to something akin to a bicycle helmet. But many technologists believe that within five years, these displays will be able to project a virtual screen on every surface.”

Envision looking at your hand and seeing your smartphone screen – but it’s not really there, just a representation of it projected to your eyes. And as Mims says, “Imagine looking at a wall and, with a gesture, transforming it into a giant display—your entire workspace, with you wherever you go. Imagine a world without screens, save the one we bring with us.”

Mind-boggling is not enough to describe this next stage of the human-machine interface. As the song goes, “the future’s so bright, I gotta wear shades.”

Learning the ropes

Main-mast, Joseph Conrad, Mystic Seaport

Mystic Seaport Village is a living museum on the Mystic River in eastern Connecticut.  The museum memorializes our rich New England maritime history and several restored wooden sailing vessels show us how fishing and whaling and trade were accomplished in the age of sail. The town, famous for Mystic Pizza and its eponymous movie, is situated several miles north of Fisher’s Island Sound and is reached from the sea by navigating a circuitous channel and negotiating several ponderous bridges.

The first, a swing bridge, carries Amtrak trains across the river.  After contacting the bridge operator by radio, the bridge swings open when train schedules allow.  After waiting for one northbound and one southbound high speed Acela to pass, the bridge slowly swings open and we slip though. The next bridge carries busy US Route 1 through the middle of Mystic.  It opens at 40 minutes past each hour – if you arrive late, you must wait.

After negotiating both of the bridges and carefully staying within marked channels (the mud flats are treacherously shallow), we arrive at the Seaport.

The museum features a number of large sailing ships, notably the Charles W. Morgan (a whaling ship built in 1841 in New Bedford, MA), the L.A. Dunton (a fishing smack built in Essex, MA), and the Joseph Conrad (a 111 foot, square rigged training ship). All of these ships share the use of wind power, intricate sails strung from masts and yardarms, hoisted and canted by multitudinous lines.  Knowledgeable docents vividly describe life at sea, the jobs that the crew performed, and how they climbed through the ranks.

On a large capital ship of the late 18^th century, twenty or more sails hung on three masts provided power to the ship. Well over 300 lines were used to control and support the sails, and an able seaman must know all of their names and their functions. These were a combination of halyards (to haul the yards, i.e., raise the sail), sheets (to control sail angle and shape), and stays (to steady the masts). More, there were cunninghams, and vangs, and topping lifts, all used to control and refine sail shape, and hence deliver power to the ship.

In days of sail, the able seamen who mastered the complexity of their ships were the highly skilled workers of their time. They were the equivalent of today’s firemen and engineers who tend the engines of huge container ships and oil tankers.

Rising though the ranks, the sailing ship officers were educated and skilled in the arcane science of navigation.  Charts and sextants and trigonometry were used to ascertain the ship’s position and plot a course to the desired destination. Those mastering these skills were the technology wizards of their time, and handsomely paid.

After several days spent pleasantly reliving our maritime heritage, we fondly bid the Seaport adieu and head down the river.  Early, sun just risen, mist hangs on the water but begins to dissolve as we negotiate the two bridges. But upon reaching the base of the Mystic and entering Fisher’s Island Sound, we encounter a heavy fog, barely able to see the bow from the stern. Time to deploy our modern miracle, an iPhone with a marine GPS navigation app. We creep through the treacherous shoals and reefs, watching the navigational buoys loom from the fog, each on time and in position as predicted by the app. After some time, we emerge into the expanse of Block Island Sound, and the fog eases.

It becomes clear that what was of value then, and now, is knowledge. The able seamen and navigators of the sailing ships were the diesel engineers and Apple programmers of their day. Knowledge and skill must be learned, and earned, and applied to our common good. Anything we can do, collectively or individually, to motivate our children to learn, to enable their academic journey, is the highest good. We, and they, will benefit mightily.