Posts

Science fiction writers have always speculated about the future -- sometimes they are quite accurate (Jules Verne predicted live newscasts, space travel, and the Internet in the 19th century) and other times not so much (flying cars haven't taken off yet). Authors typically set their science fiction stories well into the future, so they can get in a few good years of sales before the future becomes the present and their works begin to look naïve and obsolete. Far-future settings also allow an author to solicit a greater suspension of disbelief from the reader -- sure, warp drives and holodecks sound fanciful, but who's to say what wonders may be common in 400 years? However, occasionally authors will accept the difficult challenge of setting their stories in a near-future world similar to our own, where predictions about technological advancement must be grounded in reality and "science as magic" hand waving is not tolerated. It is these stories, when written by authors with expertise in the subject matter and the willingness to conduct research, that provide invaluable inspiration to those of us in the business of turning imagination into reality.

Vernor Vinge's Rainbows End and Charles Stross's Halting State are two recent books that do an excellent job of illuminating possible future paths of software technology. Both books are written by authors who are experienced in the art of software (Vinge is a computer scientist, and Stross is a recovering programmer), and both books express a very specific vision of the next steps in software based on current technology and well understood trends. These books describe a future where ubiquitous computing has become a reality. Computers have mostly disappeared into the woodwork, keyboards and mice have given way to gestures, and users live in an information-rich augmented reality world where electronic eyewear overlays data onto everyday scenes in useful, and sometimes not so useful, ways.

Rainbows End is set in the year 2025, and is told from the perspective of a recently cured Alzheimer's survivor who must come to terms with the changes in the world around him, while unknowingly becoming involved in an international conspiracy. The common operating systems and user interfaces of today, such as Microsoft Windows, are no longer commercially viable (except as legacy interfaces for old people) and have been replaced with a product called Epiphany which ties the new technologies together. Computer hardware is sown into the fabric of clothes, and the display device of choice is a pair of electronically enhanced contact lenses. As far as the story goes, I don't think this is Vinge's best work, and I found the bits about the life-extended elderly being "retrained" for new jobs alongside high school students to be somewhat hokey. However, for idea-obsessed readers, the incredible detail given to technological speculation makes up for the weak story, and I can't resist a book whose most clever chapter has one of the best titles ever: "How-to-Survive-the-Next-Thirty-Minutes.pdf". If you want a taste of Vinge's writing with a better story line, I'd recommend the excellent Marooned in Realtime.

The story of Halting State takes place in the year 2018, and revolves around a software engineer at a company that produces massively multiplayer online games. This character finds himself unwillingly tangled in a complex web of international intrigue that is made possible by the ubiquitous computer technology portrayed in the book. While Halting State describes a similar ubiquitous computing environment as Rainbows End, the hardware speculation is less ambitious -- instead of wearable computers, mobile phones are the principle unit of personal computing, and eyeglasses provide the augmented reality overlay instead of contact lenses. Halting State is unconventionally written in the second person, as a homage to early computer adventure games. While I know at least one person who found this to be distracting, the second person prose became natural and unnoticeable for myself and others after the first chapter or two.

I found a couple of the technological predictions presented in these books to be particularly clever. In the world of Rainbows End, transparent cryptography has encouraged the development of sophisticated economic systems. The authenticity provided by digital signatures and certificates not only protects the movement of money, but allows contracts for specific goods and services to be anonymously subcontracted in real time without the involvement of lawyers. The anonymity allows arbitrary individuals to interact with the economy in various agreed-upon roles -- a concept that software engineers refer to as the "separation of interface and implementation." The efficiency makes possible rich systems of multi-level affiliations, with each level providing value in well-defined ways. These new systems raise issues of their potential for criminal abuse, which are explored in the book. In the following excerpt from Vernor Vinge's Rainbows End, Juan Orozco is subcontracting a task -- which he himself was subcontracted anonymously from a third party -- to Winston Blount, who verifies the authenticity of the payment promise by checking its digital signature which is certified by a trusted third party -- in this case, Bank of America:

"Y-you know, Dean, I may be able to help. No wait -- I don't mean by myself. I have an affiliance you might be interested in."

"Oh?"

He seemed to know what affiliance was. Juan explained Big Lizard's deal. "So there could be some real money in this." He showed him the payoff certificates, and wondered how much his recruit would see there.

Blount squinted his eyes, no doubt trying to parse the certificates into a form that Bank of America could validate. After a moment he nodded, without granting Juan numerical enlightenment. "But money isn't everything, especially in my situation."

"Well, um, I bet whoever's behind these certs would have a lot of angles. Maybe you could get a conversion to help-in-kind. I mean, to something you need."

In Halting State, the intersection of refined peer-to-peer networking protocols, massively parallel execution contexts, and the increasing computational power of hand-held devices has led to the development of the Zone, an advanced distributed computing environment which relies on processing power provided by the mobile phones of its users. The phone-based Zone nodes are implemented as virtual machine sandboxes which execute distributed code written in the "Python 3000" language. In the following excerpt from Charles Stross's Halting State, the protagonist explains how the Zone uses cryptography to protect assets in online games:

“Zone games don't run on a central server, they run on distributed-processing nodes using a shared network file system. To stop people meddling with the contents, everything is locked using a cryptographic authorization system. If you ‘own' an item in the game, what it really means is that nobody else can update its location or ownership attributes without you using your digital signature to approve the transaction -- and a co-signature signed by 
a cluster of servers all spying on each other to make sure they haven't been suborned.”

Both Vinge's and Stross's visions of software's future rely on cryptographic systems which are more ubiquitous and transparent than most of the systems in use today. While secure web transactions have been quite successful and have largely met the goal of transparently verifying the authenticity of the web server, use of certificates by individuals to prove their authenticity to others (whether via the web, email, IM, or other protocols) is extraordinarily rare except among extreme power users and certain high-security corporate environments. Bringing the full power of cryptography to ordinary people, and making it so easy that they don't realize they are using it, is a necessary prerequisite to building the next-generation Internet. Perhaps we need a revival of the cypherpunk movement of the 1990's -- a sort of Cypherpunk 2.0, if you will -- only this time with an emphasis on user interface simplicity that will finally allow cryptographic technology to be accepted by the public and commonly used to provide privacy and authenticity.

The role of writers in injecting a needed dose of imagination and creativity into engineers and entrepreneurs should be taken very seriously. I'm reminded of the story about how the science fiction writer Katherine MacLean, who predicted the role of computers in communication and music in her 1947 novelette Incommunicado, accidentally stumbled into a conference of electrical engineers and found herself quickly surrounded by Bell Telephone researchers who were inspired by her ideas to build the next generation of communications equipment. I hope that the software engineers of today find similar inspiration in the works of visionaries such as Vinge and Stross, and decide that it's time to pull up the shirt sleeves and get to work building the world of tomorrow.

I'm traveling in Arizona at the moment, and decided to get in a bit of geek tourism by swinging by the Western Design Center (wikipedia) in Mesa, Arizona. The Western Design Center was founded by Bill Mensch, one of the creators of the MOS Technology 6502 series of microprocessors found in many personal computers of the 1980's, including the Commodore 64 and the Apple II. The 6502 series is somewhat nostalgic for me, as I taught myself machine language programming on the 6502 as a kid.

As far as I know, the Western Design Center is the only surviving manufacturer of 65xx microprocessors. I sort of expected to see a fabrication plant at this address instead of a house, though. According to Google Maps, there's even a pool in the back.

When I was growing up back in the 1980's, I wrote a lot of documents on my Commodore 64 including school assignments, letters, journals, and attempts at fiction. I used several different word processors during this time, but towards the end I was mostly using a product called Word Writer 4 from Timeworks. Now that I'm older, I'd like to preserve these childhood memories by converting them into a more modern, standardized format that can be easily viewed on modern computers. (You may find this shocking, but it turns out that contemporary word processors lack the ability to import from Commodore 64 file formats.)

The first step is to recover the raw data off of the 5.25-inch floppy disks before they completely rot. Since I still have my trusty Commodore 1541 disk drive, I ordered the XA1541 cable from Vintage Computer Cables which connects the drive to a PC's parallel port. While this lets me connect my antique disk drive to the best computers that the 1990's had to offer, I've found that it's actually hard to find a computer these days that still has a parallel port. Fortunately, I was able to dig up an old Thinkpad 600 (circa 1999) that has a parallel port. Using the cbm4linux (a.k.a. OpenCBM) tools, I was able to copy the data from my old floppy disks to d64 disk image files on my PC. I was then able to use the cbmconvert program with the "-d" option, which reads the specified d64 disk image and dumps each individual file into the current directory.

Each Word Writer 4 document has a filename ending with ".seq" and contains data in a seemingly proprietary format. A small amount of reverse engineering effort reveals that the format is actually fairly close to ASCII, but various character ranges are offset. (No, it's not PETSCII, either!) Some embedded control codes provide markup information about text styles and formatting. I wrote a small C program called ww4convert that reads the Word Writer 4 file on standard input, and provides an HTML version of the document on standard output.

Now I can completely peruse my tome of old documents using my web browser. However, I must provide a word of warning about resurrecting your childhood documents: You may find out that you were really, really horrible at writing as a child, contrary to your memories of writing paragraph after paragraph of elegant prose. If only I could go back in time and knock some grammar and composition sense into the head of my former self.

Download:

• ww4convert.c

Warning: The following post may contain weak metaphors and bad analogies. If these are offensive to you or illegal in your jurisdiction, you may want to skip this post.

Like many people who grew up in the forward thinking, happy go-lucky days of the twentieth century when anything seemed possible, I sometimes wonder what happened to all the futuristic luxuries that technology was supposed to have delivered by now. Science fiction promised us flying cars, ray guns, personal robots, and space colonies. I can understand speculative fiction not living up to reality -- after all, nobody has a crystal ball. However, many of us who were involved in software during the early to mid-nineties had some very specific and realistic notions of how computing technology would progress over the coming years, including the following:

• Ubiquitous computing. Computers would fade into the woodwork all around us, and human-computer interaction would take place using natural metaphors instead of clunky terminals. In some ways we're slowly moving in that direction, as powerful computers become embedded into mobile phones and other handheld devices, but we're not there yet.
• Transparent cryptography. Cryptography would be seamlessly integrated into our day-to-day communications. Today, you can use cryptography to communicate with your bank's web site or connect to your office's network in a fairly seamless fashion, but the vision was much broader than this -- all network messaging (email, instant messaging, twitter, etc.) would be encrypted to protect the contents from prying eyes, and digitally signed to provide strong assurance of the authenticity of the sender.
• Natural language processing. By this time we should have been able to verbally interact with our ubiquitous computing infrastructure, asking our computers to "turn off the coffee pot that I accidentally left on," "check on the status of my portfolio," "open the pod bay doors," etc. At present, natural language technology still seems clunky and limited.
• Intelligent agents, micropayments, effective and low-cost telepresence, and many others.

Some of the above ideas exist today in a limited fashion, although none have lived up to the potential that we once envisioned. Some ideas require the overcoming of difficult challenges (such as the ideas related to artificial intelligence: natural language processing and intelligent agents), and others are technically solved to a large degree but have not been refined enough to be successful in consumer applications (transparent cryptography and micropayments, for example). I've been mulling over a theory about why these technologies have either failed to become reality, or haven't lived up to our expectations.

In the late twentieth century, technology went through a period of great advancement in capability which outpaced the application of these capabilities as solutions to problems. An entrepreneur who is investigating ideas for a startup company will find a great many tools available, and many unsolved problems. Using existing tools to solve existing problems can be very lucrative, and the entrepreneur will likely chose to deliver these solutions instead of developing new tools. Because there is a finite (although not fixed) amount of innovative energy in the economy, this low-hanging fruit is bound to be picked first. If it's 2004 and you're looking to start a company, which would be more likely to provide a larger and quicker return on investment -- solving the problem of how to richly communicate with friends using established and well-understood web tools (Facebook), or starting an artificial intelligence research and development laboratory that may not bear fruit for years? When the low-hanging fruit becomes scarce, more energy will be spent on the harder problems, and we will be back on track to building the world of tomorrow.

And now, it's time for the bad analogy you've been waiting for. When I think about the temporary diversion of efforts into applications instead of new technologies, I can't seem to shake from my head a graph from high school chemistry class showing enthalpy of vaporization. (Enthalpy can be very roughly defined as the heat content of a thermodynamic system.) When you add energy (heat) to water, its temperature will increase continuously in proportion to the amount of energy added -- until it reaches the boiling point (201°F at any reasonable elevation), at which time the energy is redirected towards the "enthalpy of vaporization" (also called the "heat of evaporation") and the temperature flatlines at the boiling point until vaporization is complete and the energy can once again be used towards increasing the temperature, as seen in this graph:

In a similar fashion, it seems that the technology industry has been in a sort of "enthalpy of application" where energy (effort) has been redirected towards application, and the technology "temperature" has not risen much:

Before anybody accuses me of being hostile towards Web 2.0 companies, let it be clear that I think putting effort into using existing tools to solve problems is just fine and dandy, and this is the natural order of things. (Even if my graphs seem to imply that Facebook is the ramen noodles of technology.) However, it never hurts to try to make sense of these trends. If the current economic downturn begins drying up the low-hanging fruit prematurely, it may be a good idea to begin looking into the grand challenges and long-term endeavors. I'm personally looking forward to working on some of these challenges in the years ahead.

I occasionally need to login to embedded Linux devices over serial connections. One frustrating thing about using a serial login is that programs running on the remote device don't know the window size of my local terminal emulator, so 80x24 is usually assumed even if I've resized my xterm to be much bigger (or even full screen!). Most network remote login protocols, such as ssh and telnet, use protocol features to communicate window change events transparently. Unfortunately, there is no such facility for serial logins, because there is no protocol.

To help ease the pain of this problem, I wrote a small C program called fixterm which runs on the remote machine and uses a little-known feature of some terminal emulators to query the actual window size and update the kernel's notion of the tty's window size. Unfortunately, this program doesn't automatically update the tty whenever you resize the window -- you still have to run the program each time. However, it does save a few steps compared to manually figuring out the window size in characters and typing "stty rows 24 columns 80". This program seems to work fine with xterm, Gnome Terminal, and PuTTY, but not with Konsole which does not support the window size query feature.

It is important to note that fixterm does not work with GNU Screen out of the box, even if you are running screen in a supported terminal emulator such as xterm, because screen provides its own terminal emulation layer that shields the client programs from the outer terminal. As a frequent screen user, I found this annoying enough that I created a small patch to add the required window size support. If you happen to be a screen user, you may want to use this patch by performing the following steps:

1. Download screen 4.0.2 from the GNU ftp site.
2. Download my patch.
3. tar xvfpz screen-4.0.2.tar.gz
4. cd screen-4.0.2
5. patch -p1 < ../screen-4.0.2-sizereport.patch
6. ./configure
7. make
8. make install

Here are the goods:

• fixterm.c
• screen-4.0.2-sizereport.patch - a patch for screen 4.0.2 to support window size query

Prologue

I broke down and bought a Kindle, the electronic book reader designed and sold by Amazon. I was somewhat interested in this device when it launched back in November, but Amazon sold out in five hours. Whether that's because of exceptional demand, or because Amazon only made eight, we'll never know. As a voting member of this year's World Science Fiction Convention, I recently received several books in electronic format for me to read and consider for the Hugo Award. (I tell people that I get to be a "superdelegate" for the big convention in Denver this fall.) As I spend way too many hours in front of a computer screen as it is, I figured this was as good an excuse as any to spring for an ebook reader. (Well, that, and an incurable love of gadgets.) It just so happened that Amazon had finally restocked the Kindle when I checked the web site on April 19th, so I decided to place the order. I selected the free two-day shipping, and my Kindle arrived a few days later.

I find Amazon's chosen name for this device to be a little disturbing. I'm sure they had in mind associations with "sparking an interest" or "igniting a revolution," but the first thing that came to my mind was visions of book burnings. I wonder if I should expect Fireman Montag from Fahrenheit 451 to burst into my apartment and incinerate my shelves of non-Amazon-compliant paper books. Amazon should have kept their development code name for the device, Fiona, which was inspired by Neal Stephenson's The Diamond Age.

Who would buy such a thing?

The night before my Kindle arrived, I mentioned to a friend that I had ordered one. He just happened to find out about the Kindle earlier that day while browsing Amazon's web site, and wondered who on earth would buy such a thing. Now he knows.

Whenever I move, I'm reminded of one very important physical property of books. Books are heavy. I mean, sure, when you're turning a page, paper can seem light as a feather. Let me assure you that when you accumulate enough of these pages, the weight adds up. In addition to having mass, books also occupy space. Do I really need a wall full of bookshelves? What happens when I run out of walls? This is frustrating when my entire book collection, in electronic form, could probably fit on a single DVD.

Electronic books also offer the hope for broad availability of written works. There are many out-of-print books that may not have enough demand to justify reprinting, but could easily be made available electronically. Indeed, Jeff Bezos has expressed his vision for the Kindle as "every book ever printed in any language, all available in less than 60 seconds." I think that's easier said than done, but it's a noble goal and I wish him the best.

The Kindle is not without its naysayers. Steve Jobs says that the Kindle doesn't matter because no one reads anymore:

It doesn't matter how good or bad the product is, the fact is that people don't read anymore. Forty percent of the people in the U.S. read one book or less last year. The whole conception is flawed at the top because people don't read anymore.

While it's true that many people don't have the attention span for reading books these days, I think it's still possible to make good money from the segment of the population that remains actively literate. Barnes and Noble still seems to be able to pay their light bill. Amazon is still selling books. After all, consider the words commonly attributed to Mark Twain: "The man who does not read good books has no advantage over the man who can't read them."

Electronic paper

The Kindle's "electronic paper" display is the work of E Ink, a company whose flagship product of the same name can also be found in the Sony Reader. The E Ink display is radically different than the LCDs used in PDAs and cellphones. I first saw an E Ink display in 2004 at a Sony store in Tokyo, Japan. Among the gadgets available on display at the store was an early predecessor to the Sony Reader. When I saw this device, the paper-like display was so convincing that I thought I was looking at a dummy unit with a printed sheet of paper behind the screen. When I pressed a button and the content of the paper changed before my eyes, I was absolutely floored.

The E Ink display is the number one feature of the Kindle. Reading text on this display is much easier on the eyes than staring at a lit computer screen, and provides an experience much closer to reading a book. Unlike an LCD, bright light actually makes the image more readable, just like real paper, instead of washing it out. E Ink is also extremely frugal with battery power. No power is needed to maintain the image on the display -- only when you "flip a page." Because LCD screens are usually among the largest energy consumers in devices, the Kindle and other E Ink products can last a very long time before the battery needs to be recharged.

Electronic paper technology is still in its infancy, though, and there are some important drawbacks to consider, especially if you are accustomed to using devices with traditional LCD displays. The biggest drawback is the refresh rate. While the LCD screen in your notebook may repaint its image 60 times a second to deliver smooth animation and immediate feedback to the user, the E Ink display requires a noticeable delay to change anything on the screen. (More about this in "Usability," below.)

When you flip a page while reading, the Kindle momentarily clears the screen with a sort of "black flash" that some people find annoying. These people are of the opinion that it disturbs the continuity of reading. I've noticed that in a few cases where dialog boxes do not perform this flash, it's possible to look closely and see a very faint residual image of the previous screen contents. Therefore, I assume the black flash is necessary to avoid this residual image problem. It doesn't bother me, personally. I stopped noticing it after a few minutes of use.

Besides the slow refresh time and the dreaded black flash, the drawbacks are quite minor. The plastic screen surface seems to reflect a small amount of glare from light sources. The glare is greater than the page of a book, but far less than the glare on PDAs or cellphones. Also, I wouldn't mind if the display had a bit more resolution and contrast.

The E Ink display is both the Kindle's greatest asset and its primary limitation. In spite of the downsides, E Ink does deliver an incredibly beautiful display for reading text. It's important to understand and appreciate the trade-off. If you're not in any hurry to buy an ebook reader, you can probably hold off for a couple of years and get a much improved display at a lower cost.

Usability

When the Kindle was launched in November, the reaction to its design was almost universal: This thing is ugly. These days, people expect sleek designs such as found on the iPhone, and Amazon built a device that looks like it fell through a time warp from the 80's, and wouldn't look terribly out of place in Flash Gordon or Buck Rogers. I'm willing to cut them some slack for this first release, though, and not judge a book by its cover. Fortunately, it turns out that most books don't require much interaction -- as long as "next page" works, the usability concerns are minimal.

Much of the design of the Kindle exists to work around the limitations of the E Ink display. This has led to a somewhat unconventional interface which avoids screen updates like the plague. For instance, to select menu options and text, the Kindle provides a strange secondary display in the form of a thin vertical stripe to the right of the screen. They call this a "cursor bar," and it works by activating small silver squares which correspond to a selected area of the main screen. (For the life of me, I still haven't figured out what sort of display technology this is. Silver squares that appear and disappear? Maybe this thing really did come from Flash Gordon!) A thumb wheel is provided below the cursor bar to move the cursor. As you may have guessed by now, the Kindle does not have a touch screen.

E Ink's slow refresh time is the primary limitation of the Kindle, giving the device a somewhat sluggish feel while navigating pages and interacting with features. An average page flip takes about 1.5 seconds. I've found that the extent to which this a problem varies greatly depending on what sort of reading material is being consumed. The more you need to flip through random pages, the more you will be frustrated by this slowness. When my Kindle arrived, I read several short stories on it. The Kindle excels at this sort of content, and the delay was not an issue at all. I've started reading a complex novel, and I occasionally feel the need to flip through previous pages to refresh my memory about certain minor characters. This is not a pleasant activity on the Kindle. I would expect reference and technical material such as textbooks to be most negatively impacted by the slowness. (Sorry college students, you're going to have to carry around a backpack full of books for a while longer.) It is a small consolation that you can flip several pages at once by pressing the "next page" and "prev page" buttons multiple times in rapid succession. Perhaps I should explore the Kindle's search features some day.

Because page numbers are not very useful for electronic books where the text size can be changed by the user, the Kindle uses the abstract notion of "location numbers" for referencing positions in the text. For example, I'm about a quarter of the way through a novel and "Locations 2003-20" appears at the bottom of the screen to let me know which "locations" are being displayed on the current page. In lieu of real page numbers, this is probably the best that can be done, although I don't expect to see people using locations when citing references anytime soon.

Other minor nitpicks I have with the device are the screensaver, which seems too eager to activate when I'm taking too long to read a page, and the main menu for selecting content, which shows all titles on the device with no hierarchy. I could imagine this list of titles could grow to be quite large, and be cumbersome to browse.

It is inevitable that people will compare the Kindle to more responsive and feature-rich devices such as PDAs and tablet computers. I don't think this comparison is very fair, but I understand why people value the characteristics of those sorts of devices. Today, in 2008, you can have either a snappy, rich user interface, or a beautiful paper-like screen. I hope the technology advances enough to eliminate this compromise, so users can truly have it all.

Features

You can read the full feature list of the Kindle at Amazon's web site, but a couple of features are noteworthy enough to mention here. One surprisingly useful feature that I didn't know about when ordering is the online dictionary. While I'm reading, if I come across a word that I'm not familiar with, I can select it and immediately retrieve the definition from The New Oxford American Dictionary. This has come in handy quite often.

The Kindle is unique among electronic book readers for its wireless connectivity through Sprint's EV-DO network. The service, which Amazon calls "Whispernet," is used to connect to the Amazon store and download purchased books. There are no special charges for this service, other than the cost of the content you purchase. The Kindle also uses the wireless service to provide free access to Wikipedia. An experimental web browser also allows you to access simple, text-oriented web sites, but this "Basic Web" is not an advertised feature and is not guaranteed to be available forever.

Content

The biggest worry people have with acquiring content for the Kindle is the Digital Rights Management (DRM) copy protection found on works purchased from Amazon's Kindle Store. The concern is that if Amazon decides to stop supporting the Kindle, your investment in books may go up in smoke -- especially if your Kindle stops working some day. In contrast, a traditional printed book is likely to last a lifetime, even if the publisher goes out of business. While it might seem far-fetched that a major company like Amazon would stop supporting customers' purchases, it is not without precedent. For instance, Microsoft recently announced that it is discontinuing support for purchases from the MSN Music Store, which uses the ironically named PlaysForSure DRM protection. If customers want to play their music after the cutoff date, they must keep the same computer for the rest of their lives, and never reinstall or upgrade the operating system. It's no wonder that many customers are skeptical about allowing their investment to rely upon the continued goodwill and support of the retailer.

I'm probably going to dabble a little with the Kindle Store while I decide where my comfort level is with DRM encrypted content. Using the Kindle's built-in support for browsing the Kindle Store, I picked up Vernor Vinge's The Peace War, and it was delivered wirelessly to my Kindle. The process seemed to work smoothly and as advertised.

It is important to note that the Kindle is not restricted to viewing encrypted content obtained from the Kindle Store. The Kindle can directly view plain text (.txt), HTML, and unencrypted Mobipocket files that you transfer to the device from your computer. Using conversion software (or Amazon's convert-by-email service), you can view many additional formats on the Kindle, such as RTF, Word, etc. Mobipocket, a well-established ebook provider, was acquired by Amazon in 2005 and the company's ebook format is the native format for content on the Kindle. Encrypted Mobipocket files will only work on the Kindle if you bought them from the Kindle Store. However, any unencrypted Mobipocket files you may already have should work fine.

Many people have complained about the Kindle's lack of advertised PDF support. I think these people fail to truly understand the nature of PDF documents. PDF documents are stored in a strongly page-oriented fashion, ready to be sent to the printer. Unlike HTML, RTF, or Microsoft Word files, there is no concept of paragraphs or flowed text (word wrapping). Displaying a PDF document containing 8.5"x11" or A4 pages on a smaller screen with variable font sizes is not easy without applying some intelligence to turn the fixed-position lines of text back into paragraphs so they can be word-wrapped. Turning PDF content back into flowed text that can be displayed well on an ebook reader is akin to unscrambling an egg. Despite the challenges, however, it is quite possible to view PDF content on the Kindle. I use the freely available Mobipocket Creator software to convert documents into Mobipocket format for reading on the Kindle, and it will indeed perform best-effort conversion of PDF files. The PDF content that I've converted has been very simple, and it is viewable quite well on the Kindle. Anything more than the simplest of PDF files is likely to not convert well, though, so it is strongly recommended to only use PDF source material as a last resort.

My favorite source of ebook material is the Fictionwise web site. Fictionwise is an established ebook retailer, and many of the books and periodicals they carry are unencrypted so I don't have to worry about my purchases going bad. The unencrypted content is labeled as "MultiFormat." I've already spent far more money at Fictionwise than I have at the Kindle store.

Cost

Amazon is currently selling the Kindle for $399.00, which is quite a hefty sum for most people's book budgets. At that price, you could instead buy 57 paperback books at $7 a pop, and be reading happily for quite some time. This price tag is a bit much, and it will probably deter most people from jumping into the wild world of ebooks.

Undoubtedly, the $399.00 price includes a built-in "early adopter penalty." With so much room for improvement and possible technological advances in electronic paper, I think that electronic book readers are bound to get a lot better and cheaper over the next few years. In some ways, the Kindle reminds me of the first digital camera I bought, back in 1996. It was an Apple QuickTake 150 that could take a whopping twenty or so 640x480 (0.3 megapixel) photos before running out of memory and battery, all for the low price of $700. It was brilliant -- I no longer needed to have film developed and scanned to put my pictures on the web. About nine months after I made the purchase, the digital camera market exploded with a wide selection of models that were vastly superior for half the price, and I was left with an obsolete relic. Perhaps in a year or two I'll feel the same way about this first-generation Kindle.

Epilogue

The Amazon Kindle is generally a fine device for reading books, and I've enjoyed mine quite a bit so far. No longer will I have to face the possibility of running out of reading material on long trips. The Kindle is far from perfect, however, and unless you're a gadget freak like me, you might want to hold off on buying one of these for a couple of years. The technology is still in its infancy, but it has a very promising future. If you are a gadget freak like me and you're considering buying the Kindle, well... c'mon, you know you wanna!

For various reasons, I'd like to get a summary of new mail in my IM client. Since I use a traditional mail server with procmail support, and also operate a Jabber (XMPP) IM server, I decided to whip up a small python script to accomplish this. I used a Python XMPP library called xmpppy.

For what it's worth, I use the excellent Openfire Jabber/XMPP server software from folks at Jive Software. It's dead simple to install and configure, and a refreshing change from the bad old days of hacking away at jabberd. The basic version is free and open source, and Jive has recently announced that the enterprise version (which includes advanced features like reporting and logging) will become free and open source later this month.

Here's the script and a link to the required library:

• mailjab.py
• xmpppy - this library is required to run mailjab.py

When developing database-oriented business webapps, the data model can sometimes become quite complex with many database tables referencing other database tables. To maintain referential integrity, you can (and should) declare foreign key constraints so rows cannot be deleted if other rows depend on them. However, if the application is a basic CRUD webapp that provides the user a direct, non-abstract view of the data, the user may be surprised to get a confusing constraint violation error when he or she attempts to delete a row. Such errors seldom contain useful details, so even if you wrap the error in a pretty page, the user may still be left wondering exactly why that row could not be deleted. Even if your database provides such details when a constraint is violated, an important usability question must still be asked: Why was the user given the option to delete the row in the first place, if deleting the row would result in a constraint violation?

The ideal solution would be to remove the option to delete rows that cannot be deleted, gray out the delete button, or otherwise indicate that deletion can't happen. Better yet, provide some information about why the row is important and can't be deleted yet. This requires the webapp to determine which rows can be deleted when the rows are being rendered in the web page view.

The goal is to write a routine for preemptively checking foreign key constraints. Such a routine must be very general, because we don't want to hard-code data model logic that must be maintained every time we change the schema, and prevent us from easily reusing the code in other projects. It would be great if databases supported an SQL query such as "CHECK DELETE CONSTRAINTS FROM table_name WHERE id IN (42,43,44,...)". Unfortunately, I haven't come across any such feature. Fortunately, databases do allow the schema to be examined and foreign key constraints discovered. Therefore, our problem can be handled with a simple, two-step solution: (1) At application startup time, perform foreign key constraint discovery, and (2) when delete options are being rendered, call our routine to query the deletability (deleteability?) of each row.

In Java, for instance, a schema's foreign keys can be examined in a database-neutral manner with JDBC using the DatabaseMetaData.getExportedKeys() method on each table, building a list of all foreign key relationships. (The list of tables can be queried with DatabaseMetaData.getTables().) The foreign key relationships defined with ON DELETE CASCADE must be flagged based on the presence of the importedKeyCascade flag in the DELETE_RULE.

When the webapp needs to check the deletability of displayed rows, it can call the routine with the table name and a list of id's to check. The routine can check each foreign key relationship with a SELECT to see if any foreign key constraints would be thrown if any of the specified rows were deleted. Relationships defined with ON DELETE CASCADE do not throw constraint violations for the initial delete, but the cascaded delete might violate a constraint, so the "check deletability" routine must be recursed into for each of these cascade relationships. Because the directed graph of foreign key relationships is allowed to contain circular references, the set of processed rows must be passed into each recursion and checked against to prevent infinite recursion. After processing, you should have a list of row id's that cannot be deleted, along with the reasons. Implementation of this scheme is left as an exercise for the reader. For bonus points, integrate with Hibernate so the webapp can pass a collection of persisted objects instead of row id's.

I've found that this approach results in a nice, easily reusable module that seems to work well with PostgreSQL and SQL Server. For some webapps, there may be some performance issues with executing these additional queries for every page view. However, if all the relevant criteria fields are indexed, it's probably not a big deal. Many people solve this problem by never actually deleting rows, but merely setting a "deleted" boolean flag to indicate that the rows should no longer be displayed. This can be a fine solution, except when there are business reasons for making sure the user is aware of all the outstanding dependencies.

When I decided to start this blog, I installed the Roller 4.0 weblog software. Many different blogs can run in one instance of Roller, and the URLs for the blogs are arranged as subdirectories of a master Roller URL. For instance, if you installed Roller to be /roller, then your blogs might have URLs like /roller/my_blog, /roller/potato_farming_in_pocatello, and /roller/i_like_lettuce. That's fine for many uses, but I prefer to have more concise URLs. I'd like my blog to be referenced from the root of the web site, like /my_blog. Why should I have to conform to how Roller thinks I should set up my web site?

Configuring a web server to remap incoming URLs is a simple matter, and can be easily accomplished with tools like Apache's mod_rewrite. Such remapping techniques are well known and I won't bother going into detail about it here. However, what about outgoing URLs? With the rewrite rules in place, users can easily access the blog at /my_blog, but all the links on the page point back to the ugly URLs. It's a simple matter to redirect these in the web server so that they work, but it's nasty and wasteful to be constantly sending redirect messages, and besides... what would Googlebot think of such shenanigans?

I get stubborn about these sorts of things, so I decided to roll up my sleeves and see what was going on under the hood of Roller. It turns out that Roller provides a URLStrategy interface that can be swapped out programmatically with different implementations to provide different URL behaviors. Also, to my astonishment, Roller 4.0 is hooked together with Guice -- the lightweight dependency injection system developed at Google. I haven't worked with Guice, but I have used the Spring Framework's inversion of control library for dependency injection, so I know that these systems are built to make it easy to swap out components -- just what I'm looking for!

Unfortunately, there's no facility in Roller to configure the Guice dependency injection at runtime -- as far as I can tell, the Guice configuration is hard-wired in the code. (As opposed to an XML configuration file, as is common in Spring applications.) However, there is a runtime property to select the class that does the configuring -- the so-called Guice module. This means that to provide my own URLStrategy, I must supply at least two new classes: a custom replacement for Roller's JPAWebloggerModule class which is used to configure Guice, and the replacement for the MultiWeblogURLStrategy class which currently defines the URL behavior.

I start by configuring Roller to use my custom Guice module, by adding this line to my roller-custom.properties file:

guice.backend.module=com.davidsimmons.roller.CustomWebloggerModule

This property will configure Roller to use my CustomWebloggerModule class, which configures Guice with all the same bindings as Roller's own JPAWebloggerModule class does, except it binds my CustomURLStrategy class to the URLStrategy interface instead of the default MultiWeblogURLStrategy implementation. My CustomURLStrategy extends MultiWeblogURLStrategy, so it is almost the same, except that it knows about special weblogs that I want to be referenced from the root of the web site. For these weblogs, my custom class post-processes the URL to remove the first component of the URL path. Voilà! All my links now reflect the friendly version of the URL.

There is one gotcha -- when I reparented the blog, important cookies stopped working because they were tied to the /roller path. To solve this for the JSESSIONID, I configured my Tomcat servlet container to use an empty path by including emptySessionPath="true" in the Connector attributes. There are a few more path-dependent cookies that Roller uses that may come back to bite me... we'll see.

The sample code is available here: simmons-customurl.tar.bz2

Greetings, Blogosphere!

I've finally found the time to set up a blog where I can yap about technical topics that might be of interest to other people working in the wacky, wild field of software development. I actually started to do something similar back in 1997, before the word "blog" had been invented, but I ended up getting distracted by real life and abandoning it. Hopefully I'll do better this time.

I'll briefly describe my background in software. I doubt anybody is going to actually read this, but I need some content to test my blog setup, and this is as good as any.

I started programming when I was a kid in the 80's, mostly on the Commodore 64 using its BASIC interpreter, and later its 6502 machine language. I started using UNIX in 1991 -- my high school had terminal connections to a VAX running BSD4.3 at a nearby university, and it was a great opportunity to learn about UNIX and the Internet.

In early 1992, I downloaded a copy of Linux 0.12, a UNIX system for PC's which a fellow in Finland had just started working on a few months earlier. Linux was quite different in those days -- there were no "distributions" with "installers" to assist with getting it on your computer. Part of the installation process actually involved using a hex editor on the boot floppy to hard-code the major and minor device numbers of your hard drive's root partition. Nevertheless, it was exciting to see all the great things that talented hobbyists were able to do with the Intel 80386 microprocessor -- things that the established software vendors were dragging their heels on.

In the early to mid 90's, I started writing web applications using C and C++, and then eventually Perl when it became viable for such things. Most of the rest of the 90's was spent working on web application development, some UNIX system administration, and consulting. After 2000, my career turned to embedded system development, which is a large part of what I've been doing since. I do still develop web applications from time to time, and it's interesting to switch back and forth between the world of the very small -- platform work, graphics, and device drivers -- and the world of the very large -- databases, clustering, and managed environments. Today, my primary focus is software solutions for television hardware such as set-top boxes.

With any luck, I'll have something more substantial to write about next time.