Thunderbird and Vi Shortcuts

THE VI text editor is found on every Unix/Linux system, and it's also been ported to MS Windows. I used to have a manager who pronounced it like the verb vie. He would say, "Can't you just edit that with vie?", and we engineers would have our little chuckle. Most users call it by the individual letters: "V.I.". However, to be fair, it is just an abbreviation of visual, so vie is actually more logical.

A visual editor — what's that? Well in the olden days, the first text editors were line editors, and were based on terminals that printed on a continuous feed of paper. You worked on one line at a time. Want to edit another line? Well you type a command to switch to line 297 or whatever and start editing that one. You can experience that today by using ed or ex in Unix/Linux. (As primitive as that sounds, this was considered a huge improvement over punch cards!)  When CRT terminals became common, the visual editor was developed. It presented many lines at once (usually 24) on the screen and you could move about with keyboard commands.

Vi was an early visual editor for Unix and became a standard. It was later expanded into the Vim project (VI iMproved), which added a giant load of new features and conveniences, including color syntax highlighting, history mechanism, and improved pattern matching.

Using vi requires you to switch back and forth from command mode and insert mode. You are normally in command mode, where you type one or more characters to move the cursor or do some other task. If you type one of the insert or add commands, you are then in insert mode and anything you type becomes part of the file you are editing. Press Escape to exit insert mode. This is quite different from other text editors (like emacs) and word processing apps (like MS Word), where you are always in insert mode but you use modifier keys (Shift, Alt, Ctrl, etc.) or function keys or the mouse to execute commands.

So one of the interesting features of vi is its keyboard shortcuts for moving around the screen — j moves the cursor down one line, k moves it up, h one character to the left, l to the right. If you use the touch-typing method, you will quickly notice that those keys sit under your right fingers. (Yes, the l key moves right, not left, but the l key is on the right :-/  )

***

It turns out that Gmail's Custom Keyboard Shortcuts Lab has k and j built in as defaults for moving up and down in the message list — obviously borrowed from vi. Add in the the Auto-advance Lab and you have a very snappy webmail app with vi keyboard shortcuts.

What about Thunderbird? How to make it behave à la vi? (Not to be confused with, or prounced like, c'est la vie.) A year or two ago I looked for a solution — tried Muttator and Nostalgy add-ons. Nothing seemed to work very well.

But I was missing a lesser-known add-on called Keyconfig. You can download keyconfig.xpi from mozilla.dorando.at and install it into TB (Tools | Add-ons and then click the dropdown arrow next to Search box and choose Install Add-On from File).

After restarting TB, you will see Keyconfig in the Tools menu. You can play with that to get vi-like shortcuts. But here's a quicker way, suggested by Teleogistic. Add these lines to user.js in your TB profile directory (create user.js if not present):

user_pref("keyconfig.main.key_killThread", "][I][");

user_pref("keyconfig.main.key_markJunk", "meta shift][J][");

user_pref("keyconfig.main.key_markReadByDate", "meta shift][D][");

user_pref("keyconfig.main.key_markThreadAsRead", "meta shift][R][");

user_pref("keyconfig.main.key_newMessage2", "meta shift][N][");

user_pref("keyconfig.main.key_newMessage", "meta shift][M][");

user_pref("keyconfig.main.key_nextMsg", "][J][");

user_pref("keyconfig.main.key_previousMsg", "][K][");

user_pref("keyconfig.main.key_delete", "][D][");

user_pref("keyconfig.main.key_replyall", "shift][R][");

user_pref("keyconfig.main.key_reply", "][R][");

user_pref("keyconfig.main.key_toggleMessagePane", "][V][");

user_pref("keyconfig.main.xxx_key74_SwitchPaneFocus(event);", "][D][");

Then restart TB.

There's a whole boatload of key_ functions to play with.

One thing I noticed if you're using j and k with Gmail as the IMAP server — if you go too fast, Gmail will drop the current connection and you'll see a popup from TB asking for a new password. Don't be concerned, your password hasn't changed. Just press Escape to ditch the popup and continue as before.

But wait there's more! Another annoyance of TB is how it advances (or not) in the message list when you delete a message. If you sort with latest on top (as in Gmail), and you delete a message, TB opens the next older message, which you probably just read!

Well, sometimes ya get lucky. I finally found out there's a hidden setting to fix that. Go to Preferences | Advanced | Config Editor. Type "delete" into the Filter box. You will see a key called

mail.delete_matches_sort_order

Double-click on that so it turns bold and becomes user-set and = True. No need to restart TB.

Now it will open the next newer message after delete. 

I use both Gmail and TB (with IMAP connected to my Gmail account). Gmail can be faster for just checking mail. It is faster for searching. It's integrated into your Gmail Contacts. But TB has more predictable formatting for composing, and it's easier to insert images.

The Technology Bandwagon

SO, ON A single day, last May, I graduated from:

• CRT low-def TV to 32" flat-panel HDTV,
• an old VCR to DVR,
• 3Mbps/400Kbps AT&T DSL to 20Mbps/6Mbps Comcast cable Internet.

And from wired analog landline to — no landline at all.

After years of wired phones and answering machines, I got on the bandwagon.

I already had a cell phone, and so did my wife.

But I wanted an extra phone number to give out to doctors and insurance companies, etc., so they could leave voicemail. So I got a Google Voice number — it has voicemail (via email, which includes audio MP3 attachment and attempted text transcription), and I configured it to not forward inbound calls to anything. That let's me check the voice mail via email or web browser.

And then I installed the Google Voice app on my Android smartphone. Now I can check voicemail instantly without dialing the cell-phone provider.

Watching TV has changed completely in just a decade. We've gone from a 36 or 50 channels to hundreds. In the olden days you watched whatever was on, including commercials, but only when it was broadcast. Then came the VCR and now you could make a low-quality recording of whatever was on, and watch it later, fast-forwarding through ads. With DVR, by pressing one button, you can make a perfect recording of whatever you want (and it's smart enough to know when to stop recording). Even better, you can "pause" live TV, rewind it, play it in slow motion, and resume.

And you can view and record at the same time. A friend of mine will start recording a football game, then go out and work in his yard. At the beginning of the 4th quarter, he returns and goes back to the beginning of the game, while the DVR continues to record the live game. He watches play after play, skipping through huddles and ads and other delays. After 15 minutes, he's watched every play and is at the beginning of the 4th quarter. Since the DVR has meanwhile been recording that, he can skip his way through most of the 4th quarter as well. Now there's 3 minutes left and he watches that live.

The history of Internet connection speeds is well known. Mostly it's the app usage that leads and the connection technology catches up later. For example, a new website or voice app or video stream may work best at 6Mbps but you can only get 2Mbps wherever you live. (Sometimes it's the reverse.) Gmail, for example, uses about 1MB of traffic just to load its main page (after login). That takes a second or so at 20Mbps (a typical cable-Internet speed). But if Gmail had used that much data 10 years ago, when most people had a 56Kbps dial-up modem, you'd have waited about 3 minutes. However, webmail in those days (pre-Gmail) consisted of very simple HTML — no JavaScript and no AJAX. And all the goodies you now see on a Gmail page were simply not provided. (Today, Gmail does have a "Basic HTML" mode for slow connections.)

So what's not to like?

Well, consider that nice flat-panel TV with its 16:9 aspect ratio (much nicer to watch than old-school 4:3). But many movies since 1955 were filmed in wider ratios than 16:9 (which equals 1.78:1) — like Star Wars (2.2:1), Mr. Holland's Opus (2.35:1), or The Da Vinci Code (2.35:1). And yet the average HD cable channel shows them in 16:9. Which means the sides or tops (or both) have been cropped. Which takes us back to the olden days when 16:9 movies were shown on 4:3 screens.

Why can't we have a law that says all movies must be provided in the full original frame, even if that means no-content areas on top and bottom?

And consider phones and their voice quality. In the 1980s, AT&T and other wireline carriers made a big deal about how fast and reliable their connections were and how crystal-clear the sound quality was. They were right. Quality was excellent for a phone line and for average consumer phones. Now, with cell phones, the standard has dropped back to 19th century levels.

As for the Internet, the speed and quality of a cable-modem connection is excellent — but it's expensive.

Still, it's all a huge improvement and revolutionary change in just 10 years.

What will it be like in 2021?

I predict that a free or low-cost WiMax or other wide-area wireless Internet will change things even more:

• The cell phone, TV, and Internet providers will morph into competing WiMax companies.
• Mobile and fixed-location phones will use this ubiquitous WiMax Internet for voice calls (and data), instead of cellular technology. Quality and connections will be much better.
• Cable will disappear as TVs get all their content from WiMax. There will be a wider market and more distribution channels for on-demand movies — via WiMax — so we can watch more of what we want (including true aspect ratio).
• There will be multiple WiMax networks in every city, so the current wire monopoly will be a thing of the past. With true competition, connecting to the Internet will cost a lot less.

And when all that happens, people will still complain — about something. It's always something.

Who Needs the Internet?

A recent survey released by the US Commerce Department gave some startling stats.

For those of us who spend all of our waking lives directly connected to the Internet, it's hard to believe that — get this — "30 percent of all persons do not use the Internet anywhere".

Wow.

And one of the main reasons offered for steering clear of the Internet?

"I don't need it."

They don't need the Internet. They use the phone, they have TV and radio. They send letters via the post office. You know, all that mid-20th-century technology.

***

Three decades ago, when 99.999 percent of all persons did not use the Internet anywhere, a few scientists needed to send computer files across the country.

So the Internet's first app (not called an app) was file transfer, conducted in the protocol of FTP.

And folks wanted to send messages across the country without using postal letters or telegrams.

Thus came email, which used (and still uses) SMTP protocol.

Large companies and universities were the first to be directly connected to the Internet, and it was no trouble for them to use FTP and email.

And those offices and groups who couldn't afford a hard-wired Internet connection used a point-to-point protocol for file transfer and email, called uucp (Unix-to-Unix copy). This worked over ordinary phone lines. It was possible to send email and files without ever using the Internet.

Along came more interactive data transfer protocols, like HTTP, which made the Web — the third great Internet app — possible. And HTTP would only work on the Internet proper.

But until the mid-1990s, few individuals had a true Internet connection.

Then began the age of the ISP, and you could connect directly — for an hour or two — from home, via a telephone line (ie, "dialup"). Later this was improved to an always-on hookup via DSL, fiber, TV cable, and WiFi.

Why this walk down Internet's Memory Lane?

I forgot.

Oh yeah.

Like those scientists in the 1970s, we are still in the daily business of sending messages and files via the Internet.

That is still the essence of all Internet traffic, including the Web.

"I want to tell you something."

That is the need.

As humans, we've always had that need.

Only now billons of people can do it easily via the Internet.

***

In bygone ages, before the Internet, before telephones, before the telegraph, before the railroad, delivering your message to a distant recipient meant only one thing — paying someone to get it there for you. You wrote it down, put it in an envelope. Maybe you tossed in some business papers or drawings. And you handed it off to a personal courier (if you had that kind of money) or to the post office. All that took a half-hour to get ready.

Your envelope was then carried in a saddlebag on the side of horse, in a sack on the floor of a carriage, or in a bag on a runner's back.

Hours, days, or months later, your envelope would arrive. (Assuming it wasn't lost or destroyed en route.)

Today, with email, the speed has improved somewhat. When my brother sends me a message, he spends 3 minutes writing it. He then adds files and photos as attachments and clicks Send.

It goes from his computer in Santa Barbara, to Yahoo Mail in Mountain View, to my mail server in Florida. On my PC in Northern California, the Thunderbird email program gets the new message (through the IMAP IDLE connection) and I see a pop-up notifier. Amount of time for the "envelope" to travel? Less than a second.

***

So, we are only doing the same thing as our ancestors did long ago.

Only faster.

And because it's faster, more of us are doing it.

A few hundred years ago, most people couldn't even read or write. They had no interest, or even an idea, of sending a message to a distant recipient. They only spoke to their family and friends within a radius of a few miles.

Now 70 percent of us are sending billions of messages per day all over the world (and getting tons of info from the Web).

Is that good? I think so. More communication means more agreement and less contempt. It means more acceptance and less anger.

And the 30 percent who do not use the Internet anywhere? They are satisfied with some slightly older methods. And it's likely they will use email or surf the web someday, as that is obviously the trend.

But then again, there are plenty of people who would rather not get 500 emails and spams each day, or look at countless websites.

At times, I envy those 30 percent. Having less information can be rather serene.

Using IMAP Email -- and Backing It Up!

Some (many?) email users still get their messages downloaded via POP3 protocol into Outlook Express or other such client. This usually means the messages are then deleted from the server. Now the only place to view those messages is on one computer. Want to read messages later, when you're somewhere else? Forget it. Your ISP-provided webmail shows an empty Inbox.

Yes, you can tell OE to keep the messages on the server. Now you have the same messages in two places. But if you delete 10 of them in OE, you will find those same 10 are still on the server. The next time you download, your email program may be smart enough to figure out that you deleted those 10 and delete them on the server.

But now you're away from home and you go to your webmail. Hey, those 10 "deleted" messages are still on the server! That's annoying! Especially if they're spam.

A lot of folks these days just use webmail only. That solves the messages-here-but-not-there problem. But it also means you can't enjoy the benefits of a real email program — speed, message sorting, and better configuration options.

You can have it all — with IMAP. By finding a provider with IMAP protocol (used instead of POP3), you can:

• have all your messages on the server in one and only one place,
• use online folders,
• use a real email program on one or more computers,
• access your email quickly on a smartphone, and
• use webmail.

There are some good IMAP providers out there. I have used Tuffmail for many years. It's fast and reliable. Great support via email. Superb server-based anti-spam. Cost varies depending on the the size of your mailstore (ie, your entire bag of messages on the server). And everything about delivery and sending is extremely configurable — but some advanced knowledge of email is required to make those optional adjustments.

Other IMAP providers are: AOL/AIM mail (free, fast, unlimited storage, image ads on the webmail page). Easy to use and lots of pretty themes.

And Google Mail (free, 7+GB storage, very fast, text ads on the webmail page). A rather different email interface, with threaded views (no way to have views by date, etc.).

Also check out Fastmail and GMX So far, Yahoo Mail and Windows Live Mail do not have an IMAP option.

Do a web search to find out how to set up OE, Thunderbird, and other apps to connect to an IMAP server.

BTW, your outgoing email will still be handled by your ISP, so there's usually no need to change the SMTP settings. However, all of the above providers have SMTP service available.

***

Now the question comes up: "I've been using POP3 all these years and I've got my all my email in local folders. How can I get them into IMAP?"

The simplest way is to simply create the same folders in your server-based IMAP account. Then, using your local email program, copy the messages from each local folder to the corresponding IMAP folder. You can use Shift + left-click to drag-copy the messages. But it's easier and safer to use one of the menu commands, like Message | Copy ... in Thunderbird. If you are on an asymmetric DSL or cable connection, where the upload speed is a lot slower than download, then the copying will take a while.

There are some other options, like using IMAPSize (Windows only, freeware) to copy an entire local mailstore to an IMAP server.

***

Okay, so you've been using IMAP for a year and have 200 or 300 MB of messages. You would like to back those up somewhere, because, who knows, you (or a hacker) could somehow delete all your messages.   Or there is a disaster or mistake and your IMAP provider loses everything.

You can't just say:

zip aim.bk.zip imap://mail.aim.com/myusername

In an IMAP mailstore, each message consists of the actual message text and a bunch of metadata, such as sender, unique ID, date received, size, IMAP flags (the little flag or star that you can click in your email program to mark a message), and keywords (important, personal, etc.). It's a complex blob of stuff to download (and later upload).

One fairly easy way to do a backup is with the aforementioned IMAPSize. This excellent app can be used to download a whole mailstore to your computer, where each message will be in its own file, organized by your folder names. The metadata is captured in the file name and in a small data file. IMAPSize will also upload those messages to an IMAP server and restore the metadata.

What if you're on Linux?   You can have some fun with these two apps:

• Courier IMAP server installed on your PC
• imapsync

Install Courier using Synaptic or other such means. You should install courier-imap-ssl and courier-ssl as well. There are other IMAP servers to choose from, such as UWimap, dovecot, and Cyrus. But Courier is fairly simple (compared to Cyrus) and has the nice benefit of allowing a folder to contain both messages and folders (unlike UW and dovecot).

Then create a local mailstore directory (maildir) for yourself with this command (as you, not root):

cd

maildirmake Maildir

Set up an account in your email program (OE, TB, etc.) to access this new (but currently empty) local mailstore — use your Unix username and "localhost" for the IMAP server name.

Now download the latest imapsync tarball. In it you will find the excellent imapsync command, which is a perl script by Gilles Lamiral that copies an IMAP mailstore (or individual folders) to another IMAP server. You can just use the imapsync command without running make install. The command has many options, so build up a script file to hold them, such as:

perl imapsync \

—authmech1 LOGIN \

—authmech2 LOGIN \

—noauthmd5    \

—host1 imap.aim.com \

—user1 joe@aim.com \

—password1 mypassword \

—host2 localhost \

—user2 joe \

—password2 mylocalpw \

—ssl1 \

—syncinternaldates \

—subscribe \

—exclude 'Spam|Trash|Drafts' \

—expunge

The "1" settings refer to the source server. The "2" settings are the destination. Save this as imapbkup.sh. Then run:

sh imapbkup.sh

(You could make imapsync and/or the script executable with chmod, but having to use sh to run it means you can't do accidentally).

If all goes well, the messages from your Internet IMAP server will be quickly downloaded into your local mailstore. Yay!

imapsync is pretty smart, it won't download the same message again. So you can restart it and only get new messages. This means you can run it every few weeks or months and have an fast backup. But it won't delete any messages (at least not with the above imapsync options), so if you delete messages on the Internet server, they will still be in your local mailstore, if even after you re-run imapsync. That's usually not a problem.

***

Now the worst happens and your Internet IMAP server has one of those "should never happen" crashes. You can immediately access your backup in your local folders.

To upload the backup to a new Internet IMAP mailstore, just reverse the "1" and "2" settings above to produce an uploading copy:

perl imapsync \

—authmech1 LOGIN \

—authmech2 LOGIN \

—noauthmd5    \

—host2 imap.aim.com \

—user2 joe@aim.com \

—password2 mypassword \

—host1 localhost \

—user1 joe \

—password1 mylocalpw \

—ssl2 \

—syncinternaldates \

—subscribe \

—exclude 'Spam|Trash|Drafts' \

—expunge

Yet another option is to copy one Internet IMAP mailstore to another Internet IMAP mailstore. You may have one of those web-hosting accounts where you get 2,000 IMAP/POP3 accounts included. Set up one of those as your backup IMAP mailstore. Then use imapsync thusly:

perl imapsync \

—authmech1 LOGIN \

—authmech2 LOGIN \

—noauthmd5    \

—host1 imap.aim.com \

—user1 joe@aim.com \

—password1 mypassword \

—host2 imap.megahost.com \

—user2 joe@megahost.com \

—password2 mymegapw \

—ssl1 \

—ssl2 \

—syncinternaldates \

—subscribe \

—exclude 'Spam|Trash|Drafts' \

—expunge

This is the slowest of all, since it's downloading to a local temporary file, then uploading to another server. But the next time you run it, it will only get new messages and go faster.

What Happened to My vi?

If you are using Ubuntu Linux, you might be wondering why vi (er, vim) doesn't save searches and other things into a history file, or why it ignores your syntax colors (which show different colors for different elements of C or bash code, for example).

Yes, you moved your .vim* files from another box. Yes, it's vim version 7.2, not 5.7.

Dang thing don't work.

Then you run

vim --version

and discover that the viminfo (history, etc.) and syntax options are marked with a minus sign! Oh, so you have to recompile your own vi from source code?

Not a bit of it!

Look at your current version of vim using the dpkg-list command:

dpkg --list 'vim*'

un vim 2:7.2.079-1ubu Vi IMproved - enhanced vi editor

ii vim-tiny 2:7.2.079-1ubuntu5 Vi IMproved - enhanced vi editor - compact version

You will see that the standard version is not installed (the n in the second column). The Ubuntu folks decided that you wanted to start out with the "tiny" version (i = installed).

Hah!

Well, not a problem.

Use Synaptic or this shell command to install a bigger vim and its doc files (as root):

apt-get install vim vim-doc

After the package install, you will find in /usr/bin a command named vim.basic, which, as you can well imagine (and see with ls -l), is bigger than vim.tiny. The install also resets the /usr/bin/{vi,vim} symlinks to correct links in /etc/alternatives, so next time you type vi or vim, you get vim.basic and not vim.tiny.

If you want even bigger, then you can install vim-gnome (previously called vim-full) — this gets you a GUI version with a scrollbar, etc.

3 Cheers for 9.04!

Did a very smooth upgrade from Ubuntu Intrepid Ibex (8.10) to Jaunty Jackalope (9.04). So far, it's definitely better and has given no trouble at all.

• Cheer #1 — Sound is fixed! Yay! In 8.10, no matter how you configured sound, you might get it working, sort of, and then after 48 hours — no sound. Adobe Flash sound stopped. Realplayer sound stopped. Every other sound, including the Ubuntu login sound effects, stopped. And the Gnome Sound Recorder wouldn't work without a lot of re-settings (each time). The fix? You had to reboot (not just re-login). Now it all works, and works, and works, like the Energizer bunny.
• Cheer #2 — OpenOffice v3! (Actually 3.01.) I tried upgrading to v3 under 8.10, but nothing worked out. OOv2 with updates opened MS Word sort-of-XML .docx files, but OOv3 does a better job and starts faster. I'm not tied to OO for everything. Sometimes I use Gnumeric for spreadsheet, and Abiword for formatted word processing.
• Cheer #3 — Notifications! Instead of yellow popups with X-close widgets, we now have fade-in-out black-background popups. Seems like Pidgin IM, volume/mute and networking have out-of-the-box support for the new system. But for Thunderbird, see below.

Now that we're done upgrading, let's look at some details.

Sound

These days, streaming sound is delivered mainly via Adobe Flash and Realplayer. Ubuntu's PulseAudio system manages these inputs. You should be using PulseAudio, not anything older. The Jaunty upgrade seems to have forced this, which is good.

The only Adobe Flash package needed is adobe-flashplugin. In /usr/lib/mozilla/plugins, you should have a libflashplayer.so of some kind (actual file or symlink to the real file).

For Realplayer, you have to download v11 from real.com and follow the instructions. Then check again in /usr/lib/mozilla/plugins to see if you have something that looks like this:

nphelix.so -> /opt/real/RealPlayer/mozilla/nphelix.so

nphelix.xpt -> /opt/real/RealPlayer/mozilla/nphelix.xpt

That should theoretically work, but it's not enough. User adbs over in Ubuntu Forums gives these further instructions:

1. In the Realplayer go to Settings->Hardware. Under Audio Driver select OSS.
2. Edit /usr/bin/realplay (a shell script). Insert "padsp" before the $HELIX_LIBS/realplay.bin "$@" command:

padsp $HELIX_LIBS/realplay.bin "$@"

padsp is a wrapper for PulseAudio and this little mod to the realplay startup script makes it actually work! Thanks, adbs!

Here are some streaming-sound URLs that I like:

• music.aol.com — A boatload of free and excellent music channels. Scroll down about half-way and click Listen to AOL Radio Now. No login needed.
• www.democracynow.org — Excellent 1-hour daily newscast (sound and video).
• www.bbc.co.uk/worldservice/emp/pop.shtml?p=/worldservice/meta/tx/flash/live/eneuk.xml&l=en&t=audio — BBC World Service News 7x24
• www.bbc.co.uk/fivelive/live/surestream.ram — BBC 5-Live UK radio station, great hosts and interviews, no commercials.

If you can hear the sound from the all of the above, you're in good shape with Ubuntu sound.

And here's a detailed explanation of all the Linux sound stuff from markbuntu at Ubuntu Forums.

Notification

Some (many?) apps don't use the new Notify-OSD notification system. This is especially noticeable with Thunderbird. There are currently 2 TB addon's available for this:

Mozilla Notification Extensions 0.1.1

and

Ubuntu Notify 1.3

I could not get the first one to do anything. The second one works fine. Perhaps in the (very) long-awaited Thunderbird v3, this will be built in.

9.04 isn't an exciting all-new version of Ubuntu. But when you think about it, we didn't really want that, did we?

#! Crunch Bang !

A friend's Dell PC had got the Blue Screen of Death last month, and he called me up.

No big deal I thought, just boot up the console from the XP CD and run a few commands.

But this bootup error — some arcane stuff about a BAD_POOL_CALLER — I had never seen before. And nothing I tried would fix the problem.

Of course my friend did not have any recovery CDs for his 2003-era PC. Or if he did, he didn't know where they were. And if he had spent 3 hours looking through his garage, he might have said, "Here they are!" And handed me Discs 2 and 5 of a 6-disk set from a Gateway Windows 98 machine (includes Microsoft Works '98 and Microsoft Greeting Card Maker '97).

So I could have used my non-Dell XP CD to do a "re-install on top", where most of the windows folder is rewritten but personal files are left alone. But I knew that a half-dozen Dell-specific hardware drivers would not be on that CD and I would have to go hunt them down on Dell's site.

Now we'd be up to about 6 hours of recovery attempt. And not a fun recovery attempt.

It would be at this point that we discover whether the BAD_POOL_CALLER was vanquished. If not, then we go back to square zero.

But let's say the PC now booted into XP. Are we done? Not on your tintype! My XP CD is from the SP1 days. So I would then be downloading and running SP3 — about, oh, 3 hours minimum. But not 3 hours of being somewhere else. No, I have to watch the update happen (very boring), in case I have to click the "Accept" button, which I will, on multiple update components. In so clicking, I would indicate, on behalf of my friend the actual owner, agreement to Terms of Use that I certainly do not — on this day or any other — have time to read. (And certainly have no interest in.)

It would be even worse with a laptop, which always has strange device drivers that are even harder to find after the 90-day warranty period has expired.

Well let's say we got this far. Then 100% of my friend's application settings have been lost, because the "reinstall-on-top" has reset the registry's hive files to factory fresh. So some apps work, some don't. And of course my friend has no record of any license keys. So let's spend a few more hours (for application CD hunting and re-installing).

***

As you can imagine, I stopped this process after Paragraph 3 above. I said, "You can go buy a new PC that is 100 times better than this one, for about $299." Or I can install Linux. Since he had just paid a bunch of bills, he chose the Linux option.

I was thinking of Ubuntu, which I use for my own desktop. But I wanted something that required less initial configuration — to minimize the time spent on the project.

I came across a derivative of Ubuntu 8.10, called #! CrunchBang Linux. I downloaded the Lite version (450MB) and burned the CD using Brasero (on Windows, you can use the free BurnCDCC utility from TeraByte Unlimited).

The #! thing? That's the first line of a shell script under Unix. It tells the operating system what program to use to interpret the rest of the file. The most common uses are:

#!/bin/bash

and

#!/usr/bin/perl

Why #!?? It's just an unusual sequence of characters that is unlikely to be part of normal typing in any language. I had not heard of "crunchbang" before. It's more commonly called "pound-bang" or "shebang". The ! has been called "bang" for many years, especially when saying the old UUCP-style email addresses, which often contained multiple !'s:

blorkmail!ucbvax!ukcloudvax!bananapdp8!jackmail!bob

Try saying that with the words "exclamation point".

Anyway, CrunchBang Linux was easy to install. First I booted the fubar'd PC with a R.I.P. Linux rescue CD, mounted the NTFS file system, and copied off my friend's My Documents folder to an external drive. (This could've been done, perhaps even more easily, with DamnSmallLinux or other bootable desktop LiveCDs.) Then I ran CB's installer, same as Ubuntu's. It wiped out my friends NTFS partitions and formatted them with ext3 and swap. Avast ye BAD_POOL CALLER!

Once cool thing about CB Linux is that it comes with Firefox v3 and already has Flash installed. It's ready to use.

After booting, I played with the OpenBox window manager features (CB uses OpenBox instead of Gnome's regular Metacity WM). Set up some nice colors and a floral background. Then added OpenOffice word processor and spreadsheet with Synaptic.

That whole bit took about 2 hours, with an hour of actually being there and doing something. And it was fun!

***

The foregoing is not an anti-Microsoft rant. It's more of a "When Windows Goes Bad" story. And when Windows does go bad, it's sometimes very difficult to make it good.

When Linux goes bad, well, you can easily recover your files and re-install the OS. But let's face it, when has that ever happened with Linux?

mogrify That Image!

When MIME standards were established in ca. 1996, it wasn't long before most email programs could send various kinds of attachments, such as documents and photos. (And soon, troublemakers, spammers, and criminals started sending infected executables in this manner.)

But something else happened in the years since then: digital cameras and increasing image file sizes!

Your friend snaps a dozen 2MB photos while on vacation. She gets home and wants to share the images with everyone. She attaches the images to an email message.

Every recipient suffers. You suffer.

If you're using POP3 (poor you), then you wait for the 30+MB download into your email program. With IMAP, and with some webmails based on IMAP, you might do the same waiting game every time you click on that message.

Now the message is downloaded. But wait there's more. Your computer has to spend precious seconds (or longer) and oodles of memory decoding the attachments, then decoding the huge JPEGs, and then rendering them into images on your screen. Even then, the images may be too large for your screen and you'll have to scroll vertically and horizontally.

Notice I said 30+MB, which is > 12×2MB? That's another penalty of sending attachments. If the attached file is anything other than simple text, it has to be encoded into something that resembles simple text (ie, 7-bit ASCII and ordinary line lengths). Then it can be guaranteed transport through the Internet and viewing with any email program.

That encoding is called Base64. It's an ingenious method that converts successive sequences of 3 8-bit bytes (each byte can have any of 256 values) to 4 6-bit characters (each having any of 64 values). The 64 values are then mapped onto the set [A-Za-z0-9+/=], all of which are ordinary text characters. Consider these three randomly chosen bytes from the middle of a JPEG file (show in octal):

265 356 251

Here are the same bytes in binary (3 8-bit bytes):

10110101 11101110 10101001

Now re-grouped into 6-bit bytes:

101101 011110 111010 101001

Back to octal:

55 36 72 51

And then decimal:

45 30 58 41

And mapped to the Base64 "alphabet":

te6p

Each 6-bit char still needs an 8-bit byte for transport, since computer files are based on 8-bit bytes (the top 2 bits are set to zero). So you have the same 24 bits of original information, but the number of 8-bit bytes has grown from 3 to 4.

Thus the 24MB of photos has ballooned to 30+.

Note: this encoding is not the same as compression (it rather seems to be the opposite of compression :-) ). The original bitmap image from an 8MP camera has 3-bytes per pixel (representing brightness values of red, green, and blue). So that's 24MB of camera memory. Most cheaper cameras, like mine, use "lossy" JPEG compression, with a high-compression quality setting, inside the camera and that's what gets fed to the computer. (With other cameras and supplied software, you can access the bitmap images, usually in TIFF format.) So JPEG files are already compressed (and thus zipping them into a single file will not reduce the overall size).

The problem with this whole scenario is not Base64 or JPEG or camera technology. It's the fact that you didn't need an 8-megapixel image just to see your friend standing under a palm tree. The huge-MP formats are great for printing and editing, but not necessary for showing a picture in email. And there's no obvious way to properly reduce a large dense image to something much smaller.

It's arguable, but a good size for an emailed image might be 400x300 pixels (for 4:3 aspect ratio) or 400x225 (for 16:9). With these smaller dimensions and a reasonable JPEG quality setting, such an image might only be 20KB in file size — 100 times smaller than 2MB! Now those dozen images transport in about 300KB — much better! So the trick is to reduce the dimensions and quality setting.

Here's how to do it in Windows, if you right-click on a JPG file (or group of them) and choose Send To Mail Recipient, a dialog box pops up and asked if you want to use a reduced size, and gives you some options to choose from. It will make the reductions (to temporary copies, not the originals) and attach them to a new message in your default email program. If you need to upload the reduced files to a webmail app, then you can drag the reduced images from the email program's attachment area to the desktop (then cancel the new email message).

Unfortunately, Ubuntu Linux doesn't have this feature. (Its Send-To feature will simply email the original image directly, bypassing your email program.) And you will have much trouble trying to do it in Gimp, flPhoto, Gthumb, or F-Spot. Here's where the command line comes in very handy.

See if you have the ImageMagick package installed. If not, you can install it (as root) with:

apt-get install imagemagick

Then check out their site at:

www.imagemagick.org

Click on Command Line tools in the navbar. One handy command is identify:

identify somephoto.jpg

somephoto.jpg JPEG 273x365 . . . 8-bit 16.8496kb

You can spend days learning these commands. You can do almost anything that Gimp or Photoshop can do — on the command line. And thus you can operate on a large number of images at once.

Here's a command to reduce a large image to an emailable one:

convert -resize 100000@ -quality 30% bigimage.jpg email.jpg

That @ character after 100000 means the converted image (email.jpg) should have at most 100,000 pixels (not bytes). So you don't have to worry about a small image getting converted to something microscopic.

The convert command can also be used to, well, convert:

convert myimage.gif myimage.png

What could be simpler than that? :-) In this case convert uses the "png" extension to figure out what you want. Another ImageMagick command is mogrify. In its simplest form mogrify is a version of convert that modifies your original file:

mogrify -resize 100000@ -quality 30% vacay.jpg

(However big vacay.jpg was before, it's now about 20KB.) With the -path option, you can get the results to go into a subdirectory:

mogrify -monitor -resize 100000@ -path email *.jpg

(That -monitor option is like -v or —verbose.) This command will do the conversion of all .jpg files in the current dir and put the results into the email subdir — the original images are not changed.

So snap those pictures with your new 20-million-pixel camera. Then convert with convert or mogrify and email away! Your recipients may even notice and thank you.

Ubuntu, Where Art Thou C Man Pages?

You're writing a C program and forget how to call stat(3). No problem

man 3 stat

But you get this:

No manual entry for stat in section 3

Whuhhh? That function's been around since the Stone Age (ie, 1971). Turns out that the C man pages are not loaded by default during the Ubuntu desktop install.

No problemo. You can get 'em here (as root):

apt-get install manpages-posix-dev manpages-dev

Now you have man pages for all your favorite C library functions (and C include files, like stat.h).

Go nuts!

Waiting for the Jackalope

Ubuntu Linux 9.04 (code name Jaunty Jackalope) is due in the last week of April. The ingenious version numbering is well-known to Ubuntu folks -- year.month. If it slips, they will call it 9.05, but that won't happen.

Have you have ever received a postcard of a jackalope? I have. Went on the fridge door for about, oh, 3 years.

Still have never actually seen a live one.

Heh.

Anyway, here's what I'm looking for in JJ:

• OpenOffice v3 (8.10 kept v2.4). After abandoning OO v1, I've been really impressed with OO2's speed and huge compatibility matrix — even opens .docx files, thank you. OO3 should be even better. However, I still think the spreadsheet app is a little too complex (ie, overly complex in its own unique way, not overly complex in an Excel way).
• Maybe my Canon MX310 all-in-one scanner will work?
• Out-of-the-box audio — in 8.10, I had to fuss with PulseAudio and competing older technologies to have proper sound. That's dumb. Then you have to configure the right plugins/apps for Firefox sound (RealAudio + Flash + totem + I forget what else). Ubuntu install/upgrade should have an option that says, "I accept non-free software, now make audio and video work in Firefox and don't bug me."

I switched to Ubuntu/Gnome recently after using Kubuntu/KDE for 2 years. I liked KDE — Version 3, that is. It's kind of flat and plain, but very configurable. And the KDE apps like Kopete (IM), Konsole, and Kspread are very good. But KDE3 didn't play very nicely with Compiz and the other eye-candy components.

KDE4 came along. V4.0 not only looked astonishingly like Windows Vista (pre-SP1) with its shadowy dark theme and floating widgets, but it was, like Vista pre-SP1, completely unusable and un-configurable! KDE4.1 improved things, but all the joys of KDE3 are gone. And, starting with Kubuntu 8.10 (ie, Oct 2008), KDE3 is discontinued.

Meanwhile, while I wasn't looking, Gnome got a whole lot better. Faster, smarter, more configurable. And its various g* apps acquired lots of new fun features. Bravo!

Another thing about KDE4. They unceremoniously dumped workspaces in favor of viewports. Took me a while to figure this out. Switch to a new workspace (as you can in KDE3 and Gnome) and you get a separate desktop with settable background. Has its own set of tasks on the taskbar, but same panel and desktop icons. Move a window off to the side and part of it is hidden. If you have 4 workspaces, think of it as 4 pieces of paper.

The viewport model, on the other hand, gives you one giant piece of paper and switching moves around on it. Panel, background, and taskbar stay the same. But icons and open windows are, well, wherever you last put them. Move a window off the current viewport edge and it now straddles two port views. You can configure KDE4 to "fold" the piece of paper into a cube or other shape. Switching now shows up as a spinning cube (4 or maybe 6 viewports) or other fun effects. Each side of the geometric shape is a viewport, and you can leave things on any of the 4 sides or in between them.

Viewports may sound better, but I prefer the more compartmentalized workspace model. When I want to switch to another background and set of windows, I want don't want half of something else to be there.