viva la rsync

I've had automated backup processes running on my Linux boxen for a few years now. Being a developer with a fondness for automated processes that work hard while I'm asleep, I reasoned that the best way to back up a machine automatically was to write a script that's smart about how it does things, and just let it take care of itself. It worked pretty well, saving my bacon a few times.

My backup routine was thus written as a shell script that would make a list of all the files in each user's home directory, omitting files in directories listed in an "exclusion list" file for each user. Then it would zip all of the files in the list into a zip file named after the user, machine, and the current date. Each night when it ran, it would remove any backups more than 3 days old.

It worked fine, except for some problems:

First of all, if I had it back up more than 2GB of data, it would fail. Because the "zip" program uses 32-bit file pointers, it can't address a file bigger than the max value of a 32-bit integer, roughly 2GB. So I was always thinking of ways to prevent the backups from being "too big". Every time I copied a file around, the possibility that it could hose up my backup was something I thought about. Lame!

Second, the process itself was resource-intensive. Creating a gigantic zip file on a machine with a 1GB of memory puts everything in RAM into swap. So in the morning, I'd wiggle the mouse to bring up the desktop, and wait... and wait... and... wait... while the machine thrashed around swapped its consciousness back into RAM. Sometimes, I'd find that the zip program was still running, 3 hours after the backup was scheduled to run. Lame!

I finally got sick enough of these annoyances to do something about it, so I tried out rsync, the "standard" way of backing up Unix boxes. It's purpose-built to copy large numbers of files and sync directories. I used it to create mirrors of the backed-up directories on a removable USB drive. After the initial sync, subsequent rsync runs only back up changed or added files.

Additionally, you can use hidden .rsync-filter files to control how files get backed up. You can omit certain types of files, files with specific names, directory trees, and so on. You can sprinkle these files all over the place and rsync will read them wherever they are and follow the rules they describe.

So now I'm using rsync. I'm backing up over 60GB of files, the process typically takes approx. 20 seconds for each machine, and the resource load is so low it's not even noticeable. If it was a product you could buy, I would recommend buying it, but since it's free, I guess I can't. There's always a downside, I guess.

Chunkin' punkins

For various reasons, our family has always kind of bypassed the whole tradition of cutting faces in pumpkins for Halloween. We got pumpkins, and just set them out front posing as... you know... pumpkins.

This year, though, we decided to put faces on them. We got the kids together and went to work. I had forgotten how gross the inside of a pumpkin is. I cut the top off the first one, and put my hand inside to clean it out. Realizing how unpleasant that task was going to be, I exercised my "Dad Power" and told the kids, "Oops, I almost forgot. This is the fun part. You guys get to pull all the stuff out of the inside of your pumpkins, and put it in this bowl!" The kids said "Yay!" and stuck their hands in. Ha ha...

Eventually, we got them cleaned out and I helped Jiana draw the face on her pumpkin while Andrew drew on his. His design featured fangs, eyebrows, hair, and a lot of other stuff that would have looked cool, but would have been murder to carve out. I used a tomato knife to carve them out. It works great, like a coping saw for food.

Thus concludes this year's Halloween Special.

Octavarium

Just got the newest CD from Dream Theater. It's called "Octavarium".

I've always thought of Dream Theater as one of life's great ironies. Here's why:

Imagine if you took 5 of the best musicians in the world, gave them the best equipment, a big budget, and lots of time, and told them to get together in a studio and let their creativity run wild. I would think you could reasonably expect stellar results.

So, there's Dream Theater. Arguably 5 of the best musicians in the world, getting together with some really good equipment and apparently ample time, letting their creativity run wild. The irony, as you're probably expecting, is the results. Not always stellar. Sometimes the singer does this weird thing with his voice that sounds... Well, I'll be nice, and just say "not manly". There's just kind of a wide vibrato with a lady-like lilt to it. I'm pretty sure I've never heard a man make a sound like it. Then there's the synth solos. Admittedly, I'm not big on those, it's a personal thing. At best, I think they sound like a small animal being tortured. At worst, they bring to mind the image of a weirdo in a pink spandex jumpsuit and an exposed hairy chest dancing around with one of those strap-on keyboards on. Echh...

Then there's the "group" solos. Remember, these guys are amazing musicians. If I had 1 penny for each note each one of these guys plays in an average solo (and they all play solos), I could quit my job. Their later albums feature a lot of "solos" that somehow involve the whole band. Everyone joins in on the fun. The drums sound as though someone dropped them into a tornado filled with drumsticks, the guitar player is wearing out picks by the bucketful, the keyboards sound like a herd of angry cats is running around on them, the bassist sounds like he has 30 extra fingers. The singer is taking a break. The weird thing is, they're all going nuts (except the singer), but they're all in perfect sync. They all speed up and slow down at the same rate, at the same time, and never have a wreck. They change modes in perfect sync, and do some really weird stuff. One of their earlier albums has a section where the speed-metal song they're playing degenerates into ragtime for about 10 seconds before taking off in a different direction. It's the sound of 5 great musicians with total creative latitude showing off, and it's so over the top, it actually works some of the time.

But sometimes it gets annoying. I sit there listening until I just can't take it anymore, then I say "wankers!" and hit the "FF" button.

Another thing is how long the songs are. I kind of like epic pieces of music, but it's a little inconvenient when you can't get in your car, drive 40 miles, and even get through one song on the trip. Dream Theater must be making prog metal for Amish people who ride to town in horse-drawn buggies.

Before you get to thinking I don't like them, understand that I own every one of Dream Theater's albums. Why? Every one of them has moments on it that make goose bumps pop up on your skin. I think it's a "musician" thing, and I know other musicians that experience the same phenomenon. I even read an interview with Alex Lifeson where he mentions the "GB Factor", which is how he used to measure the goodness of a particular piece of music. He would watch his arm while he listened to something, and if goose bumps appeared on it, he knew it was good.

"Octavarium" is good. DT seems to alternate between making "listenable" albums, and more wankish ones. This one leans far to the listenable side. The singer (James LaBrie) is still doing weird things with his voice. On "Panic Attack", a double-bass thrashing metal song, he sounds just like Gwen Stefani! There's a song that sounds like something U2 would do. I like it. Actually, I like them all. I think this is the first time I've listened to a DT album 3 times in a row, and not hit "FF".

The title track, "Octavarium", ends the CD, and is a perfect example of Dream Theater in action: It's very long, and it contains a big crazy middle section full of solos and unbridled wankery. The song roars along for about 15 minutes, farting and belching and grinding. There's a rapping pirate at one point (I'm not joking!), and they play the main riff from "Jingle Bells" at one point. (I'm not joking here, either). Kind of ridiculous. However: The last 4 minutes of that song contain some of the highest-horsepower music ever created. It sounds huge. Imagine the biggest movie ever made, with the most climactic ending ever imagined... Something involving mountains exploding, giants roaming, clouds parting, Hobbits pillow fighting, and God Almighty riding in on a white horse. This would be the kind of music you'd expect to hear. I played that last part of the song for my mom, an operatic-voiced classically-trained music snob from way back. She said "Wow! That's cool. Let's hear it again."

No kidding.

Opiate

A friend of mine mentioned Gartner (aka The Gartner Group) on his blog, and it got me thinking.

First, some background: He and I both work in a company whose IT department is heavily dependent on Gartner for advice on everything. (Read his post if you want an eloquent explanation, I can't be bothered).

There's a lot I can say about an organization whose policy it is to let an outside firm do their thinking for them, and none of it is good. Talk about a recipe for being a loser... At best, you can expect to run mid-pack among your competitors, assuming you have competition to start with, and they're all listening to the same advice as you. I think it shows a distinct lack of imagination.

Understand that this rant isn't necessarily a slam against Gartner. In fact, I applaud their ability to make the kind of money they do doing what they do. It's good work if you can get it. And I think I'm starting to figure out how they're able to do it.

Let's start with this sample of wisdom, direct from Gartner (I swear to you, I did not make this up):

"Enterprise architects must act as catalysts that speed the formation of unified business technology strategies and their execution. The enterprise architecture process must shift gears from limiting complexity by limiting choices to accelerating innovation and execution by coordinating complexity through unified business and IT strategy, decentralized execution and loose coupling among all related stakeholder disciplines."

Uuuuuh huh.

Care to parse that? No problem, I already have. Here's what it says:

Software is complicated. Making it work on a large scale takes some planning, some discipline, some consistency, and generally requires a well-thought out and well-designed system spread across the enterprise that does a lot of things for the people writing the checks. Get used to it.

Pretty straightforward, right? Obvious enough to make most people say "Duh!" before they realize they've said it, right? Right. You don't have to pay me for that statement; I didn't expend much effort translating it for you.

The Gartner version, on the other hand, appears in an expensive "white paper" (Gartner's name for a document printed on a piece of white paper), accompanied by many other such statements. It's worth thousands of dollars, and is much more verbose. Why? Someone had to put a lot of thought into it, because the Gartner version has more than just the function of conveying a piece of information. It's also designed to lull the reader/listener into a trancelike state, where all they can do is agree. If you don't believe me, read the statement again. Notice how, as you read, your eyelids get heavy, and you feel kind of dizzy and slow-witted. That's not senility setting in, that's the effects of the skillfully-crafted phrasing you're feeling. Call it "Gartner Gas". Notice how, at the end, you kind of thought "Uuuuuh... Okay. I guess..." Skeptical? Try it on your spouse. Read the Gartner sample to them, and watch their eyes glaze over. Ask them if they agree with it. They'll most likely say "Uuuuuuh.... I guess." Tell them it's from a respected management consulting group, and notice that they're more inclined to agree, even if they don't understand it. See?

This effect is intentional, and there is great value in it if you're someone who spends a lot of your company's money on advice from Gartner or other management consulting firms.

Imagine you're an MBA in a suit, sitting meekly in your boss's office while (s)he grills you about why things aren't going according to plan. You needn't worry, because you're armed with the very best in suit-defense technology: The Gartner White Paper. Wield this weapon and start spouting quotes from it, and you will be able to repel most attacks (except in one case, which I will describe below).

When you present a white paper, especially one that costs what the Gartner ones do, you gain instant credibility. It cost a mint, it must be good. The presence of a white paper in your hands shows you've done your homework, looked around, and haven't just pulled your ideas out of thin air. Not only that, but when you quote from a white paper, the artful phrasing kicks in. Your boss will be totally mystified by the flurry of enterprisey-sounding speech coming out of you. They will be lulled into submission, and have no choice but to agree.

On the other hand, suppose you're the same MBA getting grilled by the same boss, and instead of wielding Gartner, you just use simple speech to explain yourself. My translation, for example: "Software is complicated... yadda yadda... Get used to it." Oops. You just said the same thing as before, but there's no white paper to back you up. You delivered information in a simple, short pointed burst, and totally missed out on the Gartner Gas effect. Your boss, still fully lucid, won't be the least bit docile after hearing something so pointed and short. They may even think you're being flippant, or unduly terse. You could be killed.

This, I believe, is the value of Gartner. It's a big flame-retardent suit made out of titanium-coated bullshit. In the appropriate places, used on the appropriate people, it's invincible.

Of course, there are places in the world where this approach wouldn't work. There are companies out there who have about as much use for Gartner as a fish does for a bicycle (Thank God). These are companies that are generally considered to be (to borrow a phrase from Kathy Sierra) kick ass companies. Whip out a Gartner white paper in one of those places trying to defend a dumb decision, and after the laughter dies down, all that will be left of your MBA body will be a couple of bloody stumps. Actually, I doubt this happens very often. Companies like the ones I'm thinking of probably don't employ people who would have much use for Gartner white papers either (except perhaps to line the kennels of the dogs they bring to work).

Let's pretend we could go back in time to when Google first started out, and let's also pretend that instead of being smart geeks, founders Sergey and Larry were Gartner-dependent MBAs.

So one day, they're sitting in their corner office (let's assume that, as MBA leaders of a cash-poor startup, MBA Sergey and MBA Larry share the same corner office in a rented high-rise temporarily). Sergey looks out the window and says "Larry. We're going to build a search engine that's going to need to handle a lot of traffic. I don't know what computer to run this on." MBA Larry, without even looking up from his game of Sudoku, says "Mm." Sergey says "I know, let's call Gartner and ask them! I'm a Gold Member with Gartner, and the cookies they gave me at the last Big-G Symposium were delicious!" So, they take $20K of their precious venture capital and write a check to Gartner, and then hold a conference call on their Blackberries. The guy from Gartner tells them they should use Windows 2000 Server because, based on earlier advice from Gartner, MBA Sergey and MBA Larry built the Google search engine in Visual Basic.

All rightey... multiply the number of machines in Google's datacenters by the number of dollars for a legal copy of Windows 2000 server. The answer, for those of you without access to a calculator, is "hosed".

MBA Sergey and MBA Larry can take comfort in the fact that a well-respected management consulting company told them what they should do, but it didn't work. If they had angry bosses, they could use this as an excuse. They could claim that the idea of indexing the internet must be flawed. MBA Sergey and MBA Larry can therefore close up shop, fire all the VB programmers, and go get a lap dance with clear consciences.

Companies that do something unusual, something innovative, don't rely on someone else to do their thinking for them. Fortunately, the real Larry and Sergey didn't. There is no way Gartner would have told Google, "Uh, yeah. Just use Linux and customize it so it works for you. Make your servers out of plywood and velcro, and put them in a shipping container."

Management consulting companies are an opiate for the MBA masses.

Who Killed the Electric Car?

I just watched a movie that came out a few months ago, and is now available for download. It's an environmentalist movie about how GM killed the EV-1. I watched it because I'm a sucker for a giant download off of YouTube.

The EV-1, for those who don't know, was GM's first attempt at an electric car. It was a 2-seat car that ran on lead-acid batteries, had a top speed of 70mph, a range (best case) of about 80 miles, took 12 hours to charge up, looked like a suppository with headlights, and cost something like $250K. GM leased the cars to celebrities, several of whom were interviewed throughout the movie. (Mel Gibson even appears in it, wearing a freakishly large and striped beard. Maybe he was trying to disguise himself.)

Anyway, sometime around 1999, GM cancelled the leases on the cars, and took them back. Some of the people who had EV-1s wanted to keep them, but GM said "no". They wouldn't sell the cars to customers either. They just came and took them, then hauled them all to the crusher.

So. This movie is a long tirade about how bad it was that GM did away with this great car. It paints all kinds of fanciful pictures about the oil industry, the auto industry, the Bush administration, the House of Saud, the California Air Resources Board, and other members of the Legion of Doom. According to the movie, these evil forces all acted together to kill the electric car. Their apparent motivation in doing so was to kill bunnies, cloud the sky with smoke, burn oil, and enslave the American people. Killing the EV-1 was but one step in their master plan for humanity's downfall.

It also put some of the blame for the murder of the EV-1 on the American consumer. The average American consumer, according to the movie, is pretty dumb, and could have been convinced to buy EV-1s by the zillions if sufficient marketing muscle had been applied. One voice-over in the movie even said "Look. The American people will buy what you convince them to buy. You feed them something enough, they're going to start to believe that's the diet, know what I'm sayin'?"

Puh-leeeeze. Here's what the EV-1 was. It was a 7000-pound golf cart with no air conditioning, pathetic range, lame performance, an exceptionally high price and no luggage space, in a world whose population isn't inclined to wait around for 12 hours for it to charge its batteries. The only people who had EV-1s were celebrities who only took them out once in a while to show off how "green" they are, and who normally spend their time in gas-guzzling limousines or private jets. GM killed the EV-1 because it deserved to die.

Any "dumb American consumer" can use their rudimentary math skills to figure that their money is far better spent on a sensible gas-powered car. For about 1/15th the EV-1's purchase price, you can buy a car that can outrun, out-distance, out-haul and out-last an EV-1, and it will get somewhere in the neighborhood of 30 mpg. You can also fit it with loud exhaust and throw one of those colorful fireplace logs in the exhaust stream, and get the side benefit of annoying the kind of ninnies that appear in movies like this.

I think electric cars are a cool idea, and someday, they might actually be practical. And by "practical", I mean "have enough of the ability that a conventional car has that a normal person might actually want to use one". The EV-1 didn't have it, and neither do other electric cars. The EV-1 was a nice experiment, it's over, and everyone (except the misty-eyed redhead in this idiotic whinefest flick) has moved on. Until you can drive an electric car a respectable distance on a charge, and recharge it in about the same time it takes to pump fuel into a conventional car, it's not going to replace the conventional car. Ironically, the only people dumb enough to think otherwise are the people in this movie who are convinced that the rest of us are stupid.

If I were GM, I might have chosen to let the whiners in this movie keep their EV-1s, in exchange for having their mouths surgically removed. That would go a long way toward reducing some real pollution.

Classic Joel

This is the second post I've written in a week's time about Joel Spolsky. I can hear the whispers now: "Fanboy... fanboy..." Whatever.

I found a big archive of his old posts the other day, and I've been reading them.

This guy gets it.

He talks about technical stuff, how to hire people, what kind of environment and tools to provide for them, what to expect of them in return, how to compensate them, how to grow a company so it lasts, how money and toys aren't the prime motivators for people (if you can provide them with other things to hang on to), "Architecture Astronauts", and on, and on, and on. The stuff makes sense, too.

I want to live on same planet as this guy. Seriously. I live on a toally different planet right now. Sitting in a half-height cubicle in Dockers, surrounded by hostile, thin-skinned, largely incompetent assclowns and blowhards, mired in ill-conceived/ill-defined projects that never end, provided with crappy tools that cost an arm and a leg and don't work, and being continually lectured on The Company Way, it's easy to get annoyed and start thinking everything sucks. Some days, going through the mental process of trying to come up with a way to tolerate the annoyances without going insane, you can actually feel yourself getting dumber. "Eh... Maybe this isn't so bad.... Maybe I'm just being... you know... not good... attitude... duuuuuuuuuuuuuuh."

Reading this stuff is a breath of fresh air. It gets me kind of fired up, actually. Just knowing that someone out there has a grasp of how things ought to be is encouraging.

One thing I did find kind of amusing was a series of posts about how Fog Creek Software (Joel's company) plans to compensate employees, what their benefits are going to be, how they'll all have offices with doors, all situated in a bustling metropolitan area. It all sounds great. At the end, it says "when we are able to afford it." I hope they can someday. I hope they become gigantic and buy up all the insurance companies in the world. Of course, they wouldn't be cool then.

Turning over a new leaf

Well, not really. I am going to stop beating up on Robert Scoble for awhile. I happened to see my own blog today, when I stopped by Jonathan Schwartz's house. He was reading it. Anyway, I didn't like the way my last post looked on the screen. "Hey look at me, I'm Robert Scoble!" It seems kind of hostile and petty.

So I've decided to talk about more constructive things, like focussing on the consumerization impact being driven by the digital natives. This is important, whether you're talking about people, processes, or technology. I saw a really good 2-slide presentation on this today, and although it looked oddly familiar, it struck me as totally original and innovative. Whoever came up with this stuff should be working for Gartner!