PhotoDude.com

Sat. Sep 13, 2003

Changes Under The Hood

Changes Under The Hood – This weekend, I am in the process of either [1] substantially restructuring this 7 year old site in order to move it to a new web host, OR [2] completely hosing years worth of work by misusing powerful software.

So if you are experiencing any strange redirects to nowhere, missing pages, or other oddities at this site, it’s OK. I’m doing it on purpose. Or rather, it is an unavoidable consequence of the structural contortions I must perform simply to prepare this site to be “moved.” Perversely, if I am successful, you will hardly be able to notice anything has changed … visibly.

If you’re not a web geek, that’s all you need to know; things may look strange, but it’s not your computer, and all will soon return to normal here without you having to do a thing. If you are a web geek, of course, you might be curious why I’m structurally screwing with a site just before I move it.

I blame it on Movable Type.

As always, it’s the tool that gets the blame, not the operator. But I’m also not the first person to face this dilemma. In fact, you’ll find a clear tutorial here on exactly how to fix this problem should you encounter it, but here’s the gist of it.

MT gives entries sequential ID numbers, which many of us started using within our archive structure. For example, your second entry in your weblog would end up with a filename of 000002.html. MT will happily pump out new entries in sequential order.

The problem comes when you add a second weblog served by the same installation of MT. Suppose you’ve made 52 entries in your first weblog, and then you add a photoblog. Your first entry in your photoblog won’t be named 000001.html, it will be 000053.html. The new weblog becomes a part of the already ongoing number sequence. If you end up with 198 entries in your first weblog, and 42 in your second weblog, the numeric ID’s between 000052 and 000250 will be a jumbled mix of the two weblogs.

So what’s the big deal? Well, when you move to a new host and set up a new install of MT, you’ll have one large exported file for each of your two weblogs, the backups output by your install of MT at your old web host. Those files must be imported one weblog at a time, and therefore the numeric ID’s will be entirely different than they were on the other server, as they won’t be “created” in the original sequence.

This will cause every link anyone has ever given your work to break, and for those of us who “subreference” one entry we’ve written within another new one, those links will break as well. What a %$@?! mess.

I know there’s a lot of site owners out there who simply don’t care if they create such link rot, in fact, almost brag about it; “All the archives are gone, deleted, as I’m doing something new now.” To me, frankly, that’s rude! You know how bloggers are. We covet links from others. We hope our witty insights inspire others to point to us. And then when people do go to the trouble to link your words via the specific “permalink” you’ve provided, you brusquely convert them to “deadlinks.” It’s not only rude, it degrades the entire concept of “linking” on which this wonder is all built, and is counterproductive to your own web wellness.

Rant over. Suffice it to say, I hate link rot, yet was faced with a situation rife with opportunity to generate it. Finding that others have faced the same thing, and with a wealth of somewhat competing advice on just how I should best structure my archives and their URL’s (“Really, enough, guys, I get the point”), as usual, I came up with my own unique way of doing things.

The problem at many MT sites is the “default” archive structure can give you URL’s like this:
../archives/003137.shtml

As you might expect, when I set up this archive structure in February of 2002, I started out with my own hybrid that is now biting me in the ass:
../dumped/2003/September/003137.shtml
I managed to include the date, but with the month capitalized, and the poison pill of the entry’s numeric ID.

Now, a real purist would say I need to establish a structure that would generate this kind of URL:
../2003/september/13/changes_under_the_hood.shtml
...or even a “slugged” URL stripped of file type:
../2003/september/13/changes
...while others suggest the only URL scheme that will never change is one solely based on the date and time of the entry (the argument being if you ever change the entry title, it will break the URL).

I’m not a purist (well, I’m a selective purist). I don’t change entry titles. And I don’t want to generate 365 directories for every year this weblog has been around (there would be just shy of a thousand for the past 2.5 years). So I decided on this:
../2003/september/13_changes_under_the_hood.shtml
Only 12 directories generated per year, a human readable URL, and if the web world someday switches from *.shtml to *.futurefile, I’ll rely on the fact some smart soul will offer a server side solution. But he/she is likely in 6th grade right now, so I’m not going to worry about it too much today.

Now that we’ve solved that, how do we get from
../dumped/2003/September/003137.shtml
...to…
..2003/september/13_changes_under_the_hood.shtml
...and do it from one server to another, without busting every link we’ve ever gotten? First, we note one of the reasons we’re switching web hosts … a lack of web space. Reading through the MT forums has revealed that MT really really doesn’t like it when you run out of web space. It gets angrily constipated and/or starts spewing out crap you never requested. My plan involved duplicating nearly 2000 HTML documents in a new directory structure, and I was using nearly 290 MB of my allotted 300 MB (I’ve had a site for seven years, get off my back).

So step one was a crash diet, in which I temporarily stripped everything I could in the “low-traffic, large-file-size” category. Despite being relatively ruthless (as much as I could bring myself to be), I carved a mere 20 MB from the site. My rough but conservative estimates (1900 pages at 20-25 Kb each equals 35 to 45 MB) indicated the free 30 MB might not be enough.

So I decided I’d have to institute site wide link rot … for ten to fifteen minutes. In a manner similar to the one shown in the tutorial, I created a template for the individual archives that contained nothing but a redirect built using the archive tags to point to the new location, and also a visible text link to the new location [example redirect for this page]. This template was 2 KB, so 2000 redirects would only add about 4 MB to the site. Unfortunately, I had to build these redirect pages first, so they’d be in the proper place in the old directory structure. Therefore, there would be a short period of time when the redirects would point to pages that had not yet been built in their new location.

And since I was going to rebuild the entire weblog in new directories, I decided to rework all the templates before that rebuild. I had a couple of things I wanted to accomplish. When I first converted to using CSS for all presentation and structure, I was so focused on getting the layout right that I was a bit sloppy about properly separating the content and the presentation. I wanted to strip out some of the loopy and/or redundant CSS I’d created to “make things work” the first time around. And I also wanted to improve the semantic structure and utilize CSS Image Replacement (turn off style sheets, and instead of images for PhotoDude.com and PhotoDude’s Weblog, you’ll see those words as headings). There’s no longer any coddling for non-compliant browsers (yeah, I’m talkin’ to you, Netscape 4.x), but my aren’t those semantics pretty without that hard-to-understand CSS! [cough]

With fresh new templates to rebuild, I decided to also test my template code for the meta refresh tag by hiding it within a comment in the head of the newly redesigned documents. Now that I had nice clean templates and tested redirect code, I was ready to replace each of the templates (individual, weekly, and category archives) with their corresponding redirect template. After, of course, a major league backup by exporting the weblog via MT, and downloading all the current working archives via FTP (so, worst case, I could put them back in place while I picked up the pieces).

[Now we reach the point where Reid talks in the past tense about things he hasn’t yet done … call it “previsualizing success”]

After sacrificing a goat (easier to find than a virgin) at the Altar of the Server Gods, I opened up four browser windows; an edit window of one weblog entry, and the individual, weekly, and category archive pages for that entry on my site. I replaced the templates with the redirect templates, and saved them, but did not rebuild the entire site. Instead, I made a simple change in that entry I had open (I usually just change the punctuation), and saved it to rebuild just that entry. Then I refreshed those three archive pages I had open to see if the redirect worked, and pointed to the proper URL (and at that time, generating a “404 – Page Not Found”). Doing it this way allows you to catch any template error before you propagate it by rebuilding all 2000 pages.

Satisfied that my redirect templates were functioning properly, I rebuilt the individual, weekly, and category archives. Once I’d done that, I then altered the archive directory structure to the new scheme, and put the original templates back in place. Rebuilding the site again then put the final destination pages in the place that the redirect pages were pointing to.

At that point, I just FTP’ed the entire old directory structure containing the redirects to my hard drive, ready to upload at the new web host. When I import my weblog(s) into the new MT install there, I’ll use the same date/title archive scheme to build the new directory structure like it now exists here. Any link anyone has given a page in my weblog will redirect to the proper entry

Yes, you might say it’s a lot of effort to accomplish that. But I do it for you, so you’ll always know, “When you link PhotoDude, it sticks.”