Sometimes my users aren't the ones who are dumb. Sometimes it's me.

So I have this new off-site executive from an affiliated organization (AO) crawling up my butt trying to get into our intranet, a funky slash-coldfusion hybrid. Since he's a) pushy, b) just smart enough to be dangerous, and c) picking up the pieces from a big political meltdown somewhere else, everyone else is crawling up my butt to make sure it gets done too.

Of course, as expected, he can't get in. The guy used to work for a different AO in a different state, so he was already in there, but de-activated. I wipe out this old record, and then try to log in using his information. No luck.

So, I had a user who should be able to log in, but couldn't, and all the logical things that would be keeping him out aren't there. At times like this I drop back 10 yards and punt, moving on to other projects to let my brain rest for a while.

A few hours pass and yet another executive comes by asking how's it going getting this guy on. Well, it's not going on at all 'cos I don't have a clue what's wrong (which is what I say on the inside). Instead I say, "Working on it, should be fixed soon" and turn back to the project.

So there I am with my little database tools trying to get a look inside the files that contain the user information. What I wanted to do was get a look at the entire contents of that file, which would require me to run this technical command (called a 'query'):

select * from users

I had a query already very similar to that, "blah blah blah where user = [username]". So I just erased the "where user" line and ran the query.

It was only after a few seconds with no results that I realized what was going wrong. Sitting in front of me was not

select * from users

Instead it was

delete from users

Those of you used to SQL will already be slapping your head. Those of you who don't, a brief explanation:

SQL is a language used to talk to databases, giving you access to the information in their files. You use it by typing commands and pressing ENTER (a "command line interface"). Like all command line interfaces (CLIs), it's very powerful, very weird, and very unprotected. CLIs were designed by geeks, for geeks, and geeks don't need safety nets.

The DELETE command is about as powerful a command as you can get. By default it wants to get rid of everything. You have to tell it special conditions to stop it from doing this. Imagine a bomb that is powerful enough to blow up a city, but sophisticated enough to erase a pencil mark as long as you twiddle the knobs correctly. Now imagine the designers of that bomb making sure that by default it does the former, not the latter, without so much as a f*ck you on its way out the door.

What I had done was told it to delete everything in the user table. All the names, all the e-mail addresses, all the passwords, everything. Kaplooie! BeepBeepZipBANG! One second, a computer file full of information about the most technically clueless users on the planet. The next millisecond nada, nothing, a big empty warehouse with just shadows on the floor where the shelves used to go.

Any sysadmin worth a dime knows the greatest danger to his or her network is not weather, electrical problems, or software glitches. It's users. You may judge an admin's quality by how quickly he or she fixes your computer, but we judge each other's quality by how difficult it is for you to blow sh*t up in the first place. This means locking you out of all sorts of amusing and fun-to-play-with toys like system files, certain (in my case most) utilities, even some kinds of data files like... you guessed it, user database tables.

But of course there's little to protect a system from the admins, who need all these tools when a user has been extra-special clever and managed to fark it all up anyway. Normally we're smart enough to know where the cables are buried in the back yard, but every once in awhile we go ahead and dig anyway.

So after saying oh-Sh*t a few times (out loud, to judge by the looks out my door), I went into damage control mode. The rest of you sysadmins are saying "no problemo, just restore it from backup". Ah, my friends, that means you are not inflicted with Arcserve, the (IMO) Worlds Worst Backup software. I'd just spent the past month banging my head bloody on it trying to get it to back up anything, and all it did was merrily give me the finger and fling boogers on the wall. I hadn't managed to get it backing up the slash box yet. It was barely backing up even more important stuff.

In other words, I was the front man on one of those flying Wallenda pyramids and had just sneezed a hundred feet in the air, nothing below me but a nice hard concrete slab. The pole was swaying, and falling off meant taking everyone and everything with me.

It's times like these where the design decisions you made laying out the network either make you or break you. At that point you can't save the network, but sometimes it can save you. And, actually, it did.

See, the slash site is just part of where user files "live" on this network. The core information is stored somewhere else, in an "LDAP Tree" (fancy technojargon for "other, safer, place"). Without even missing a beat the LDAP Tree started to synch up the slash site as if nothing special had happened. The phone didn't start ringing off the hook, people didn't start showing up to my door with pitchforks and torches, really nothing special happened at all.

The account that triggered this whole mess even started working again, but that kind of felt like watching your linebacker fall on a fumbled football after it rolled across the first-down line... you'll take it, but that's not normally a real good way to get it done.

So there ya go folks, a brief look at what happens behind the scenes WHEN SYSADMINS ATTACK.