Josh Work Professional Organizations Trip Reports Conference Report: 2003 LISA

The following document is intended as the general trip report for Josh Simon at the 17th Systems Administration Conference (LISA 2003) in San Diego, CA from October 26-November 1, 2003. It is going to a variety of audiences, so feel free to skip the parts that don't concern you.

Sunday October 26

Despite wildfires reportedly burning ten miles from the conference center, and reports on the #sage-members IRC channel and on LiveJournal of a rain of ash and smell of brimstone, I decided to come to San Diego for the conference anyway. I was all packed and ready to rock-and-roll, except my 5pm ride to the airport didn't show up. Luckily we'd built some lead time into the schedule for just such an emergency, so I called my ride.

And woke him up.

And then he called me back five minutes later to say his car was blocked into his driveway and he couldn't pick me up. So I drove my car to his house, picked him up (as well as another friend of ours who was there so he'd have company), and then I drove them to the airport. I bailed from the car at the terminal and he was going to drive it back to my place then take the T home from there.0

With all that, check-in at Logan was complicated. I used the self-service check-in machines but while they gave me a boarding pass they wouldnt print a baggage tag for me. Turns out that I'd checked in using American Eagle's machines. (Nice of them to have had only "American" signage, eh?) Found the right desk, got them to manually print the tags, and handed off my baggage to be x-rayed and scanned and manhandled by TSA people, then went through Security.

Clearing Security I got wanded for the third time. It was reasonably straightforward. Had to unbuckle the belt to make sure I wasn't, oh, I don't know, hiding a knife or something in it. Didn't have to remove the shoes, though I did have to let them wand each leg and foot while sitting down.

Boarded the (overfull) flight. When the flight attendant started with the San Diego weather report ("Partly cloudy") about two-thirds of the passengers laughed, since we'd all heard about the fires and smoke and ash. One flight attendant mumbled about where we would be landing since there was a rumor that the San Diego airport was closed. I told her that what I'd heard was that it was closed for departures but arrivals were still landing. Which makes some sense; if there was nowhere to land, we'd probably have been on an FAA-mandated ground hold in Boston.

The flight itself was a little bumpy. Dinner (a rarity when flying nowadays, but it was a 6-hour scheduled flight) was a choice between chicken and rice (more accurately, arroz con pollo) that was actually pretty good, and cheese tortellini. The salad was the usual iceberg lettuce with shredded carrots and radishes but at least the vegetables were fresh. The flatware was plastic (unsurprisingly in this post-9/11 era of paranoia).

Got into San Diego early. We had a beautiful view of the wildfires (bright orange against the black of the ground and sky) on approach into the airport. Gasps of surprise, dismay, and wonder were heard throughout the plane. Met up with Adam and shared a taxi to the Town & Country without incident.

Then I tried to check in.

First, the trainee working the check-in desk (Mayra) couldnt find my reservation. Considering I made it in July and updated it in August, I found this somewhat surprising. After a bit of a wait and very little help from the manager (who was chatting merrily with another employee, not helping guests, over at the Diamond Check-In desk) we found my reservation. She apologized for the inconvenience and gave me a room (1208). So I hiked to the tower and got to room 1208.

Which was a smoking room1 and had no bed.

Well, it had a pull-out sofa-bed (though they insisted on calling it a Murphy bed, but those pull out of the walls, not out of the furniture). So I called down to the front desk and spoke with someone else (Nancy). I explained how I needed a room with two beds ("double occupancy") since I had a roommate arriving and I didn't think he'd want to share a bed with me. She said she was sorry for the mix up and she'd move me next door to 1209, and I could just pick up a key at the desk and then swap my belongings over, since they had only one bellman on duty and he was a bit busy what with the property being used as a safe haven for evacuees from the fire areas. So I went back down to the front desk to pick up the key to 1209, and ran into the arriving Tom Limoncelli (who had his own degree of difficulty with his room reservations, but I'll let him and his roommate share that story).

Which had only a king-sized bed.

Now at this point I'm getting a little bit beyond aggravated. I've had a reservation for easily 2 months for a nonsmoking room and 2 queen-sized beds. I don't think its too much to ask for to actually have a nonsmoking room with 2 queen-sized beds available for me when I arrive. It's not that I arrived on the wrong day, or that I'm asking for anything unusual in terms of hotel space. She again apologizes for misunderstanding me on the phone but the computer doesn't show any 2-bed rooms, smoking or non, available this evening.

Finally the duty manager comes over to assist. I explain again that I want a nonsmoking room with two beds since my roommate is arriving tomorrow (Monday) morning; I suggested that they give us a free room for the week, one for each of us, if that's not possible. Aha! A possible solution! Could I accept a king bedroom tonight and move in the morning, once the evacuees are gone, to a 2-bedroom nonsmoking room? Short answer, yes. So I've got a certificate good for a free breakfast in the morning and a king-sized room tonight.2

Go back to the tower, move my stuff from 1208 to 1209, and head back down to the Lanai Party Suite (which is apparently again 1114 this year). Gulfie pours me a large (and I mean large, as in a triple) rye and I drink most of it on a not-quite-empty stomach. Chat with folks (including a lot of Online Services people like JD, Nick, and David, who's our new Board liaison for the team), plan when to meet up with JD for more in-depth planning about the Wednesday announcement, and crash around 3am Pacific (which is closer to 7am biological, thanks to a 3-hour time zone shift and the hour change from Daylight Saving Time).

Monday October 27

Today was supposed to have been my free day and a day to sleep in. Instead, thanks to the smoke in the air, I'm awake and functional at 6am (though it's an open issue as to how long it'll last) and spent a lot of time in web-related meetings.

Ate breakfast with Pete and also spent time chatting with folks like Nomad. Spent some time with JD doing web stuff again, and some time with various SAGE Executives and Staff people on various politicky things.

Had lunch with Trey, JD, Jane-Ellen, and Pat Wilson. After lunch I updated SAGEweb to point to the SAGEwire story about LISA still happening despite raging wildfires and raining ash and brimstone and so forth. Around 2pm I called the front desk and found our new room was ready, so I grabbed Philip on his way back from lunch and we dragged his stuff out of Nomad's room (1207) and my stuff out of my old room (1209) and moved it into our new room (1222), which is indeed a nonsmoking room with 2 queen-sized beds. Yay!

After unpacking, he and I joined a bunch of folks — Gus, Aaron, Peg, Trey, JD, Brian, and Ellen — hanging out in the hot tub. After breaking up around 4:15, we went off to shower and then meet up for dinners. The Executives and Hangers On were having a joint USENIX/SAGE Board dinner, so, assuming many local restaurants would be closed due to the ongoing state of emergency, I invited others to join me for beef at Kelly's Steakhouse on campus. Eight of us wound up going, including Brian, Philip, Aaron, Jonathan, and three others whose names are escaping me. Dinner was pretty good (yay, prime rib on the bone), and afterwards I headed to the Lanai Party Suite for some bourbon. Got waylaid into another longish Online Services planning meeting, returned for another drink, then went to collapse in bed.

Tuesday October 28

The Advanced Topics Workshop (ATW), my regular Tuesday event at LISA, went as smoothly as always. I wound up being the on-screen scribe using Rob's laptop for the introductions (when Rob was at the USENIX Board meeting next door, for the whole first session) and the tail end of the predictions at the end. A detailed summary will show up on my web site and in ;login:.

After a quick dinner at Hunter's with JD, Trey, Pete, Peg, and Chris, I headed (with all but Peg) over to the GLBT BOF, known to others as the GBLTUVWXYZ BOF, the queer BOF, the motss BOF, and the Alphabet Soup BOF. We had a smallish turnout this year (about 25 people all told) and had several lively discussions about benefits, politics, and what we can do in those areas as well as what SAGE can do for this group of our membership.

After the BOF I hottubbed for a while then visited the Lanai Party Suite before heading off to bed.

Wednesday October 29

Today the technical sessions began. We started with the usual announcements (statistics on attendance and abstracts submitted and accepted, awards for Best Papers and for SAGE) and the keynote address by Paul Kilmartin of eBay.

Paul gave an interesting (if a bit overlong) talk about eBay's growth from one machine in 1995 to several hundred machines and tens of petabytes of the SAN, how they managed growth over time, and some recommendations about dealing with vendors and what you should and shouldn't think about.

After the abbreviated morning break I got to be session chair for Dave Plonka's invited talk on Internet Pathology. For those who don't know, my job as session chair is to nag the speaker in advance for any special audio/visual needs (to make sure they're ordered before the conference) and an introductory biography; then at the conference to make sure his laptop's hooked up to run his slides, do the introduction, flag the speaker when he's running out of time, and moderate the questions from the audience afterwards. In this case we started a bit late because of some projector problems on one side but the talk otherwise went smoothly. He spoke about the process he used to determine how the University of Wisconsin's NTP server was being flooded by off-the-shelf Netgear products and how he and they worked to resolve the problem to everyone's satisfaction.

I had a quick lunch with David, Trey, and JD at the Terrace Café, and chatted afterwards with folks on the patio. After lunch, I attended the Information and Content Management refereed papers track. I didn't have a lot of choice in the matter since I was presenting my paper in the middle of the block of three. My paper, "Designing, Developing, and Implementing a Document Repository." It seemed to go over well, though the talk itself ran only about 12 or 13 minutes and I only had 4 questioners. At this point, other than the Roundtable after the next session, my official duties were ended.3

One of the ideas that came out of the conference planning this year was to reserve a time — lunchtime for the morning sessions and 5:30-6:30 for the afternoon sessions — for speakers to meet in small focus groups with interested parties to talk in more detail about their papers or talks. These roundtable sessions were a great idea, formalizing the hallway track relative to the speakers, but the implementation needed some work. (The lunchtime meeting didn't actually provide lunch for the speakers, and a couple of speakers left post-session discussions to go to the roundtable and have nobody there. The consensus I received was that the idea was good but the execution didn't work.)

Of course, also from 5:30 to 6pm, in addition to the roundtable (for which nobody signed up to see me so I had no problems blowing it off), I got to move a bunch of files from development to production on SAGEweb. And since the list of affected files was incomplete (by a couple of directories), and the versions of Perl on the two servers doesn't match (5.6.1 on development and 5.8.1 on production, in case you care), and the environments don't match at all, and so on, things broke horribly. I skipped out to dinner leaving JD and Nick to fix the remaining problems as well as to update the SAGEwire user interface to be the same as SAGEweb's interface.

Dinner was at Forever Fondue with Moose, Adam, Kevin, and J. Yummy.

The new logo and tagline for SAGE are visible on our websites. Check out both SAGEweb and SAGEwire for details. You'll also note this updates SAGEwire's user interface to the standard defined for SAGEweb back at the beginning of August. We've got versions of the logo graphic available for the public, as well as member-only versions for affiliated local groups [member only] and our members [member only].

Wednesday night at 9pm we had the SAGE Community Meeting. This is the organization's annual public meeting with the Board, the Executive Director, and the membership, where the board and ED get to say what they've accomplished since the last meeting and the members get to ask questions. One of the things that we announced at this year's meeting was the new logo and tagline. (I'd used the new logo — at least the graphic element of it — in my slides earlier in the afternoon to tantalize folks. I'd removed the SAGE logo with tagline, though, since even though I'm Online Services Chair I don't officially speak for SAGE so their name shouldn't be on my slides.) Just before the meeting, JD, Nick, and I updated the SAGEweb and SAGEwire user interfaces to reflect the new logo, new graphic element, and new tagline: "The People Who Make IT Work."4 In summary, the number one priority for SAGE in 2004 is to grow the membership. The focus on the SAGE Online Services front will be to support growing the membership (and providing hardware, web applications, and so on to let it actually happen).

After the Community Meeting I did some hottubbing then went to be social at the Lanai Party Suite. When I realized I was falling asleep on the couch I decided heading off to bed made more sense.

Thursday October 30

This morning I was going to sleep in, since the Internet History guru session with Peter Salus was replaced by Greg Rose on PKI and Cyrptography. (Nothing against Greg; I've taken his tutorial on the subject and been to his guru session before. Peter decided, on his doctor's advice, not to come out to overly-smoky San Diego, so Greg presided over a PKI/Crypto guru session.) Unfortunately, housekeeping was too noisy outside the room so I was up in time to catch up on email (not having been online other than for SAGEweb changes yesterday), catch the middle paper in the "Making Difficult Tasks Easier" session (because (a) I shepherded the paper and (b) both its authors are volunteers doing ;login: summaries for me), and grab some pain au chocolat from the vendor exhibition at the break.

I stuck my head in the "Eroding Network Boundaries" invited talk, but the speaker effectively said he'd have nothing new, so I came back out to the Hallway Track and heckled sessions on the IRC channel. I was going to do more SAGEweb stuff — some minor copy corrections, mostly — but I didn't bother.

After lunch (at the Terrace Café buffet with Bob Gill and David Parter) I hung out in the Practicum session for Kevin Miller's talk. Stayed in the back (where the power outlet was) and shared the power strip with folks around me. It's amazing how popular you can be with a shared power strip.

At 4pm I attended the plenary session. kc claffy from CAIDA spoke about problems on the Internet and what we as sysadmins (and researchers) can do about it. (One of the joys of having IRC access is that during the talk, the session chair asked if we could get the slides online. The placeholder page was up before more than 3 minutes had passed. kc was going to edit down her slides and notes before providing them for SAGEweb.)

Thursday evening, I attended the Costume Reception in my usual Halloween costume of "Doctor." This year it was surgical scrubs and a white lab coat (which used to be my late grandfather's), with a real stethescope (also my late grandfather's) around my neck and a photo-ID badge with magnetic stripe and barcode (my old University of Michigan student ID) clipped to the coat pocket.5,6 I apparently shocked people by how little my 1989-ish photo on the student ID looks like me now.

After the Reception ended I was once again double-booked. I had an 8pm Quiz Show run-through and an 8pm party in Æleen's room. Since we'd apparently managed to get enough people there to help with the run-through (I was aware of John Orthoefer, John Sellens, Steve Traugott, and Ellen Mitchell, though I found out later that Ellen couldn't make it and Robyn Landers helped out too), I stayed at the party upstairs in 1012. We kicked people out in time for the scotch BOF and had some really good whiskies (including a surprisingly nice Californian whiskey, a Balvenie 12-year, and a very nice bottle of Midleton). Hung out there 'til about midnight, then swung by the Lanai Party Suite, spent some time in the hottub, got a job lead from someone, and headed off to bed around 2am.

Friday October 31

Woke up in time (luckily) for David Blank-Edelman's invited talk, "Through the Lens Geekly: How System Administrators Are Portrayed in Popular Culture." It was amusing, and included a lot of movie and TV clips that portrayed system administrators in (US) popular culture. DnB is a great speaker and the talk was amusing, and it was indeed standing-room only, so someone must've enjoyed it. (I had lunch with David, among others, and he noted he'd probably do the session summary for ;login: instead of having his student do it, since David knows what he wants it to say.)

I skipped out on the 11am session to start preparing for the Quiz Show. I confirmed that John Orthoefer and Steve Willoughby would tke care of the quiz grading at 12:30 then headed off to the maul, er, mall with a group of people (David Blank-Edelman, Doug Hughes, John Sellens, David Williamson, Pat Wilson, Elizabeth Zwicky) for lunch at Crocodile. I had a rigatoni in meat sauce that was quite tasty and just what I wanted. The wait staff were all in costume (it being, of course, Halloween); our waitress was Thing 1 from the Dr. Seuss books.

After lunch I peeked in on the 2pm session Tim Hunter had for his paper (which was sparsely attended due in part to lunch running long and in part for being opposite the WIPs), then ran around doing preproduction for the Quiz Show. Managed to get $200 in cash from USENIX for prize money, arrange to get people to Rob's room to carry the equipment — including two flat-screen monitors for the contestants which receive signal over ethernet from the driving laptop (cool, huh?) — and finished the audio questions and did the final (we thought) bug fixes on the software.

At 4pm we started the Quiz Show. This year's show was more exciting than in years past for a few reasons. On Monday, Rob's laptop — the ancient piece of crap with a broken screen — was stolen out of a locked room which was supposedly guarded by Security as well. He didn't have the most-current version of the code or questions backed up to his home network. (Lesson: Back up your laptop frequently!) So Rob was more invisible than usual this conference, rewriting the game show software, writing new questions, choosing audio songs involving smoke and fire (because of the nearby wildfires and the ashfall the first half of the week), and trying nto to go completely insane. In addition to the hardware and software issues, we'd changed the format slightly. We now had 4 rounds of 4 contestants (involving 16 people) instead of the 3 rounds of 3 (9 people). Consensus after the fact was that it kept Rob from spending time with the contestants and in the banter that's very popular.

Things in the show itself were going okay, modulo a "wrong answer" buzzer effect every time we exited a question to go back to the board, regardless of the correctness (or not) of the answer, until for no apparent reason the software crashed just before the midpoint of game one. Luckily I'd been doing the manual-scoring as judge so we had a transaction log to recover from. (I don't know if the current software version has the built-in checkpointing the version from last year had.) We resumed (after Rob did a code fix in real time with the main monitors off and Dan Klein did an improvisational comedy routine to keep folks entertained) only to have the buzzer system fail spectacularly in the middle of another game. So Dan and I went to the backup system of contestants raising their hands. We had a couple of instances where the contestants didn't wait to be acknowleged and so the wrong person answered, but it didn't seem to affect the final scoring much.

The first- and second-place finishers in each round won one of the Linux adapter kits for their Playstations (courtesy of Sony Entertainment Corporation America); the third- and fourth-place contestants in each round won a variety of books from several publishers.

The final round (with the winners from the first four rounds) ended in a tie for first and second place, so we played a tie breaker catgeory. That caused us to end in a tie for second and third place so we played another tie breaker category. When all was said and done, we declared Ken Hornstein the winner, and he walked away with his Linux adapter kit, a satellite photo of the smoke plumes from the San Diego fire (incidentally with the Town & Country more or less centered on the map), and an autographed photo of the Sunday sun, with the visible sunspots.7 Final-round winners also received valuable cash prizes in the form of pictures of dead presidents ($25 each for third- and fourth-place, $50 for second place, and $100 for the grand winner).

Dinner after the Quiz Show was put together by Tall Nate8 at a Brazilian restaurant. They serve grilled meat. Lots of meat. On swords. $34.95 (plus drinks and desserts) for all you can eat. So we had top sirloin, garlic-crusted sirloin, bacon-wrapped filet mignon, bacon-wrapped turkey, pork loin, sausages, baby back ribs, and chicken and cheese, in addition to the salad bar (which included a shrimp in tomato sauce and some cold cuts in addition to the potato and vegetable salads). In addition to the meats I had some potatoes au gratin, sesame-grilled asparagus, a tomato-red onion salad, and some cucumber salad. Nibbled some of the desserts at our end of the table (tiramisu, chocolate cake, and chocolate mousse cake; I think the mousse cake was the best of the three). Didn't drink wine or have alcohol, but we used the div-14 method of bistromathics and it came out to $60 per person (tax and tip included). Managed to let others pay for the cab rides between the hotel and the restaurant, and only borrowed $40 from Lois so I'd have cash for Saturday.

After getting back from dinner, I attended the Dead Dog (or as Moose insisted, Dead Daggit) party. Bartended, as usual, wearing my usual bartender drag.9 But not for long. Because of the limited sizes of the Lanai Party Suite (1114) and the presidential suite Æleen was in (1012), we co-located the party between them. Both halves of the party were fairly low-attendance, so I skipped out on bartending to grab some port from downstairs. The Fonseca was crap, and the Taylor-Fladgate's cork had self-destructed in the bottle. Greg Rose and I managed to decant and filter it through paper towels into an unused clean coffee pot to rescue it, and it wasn't bad. (I can hear the cries of the purists among you. It was either cork floaters or filtered decanted. It didn't suck.) I was social between the two halves until about midnight.

Saturday November 1

The flight home was scheduled for 8:40am (and rescheduled by the airline, without notification that I received in time, to 8:35am). Ugh. Woke up a bit before 5:30, finished packing, checked out,10 and met up with Doug Hughes and James Clark (his successor at Auburn) to share a taxi to the airport. Got through check-in and Security with a minimum of hassle (letting a woman in a wheelchair cut in front of me without complaint) and set up shop near a power outlet near my gate. Managed to work on this trip report (especially Thursday, Friday, and Saturday) before William and Aaron showed up. Chatted with them for a while before boarding my eastbound flight to Boston.

The flight itself was pretty empty. I was expecting a delayed meal and a lunch service, but we got a breakfast service (choice of cinnamon French toast or cereal, with canteloupe and yogurt as accompaniments). I will admit even though carping about the terrible food service on airlines, I was impressed with the reasonable quality of coach-class food this trip. The Sunday dinner service (arroz con pollo) was spicy enough to have a little kick without overpowering the wary flying public, and the cinnamon French toast was neither stale nor soggy. The vegetables (salad) and fruit (breakfast) were fresh. Considering these are mass-produced and prepared in advance to heat and serve aboard aircraft not known for their kitchen equipment, I was pleasantly surprised at the high quality. Public kudos to American Airlines (or more accurately, to whoever they've subcontracted the food service out to).

According to the pilot in flight, we had a 100-knot tailwind and were planning on arriving 45 minutes to an hour early in Boston. We got a little bit of chop now and then, but it was a pretty smooth flight overall. (Screamingly unhappy 4-month old baby across the aisle notwithstanding.)

0   This was the first time I've ever lent out this car to someone else. The only other time I'd lent out my car, ever, was when I moved from Michigan to Texas and the moving company sent the wrong truck to load up my belongings. They wound up driving my brand new, only 45 miles on it, car to their warehouse to load it on the right truck. It arrived, 4 days late, at 1am Wednesday, July 4th, 1990, with some $1,100 in damage to it from mishandling.
1   With the overlaying scent of mesquite barbecue and the high smoke content of the air, not to mention the ash on every surface outside and the fire being only 5 miles from the hotel grounds, the smoking or nonsmoking distinction is going to be pretty meaningless this week.
2   It's just as well that an IRC friend wasn't coming in tonight after all, else I'd have had nowhere for him to sleep either.
3   I still had the optional duties involving the Quiz Show preparation — distributing, collecting, and grading the qualifying questionnaire, selecting contestants and alternates, writing questions, running through the questions and answers, and helping with the setup and tear down of the Quiz Show itself — which tended historically to take up most of my Thursday and Friday time. We had enough help to let me get out of this for the most part; see Thursday and Friday's entries.
4   This is the set of changes we've had ready to roll out for the past two weeks and been unable to do so since Rob wanted to make a big splash with it at the conference itself. So JD and I got to jump through extra hoops and deal with website changes and subsequent QA testing while at the conference itself.
5   I first wore this costume with pockets full of condoms and called myself "Doctor Love," passing out said prophylactics to all and sundry, back in college.
6   When I wore this last year, over dress shoes and socks instead of sneakers, one of the neighbor children asked me if I was a real doctor. My reply was. "Tonight I am." I was amused. His parents were amused.
7   Dan Klein, the photographer, autographed and provided the picture. The sun did not autograph the picture. It's a cool picture and available online at
8   As opposed to Fabulous Nate or Young Nate, who are two other Nates. The joys of telling the players without a scorecard.
9   For those who don't know, it's a leather body harness, leather jockstrap, and black jeans, flagging gray right. This year I added the leather collar as an accent piece. Nobody took advantage of me, er, the accoutrements, though. Pity.
10   I paid in full and plan on providing the receipt to Philip so he can get his portion of the room reimbursed and then mail me a check, rather than trying to add his name to the folio. Given the events of Sunday when I arrived, I didn't think the risk of their screwing it up completely was worth trying to get his name on the account and splitting the charges.

Back to my conference reports page
Back to my professional organizations page
Back to my work page
Back to my home page

Last update Apr15/04 by Josh Simon (<>).