The LISA ‘05 Conference Blog

I’m mindful of the promise I made when I started this blog to reveal how things really work behind the scenes at the LISA conference. Here’s another little tidbit in that category.

Each program chair gets to choose her or his method for a) keeping track of all of the kerjillions of details associated with planning the program and b) coordinating all the communication that has to take place between the various volunteers during the planning process.

I knew email alone wasn’t going to cut it. Sure, we exchanged tons of it (and continue to keep port 25 hot) but email doesn’t provide an easy repository of information for the entire set of people involved to reference/edit. A collection of email also doesn’t do a good job of representing the current state at any one time (e.g. a concise summary of what was decided on a particular subject). We really needed a canonical data store and rendezvous point for all of the activity taking place.

I chose to put up a Wiki for the organizers to use because it seemed to be a perfect application for that model. Originally I planned to use MediaWiki because that’s what Wikipedia uses but a test instance ran way too slow for me when I tried it. (This speed problem, I’m certain, was a reflection of the particular way I had it set up and has nothing to do with the software itself). At that point I found TWiki and decided to use it instead because it looked fairly mature and my Perl background would be helpful if I needed to hack/debug it.

In general we used a stock version of the production release with only a few small tweaks beyond the normal configuration work:

• added the calendar plugin
• added the mailer contrib plugin (for better notification of changed topics)
• added the EditTable plugin(for a reason you’ll see in one sec)

So how well has this worked for us? Here’s my impressions as program chair, other conference organizers who were in the process may have different opinions:

Bad: several of the committee members despise wikis (for reasons that I never quite investigated), but they managed to use it through gritted teeth just to humor me. I thank them for this, though I wasn’t happy using a method that other people didn’t like. (On the flip side, several committee members really dig wikis, so perhaps it all evened out?).

Less good: As an experiment, I tried using the wiki as a annotation tool. I spent a bunch of time converting the results of the 2004 LISA attendees survey into wiki format, commented the heck out of it, and then asked other people to add their own annotations. The wiki was not the ideal tool for this, probably because its (intentional) lack of tools to deal with structured content. Ideally you’d like to be able to freeze the source document and just let people freely edit comments on it. I’m sure I could have hacked TWiki into doing a better job of this ala QuickTopic Document review, but I didn’t spend the time. [As an aside: yes, we really do pay attention to the results we get back from the survey we ask attendees to fill out. I spent a lot of time pouring over it to help me understand ways to improve this year’s conference. Please fill out the 2005 version so my successor also has that kind of data.]

Less good: The wiki got used at times as a discussion/conversation forum about various topics. People would leave comments on the content (sometimes interleaved into the content itself) or in response to other people’s comments. While this worked ok, my experience was this tended to get unwieldy or just graphically unpleasant. Perhaps this was just bad typographical conventions on our part, but my experience with wikis in general suggests they don’t function nearly as well as threaded discussion forums do for this purpose. We never got to the point where one had to perform archaeological forays to find things (as one does sometimes on TWiki.org) but that was a likely endpoint. I took a very strong hand at times and shuffled off past discussion to separate wiki pages whenever it seemed appropriate.

Good: Besides the issue with discussions mentioned in the last item, the wiki functioned well for general brainstorming (e.g. where people contribute to a list of items).

Very good: The wiki functioned well in two cases: scheduling and status reporting. I find the process (especially with people this busy) of scheduling meetings and conference calls via email to be pretty painful. It worked very well to ask everyone to go to a wiki page and edit a pre-made page where they could indicate their availability. Yes, this could have been done with either polling software or calendar software, but this was a quick and easy (for me) solution that didn’t require a bunch of round-trip email transactions. Similarly, we used the EditTable plugin mentioned above to allow people to easily report back the status of papers as they moved through the shepherding process.

Very good: (all of the stuff regarding ad-hoc content creation/editing wikis are good for)

I had thoughts about bringing in other cool web-based planning software (e.g. Basecamp) into the picture, but I ran out of time to deal with the infrastructure (vs. just planning the conference). Still, I’m pretty happy with what we did use and I plan to apply the knowledge I gained from working with the software to my day job.

Speaking of amazing teacher opportunities, Richard Bejtlich, formerly of Foundstone and well known for his network security blog, is offering a training class on using the cornucopia of open source security tools for network security monitoring. There are a tremendous number of tools now available in this space which makes keeping up with them tricky. Having someone like Bejtlich tell you which ones to pay attention to and how to best use them is a superb jumpstart.

More info on the training class here. Register for the conference and this class here.

If you want to know how the tech behind-the-scenes at LISA works, the right guy to talk to is Tony Del Porto, the USENIX system administrator and conference network administrator. He’s the laid-back, ultra-capable guy you see moving at warp speed during the conference keeping everything running.

I was curious about what it took to provide a network for a conference full of sys and netadmins so I asked Tony to describe the setup he uses for LISA. Here’s what Tony, the sysadmin’s sysadmin, wrote back:

David asked me to talk a little about the USENIX conference LAN. I’ve tried to limit the following to the bits that are somewhat unique to a conference network, and LISA at the Town and Country specifically though most of it applies to every USENIX conference LAN.

The most crucial bit of the LISA ‘05 conference LAN is the internet connection, without which there really isn’t much point in having a network. Attendees used to corporate LANs or cable modems don’t think twice about downloading ISOs or pulling large chunks of code from CVS while at a conference, so having plenty of bandwidth is an obvious primary concern. ISP contracts being what they are, USENIX can’t order up a T3 for a week, or even a month, so we’re largely reliant on what the venue has to offer. The T&C has a shiny new T3 which is wonderful compared to the T1 we’ve used in previous years.

Second to the connection is the site network infrastructure and how much leave I have to use and alter it. The T&C is a property (meeting planners call hotels “properties”) USENIX has visited many, many times and, unlike some properties I’ll not mention, is very accommodating in granting access to its infrastructure. The T&C’s ethernet isn’t great, but isn’t non-existent either. There are always challenges in making a network that is designed to work a certain way work the way I need it to. Most of the resolutions to those challenges involve me on my hands and knees taping down several hundred feet of CAT5. Don’t walk barefoot at a conference. Trust me. The T&C requires three such runs of cable to work around the way the room the router sits in is wired. Why not move the router to some central location you ask? Access. The main wiring closet of the hotel is in a locked cage that only a few people have unrestricted access to, and I don’t number amoung them.

A bit on the hardware and software I use. The “router” for the conference LAN in past years has been an 800 PIII Mhz Dell laptop with three interfaces running OpenBSD. A little over a year ago I discovered the hard way that PCMCIA cards are pretty limited in the amount of traffic they can handle. Thus the current “router” is a 700Mhz PIII desktop with a gigabit interface for the conference LAN and a four port Soekris card for the internet connection and registration LAN. A note on the Soekris card: it buffer underruns under load. I have a cron job that ifconfigs the active interfaces up and down every five minutes. The next conference router will not have a four port Soekris card.

The “router” runs the usual collection of network software: Bind 9, ISC dhcpd (the OpenBSD version), Squid, and an ftp proxy. NAT, packet filtering and redirection is done by OpenBSD’s packet filter, PF. A laptop provides a second dns server for the network and doubles as a router and firewall for hands-on security training classroom. The Squid proxy has been voluntary at past conferences but became transparent for our Security conference. 400 people on a 1.1Mbit DSL line without caching is not pretty.

Wireless hardware is a collection of four old Aironet 4800 series access points, five Cisco 1200 series access points, and an Airport Base Station for small isolated meetings. The T&C presents more of a challenge than most venues because it is so spread out, thus requiring more hardware than any other property. The LISA conference format recently changed such that training and technical sessions happen on the same days which requires additional hardware. In short, I need more access points for LISA at the T&C than any other USENIX conference, and I don’t have them. I’ve tried using borrowed SOHO access points, but they fall apart with more than about 10 active connections. The Ciscos can handle 40 to 50 active connections on a single power outlet and ethernet connection. If you have spare Cisco gear laying idle you’d like to lend to the network please let me know.

What I do is based on the work of many others, my own experience, and the suggestions of attendees at each USENIX conference. LISA is the most challenging USENIX conference as its attendees use the most bandwidth, use “security evaluation” tools the most, and have the highest percentage of laptop usage. LISA is also the USENIX conference I learn the most at, and have the most fun at. This year a network team is forming to provide additional services on the Conference LAN. If you are interested in helping or have an idea for a service to provide, please send e-mail to wireless at usenix dot org.

Casey (of the super-cool USENIX production staff) informs me that the official web site for the conference, complete with registration is now live at: http://www.usenix.org/events/lisa05/.

Be sure to check back periodically (and watch this blog) because we’ll be adding more stuff to the site as the conference continues to gel. But in the meantime if you want to jump on the early registration discount, now’s a great time to do it!

And…we’re back. With that icky vacation thing out of the way I can get back to keeping you in the thick of LISA 2005. Have a number of things queued to tell you about, will post them as soon as I can. In the meantime, comments have been turned back on, so have at it.

P.S. Note to self:

Dear Self,

Maybe that vacation thing without the computer thing wasn’t so bad. Probably should try to replicate the experiment in another few years to be sure this wasn’t a fluke.

For the first time in at least 5 years (perhaps more), I’m going to be taking a one week vacation during which I’ll be completely off the net. I’m going to try to avoid anything computer-like. If I start to have too many withdrawal symptoms I suspect I’ll find myself constructing a turing machine simulator out of seashells and kelp.

Ok, sorry, back to the point. I’m going to turn off comments on this blog for the duration of my absence just because I won’t be around to keep an eye on things. Please save up your pith for my return when we’ll resume our usual program.

P.S. For those of you who are curious, the new word I used in this post was vacation. Be sure to go to that page and click on the link to have the word read to you if you haven’t heard it spoken in a long time.

Welcome to the LISA 2005 conference blog. My name is David N. Blank-Edelman and I’ll be your server host for this blog. Let me tell you a little bit about this blog and myself as a means of introduction.

As program chair for LISA 2005, my main goal is to help make all of the facets of the conference as stupendous as possible. There’s a lot of work that goes on behind the scenes to make this happen. If there’s anything sysadmins love, it’s knowing how things work behind the scenes so I thought I’d start this blog. Along the way, you’ll get a preview of some of the fabulous stuff we have planned for the conference. If you haven’t decided to attend LISA, the biggest systems administration conference in the world, maybe this blog will give you a few good reasons to join us in San Diego in December.

I plan to try and cajole some of the other hard-working people associated with the conference to post here (like the program committee members and maybe the USENIX staff), but in the meantime you are stuck with my purple prose. For as long as I can keep the comment spam no-goodniks at bay, comments will be allowed on this blog so pretty please register and take part in this experiment. You are also always welcome to email me at lisa05chair@usenix.org.

Let me end this first post with a quick disclaimer: this blog (and the domain lisaconference.org) springs forth from my head alone. It is not an official mouthpiece of SAGE, USENIX or any of the other conference sponsors. It will be more fun that way.