Concepts in the Paradox

Answer: A $40 citation.

As one of my co-workers said to me, “They’ll get you sometime man, they’ll get you, I promise. A car in the city? They’ll find a way to nail you.”

I gave my typical defensive reply, “Yeah right, I’m more on the ball than that. I can beat the man. I’m not going to get a street sweeping ticket. I read the signs. I move my car in time.”

What did I find on my car this morning? A $40 citation from the city of San Francisco Department of Parking and Traffic, labeled TC37C STREET CLEANING VIOLATION. Woops. I thought that sign said “Friday, 12noon - 2pm”, but it really said “Wednesday, 12noon - 2pm”.

It would seem that the city has given me a little new year’s gift of it’s own for my absent mindedness.

Tonight I decided to go on a “San Francisco adventure” and give yoga a try. On Second and Howard there is a place called Funky Door Yoga. It is heated yoga, which means that all 26 positions are done in a 105-110F room. The purpose of the heat is to help clean out the toxins in the system as well as make the muscles easier to stretch.

Upon walking into FDY, I was asked to take off my shoes and sign up for a membership. The employees were friendly, even suspiciously so. As it would turn out, the vast majority of the people that work there, mostly female it would seem, are volunteers. No volunteer is turned away, and they get free yoga in exchange for their services. The result is a facility that is friendly and clean, a far cry from 24 Hour Fitness.

I’ve always thought that these guys are girls doing yoga were hardly doing anything stressful or complicated. I would like to take this paragraph to say that I was very, very wrong. After only the second or third position, I was not feeling very well. Call it being unhealthy, blame the heat, or eating beforehand, but yoga is really challenging. I had to sit out numerous times to let the nausea wear off.

About halfway through the 90 minute session, I stopped noticing the heat and was able to actually get into what I was told to do. I watched what others did, had the instructor show me what I did wrong, and I started to get into this whole yoga thing. I started to get it, instead of thinking “Why the hell do I put myself through this torture?”. I think everyone knows why I put myself through torture, but that is a different topic.

I’m sore, spacey, and tired now, but in all I think it was a worthwhile experience. I’m going to try yoga a few more times, I see the benefits that it can give me, and sweating it out makes it all worth it by feeling so much better after the session is over.

I fell for a Sprint salesman Jedi mind trick today at the Market St. Sprint store. I’ve been lusting over Motorola’s RAZR for a few weeks now, but my carrier, Sprint, doesn’t offer that phone. They instead offer the very comparable Samsung a900. EV-DO network access, small form factor, bluetooth. Perfect, but no Sprint store in town had them in stock. I called everywhere.

Finally, I called the Market St. store and asked if they had a demo I could play with. The answer was yes, so I headed straight there after work. The demo was hidden in the back, not on the floor, and the sales drone hovered over me as I pushed the buttons with glee and test drove all the features.

“I want this phone.” I told the drone.
“Are you a current Sprint customer?”
“Yes, I have a Treo 600 and I hate it.” I answered.
“Oh, well the a900 is a really popular phone and it just came out. We don’t have any in stock.”
“That’s a shame, could you give me a quote on how much it would be and when I can get it?”

Clearly, I was serious about the phone.

“You know, if you ordered another line to your existing plan, I could call another store and see if they had any in stock.” the salesman said, oh so slightly showing his hand to me. I didn’t pick up his hint, but I knew what I had come to buy and I wanted now.

“Yeah sure, whatever. I want this phone.” I grumbled out.

And then like magic, his manager appeared out of the back room with an a900 with my name on it. This was a “business account phone”, that’s why they had “just one left”.

This is a typical ploy from Sprint sales drones. They only want to sell these sexy new phones to new customers, because that’s were the money is at: Two year contracts, activation fees, and insurance policies.

Never mind all that though. With much enjoyment in my heart, I carried the phone out of the store and I’ve been playing with it ever since. It does almost everything I want. I paired it with my Powerbook and my Palm TX, and when I go to surf or ssh, the phone blinks for a second, and then I have ~250ms latency with a 700k/s transfer rate to either computer. What doesn’t work is the dialer in the Palm, which is very annoying since my contacts are on the Palm. To make matters worse, iSync and OnSync both don’t support my phone, so I’ll either have to enter contacts manually or export and manually upload the contacts data to my phone.

I figure this will be fixed in a few weeks, and other than these few technical problems, I think this is the best phone I’ve had since the Treo600. Recommended.

While waiting for my late flight to Oakland and on the flight itself, I spent time blitzing through the Time Management book I picked up at LISA last week. I think meeting the author of the book helped inspire confidence in the ideas he has presented in his book. I’m about halfway through this 210 page book, and I’ve already started to implement some of his suggestions in my daily life.

The problem with being a system admin, and in my personal life as well, is that I have a flood of projects I need or want to do, family matters that have to be attended to, yet still manage to keep a social life. My apartment is a mess, I’m unorganized, and it isn’t a problem that I’ve spent much time fixing. Routine is something I try to keep, but during the next few weeks of shopping for gifts, holiday parties, and family get-togethers, I’m likely to fall apart.

So far I’ve adopted establishing a routine. The first is to start using my PDA, the Treo 600. Yes, I use it as a phone, but I almost never use the calendar or todo list. I’m going to get Missing Sync so I can keep a mirror-copy of what I keep on my Mac when it comes to scheduling. Second, I’m going to put events in the calendar as soon as I commit to them, and also check to make sure I haven’t double booked.

I’ve already constructed a todo list, based on what needs to be done today, what can be deferred, and what can be done over time. Did everyone notice I called them back today? I made and kept an “A” todo list. I might actually get to bed at a reasonable hour tonight; I set a warning alarm to tell me to snap out of my online daze about bed time.

The ultimate plan is to unload all the ideas in my head as soon as they occur. If I need to write about it, I’ll write it down and get it out of my head. Instead of keeping my schedule in the precious little space left in my frontal lobe, I’ll put it in the PDA. I’ll track my short term and life goals as todos that have to be moved from day to day. I’ll schedule time to read, research, and stave off zoning out. The Treo can do all the hard work, and being synced to my computer will keep data from being lost. Failing that, I’ll get a PAA: Personal Analog Assistant. Otherwise known as a datebook.

Days 5 and 6 of LISA flew by in a flurry of refereed papers, invited talks, meeting peers, and BoF sessions. For Thursday, I attended Modern Trends in UNIX and Linux Infrastructure Management, followed by Automatic PC Desktop Management with Virtualization Technology. Both talks I was quite interested in, but both fell flat on my expectations. The first invited talk went through common problems with scaling; the same thing that has been iterated in the training series. For the second talk, it was basically VMware player as a portable network client. That is great and all, but VMware is slow, and this was mainly for Windows. No one even touched on license issues.

Not all was lost though, after noon, when the sharp geeks finally rise from their night of drinking, the real fun began. During the lunch break, Dan Kaminsky gave another one of his awe-inspiring talks: Network Black Ops: Extracting Unexpected Functionality from Existing Networks. A little history in case you don’t know Dan. He has a database that he has collected of every DNS server on the Internet, and some not even on the Internet. He has a lot of bandwidth, and he likes to mis-use the Internet standards and bend them to some new and very interesting ways. In short, he covered in detail: bypassing IDS’s, using IPS’s to attack the host, redirecting traffic for entire sites, and of course video-over-DNS. Don’t worry, he’s the good guy.

On the note of security, Lance Cottrell, famous for Anonymizer, gave a chilling talk on Internet Counter-Intelligence: Offense and Defense. In this presentation, Lance outlined how personal security is violated via logs, how online stores adjust prices based on your history and loyalty, along with your country. On top of that, he explained how companies can use their logs to track how competition has been visiting their site, detect a hostile take over, product launch, or people jumping ship, to gain an edge in the market. Before I cut out early, he spoke about the political implications of logs, and how in some countries they are used to hunt down and murder anonymous political opponents on message boards. And you thought trolling was fun?

So what was the talk I cut out of such a spooky talk for? Tom Limoncelli’s Time Management guru session. On the way to the session, I stopped by the O’Reilly booth and picked up a copy of his new book, aptly titled Time Management for System Administrators so I could get it signed. Tom gave an entire tutorial on Time Management earlier in the week, and now I regret missing out on it. He presented techniques on when it is appropriate to buy a solution, script a solution, do it by hand, or delicate it to someone else. He also hammered in the idea of a Wiki for documentation, something that I wish more people would use.

On Friday, I went to the Production Change Management: To Each, His or Her Own tutorial. I know this sounds like a very dry topic, and why would I subject myself to such torment? Well, I see in the next phase of my career a lot of managerial wrangling along-side some very important and very publicly visible servers. That is, someone might die if I don’t get it right, and I want to know how to deal with this in the face of political issues along with issues with developers and users. That said, it was a really good session, and gave me some sound insights on how to not only progress into a dangerous maintenance problem, but also how to prepare for the fallout: including what to do about Murphy’s law.

It is worth mentioning a little discussed but very important part about LISA: The Hallway Track. This isn’t an organized speaker, nor is it a paper presentation or a BoF. It is stopping people in the hall, at lunch, at dinner, or just sitting drinking coffee, and talking to them. During the hallway track I met people from the Stanford Linear Accelerator, GaTech, MIT, Google, Match.com, Sun, Vmware, XenSource, Austria, Australia, Germany, Texas, New York City, and San Jose, to name a few. Some people were famous and I didn’t even know it until later, others gave me new ideas and viewpoints that I don’t know where else I would have gotten.

As David, a LISA attendee who happened to be on the plane back from LISA said to me, “I’ve been going to LISA for a decade, and I don’t know where else you can learn new skills in the industry like this.”

And I couldn’t agree more.

It has been an interesting few days at the LISA conference in San Diego. I learned more than I will probably ever use with CFengine in the Advanced Topics in Host Configuration and Maintenance with Cfengine talk, followed in the afternoon by the very informative Solaris 10 Security Features Workshop. I feel as though I actually could comfortably start using Solaris 10 and containers, something that I’ve been meaning to get around to.

Today was a general track day. I went to a bunch of refereed papers talks, then I drank from the LISA knowledge fire-hose that is known as “Hit the ground running” and learned way more about AFS and VOIP than will ever be useful to me.

Come nightfall on wednesday, I prepared for my Virtualization in Production BoF. I came 15 minutes early, and the room was already half-full of people interested in just virtualization. Two employees from XenSource were there, along with three or four people from VMware, ready to field questions. Come 7pm, the room was full. I expected about 10 people, but over 100 attended! It was a very informative workshop, with exchanges about VMware, Xen, and UML. About 36 people joined my virtualization mailing list, so the post BoF discussion can continue. People actually thanked me after the BoF for suggesting the talk, wondering why there were no talks on Xen during LISA: the one place perfect for such a topic.

After the BoF, I had a great conversation with a Sun N1 developer about his project. I find it very interesting that Sun is using Xen in their grid product, showing that Sun really does know the line between containers and virtualization when it comes to large computational tasks. I’m going to do a bit of digging around between the refereed talks I went to and the N1 discussion I had: I think I may have a new idea for a thesis project.

Monday seems to be the day when people arrive. The wireless is up and down, and the diet coke flies off the snack trays during the breaks. I think I had a little bit to drink whiel socializing with attendees at the bar last night; my hang over from last night was a little pounding, but learning new things tends to wake me up a bit.

Today I went to two separate tutorials. The first talk was on Security without Firewalls, by the same speaker from yesterday, Abe Singer. In this talk, Abe made the argument that firewalls are often unnecessary and are really fallout from managerial peer pressure. Who needs firewalls if your network hosts are secure?

The tutorial went on about how you can secure your networks, like using IP entries in your exports file for NFS, using CFengine for change management, and building model hosts for images. Where I felt the talk fell flat was where what I felt were ancient administrative techniques were used, like compiling packages from source (what about vendor patches to ssh?) and using an a read-only NFS server to distribution system binaries (single point of failure, anyone?).

The second talk was part one of two on CFengine entitled Introduction to Host Configuration and Maintenance with Cfengine by the software’s author, Mark Burgess. The talk was spectacular, and answered many of my questions on how the hell CFengine works. It seems like a powerful tool, and I can’t wait to get back into the lab and try it out. Tomorrow’s advanced talk sounds even more interesting.

During the CFengine break, I lucked out and ran into Star from XenSource, and we had a long discussion on what is going on with Xen, and where I could find community resources on it. It was enough for me to post my first BoF: this Wednesday at 7pm. I hope enough people come so I can get more input on creative ways to deploy virtualization into production. I’ve lead a group of cats before when I ran RLUG, so I’m use I can do it with Lisa geeks.

Not many people are here yet, so the conference hall seems a bit empty and low key. I went to the Building a Logging Infrastructure and Log Analysis for Security tutorial presented by Abe Singer from SDSC. The key things I brought back from this tutorial were:

1. Don’t expect to be able to parse your logs with Regex

2. Programmers create uninformative error log messages
3. Log hosts require a lot of resources
4. Everyone uses syslog/syslog-ng, even thought alternatives should be considered
5. With udp, you may not get all your logs in even ideal circumstances. tcp may exhaust a log host’s resources

I have some new tools under my belt, and when I return next week I can start putting the pieces together to redo work’s logging infrastructure.

I’ve posted a note on the message board looking for people who are using Xen or have deployed it into production. I hope I get some calls or emails from people, with the off-chance I can do a BoF on it. We shall see.

I’m off to LISA’05 in San Diego, but before I leave I wanted to share some amusing photos I came across from EuroBSDCon. Undeadly pointed out how Ryan McBride demonstrated the ability of OpenBSD’s CARP to stand up against hackers during his Building Robust Firewalls with OpenBSD and PF talk.

With an axe!

And the network stayed alive…

To the airport! I’ll be reporting from LISA anything interesting I come across.