Tuesday was the Advanced Topics Workshop, ably hosted and moderated once again by Adam Moskowitz. The 25 of us went around the room doing introductions (who we are, what we do, and what problems we're facing). The introductions generated interesting questions and topics for discussions.
One interesting discussion was on the professionalization of systems administration. The comparison was made to doctors. We use similar skillsets — diagnosis, comparability, problem solving, and so on. But can it be said that lives are at stake when systems administrators do their job? Doctors charge by the visit or the procedure; systems administrators don't. The models are, however, somewhat converging in some ways. Many systems administrators do more architecture than doctors. Differences in scale: doctors are like help desks, while systems administrators sometimes deal on larger scales regarding number of people served. Patients are sometimes a bit more standardized. Doctors are, in fact, certified. Some systems administrators contravene organizational policies. Doctors are liable, lawyers are liable, engineers are liable; systems administrators are rarely liable. This led to a discussion on professions: professions have standards for training and knowledge (certification); there's a fixed set of information. Sysadmins are often grass-roots with self-training and apprenticeship. Certification is a required stepping stone. Maybe systems administration should be a "guild." Or maybe we should form a union.
Our second area of discussion was whether or not ISPs are now perceived as commodities and whether they can be run as commodities. The concensus was that they can, but your should be sure to check out their long-term business prospects because business models change rapidly. Finding a provider for services "beyond the basics" is hard. Consider NetLedger: They will run your small-business books for you on the web for a small monthly fee (personal service is free) based on number of users. These guys might not succeed. Sharing your data with them for several years might end up at a very bad end if they suddenly fold. The ISP consolidation is in progress. Any new ISP will require new technology. Not only are ISPs perceived to be commodities, so are their users (who are traded). Local and national ISPs can survive; but it's tough for regional ISPs, who are neither local nor a brand name. Are there brokers for customers? There're special deals among ISPs, but no B2B site. We think we'll move some DSL subscribers around but we see stability showing up in six months. A new technology could disrupt this. DSL was enabled by aggregating terminations at the central office. Those who can scale will survive. You can now purchase a turn-key 10-50Kuser ISP solution that requires very low levels of sysadmin skill. Shell accounts are a thing of the past; people are running their own servers at the end of a DSL line.
Our third major area of discussion was on separating policy and implementation. One possible solution is to have an interpreted "policy language." Maybe you can use general principles then color the bottom level implementation to match existing policies. This is more of a mindset problem than a coding problem. Let's build policy engines, not engineer accounting (or whatever) systems. Cfengine has features that can help you implement policies. You must codify the policy in a way that's measurable so you know if you're "on policy" or not (and then you can get back on-policy if you get far enough "off track"). We're already adapting host-based tools that query directories. Maybe we can graft policy engines onto directory responses. LDAP is insecure, though — we should address this. Microsoft thought Kerberos authorization was the big deal, not authentication. Changed Kerberos to TCP. They put policy at the Kerberos server.
Our fourth area of discussion was on how new technologies in the last few years seem to be languages. This is since languages can express extensible ideas — build from primitives and move to greater complexity. Some people say "use a database for policy" but that's hard because databases too often require predefinitions. Languages, on the other hand, are built from primitives and are infinitely extensible. We think this is the solution for policy expression. A well-crafted language could potentially address this problem, but we don't know of one right now. We think languages can express these specifications at the proper level. There are results here in Academia — see the Intrusion Detection literature. Perl6 will have the ability to make a "little language." This moves to per-application languages; specs for the perl6 sub-languages lead us to believe we could write "Authenticate all users for all machines" or somesuch. The real problem is the ability to describe when a particular operation is authorized. We need to agitate for richness-of-expression in commercial tools. Windows has a lot of configurable options under the hood that were difficult to access via the desktop or command line, even though an API was available. Declarative languages like Prolog might be able to help here. Exceptions are surely the difficult and important part of this problem.
We wrapped up in looking at our 1998 and 1999 predictions to see if we were late or still wrong. We're still batting a low average. In 1999, 9 of our 19 predictions came true (or mostly true), for a 47% success rate. More of our 1998 predictions came true in 2000, but we're still looking at about a 50% success rate.
Our 2000 predictions are:
- Peer to peer (including systems administration) will grow then shrink. (85%)
- DSL-based ISPs will grow in popularity then die as the tech-savvy turn their DSL to a friends-and-family ISP. (65%)
- Alternate dialtone-to-home competition will break loose (50% of the market) this year — telephone companies will have to change their business plans. (100%)
- The number of purported PDA- or home-appliance systems but which need management will double in the next 12 months. (75%)
- Some time this Christmas season things will go well with e-Commerce. (100%)
- There'll be an e-Commerce disaster some time in the next year, though. Or, a Fortune 100 company will have an above-the-fold e-Disaster. (40% think it'll show up in print)
- We will have at least one Microsoft-facilitated security bug a la Melissa or ILOVEYOU in the next year. (100%)
- There'll be at least one major Silicon Valley power outage that provides above-the-fold problems for at least one company. (85%)
- Many dot-coms with otherwise-profitable, viable business models will fail because their names aren't AOL or Yahoo (investor confidence will drop further). (70%)
- This is NOT the year that Silicon Valley loses its shine and people start bailing from there (mass exodus). (75%)
- 802.11b will become standard on all business desktop and laptops in the next year. (80% on desktops, 100% on laptops)
- 802.11b public Internet access will be available in the top 25 US airports by December 2001. (100%)
- A huge mobile phone will be dug up on the surface of the moon.
- You will not see networks on airplanes in 2001 (not counting dial-out via airphone). (100%)
- Businesses will find their storage (at least) doubling this year, with the concomitant backup problems. (75%)
- Linux will splinter (speciate). (10%)
- 17-inch flat panels will decline to $700 in price. (100%)
- 200 dpi resolution displays will appear on desktops. (10%)
- Gigabit ethernet hubs will dramatically decline in price in 2001. (35%)
- Serialization of Object Application Protocol (SOAP) — RPC over HTTP; aka exefuckacutable code, per Rob Kolstad — will ascend to wide acceptance. (15%)
- Official or commercial music delivery services will fail. (85%)
Finally, we listed some cool tools we're using:
- Rethinking the world in terms of PHP, MySQL, and HTTP
- Wireless everything (802.11b)
- Some of the new load management tools (batch queuing tools) for clusters
- 65-pound brute-breaker demolition hammer with its own cart
- Wiki (a very simple browser-based editing environment)
- VNC (virtual network consoles for NT)
- Baytech power strips with Ethernet access to remotely reboot via power outlet [vector for the Microsoft security outage were predicting]
- Blackberry (2-way) pager
- Unison (file synchronizer tool between desktop and laptop, 2-way comparison, etc.)
- ssh in IOS
- Netflow (Cisco-created protocol gaining acceptance) tools that've come out in the past year; you can now do accounting without trashing performance
- tangram (www.tangram-persistance.org); a perl module that makes variables to persistent storage (SQL database)
- Herman-Miller Aeron chair; its really worth it if you sit on your ass all day
- Authenticated web environment that keeps the authentication token on all the time
- Win32's NetCaptor; tabbed web-browsing web interface
- Win32's PowerMarks; bookmark manager that has keyword-based searches
- Newsbytes-style column