Warhorn was offline for about 4 hours this morning. The outage was caused by the server’s disk filling up as a result of 1) last night’s automated backup and 2) a proliferation of old log files. Because we don’t have any real alerting systems in place, we didn’t notice the outage until users started sending complaints in email. Once we were aware of the situation, we made some space on the disk and got the service running again.

Takeaways from this incident:

1. We need real alerting systems in place to let us know as soon as some part of the system fails. It is unacceptable for hours to go by before we discover that something’s wrong. We have two options: set up some basic monitoring and alerting tools, or pay some third party to handle monitoring and alerting for us. Given my lack of free time and experience with managing that kind of thing by hand, we’d prefer the second option, but that depends on you folks giving a little more money a little more often than happens now.
2. Backups need to be stored somewhere other than on the server. This is already in place; I set it up just after this morning’s incident. ~10G of space has been reclaimed, and our hosting provider’s automated backup system has now replaced our old system (at an additional monthly cost, which we’re crossing our fingers will be covered by donations).
3. Log files need to be pruned after some amount of time. I gotta just roll up my sleeves and make that happen; nobody’s gonna do it for me.

I think it’s pretty obvious that we’re running Warhorn on a shoestring budget. We get very little in the way of donations and therefore don’t have the budget to afford to pay other folks to help us with the system level stuff that I don’t have time and/or expertise to deal with. So if you value the service we’re providing, please consider donating a few bucks to help us get Warhorn working better. If we could bring in even just $100 a month we could afford to run this outfit in a much more professional manner Thanks very much to those of you who do support us financially, we sure do appreciate the help. And sorry for the outage – we’ll keep doing the best we can to keep this creaky old ship afloat.

Wow, it’s been a year since the last blog post. That’s just sad. Although I did give a quick update on Facebook a few days ago, I wanted to spend a little more time over the holiday weekend to share a little more about what’s been going on (or not) behind the scenes with us. You guys deserve better than a year of silence between updates.

Before I get to where we are now, let me review how we got here. As you may recall, we spent several months back in 2009 plotting and building “Warhorn 2e”. We talked to a wide variety of event organizers and long-time site users about how they used Warhorn and how it could serve them better. We began to conceive of Warhorn not only as a tool for helping organizers schedule and manage events but also as a community site and social tool for gamers. We also thought of a number of business ideas that could be layered on top of the core functionality that you’ve enjoyed over the last ten years. It was a super exciting time, but ultimately we were unable to deliver on our vision.

We never shipped 2e because we were much too ambitious given our limited resources. We chose to build a brand new site from scratch with all of the existing site’s functionality but with a wholly different interface and all new technology. There were excellent reasons for redesigning both the interface and the tech – they were both ten years old, the interface was far inferior to users’ expectations of how web sites work in the modern world, and the tech was limited in certain ways that effectively prevented the development of new features and even made bug fixing a great challenge. Unfortunately, we couldn’t afford to take enough time away from our day jobs see this project through to a point where it could be released and take the place of the existing site. It didn’t make financial sense to try to raise the money we would have needed to get to completion, and the experience kind of burned us out. That’s why you didn’t hear from us during the first few months of 2010.

After some time away from the project, we regrouped last spring and made a dramatic change to our strategy. We decided that since the “scorched earth” approach of rebuilding the tech and redesigning the interface simultaneously wasn’t possible, we’d try instead to start migrating the site to a new technology platform but keep essentially the same interface. The first major goal would be to achieve “single sign-on” – rather than having to set up a separate account for every event as you do today, you’d have one site-wide account that worked for all events, and a dashboard giving you instant access to every event you’re organizing and attending – far and away the most frequent feature request. We hoped this approach would give us a stable basis for incrementally updating the interface over time as well as solving the number one issue that was keeping us from being able to deliver truly new features. We called this project “v2″ to distinguish it in our minds from the failed 2e project.

Our progress over the next several months seemed to prove out the strategy. The v2 rebuild proceeded very well, and we also were able to make some small but really nice improvements to the existing version of the site, including targeted redesigns of the home page and event listings. However, this project also eventually stalled toward the end of the year, as evidenced by the blog post previous to this one being date September 18, 2010.

Even though the v2 project was less ambitious than 2e, I think the fact that we weren’t able to take time off from our day jobs to work on Warhorn really killed our momentum over the months. There was a time in my life that building software and creating web sites was all I wanted to do, and I was happy to work a 10 hour day for somebody else and then come home and put more hours into my hobby projects – that’s how the original version of Warhorn was built. But these days I have a lot of other interests, and frankly I have less energy to code all day and then come home and code some more on nights and weekends; most of the time I’d rather go see a movie or a comedy show or hike up a mountain or spend time with friends. And Finn feels the same way. Consequently, the enjoyment we got from working on the project diminished over the months, as did our progress.

On top of that, I wound up moving back to San Francisco this spring to become the first employee of a brand new company, which as you can imagine requires as much commitment and energy as one can possibly give to their profession, further intensifying my need to get away from the keyboard whenever possible and stifling even the minimal progress we had been making on v2.

So, where do we stand today? The v2 project is actually mostly done. Single sign-on is functionally complete (albeit only tested as well as Finn and I can – we’d need lots of other users helping us bang on it before we’d feel comfortable rolling it out to everyone). The site is still missing the core schedule and signup functionality, and organizers can’t yet manage games, but that’s actually only 10-15% of the functionality of the site (if you’ve ever set up an event site on the live Warhorn site, you know that there are a lot of tools for managing the event that regular users never see, and single sign-on required a lot of new user features – dashboard, account management and event registration management). So with some sustained effort, maybe as little as two to three weeks of full-time work, I think we could get over the hump.

That said, as things stand now, we’re not likely to have that last 10-15% ready anytime soon. The factors that caused our progress to slow to a slow crawl are still in play. I’m working 60 hours on my startup every week, Finn has lots of balls in the air, and we’re both trying to have lives outside of that. I do get the urge to spend some weekend time hacking on v2 every once in a while, but the fact of the matter is, unless something dramatic happens in my professional life, the status of the v2 project is unlikely to change.

Now, if I somehow find myself unemployed, it’s certainly possible that I may try to raise enough money with a Kickstarter campaign to fund the completion of the v2 project. We’ve also kicked around the idea of releasing the v2 code under an open source license and/or setting up a nonprofit to allow more people to contribute to the operation and ongoing development of Warhorn. So there are a number of longer term options. We’d love to hear your thoughts and ideas.

And just to be clear, we are absolutely dedicated to keeping the current site running, and we’re happy to add new game systems and campaigns whenever you guys want them. The DNS hiccup this week was my fault – I knew that we needed to migrate from the old provider to the new one a long time ago (although in my defense the last communication I received on that topic was in March, and I got no heads up about the planned outages that we experienced). In any event, rest assured that the Warhorn site you’ve been using for ten years isn’t going away.

So there you go. The more things change, the more they stay the same. I know I’ve promised to communicate more frequently before, but I don’t seem to be very good at that when I’m not putting in the effort on the coding side. I’m thinking maybe shorter updates over on Facebook may be a more sustainable practice – we’ll see. Definitely make sure to check in over there from time to time as well as here. And as always, let us know what you’re thinking, good or bad.

Thanks, and good gaming!

Brian

Attention graphic designers: we’re looking for some folks to work with us on a few t-shirt projects. If you’re interested email us at info@warhorn.net for details. Thanks!

Just wanted to take a moment to highlight a really nice Warhorn integration over at tampadnd.org, a web site for a group of Living Forgotten Realms players in Tampa, FL. The information in the right column’s “upcoming games” section comes directly from their Warhorn site by way of its news feed – when a game is added to the Warhorn schedule, it automatically shows up in the tampadnd site too. Great stuff! Kudos to Scott Sevener for his work on the site.

With a little programming, or your publishing platform’s feed widget, you can juice up your site with game listings too. Drop us a line at info@warhorn.net if you want to learn more. Have fun!

This post is coming to you from lovely Seattle, WA, where some of our team will be spending the weekend at PAX 2010. If you’re at the show, give a shout on Twitter (@warhornllc) or through email if you want to get together for a game or just shoot the breeze, and keep an eye on Twitter to see what we’re up to.

As we’ve mentioned before, we’ve been working on a new look for the main site for the past several weeks. It recently occurred to us that our trip to PAX is the perfect excuse to launch the new look, so we burned the midnight oil all week long, and the result is now live. Check out the site if you haven’t already and let us know what you think. If you like what you see, click on over to the sponsorship page and consider donating to the cause.

You’re almost certainly wondering what’s next on the horizon. Well, we’re putting the finishing touches on our most ambitious feature to date: single sign-on. When we roll it out, you’ll be able to log in once and register for any and all event sites using a single central Warhorn account. Among other benefits, you’ll have a dashboard showing all of the events for which you’re registered and those you’re organizing. The process of registering for an event and paying registration fees will be streamlined and easier to understand. This will remove one of the biggest pain points consistently underlined by our users – the need to set up a new account for each event site, all of which are totally disconnected from each other. We’re super stoked to be getting ready to roll this thing out and have just a few bits and bobs left to finish up, so look for it in the near future!

We added “Shroud of the Ancients RPG” to the game system list. Enjoy!

We added Living Traveller to the campaign list. See you in the Spinward Marches!

We’ve added Shattered Empires RPG to the list of game systems and Chronicles of the Shattered Empires to the available campaigns. Let us know if there are any other new games or organized play campaigns that you’d like to have added to the system. Have fun with the new campaign!

As I mentioned on Facebook a while back, we’ve had our heads down on a partial redesign of a couple key parts of the main site. It only took a few weeks longer than expected, mainly due to a barrage of bugs in some of the open source software we use, but here we are!

Home page: We’ve gotten some feedback from non-users that the home page wasn’t particularly clear about how Warhorn could be helpful to them. As a short-term measure, we added some text that attempts to hit the highlights for organizers and attendees. We also show the events that are occurring this week so you can click around to see some timely examples of event sites. This is a temporary redesign of the home page; we’ll be doing a complete overhaul later this year, so if you have suggestions, let us know.

Event listings: The listings have been completely rewritten to:

• Show only 25 listings per page so that you don’t have to wait for a giant page with thousands of listings to load
• Filter listings by location so that you can quickly find the events happening in your city, state or country
• Sort listings by title, location or date
• Update the listings table in-place when you’re paging, sorting or filtering so that the site responds faster

We’ve also made it possible for you to specify whether or not your event should be visible in the listings when you request your site. Finally, we’ve moved the blog to its own domain – be sure to update your bookmarks and RSS subscriptions.

We’re super excited to make this update because it marks the first time we’ve put 2e code into production. Originally we were going to wait and ship a completely new, totally rewritten 2e site, but as you can probably tell, life has continually gotten in the way for both Finn and me, and we haven’t been able to make progress as quickly as we’d have liked. We decided recently that instead of making you guys wait forever for vaporware, we’re going to start giving you 2e features incrementally so that you can take advantage of the improvements we’ve been working on for the last year and so that we can get some real experience operating the new software.

I’m also stoked to announce that the next feature we’re working on is hands down the most-requested feature of all: single sign-on! No more setting up an account for each event site – you’ll have one central Warhorn account that will give you access to every single event. We’re still designing this feature and thinking through how migrating everyone to the new central accounts is going to work, but we’ll have more information available soon.

We’re also planning to update the look of the site in the near future. We’ve been living with this temporary look for too long, not to mention the fact that the event site design hasn’t changed in ten years! Again, we’ll let you know more when we’re closer to launching the new look.

Once these projects are done, we’ll have a great base to start bringing over the 2e features that we built last summer and fall. I hope that by the end of the year the site will hardly resemble its 2009 incarnation.

Anyway, let us know what you think about the new stuff. Thanks!

Made a small change to the calendar view to show the current month by default. Not sure why it didn’t work that way already – I know I changed how it worked at least once before, but it was so long ago I’ve forgotten why! Anyway, let us know if there are any issues. Thanks to Todd Morgan for the poke.