Comments on: What happened today…

By: Rod Drury

Rod Drury — Tue, 11 Jan 2011 22:13:43 +0000

Joshua, we believe we are best in class with our redundant technology infrastructure.

See http://www.xero.com/accounting-software/compare/

Some googling will show you that QuickBooks Online has a far from impressive uptime performance.

Rod

By: Joshua Milne

Joshua Milne — Tue, 11 Jan 2011 12:27:27 +0000

Understanding downtime can happen, mirrored hosting to different city/country would be a better solution with data so important to customers.

Are the backups happening to a different location via mirrored hosting or some other method?
If a city got blown up in a war where does your services run from and where can you find the offsite/country backups?

Google and other SaaS Cloud provisions have these areas covered. Can we please get your clear break down regarding this.

Quickbooks online have mirrored servers around the world to ensure uptime. If power goes out it make no difference to users stuck with no server availablity. Can you please comment on this?

Xero is the best accounting software package I have seen for small business!! keep up the great work.

By: Rod Drury

Rod Drury — Wed, 08 Jul 2009 00:47:34 +0000

Our team is small enough that those who blog know our tone of voice and to ‘blog safely’.

By: PJ

PJ — Tue, 07 Jul 2009 22:51:09 +0000

Nice example of honesty and transparency in a corporate blog. Do you have set editorial guidelines for what you post on your blog or do you go with the flow?

By: Ben Kepes

Ben Kepes — Sat, 04 Jul 2009 05:25:44 +0000

Sheryl – sounds interesting – I’d be keen to hear the backstory of the system that you use. Flick me an e on ben AT diversity DOT net DOT enzed

By: Sheryl

Sheryl — Sat, 04 Jul 2009 02:32:05 +0000

My internet accounting system (not Xero and not by my choice) goes down for maintenance regularly during the day. Despite repeated requests to not have upgrades completed during working hours, it can be quite common for it to be down for more than 30 mins at a time, often at crucial times. On top of that we do not get any warning that the system is going to be down, we start to do something and all of a sudden we get the “down for maintenance” page.
One 45min unscheduled downtime in two years sounds like complete bliss to me!

By: Rod Drury

Rod Drury — Sat, 04 Jul 2009 01:08:15 +0000

Hi Adrian. Multi-currency was our biggest release ever. So like all releases we monitor and tweak. The few issues we noticed during the week we’ve fixed but have to wait until the quietest time of the week to apply. MC was only 5 days ago so we’re onto it.

We don’t anticipate another big engine change for a period of time so this is a bit more intense than usual.

With Xero you should be outside on Sunday.

We don’t think we’re doing too bad: http://www.digitalwpc.com/News/Permalink/WPC-Connect-Outage

Cheers

Rod

By: Adrian Pearson

Adrian Pearson — Fri, 03 Jul 2009 09:31:27 +0000

I have just noticed, logging into Xero, that there is to be more server maintenance (and therefore a short downtime) this coming Sunday. The server maintenance events seem to be coming closer and more frequently. It would be nice to know more information. Knowing if it’s planned or remedial work for instance – just so we know how much or little to be worried!

By: Craig Walker

Craig Walker — Thu, 02 Jul 2009 00:54:14 +0000

@Cecil @Simon We have blogged about it before (http://blog.xero.com/2008/10/rackspace-saas-event/) and we make no secret of the fact we use Rackspace. In actual fact it’s something we’re very proud of – our partnership with Rackspace has been one of the best things we’ve ever done. I’m sorry it’s not more obvious.

As you say real-time site-to-site redundancy is actually very difficult to achieve and can lead to situations where the app is more available but data consistency is diminished. This is something we are definitely looking at though – we are continually trying to improve our processes to make Xero the best experience it can be.

By: Simon

Simon — Wed, 01 Jul 2009 23:30:24 +0000

@Cecil – it does say they’re a RackSpace premium partner on the Xero homepage, but you’re right it’s not very obvious

Xero have you considered offering a premium service involving site-to-site failover? I personally wouldn’t want to pay for it as I’m only a small business, but some of the bigger companies who can’t handle 45 minutes of downtime might be interest. I’ve set it up in the past and I know that it’s not an easy (or cheap) thing to do, but it could be something to look at.

By: Cecil

Cecil — Wed, 01 Jul 2009 23:13:12 +0000

I am not sure about how the backup process works in Rackspace services. I think Rackspace should have supposed to provide more reliable backup services as it has multiple data centers in the world. I have two questions based on this.

1. Is Xero currently solely relying on the services provided by Rackspace including backup.

2. Would it be realistic for Xero in the future to use another services provider as the backup service so that it can shorten the outage time as much as possible?

I guess the users would feel happier to bear with slower backup services when an emergency occurs rather than waiting for long outage time.

And I think it would also be good for Xero to be more transparent in providing information about its service provider in the future. This is the first time I got to know Xero is using the services from Rackspace(at least I didn’t see it in this year’s annual report, but please excuse me if I am wrong on this point). I personally think all customers and shareholders would love to know more about this.

By: Keith Patton

Keith Patton — Wed, 01 Jul 2009 08:11:00 +0000

if you host within a single data centre you can expect outages.

By: Never fear transparency | AccMan

Never fear transparency | AccMan — Tue, 30 Jun 2009 14:25:07 +0000

[...] Xero had an outage last evening UK time, 8.15am in New Zealand. The outage lasted 45 minutes and was attributed to data centre issues at Rackspace Dallas, its hosting provider. Xero isn’t alone. The Rackspace issue spread to others. As you can see from the sponsored feeds, the company kept users informed via its blog and Twitter. [...]

By: Nic Wise

Nic Wise — Tue, 30 Jun 2009 12:37:48 +0000

Nice transparent post, Ali. Good to see the transparency permeates thru all parts of the business

And yes – outages suck, but 45 mins in what? 2 years? Not exactly a bad record, esp if there is no data loss.

By: Craig Walker

Craig Walker — Tue, 30 Jun 2009 07:25:43 +0000

@Paul Yes – backups reviewed immediately as part of our consistency checks. Also took a full backup when we came back online just to make sure.

@Campbell Yes – we pride ourselves on our openness – Twitter is a great tool for that and it was also fantastic that Rackspace were blogging and tweeting through the whole thing – no point in hiding from the truth – just gets you into more trouble!

By: Campbell

Campbell — Tue, 30 Jun 2009 07:05:00 +0000

Interested to hear what Rackspace have to say. I am sure they plan for all sorts of failures and have so much redundancy it isn’t funny, yet things like this can still happen.

I have to also say having your tweets certainly saved me time in letting staff know when things were or were not online.

For your handling of a bad situation you still score top marks!

By: Paul Lattimore

Paul Lattimore — Tue, 30 Jun 2009 07:03:35 +0000

Question though… Are the backups being reviewed?

By: Paul Lattimore

Paul Lattimore — Tue, 30 Jun 2009 07:01:44 +0000

Show me an internet based entity that hasn’t had an outage. Appreciate the updates – certainly a refreshing change.