Whirlpool

Tassie has had no dataflow for 14 of the last 24h.

While this is an issue with Netspace's upstream provider I hope some butt kicking happens as this is unacceptable.

And i just signed a mate up for netspace a few days ago. doh!

+1 to the butt kicking...

Someone has really dropped the ball with this one.

Is this just Netspace no broadband or is every ISP the same?

Hi Patches,

So far I am only aware of NS. Frogpond seems okay.

Cheers,
Tas

Yeah, and the fact it's been down for over 12 hours straight now..

How about some compensation, ie money back for the time we didn't have access..

Thanks Tas.

I wonder who is the upstream provider that Netspace are blaming?
Why is there no backup system?
This is not good enough, I'm really cranky about this.

paraniomia writes...

Yeah, and the fact it's been down for over 12 hours straight now..

How about some compensation, ie money back for the time we didn't have access..

If you're paying $80/m for your ADSL, that's 0.11c/hour.

Yep, I have 2 x Netspace connections (home and a secondary connection at work) and have not had a solid connection on either for the last 24+ hours (modem cannot authenticate).

I also have Internode at work (the primary connection) which is connecting up fine. As far as I knew there was only one "upstream provider" out of this state?

It is not about the money. If this happens frequently, even if the down time duration may not be as lengthy as this 1 we're having, lets see what Netspace think about the trouble going through refunding its customers for the down time. I understand that the problem did not happen on Netspace's side, but I don't really care whose fault is it as a consumer. I signed up to Netspace and Netspace is the only 1 I think I'll have to deal with, not their carrier.

15 hours of down time, how is that acceptable? Not to mention those FREQUENT annoying scheduled or unscheduled down times we had a couple months back. What's the deal??

Oh, you'll like this...

On the phone at the moment with Netspace.

Will they be reimbursing customers for the downtime? NO

What is the problem with the "upstream provider" ? IT'S NOT A PROBLEM WITH UPSTREAM PROVIDER. IT'S A SERVER PROBLEM. THE UPSTREAM PROVIDER IS INVESTIGATING.

Last night I phoned Tech Support to get 'further' information, as I am 'on call' for my company and cannot perform the 'on call' duties required. (thus financially affected). I called at 7:05pm and got through at 7:27pm to a Netspace Rep who could not identify what the issue actually was. He just recited the recorded message. When I asked who the upstream provider is (knowing who it is) he was unable to tell me that either. So I've lost 2 days of 'on call' allowance, which is not a small amount.

Obviously no companies in Tassie use Netspace, otherwise this would have been resolved long ago.

I'll be showing my support of NetSpace by fast churning to another ISP

Please refer to the following thread for current notices from Netspace regarding the UnPlanned ADSL Dataflow Issue affecting Tasmania:

/forum-replies.cfm?t=760430

I'd say they're giving us a bullshit story.
You can't tell me a server takes this long to replace.
If the upstream provider, which presumably is Telstra, is providing to every ISP except Netspace then there must be some kind of dispute between the two and we are the pawns.
This is pathetic Netspace, get it sorted immediately or you'll lose another customer.

A mate called me asking about it as I had previously referred Netspace to them a few years ago. He said he rang the helpdesk number early this morning, and got the auto message about the problem, but called again a few minutes ago and got a message saying the number was unavailable.

He is still down

Netspace advices our Tasmanian customers that our wholesale provider has experienced a partial failure of their link between Tassie and the mainland. This is causing a service disruption for Netspace, and as we understand it, other providers using the affected part/s of the link.

Our advice is that restoration of the service requires the replacement of a hardware component. Our wholesale provider is attending to the problem as a matter of urgency however there is no estimated time of completion at this time. We will post updates on our Newsdesk for those that have access throughout the day.

We ask for your understanding as we work with our wholesaler to restore the link as soon as possible.

Joe

(Update: our wholesale provider is on site. Expect restoration sooner than later.)

There is currently only one physical carrier out of Tasmania i.e. Telstra.

I think Netspace still use UUnet/MCI/Verizon for IP connectivity to Melbourne.

One can assume that a core router or similar between Netspace and UUnet/MCI/Verizon.

I think both are co-located in the City Mill Building near the waterfront.

Maybe someone has cut a fiber in the street ;-)

The fact that this has barely caused a ripple of discontent on WP shows the extent of reach in Tasmania that Netspace no longer has.

Tasmania used to be its domain.

enima writes...

I think Netspace still use UUnet/MCI/Verizon for IP connectivity to Melbourne.

Incorrect (though we have in the past).

Ozzymosis writes...

Obviously no companies in Tassie use Netspace, otherwise this would have been resolved long ago.

We only have our email coming through Netspace – that si the only thing we are out on. But then we have moved to ADSL2+ via Telstra for our main outbound internet connection for http, and have other links interoffice.

From 11:15pm Tuesday 2/9 until now (12:45pm) Tasmanian netspace users have had 10h uptime (8:30am-6:30pm 3/9) total! 10h out of 37.5h, since this issue first appeared, so far......

Where is the backup!!!!!!

Unfortunatly myself, our company and many companies we support still use Netspace. As you can imagine the phones have been running hot in our office. The only bonus is the free dialup account that is provided with the account which is allowing me to make this post.

By the way the dialup number is 0198333003 and uses your normal ADSL username and password.

Gday .. this is my 1st post so firstly I would like to say Gday to everyone.

I have been a netspace customer for only 2weeks so far and I would like to say, this is horrible. It is the most I have ever paid for a server and apart from it being a bit faster, so far its HOPELESS. Im not paying $50 a month for this crap, I have NEVER had a connection go down for 15 hours in my whole 7 or 8 years of being online.

I recommended my dad churn to you guys only last weekend, JEEZUS am I hearing about it, my head is POUNDING .. really starting to WISH i had gone with ii or inter.

Shame on you netspace.. you really SHOULD provide a better backup, not paying $50 a month for a damn dialup server.

I need a reason to "NOT" churn again instantly, I have NOT got the patience required to deal with this crap.

aitchey.

Mark (NS Sysadmin) writes...

Incorrect (though we have in the past).

Instead of spending time correcting your customers, how about providing them with some real information about the problem with the service you provide, such as an ETA for resolution?

Well put. I also advised my sister to churn to Netspace (hope she hasn't yet).
I have been on Netspace (personally) for 12 months with no real issues. As a company it is more like 7 years in three offices with very limited issues mostly caused by exchange issues i.e. some telsta contractor tripping over a cable etc..

I little more information posted here from Netspace would be nice though....

REDUNDANCY, guys! Ever heard of that word?

Having dealt with many ISPs trying to buy business in the past, they used to all harp on about REDUNDANCY...if our main link goes down, anywhere, we can reroute.

Fine in principle, but it smacks of the same people saying....oh yes, we back our databases/servers to tape every night. What good are the backups if they cant be restored when they really need to?

Seriously, Netspace, it may not be your property that has failed, but I cant believe you dont have REDUNDANCY in place, unless you dont really care about your Tasmanian customers.

Rest assured, I am drafting a response to the Ombudsmens office as I would hope everyone who has experienced this problem is as well.

Perhaps it's time to change the hamster in the wheel powering the Tasmanian link!

(Or at the very least, join us in the 21st Century)

it went down on tuesday night too.
and it was up for about a minute on wednesday night.

it reminds me of a couple of years ago, about a month after i joined with netspace – net went down and then was on but only at dialup speeds (less than). it was like that for 2-4 weeks – compensation.....nup.

thankyou netspace – i needed a reason to churn.

This is the current info from the netspace website.

04-09-2008 13:07

Our Upstream provider is continuing to work on resolving this issue with
the highest priority. This issue is currently believed to be sourced
from a faulty card which terminates our connection providing interstate
connectivity between Tasmania and Victoria.

Replacement cards are en-route, however technicians are working to
re-splice our connection to alternate cards to try and rectify the issue
as soon as possible with no current E.T.R.

Netspace apologises for any inconvenience.

yipee we are online again.

it's back up. Only a 20h straight outage......

+1

My customers are reporting that they can access the net again.

johnd writes...

such as an ETA for resolution?

That information has been provided via the Netspace news desk: https://my.netspace.net.au/myNetspace/newsdesk.do

Regards,

Mark
Netspace Online Systems

Tassie_IT writes...

yipee we are online again.

The link is restored however we are monitoring/speaking with our U.P. before this is made official.

Regards,

Mark
Netspace Online Systems

Ozzymosis,

Netspace has a long and well documented history of trying to improve connection options and costs between Tassie and the mainland. This includes numerous representations to State and Federal governments, press statements, service variations, lodging an access dispute with the ACCC, and more.

This situation is an unacceptable yet inevitable consequence of the lack of competition in connecting Tassie to the mainland. Whilst there is ultimately only one supplier on this route we are all exposed to the risk of failure from time to time.

We ask for your understanding as we work with our wholesaler to restore the link as soon as possible.

Joe

(Update: /forum-replies.cfm?t=1045544&p=2#r32)

Mark (NS Sysadmin) writes...

That information has been provided via the Netspace news desk: https://my.netspace.net.au/myNetspace/newsdesk.do

Excellent. Refer people to a web page when they can't access the internet. I've fielded a number of calls from friends who say that not only can't they connect, but that your phones ring out.

BTW, that page is for members only. What about people trying to make a decision about changing to Netspace and are researching how you handle problems. Keeping things secret doesn't help your image.

Joe (Management) writes...

Netspace has a long and well documented history of trying to improve connection options and costs between Tassie and the mainland.

Such as removing any speeds above 512k? Strange way of improving things. That decision was one of the things that drove me away after many years being a customer.
I've seen your submissions. Instead of proposing real, achievable solutions, they just whine about Telstra.

johnd writes...

Such as removing any speeds above 512k?

That was a temporary measure to manage available bandwidth at the time. All plans (including 1500 and 8000) have been available to new and existing customer since late last year.

Joe

johnd writes...

Refer people to a web page when they can't access the internet.

Considering this is an Internet based forum, I would assume if they were reading this thread they would have some form of access to the Internet (otherwise those phoning in would have been given a recorded message noting the issue). You will find also there is a Notifications thread also provided in our WP forum: /forum-replies.cfm?t=760430 however the Netspace news desk is the official source as mentioned.

As for the contact centre I'm not aware of any current issues reaching those numbers.

Regards,

Mark
Netspace Online Systems

Joe from Netspace,

I certainly do appreciate that Netspace have tried to the best of their ability to provide the best of services, where possible, for Tasmanian clients.

And for the majority, I have never had any issues, apart from limited hours of telephone support. (which for me, isn't a biggie)

I guess I would need to know exactly what caused the recent failure, but I hope I can sum up here

1. If failure was equipment that is owned and managed by Netspace...
Well I would hope in any service that you provide to customers, be they in remote Tasmania, or metro Sydney, you would have spares/backups etc

2. If the failure was equipment that is owned/managed by third party, but is solely used by Netspace...
Again, if you had no redundancy or high level SLA, built into the agreement with the third party, to circumvent issues and minimise downtime, I would be very surprised.

3. If failure was equipment that is owned/managed by third party, and used by multiple clients...
Well, I know they have redundancy and contingencies in place. It's all dependent on their relationship with each client.

So if any one of these occurred, given the length of time it took to resolve, I would say there is a serious flaw in the delivery standards of ISP services to customers in Tasmania by Netspace and/or their relationship with their wholesaler/provider.

You have my understanding at the frustration you guys have working with the 'wholesaler', but you dont have my confidence that Netspace have gone as far as they can to minimise risk to their customers in delivering a quality service.

I'll be interested to see if Netspace learn anything from this episode.

As the saying goes – we all learn from our mistakes – but it's only the true professionals who do something about stopping it from happening again.

Ozzymosis writes...

and/or their relationship with their wholesaler/provider.

Given that Telstra is the only current working fiber provider between TAS and the mainland, I doubt Netspace can do anything to force Telstra to change their ways.

Well, I know they have redundancy and contingencies in place. It's all dependent on their relationship with each client.

You can't have redundancy at any price though! Otherwise Optus and various other Telcos would have built more fiber to Tassie years ago.

johnd writes...

Excellent. Refer people to a web page when they can't access the internet.

Ummm, hello, this is an internet web page? If you're reading this, you've got access to the internet!

It died about 11:45pm on Tuesday night too. Came back sometime before 830am when I woke up.

Then died again on Wednesday night as this thread is discussing. Surely there would have been some sort of investigation done re: Tuesday night instead of waiting for it to happen again at a more noticable time? No wait, this is Netspace, who have been consistently rubbish in Tasmania for years.

Matt writes...

Surely there would have been some sort of investigation done

Yes, surely NS or the exclusive supplier of the Tassie link (read Telstra) would have done some investigating. Maybe that's why it came back up the first time, someone doing something to try fix the problem?

No wait, this is Netspace, who have been consistently rubbish in Tasmania for years.

Then you would be a fool for putting up with it then!

There was a long Tassie thread in here ages ago. Though no sign of it or any other Tassie issues for how long, a year or more? And NS even resumed selling their higher speed plans a while ago.

Perhaps you should start a poll of the best ISP deals in Tassie?

Obviously was some sort of disruption today which they say was some fault in the ONLY link available to them and everyone else.

Still all comes back to the need for a second link to Tassie. Spend less time bashing ISP's and more time hassling your local member of parliament.

Sols

Totally agreed, Netspace have been a great ISP to me for years and I will still recommend them. Every ISP has their issues, Netspace are not the first and will certainly not be the last.

For everyone whinging about them constantly, have you heard of churning or actually calling them for help? I've only had to call about 3 times in 4 years and my issue has always been resolved. Stop the bitching and contact them and atleast give them the chance to help you.

A couple of my friends are with Netspace. One said his net has been down since Saturday, the other said his started being dodgey last night. They don't live too far from each other either. This must suck for the Netspace users of Tassie, hopefully it gets back up and running soon.

jtickner1 writes...

hopefully it gets back up and running soon

Dude,

It's was restored around 3pm.

/forum-replies.cfm?t=1045544&p=2#r32

Sols

seanirish writes...

I will still recommend them.

I too will continue to recommend NS.

Despite having to work all day using a dial up connection, I do appreciate having that back up in place in addition to the newsdesk that's updated when relevant info about the problem comes to hand. I'm not sure if any of my previous ISPs had these backups (presumably they did but they certainly didn't bring them to my attention). However, I was very aware of these things at NS due to the great job of the salesperson I spoke with when signing up.

In hindsight, that dial up connection was laughably slow though :)

solsgp writes...

Dude,

It's was restored around 3pm.

/forum-replies.cfm?t=1045544&p=2#r32

Sols

Sorry, I'm not with NetSpace.

JohnXYZ writes...

Ummm, hello, this is an internet web page? If you're reading this, you've got access to the internet!

Ummm, hello, I've got access to the internet because I'M NOT WITH NETSPACE!

What I am doing is trying to find out what is going on for several of my friends and collegues who are with Netspace but are unable to get through on the phone.

Mark (NS Sysadmin) writes...

Considering this is an Internet based forum, I would assume if they were reading this thread they would have some form of access to the Internet

Nope. Since I am not with Netspace, they are asking me to investigate. Several of them will now no longer be with Netspace....

(otherwise those phoning in would have been given a recorded message noting the issue)

No. One tried several times and, even after confirming the number, got "The number you have dialled is not connected" message. Others get a ring out, or get stuck in phone menu hell, unable to access the section they want.

You will find also there is a Notifications thread also provided in our WP forum:

Which does not mention the outage.

however the Netspace news desk is the official source as mentioned.

Which can't be accessed unless you are a Netspace subscriber, all of which cannot access the internet, and so cannot read the page.

Is this fixed? I still couldn't connect at home this morning.

Ingall writes...

Is this fixed?

Yes the issue reported in this thread has been resolved as per: https://my.netspace.net.au/myNetspace/newsdesk.do

If you're still having issues please contact Technical Support.

Regards,

Mark
Netspace Online Systems

Ingall writes...

Is this fixed? I still couldn't connect at home this morning.

If you haven't already, turn off and on the modem to let it sync up again. I had to do this on one of our connections before it would authenticate.

Kim

Hi

The following communication has been sent to Tasmanian customers.

Dear Tasmanian Customer,

On Wednesday 3 September 2008 and again on Thursday 4 September 2008 there was a partial failure of the main telecommunications link between Tasmania and the mainland.

This caused disruption to the Internet service provided by Netspace in Tasmania. Other Internet providers that rely on the same part of this vital link were similarly affected.

While this Tasmania to mainland route remains at the mercy of only one supplier, all services will continue to be exposed to the risk of periodic failure.

We agree that this situation is completely unacceptable. Regrettably, it is an inevitable consequence of the lack of competition in telecommunications infrastructure between Tasmania and the mainland.

Netspace has been a leading advocate for increased competition in this area for a number of years. We continue to campaign for the provision of a competitive alternative to the current single supplier and are encouraged to hear that a second link will finally 'come online' in the not too distant future.

If you were affected by this disruption we sincerely apologise for any inconvenience caused and hope that you will appreciate this was outside of our direct control.

We thank you for your patience and understanding in relation to this matter.

Regards,

The Team at Netspace

Kim writes...

If you haven't already, turn off and on the modem to let it sync up again. I had to do this on one of our connections before it would authenticate.

Kim

Did that, still no go. I'm beginning to think that the modem is on the fritz, but have tried another one (USB) and that hasn't worked either.

David (NS Marketing) writes...

While this Tasmania to mainland route remains at the mercy of only one supplier, all services will continue to be exposed to the risk of periodic failure.

I truely sympathise with the difficulties you encounter trying to run an ISP business in Tasmania under these conditions, but as someone responsible for keeping websites running within our organisation, I spent thursday frantically trying to find out the cause and ETR for the problem, and secondly developing contingency measures due to the information void from your organisation.

Twice, I spent 30 minutes in your IVR attempting to contact your customer support, only to get there and be given exactly the same ambiguous and vague statement about the problem as appears on your website.

I had heard on the rumour mill that the problem was with a card in a core router at Telstra at 10:30am, yet this information wasn't available until 1:00pm on your site – even then it gave no details of a proposed fix time. Whilst not providing an estimated fix time in your statements isolates you from customers kicking you when that time passes, it does nothing for the people trying to deal with the consequences of that problem – i.e. the users.

As it happens, I had a website we hosted due to be launched at 10:00am on friday by the federal tourism minister, so I did the only thing I could do in the circumstances, and picked the server up at 1:00pm and relocated it to another datacenter and ISP in Hobart which was not impacted by the problem. There it shall stay.

So this is a case of two failings on your part – 1) not providing sufficient information to your users for them to make considered assessments of the situation, and 2) not managing your upstream provider by a) getting them to provide more information on these incidents and proposed fix times, and b) them not providing redunancy where it is obviously necessary.

Yours in frustration,

MGC

Here here, MGC!

This is along the same lines of my posts.

Simply put, Netspace, this ONLY affected Tasmanian Netspace customers, so I dont have sympathy for you pushing the blame on the 'wholesaler' and saying...

Oh, you know, we try to give our customers the best service possible in Tasmania but our hands are tied by our upstream provide/wholesaler.

If I had the risk of a 'single point of failure' in my work, I wouldnt be in business.

MGC is a customer of yours, as I am, but had a lot more at stake than most home users.

How about working 'with' some of your competitors here in Tassie, to offer better contingencies for customers, and at the same time rally together against the 'beast'.

MGC, please let us know if Netspace offer any compensation to you.

Ingall writes...

Did that, still no go. I'm beginning to think that the modem is on the fritz, but have tried another one (USB) and that hasn't worked either.

There i was patiently waiting for normal service to be resumed as soon as possible having performed all the necessary rituals – power cycle etc, modem self-testings, couldn't ping of course. etc etc.

Even scrubbed the hard drive and reinstalled Windows.

About 4.30 this morning it dawned on me that perhaps i might be having a modem problem so i changed to the spare and voila, the WWW is back. Hooray. Another flash of light and scurry over to Whirlpool to see what's doing and, burglar me if the problem wasn't a whole of Tasmania one. But i am still stuck for email and lack of authorization – anyone know if that server still on the fritz?

Did you ever think that the Tasmanian market is not worth putting in backups/rerouting through other fibre lines to the mainland?

I mean, TPG Soul is pulling out of Tasmania altogether. This maybe because the market there is not worth it? I dont know, but you don't make a business decision like that lightly.

If that is the case, soon you will be left with one choice only, Telstra, and that is only because they probably have to supply you with it due to government regs.

johnd writes...

Nope. Since I am not with Netspace, they are asking me to investigate. Several of them will now no longer be with Netspace....

No. One tried several times and, even after confirming the number, got "The number you have dialled is not connected" message. Others get a ring out, or get stuck in phone menu hell, unable to access the section they want.

Which does not mention the outage.

Does this mean you are asking for Netspace to ring each customer individually and advise them that their is an outage and they wont be able to access the internet until its fixed?

Tashammer writes...

There i was patiently waiting for normal service to be resumed as soon as possible having performed all the necessary rituals – power cycle etc, modem self-testings, couldn't ping of course. etc etc.

Even scrubbed the hard drive and reinstalled Windows.

I didn't go as far as reinstalling Windows, but did all the rest. Rang NS on the weekend, did all the checks etc and they reckon it's at the exchange. Should be sorted by tomorrow hopefully *fingers crossed because dial up sucks*

The cancer at the core of all non-telstra ISP's in Tasmania is always going to be their commercial viability since they apply their national pricing models to a region which incurs greater operating costs. While they do this, they are always going to provide a second class service as they try and reduce these costs to make the same margins as their other markets.

Personally (and perhaps controversially), I would be in favour of Tasmanian ISP's applying a surcharge to cover these additional costs, in return for a better class of service. The "other" ISP would need to be brought into line by threats against their anti-competitive behaviour – however once a surcharge was applied, the monopoly cost of the single link provider becomes much more tangible, and easier to politicise as something that needs a resolution.

Let's not forget the second fibre link that the Tasmanian government has been sitting on for 5 years ...