EuroBillTracker

Looks like there are some network problems at the moment, the netblock where the EBT webserver (and our company) is located has been unreachable for the last 15 minutes or so.

I'll investigate this further, but it looks like there's a problem between our router and the ISP. This will probably be corrected in the morning at the latest. Sorry for the downtime.

We're back on the net again. Total downtime was about two hours.

The reason was probably a regular maintenance break at our current ISP. If there's a need for a maintenance break, they are scheduled to the second Wednesday of each month.

It's the second Wednesday of this month, and Finnetcom seems to be fixing something again. The break started at about 02:15 Finnish time, no info yet on how long it's going to take this time.

Edit: the connection was restored at about 02:30.

Edit 2: still some short breaks every now and then..

I guess I'm going to blame this on Friday the 13th.

First of all, the network connection to our office (and EBT) went down just before my lunch and came back up conveniently right after my lunch. The official explanation is that our ISP screwed something up in their Wednesday night router upgrade and they had to revert the changes.

An unrelated break happened later this evening when the network connection to EBTForum disappeared. This lasted for about one hour. The contact person at the colocation company blamed Auria for the break after I sent him an SMS informing him of this. Apparently they fixed something as I'm able to write this message. The whole colocation company and all their clients were offline, so we weren't the only ones suffering from this. Edit 2: the news item at our hosting company is no longer available.

All of this was once again beyond my control. If I have to shut down something for maintenance reasons some day, it'll happen some time between 03-07 AM Finnish time.

For the record, the forum was unreachable today between 18-20 Finnish time. The network connection to the forum has been somewhat unstable (some packet loss) lately and now the problem was isolated to a faulty router which was replaced. As far as I can see, there is no more packet loss so I guess the problems are gone now.

avij wrote:It's the second Wednesday of this month, and Finnetcom seems to be fixing something again. The break started at about 02:15 Finnish time, no info yet on how long it's going to take this time.

It's now the second Wednesday of this month again, and time for a regular service break. This time it happened between 00:20 and 08:15 Finnish time, although it probably should have been quite a bit shorter. Sigh..

We were offline for about 8 and a half hours due to someone rearranging switches a bit on the other side of the building. See http://www.eurobilltracker.net/

There may be some connectivity problems again this night due to our ISP's regular maintenance break.

EBT and EBTF may be unreachable for short periods of time on the night between Tuesday and Wednesday due to a scheduled maintenance break at our ISP.

If there is a break, it'll most probably happen just after midnight Finnish time (23:00 for most of you) and last for about half an hour.

1) There's going to be a scheduled service break at our ISP, EBTForum may be unavailable for a while after midnight Finnish time.

2) EBTForum will be unavailable for a few hours on early Wednesday morning due to some construction work in our building.

EBT is located elsewhere, it won't suffer from these problems.

EBTF may be unreachable for short periods of time on the night between Tuesday and Wednesday due to a scheduled maintenance break at our ISP.

If there is a break, it'll most probably happen just after midnight Finnish time (23:00 for most of you) and last for about half an hour.

Our ISP is apparently having some network problems, EBT and a bunch of nearby servers have been unreachable from the 'net since 05:00am.

EDIT: Looks like this is part of a bigger problem, I can't currently reach the ISP of our own ISP. Damn backhoes..

We're back online, Sonera had some routing problems between 05:40-10:25 Finnish time.

The ISP writes:

We are investigating networking issues in our Helsinki datacenter. We will get back as soon as we have more information.

EBT may appear slow or unreachable because of this.

edit1: The situation looks better now, but not fully normal yet. Looking at the statistics of FICIX1 and FICIX2 (internet exchanges), I see some unusual traffic for Elisa, MTS, TDC and Rascom. I'd guess this is a larger issue that affects other ISPs in the area as well.

edit2: Everything seems to be working now. Depending on how your connection was routed to EBT, the total time for degraded network performance was something between 0,5 and 1,5 hours.

edit3: The ISP says this was a large scale DDoS attack.

There was apparently some network outage on Tuesday, causing EBT to appear unreachable from some locations. Even though the posted notice does not currently mention it, it appears that IPv6 routing is still somewhat broken. I have now temporarily removed EBT's IPv6 address from DNS until this issue has been resolved.