Planned minor service interruption

Announcements about Eurobilltracker or this forum

Moderators: avij, Phaseolus, Nerzhul, claudio vda, eddydevries, dserrano5

User avatar
avij
Forum Moderator
Forum Moderator
Posts: 5619
Joined: Mon May 27, 2002 10:45 pm
Location: Helsinki Finland
Contact:

Re: Planned minor service interruption

Postby avij » Tue Nov 24, 2015 2:29 pm

Our ISP will upgrade some routers and switches on Wednesday-Thursday night. EBT and the forum may be briefly unavailable within this timeframe:

  • Wednesday 22:00 - Thursday 02:00 WET
  • Wednesday 23:00 - Thursday 03:00 CET
  • Thursday 00:00 - Thursday 04:00 EET
This does not mean that EBT would be unavailable for four hours that night, but that there may be short interruptions (typically up to 10 minutes) within that timeframe.

Reference: https://www.sigmatic.fi/asiakassivut/tiedotteet/2015#notice_400

Edit: Looks like we were unavailable for about a minute shortly before midnight CET.
Money makes the world go round. We track how the money goes round the world.
EBT Tech WG leader. Do not PM me if your question is not related to Tech WG or the association.

User avatar
avij
Forum Moderator
Forum Moderator
Posts: 5619
Joined: Mon May 27, 2002 10:45 pm
Location: Helsinki Finland
Contact:

Re: Planned minor service interruption

Postby avij » Thu Feb 11, 2016 12:03 am

EBT's ISP wrote:We will be performing planned maintenance in our FI-HEL1 datacenter on 12.2.2016 between 01.00-06.00 EET to update our core routers' supervisor units to offer better performance. This network maintenance is related to the incident we suffered on January 26th.

During the maintenance, public and private network connections may experience occasional outages up to 15 minutes.

Edit: We were unreachable for around 5-10 minutes that night, depending on how your connection was routed to EBT.
Money makes the world go round. We track how the money goes round the world.
EBT Tech WG leader. Do not PM me if your question is not related to Tech WG or the association.

User avatar
avij
Forum Moderator
Forum Moderator
Posts: 5619
Joined: Mon May 27, 2002 10:45 pm
Location: Helsinki Finland
Contact:

Re: Planned minor service interruption

Postby avij » Sat Apr 09, 2016 4:12 pm

There is one specific statistics query that is taking way too long -- the query should finish in less than a second, but may take even minutes for some users. To fix this, I'm adding an additional database index to our database, which should speed this up considerably.

Adding the index will take up to 30 minutes and some parts of EBT will be unusable during that time. In particular, entering notes won't be possible. The index will be added tonight at around 04:00 CEST.

edit: Done, took around 12 minutes (04:08 - 04:20). The country filter in "My Notes" should work quicker now.
Money makes the world go round. We track how the money goes round the world.
EBT Tech WG leader. Do not PM me if your question is not related to Tech WG or the association.

User avatar
avij
Forum Moderator
Forum Moderator
Posts: 5619
Joined: Mon May 27, 2002 10:45 pm
Location: Helsinki Finland
Contact:

Re: Planned minor service interruption

Postby avij » Wed Aug 03, 2016 3:51 pm

EBT and the forum will be unavailable for about half an hour early Thursday morning due to maintenance work. The break is scheduled to start at about 02:00 UTC, 03:00 WEST, 04:00 CEST, 05:00 EEST.

During the maintenance window the operating system's kernel will be updated to the latest one, and the network adapter's firmware will also be updated. These updates may require rebooting the server a few times.
Money makes the world go round. We track how the money goes round the world.
EBT Tech WG leader. Do not PM me if your question is not related to Tech WG or the association.

User avatar
avij
Forum Moderator
Forum Moderator
Posts: 5619
Joined: Mon May 27, 2002 10:45 pm
Location: Helsinki Finland
Contact:

Re: Planned minor service interruption

Postby avij » Sat Nov 19, 2016 8:43 pm

EBT and the forum will be unavailable early tomorrow morning between 03:30 - 04:00 CET (approx) due to an operating system upgrade and the required server reboot. We will be running CentOS 7.3.1611 after the upgrade.
Money makes the world go round. We track how the money goes round the world.
EBT Tech WG leader. Do not PM me if your question is not related to Tech WG or the association.

User avatar
avij
Forum Moderator
Forum Moderator
Posts: 5619
Joined: Mon May 27, 2002 10:45 pm
Location: Helsinki Finland
Contact:

Re: Planned minor service interruption

Postby avij » Tue Dec 13, 2016 11:54 pm

EBT and the forum will be unavailable early Thursday morning between 03:30 - 04:00 CET (approx) for rebooting to the newest CentOS kernel. The kernel that we're currently using may have issues with the upcoming leap second at the end of the year. I will also update the server's RAID adapter's firmware at the same time, as it would also require a reboot.

Relatedly, it looks like MariaDB still can't handle leap seconds properly so you won't be able to enter a note with a timestamp of xx:59:60. When I tried to set some entry's timestamp to the leap second moment manually, the timestamp ended up being 0000-00-00 00:00:00. This is unfortunate, but there's nothing I can do about this.

Edit: Downtime was approx 03:39 - 03:59 CET, or 20 minutes.
Money makes the world go round. We track how the money goes round the world.
EBT Tech WG leader. Do not PM me if your question is not related to Tech WG or the association.

User avatar
avij
Forum Moderator
Forum Moderator
Posts: 5619
Joined: Mon May 27, 2002 10:45 pm
Location: Helsinki Finland
Contact:

Re: Planned minor service interruption

Postby avij » Mon Jan 23, 2017 3:30 pm

Our ISP will "upgrade some network devices and change some connections" on Wednesday-Thursday night. EBT and the forum may be briefly unavailable within this timeframe:

  • Wednesday 22:00 - Thursday 04:00 WET
  • Wednesday 23:00 - Thursday 05:00 CET
  • Thursday 00:00 - Thursday 06:00 EET
This does not mean that EBT would be unavailable for six hours that night, but that there may be short interruptions (typically up to 10 minutes) within that timeframe.

Reference: https://www.sigmatic.fi/asiakassivut/tiedotteet/2017#notice_443

Edit: No disruptions were noticed.
Money makes the world go round. We track how the money goes round the world.
EBT Tech WG leader. Do not PM me if your question is not related to Tech WG or the association.

User avatar
avij
Forum Moderator
Forum Moderator
Posts: 5619
Joined: Mon May 27, 2002 10:45 pm
Location: Helsinki Finland
Contact:

Re: Planned minor service interruption

Postby avij » Tue Jun 20, 2017 3:23 pm

EBT and the forum will be unavailable early Wednesday morning between 03:30 - 04:00 CEST (approx) for applying some security fixes to the operating system. This requires rebooting the server. I will also update the network adapter's firmware at the same time, as it would also require a reboot.

EDIT: Reboot postponed to some yet to be specified date due to some compatibility concerns, which I want to clarify first.
EDIT2: Waiting for a kernel that fixes this bug, which was introduced in the kernel that I was going to install.
Money makes the world go round. We track how the money goes round the world.
EBT Tech WG leader. Do not PM me if your question is not related to Tech WG or the association.

User avatar
avij
Forum Moderator
Forum Moderator
Posts: 5619
Joined: Mon May 27, 2002 10:45 pm
Location: Helsinki Finland
Contact:

Re: Planned minor service interruption

Postby avij » Tue Jul 04, 2017 9:08 pm

avij wrote:EBT and the forum will be unavailable early Wednesday morning between 03:30 - 04:00 CEST (approx) for applying some security fixes to the operating system. This requires rebooting the server. I will also update the network adapter's firmware at the same time, as it would also require a reboot.

EDIT: Reboot postponed to some yet to be specified date due to some compatibility concerns, which I want to clarify first.
EDIT2: Waiting for a kernel that fixes this bug, which was introduced in the kernel that I was going to install.

OK, a fix has been released and I will proceed with the update early Thursday morning.

EDIT: Downtime was approx 03:30 - 03:40.
Money makes the world go round. We track how the money goes round the world.
EBT Tech WG leader. Do not PM me if your question is not related to Tech WG or the association.

User avatar
avij
Forum Moderator
Forum Moderator
Posts: 5619
Joined: Mon May 27, 2002 10:45 pm
Location: Helsinki Finland
Contact:

Re: Planned minor service interruption

Postby avij » Sat Aug 12, 2017 11:42 pm

Sorry for the short notice, but the web server will be rebooted tonight at around 03:30 CEST for upgrading to CentOS 7.4.1708. Expected downtime is probably around 20 minutes.

Edit: Downtime was 03:40 - 03:47.
Money makes the world go round. We track how the money goes round the world.
EBT Tech WG leader. Do not PM me if your question is not related to Tech WG or the association.

User avatar
avij
Forum Moderator
Forum Moderator
Posts: 5619
Joined: Mon May 27, 2002 10:45 pm
Location: Helsinki Finland
Contact:

Re: Planned minor service interruption

Postby avij » Tue Dec 05, 2017 8:46 pm

The web server will be rebooted early Thursday morning (7.12.) to switch to a newer kernel (security update).

Edit: Downtime was approximately 03:31 - 03:36 CET.
Money makes the world go round. We track how the money goes round the world.
EBT Tech WG leader. Do not PM me if your question is not related to Tech WG or the association.


Return to “Announcements”