Page 4 of 4

Re: Planned minor service interruption

Posted: Tue Nov 24, 2015 2:29 pm
by avij
Our ISP will upgrade some routers and switches on Wednesday-Thursday night. EBT and the forum may be briefly unavailable within this timeframe:

  • Wednesday 22:00 - Thursday 02:00 WET
  • Wednesday 23:00 - Thursday 03:00 CET
  • Thursday 00:00 - Thursday 04:00 EET
This does not mean that EBT would be unavailable for four hours that night, but that there may be short interruptions (typically up to 10 minutes) within that timeframe.

Reference: https://www.sigmatic.fi/asiakassivut/tiedotteet/2015#notice_400

Edit: Looks like we were unavailable for about a minute shortly before midnight CET.

Re: Planned minor service interruption

Posted: Thu Feb 11, 2016 12:03 am
by avij
EBT's ISP wrote:We will be performing planned maintenance in our FI-HEL1 datacenter on 12.2.2016 between 01.00-06.00 EET to update our core routers' supervisor units to offer better performance. This network maintenance is related to the incident we suffered on January 26th.

During the maintenance, public and private network connections may experience occasional outages up to 15 minutes.

Edit: We were unreachable for around 5-10 minutes that night, depending on how your connection was routed to EBT.

Re: Planned minor service interruption

Posted: Sat Apr 09, 2016 4:12 pm
by avij
There is one specific statistics query that is taking way too long -- the query should finish in less than a second, but may take even minutes for some users. To fix this, I'm adding an additional database index to our database, which should speed this up considerably.

Adding the index will take up to 30 minutes and some parts of EBT will be unusable during that time. In particular, entering notes won't be possible. The index will be added tonight at around 04:00 CEST.

edit: Done, took around 12 minutes (04:08 - 04:20). The country filter in "My Notes" should work quicker now.

Re: Planned minor service interruption

Posted: Wed Aug 03, 2016 3:51 pm
by avij
EBT and the forum will be unavailable for about half an hour early Thursday morning due to maintenance work. The break is scheduled to start at about 02:00 UTC, 03:00 WEST, 04:00 CEST, 05:00 EEST.

During the maintenance window the operating system's kernel will be updated to the latest one, and the network adapter's firmware will also be updated. These updates may require rebooting the server a few times.

Re: Planned minor service interruption

Posted: Sat Nov 19, 2016 8:43 pm
by avij
EBT and the forum will be unavailable early tomorrow morning between 03:30 - 04:00 CET (approx) due to an operating system upgrade and the required server reboot. We will be running CentOS 7.3.1611 after the upgrade.

Re: Planned minor service interruption

Posted: Tue Dec 13, 2016 11:54 pm
by avij
EBT and the forum will be unavailable early Thursday morning between 03:30 - 04:00 CET (approx) for rebooting to the newest CentOS kernel. The kernel that we're currently using may have issues with the upcoming leap second at the end of the year. I will also update the server's RAID adapter's firmware at the same time, as it would also require a reboot.

Relatedly, it looks like MariaDB still can't handle leap seconds properly so you won't be able to enter a note with a timestamp of xx:59:60. When I tried to set some entry's timestamp to the leap second moment manually, the timestamp ended up being 0000-00-00 00:00:00. This is unfortunate, but there's nothing I can do about this.

Edit: Downtime was approx 03:39 - 03:59 CET, or 20 minutes.

Re: Planned minor service interruption

Posted: Mon Jan 23, 2017 3:30 pm
by avij
Our ISP will "upgrade some network devices and change some connections" on Wednesday-Thursday night. EBT and the forum may be briefly unavailable within this timeframe:

  • Wednesday 22:00 - Thursday 04:00 WET
  • Wednesday 23:00 - Thursday 05:00 CET
  • Thursday 00:00 - Thursday 06:00 EET
This does not mean that EBT would be unavailable for six hours that night, but that there may be short interruptions (typically up to 10 minutes) within that timeframe.

Reference: https://www.sigmatic.fi/asiakassivut/tiedotteet/2017#notice_443

Edit: No disruptions were noticed.

Re: Planned minor service interruption

Posted: Tue Jun 20, 2017 3:23 pm
by avij
EBT and the forum will be unavailable early Wednesday morning between 03:30 - 04:00 CEST (approx) for applying some security fixes to the operating system. This requires rebooting the server. I will also update the network adapter's firmware at the same time, as it would also require a reboot.

EDIT: Reboot postponed to some yet to be specified date due to some compatibility concerns, which I want to clarify first.
EDIT2: Waiting for a kernel that fixes this bug, which was introduced in the kernel that I was going to install.

Re: Planned minor service interruption

Posted: Tue Jul 04, 2017 9:08 pm
by avij
avij wrote:EBT and the forum will be unavailable early Wednesday morning between 03:30 - 04:00 CEST (approx) for applying some security fixes to the operating system. This requires rebooting the server. I will also update the network adapter's firmware at the same time, as it would also require a reboot.

EDIT: Reboot postponed to some yet to be specified date due to some compatibility concerns, which I want to clarify first.
EDIT2: Waiting for a kernel that fixes this bug, which was introduced in the kernel that I was going to install.

OK, a fix has been released and I will proceed with the update early Thursday morning.

EDIT: Downtime was approx 03:30 - 03:40.

Re: Planned minor service interruption

Posted: Sat Aug 12, 2017 11:42 pm
by avij
Sorry for the short notice, but the web server will be rebooted tonight at around 03:30 CEST for upgrading to CentOS 7.4.1708. Expected downtime is probably around 20 minutes.

Edit: Downtime was 03:40 - 03:47.

Re: Planned minor service interruption

Posted: Tue Dec 05, 2017 8:46 pm
by avij
The web server will be rebooted early Thursday morning (7.12.) to switch to a newer kernel (security update).

Edit: Downtime was approximately 03:31 - 03:36 CET.