Strange World Amnesia: Difference between revisions
writing. plz respond in talk page i will update this to be moar accurate |
|||
| Line 1: | Line 1: | ||
{{Template:Culture}} {{Template:Construction}} | {{Template:Culture}} {{Template:Construction}} | ||
On November 30th, 2025. Heyuri's [[Strange World#Strange World@Heyuri|/sw/]] board disappeared. | On November 30th, 2025. Heyuri's [[Strange World#Strange World@Heyuri|/sw/]] board disappeared. All posts suddenly went out of existence, and for seemingly no reason. In actuality, error logs from NGNIX had filled the disk of the server /sw/ was being hosted on to such a degree that any further writes corrupted files. Luckily, thanks to the wise monks at [[HARD]], alongside some help from a suprise guest, next to no data was fully lost, only the front-end cache. | ||
== | == TRAGEDY! == | ||
At some point in time (i dunno... i don't work there!), updates to the error log file in the nginx directory had ballooned up to a size reaching above 120+ GB. The exact cause is unknown, but this maxed out the disks that /sw/ was hosted on. (i'm assuming /sw/ is on a seperate disk to the imgboard...) | |||
As a result, /sw/ suddenly no longer worked properly. The strange world script relies on a 5000-post long message log (I'll call it the warmlog) that's used for the frontend. Due to there being literally zero disk space, any further writes corrupted this frontend log. The script would (presumably) then try to load the recently corrupted log as a person connected, fail, and then error out; resulting in weird behaviour when trying to connect. | |||
This would vary from getting a white screen, to getting some of the site to work, to seeing weird errors. I'd have to scroll further than 1000 lines on Chat@Heyuri to find the people complaining about it though... | |||
This got reported quite quickly, and HARD devs quickly got to work fixing it. [[kuz]] also came in to assist, helping to find the bloated nginx error log. | |||
== Oh, it's over? That was kinda fast. == | |||
This was solved kind of quickly. I think it was only a downtime of a few hours, and people just used [[Chat@Heyuri]] to whine about it while HARD got to work. All that needed to happen to fix the issue was to free up disk space, and restart the scripts. It was as simple as kaguya deleting the old inflated error log, and kuz and him throw away and replace the corrupted warmlog. The script initially starts entirely new posts, as if the instance was completely new, but this is quickly corrected to have their posts line up correctly with the message log. | |||
No data was actually lost. This is because the backend, KuzuhaScriptPHP+, writes posts to the warmlog, and then also to cold, long term logs that aren't limited, so every message dating back to 2022 was still there. [https://dis.heyuri.net/sw/bbs.php?c=58&m=g You can access them here.] | |||
However, the warmlog was lost. I'm not sure if it means any posts were lost, but it now means that you can't just flip pages on the frontend all the way back 5000 posts, until the warmlog fills up. | |||
Kuz says that they'll be doing smart server admin stuff to make sure it doesn't happen again, which is probably just cleaning the dust off his 5 year old server configs. Not like I'd know, I haven't hosted shit. | |||
Revision as of 06:33, 8 December 2025
|
This page is about Heyuri or Heyuri Culture |
|
|
This page is under construction or broken... |
On November 30th, 2025. Heyuri's /sw/ board disappeared. All posts suddenly went out of existence, and for seemingly no reason. In actuality, error logs from NGNIX had filled the disk of the server /sw/ was being hosted on to such a degree that any further writes corrupted files. Luckily, thanks to the wise monks at HARD, alongside some help from a suprise guest, next to no data was fully lost, only the front-end cache.
TRAGEDY!
At some point in time (i dunno... i don't work there!), updates to the error log file in the nginx directory had ballooned up to a size reaching above 120+ GB. The exact cause is unknown, but this maxed out the disks that /sw/ was hosted on. (i'm assuming /sw/ is on a seperate disk to the imgboard...)
As a result, /sw/ suddenly no longer worked properly. The strange world script relies on a 5000-post long message log (I'll call it the warmlog) that's used for the frontend. Due to there being literally zero disk space, any further writes corrupted this frontend log. The script would (presumably) then try to load the recently corrupted log as a person connected, fail, and then error out; resulting in weird behaviour when trying to connect. This would vary from getting a white screen, to getting some of the site to work, to seeing weird errors. I'd have to scroll further than 1000 lines on Chat@Heyuri to find the people complaining about it though...
This got reported quite quickly, and HARD devs quickly got to work fixing it. kuz also came in to assist, helping to find the bloated nginx error log.
Oh, it's over? That was kinda fast.
This was solved kind of quickly. I think it was only a downtime of a few hours, and people just used Chat@Heyuri to whine about it while HARD got to work. All that needed to happen to fix the issue was to free up disk space, and restart the scripts. It was as simple as kaguya deleting the old inflated error log, and kuz and him throw away and replace the corrupted warmlog. The script initially starts entirely new posts, as if the instance was completely new, but this is quickly corrected to have their posts line up correctly with the message log.
No data was actually lost. This is because the backend, KuzuhaScriptPHP+, writes posts to the warmlog, and then also to cold, long term logs that aren't limited, so every message dating back to 2022 was still there. You can access them here. However, the warmlog was lost. I'm not sure if it means any posts were lost, but it now means that you can't just flip pages on the frontend all the way back 5000 posts, until the warmlog fills up.
Kuz says that they'll be doing smart server admin stuff to make sure it doesn't happen again, which is probably just cleaning the dust off his 5 year old server configs. Not like I'd know, I haven't hosted shit.
