Might have something to do with peak activity times affecting it, ill see if I can speed it up more

Yeah we added another admin onto the team

Yeah it uses a system similar to github actions

The forgejo instance here is meant for repositories related to programming.dev and the fediverse

Cleared all the spam repos so you shouldnt see any more, will do a more permanent solution later (will see if I can do an application process using a third party form system since forgejo doesnt have that built in)

Yeah I havent gone through and cleared it in a bit since there isnt much of a point to due to it being mostly used internally only rn (and forgejo makes the process very annoying unless you do it directly through the db)

Ill purge things and adjust settings again soon

I couldnt find anything in the post details from looking at it that would differentiate it from other posts

Cant find anything that would cause this in the db. Thought the dates got messed up in federation but they seem to be perfectly normal. Still pinned even if I manually change the date to a year in the past

Im assuming something got messed up from the nodebb->lemmy federation

Im going to remove the post as thats the only way I see to get rid of it

!emacs@programming.dev exists already

Ill go through and remove the hides from the communities that dont have bot content anymore

Not possible in lemmy atm past just rehiding everything from the instance every so often

edit: just did another round of hiding things from lemmynsfw, should be reflected in the hidden communities doc

I had @icon there and the cache didnt fully update yet

Sites back. I reindexed every table although the only one that seemed to be affected by the issues was the comment table (it had duplicates I had to remove similar to what I did for the post and community tables before)

Site will slowly catch up in activities over the next couple hours so youll see posts, etc. that were made in the few hours it was down slowly come in

No votes or comments were connected to the duplicates but yeah if there was some you would move it over to the other one

The posts that had comments were the posts that were uniquely on the other community and those moved over with the post when I updated the community of the post

Sites back, communities that previously had their community page not vieweable are now viewable

An example of this is https://programming.dev/c/a11y

Basically what I did was before the maintenance exported every single post in the post table and use a js script to check for duplicate ap_ids (checking using sql itself didnt work due to broken indices so it was always missing some). Then I generated sql statements to remove duplicate posts that had higher ids than the other posts theyre a duplicate of. Ran all those during maintenance and since all duplicate ap_ids were removed I could now reindex the table to fix the broken indices

Did the same for communities after the posts (but also did some statements to move posts from one version of the community to the other)

Site might be slow for a tiny bit as it processes the 2 hour of activities that were generated while it was down

select them all, then ctrl+/

what frontend are you using, default lemmy one or one of the alt frontends .world has? Default frontend should handle it fine but havent checked the alts

versions that support (lower rectangle / higher number is more recent):

usage per version:

last major browser that didnt support which was safari updated to support it 4 years ago. update your browser

Sites back up

Got a lot of information about what it actually causing the issues which should help me automate some fixes for tomorrow + the weekend

There likely will be some more downtime tomorrow if I have time (if not itll be on saturday)

The issue is essentially a chain of issues with one needing to be fixed before I can fix the others. Some tables have duplicates entries which is preventing reindexing (since the indexes are broken). First one im going to try to fix is the post table.

I did some changes to remove a duplicate community to test which is https://programming.dev/c/photography@lemmy.world and ive managed to make it merged and now comments will show if you sort by comments in the community. Posts however still aren’t showing up due to the posts table having the issue with duplicates as well. (fixing posts first so I can set all of the posts to the correct community in an automated way later for the communities)

around half the instances I checked were affected, no idea why only them were affected. world is also currently behind in inbound federation due to the pure amount of activities its generating atm (only instance that is atm). We fixed the issues so it shouldnt be happening anymore (and I can see this post on world)

world, lemm.ee dont have them, beehaw does for example

no problem, been using it as an excuse to learn about godot as well

For people on other instances, you might not see posts 29 and 30 due to some issues (e.g. people on .world) but you can see them by going to https://programming.dev/c/godot

I dont

⠀⠀⠀⠀know what you’re

talking

⠀⠀ab out