Is decentralizing upgradeability possible?

Currently a lot of the big sidechains (polygon, arbitrum, etc) use ownable upgradeable smart contracts. They all have the goal to scale the ethereum network and they actually are scalining it and making transactions faster and cheaper. But so are centralized exchanges like binance. A handful of polygon or arbitrum devs can remove your ability to withdraw your funds and in some cases actually steal your money. I'm not saying they will but it's very easily possible. This will only be fixed if upgradeability is removed or becomes decentralized.

Since the sidechains and all the used contracts are very complex it will never be possible to be 100% sure that there are no bugs. Removing upgradeability could be very risky.

Implementing decentralized upgradeability introduces a lot of questions to which seemingly no one has a good answer (morning 1:39:20)

What do you think how this will develop? I like using sidechains but it honestly feels like using ethereum on a AWS server at the moment. It might run on multiple nodes and everything seems decentralized but as long as a small amount of people can change the code just like that, it will never be decentralized and is therefore insecure.