ELI5 why x090 GPUs are so bad at scaling performance / core count

I'm not really educated at this so please help me understand it guys.

4080->4090 is ~30% performance uplift while having >60% more cores and 1.5x mem bandwidth
5080->5090 is ~50% performance uplift while 5090 is literally 2x at every spec possible

There surely has to be the reason besides deminishing returns to why that's the case. Why do those flagship-flagship GPUs scale so badly?