SSH agent doesn't work when running inside zellij or tmux

I use a terminal multiplexor most of the time and would like to make use of the new ssh agent. However, it doesn't work when run inside a terminal multiplexer (Well I tried zellij and then tmux). It ...