You’re right, this is not the “Docker way” of doing things which is why I wouldn’t recommend it. But since there is a regular OTP Release inside of the container, you could upgrade it just like any other Release: Nothing keeps you from ssh-ing into the container and performing the upgrade.
But really, don’t do it 😉