Should `X-Real-IP` header trust be restricted to an IP range?

With some other Dockerized services I've worked with that are behind a reverse proxy (uvicorn behind caddy for example) there is an option to restrict what connections are trusted to take the X-Real-IP header from.

E.g.

While in a reverse-proxy + Docker setup there isn't really a risk due to the container network isolation, other setups might be vulnerable to spoofed X-Real-IP headers bypassing rate limiting/fail2ban blocking.

I think this could be a worthwhile security improvement.