RabbitMQ Java client accepts broker frames larger than the negotiated AMQP frame_max

Summary

The max body size was enforced to patch CVE-2023-46120, but even though that limit still works, the frame size itself still exceeds the given max size.

Root cause

The Java client records the AMQP 0-9-1 frame_max negotiated during connection tuning, but the socket inbound frame reader continues to validate broker-controlled payload lengths against the much larger maxInboundMessageBodySize limit. A broker peer can therefore send a method frame whose payload is larger than the negotiated frame_max, have it allocated and decoded, and complete the connection handshake instead of being rejected as a protocol violation.

Reported by Team Atlanta.