The first Remote Desktop Service to upgrade is the RD Connection Broker servers. This is because they are the central component that manages and directs user connections, and upgrading them first is critical for the success of the deployment upgrade.
Why upgrade the RD Connection Broker first
Central role: The Connection Broker directs incoming user connections to the appropriate session host, and must be updated to point to the new, upgraded infrastructure.
Deployment management: A successful upgrade of the entire deployment depends on the Connection Broker being updated to manage the new servers and session collections.
Important for migration: Upgrading the RD Connection Broker is listed as the "first and most important step" in the process of migrating the deployment to a new version.
Reference:
https://learn.microsoft.com/en-us/windows-server/remote/remote-desktop-services/upgrade-to-rds