While not recommended, it is possible to use one RDS gateway for multiple RDS farms, each with their own broker. In this case, the RDG must use the same FQDN in all deployments, and you should only make changes to its configuration from only one of the deployments.

In 2012, it is not possible to use one RDS WebAccess to service multiple brokers. It can only be used on a single deployment. Attempting to add the WebAccess server to another deployment will overwrite the configuration of the first deployment.

A better approach would be to create a single deployment with an RDS WebAccess, RDG and broker (single or HA), and add your "different deployments" as collections to the broker. You will still maintain the separation of the different farms if you use different security groups for the different farms.