Online modding (Xbox Live) on an RGH is a ban risk. Always use a stealth server (like Proto or XBOnline) and never use free menus on the official network.
If you have the ISO files, you will need to convert them. GTA V usually comes in two discs (Disc 1 is the "Install" disc, Disc 2 is the "Play" disc). For RGH consoles, you essentially need the content of Disc 2, or a merged version of the game created specifically for JTAG/RGH users (often labeled as "JTAG/RGH Ready"). gta v xbox 360 rgh install