I have added the second language (Japanese for example) to Rich Media template for localization. After that, my Rich Media is displayed on this language only despite language in device settings.
When you changed the content for Japanese in Rich Media, this language was set to default and localization was applied for it. But before that, there was no information in Rich Media file about languages. So, only the Japanese language was added to the .json file. That is why all In-Apps were shown in Japanese.
Thus, in order to avoid such situation, you should modify English content as well. The corresponding sting with changes will be added to the file. Thus for different devices, there will be different content.