This plugin will allow you to turn off collisions between indicated pairs of physical objects in runtime.
Features:
Disable collision between physical objects
Works with both c++ and Blueprint only projects
Works with CCD
Works with Substepping
Chaos (from 4.26)