The FASS sounds like a separate issue. When you turn the key off the pump should stop, unless it's incorrectly wired.
The top CP3 cannot leak into the engine. The bottom one can via the timing cover if it has a bad seal. Do you have a RP gauge? Is the RP sensor in the top pump or on the rail?
If there's not a miss, white smoke, pinging, knocking, ect, it is probably the seal for the bottom CP3. While it's not common, it is a "more than likely factor" if the injectors are good. Easy way to check is to feel the bottom of the CP3 for diesel. Sometimes it doesn't leak out of the cover so it might be best to pull the CP3 (not that hard).