You’re right that the fuel pump relay is located in a fuse box, but the 2002 Ford Explorer’s fuel pump relay is located in the engine compartment fuse box. The fuse you need should be located in slot 43, but to be sure of the number, you should check the diagram on the box cover and/or your owner’s manual.
A faulty fuel pump relay can often cause your engine to stall, and could even lead to a complete breakdown, so it’s a good thing you know what to do to fix it. However, if changing the fuse doesn’t solve your problem you may want to head to a garage to get an expert’s help. It may take some time and cost a bit more money, but an expert can make sure all your truck’s systems are working properly.