There wouldn't be much point removing the rest, if you leave the pump in place.
That will be one of the heaviest items, is way out ahead of the front axle so, puts the weight in the wrong place. And will offer substantial parasytic losses whilst still being 'driven' by its drive belt.
There's also the risk of it seizing (the gas has oil in it, to lubricate the pump) and snapping a drive belt.
If you're going to the trouble of removing the rest, take the pump and pump bracket off too.
Paul
P.s. Up until reading some of the questions on this forum of late, I'd have said there is no such thing as a stupid question but...... :lol: This is a perfectly sensible one though. ;-)