Yes, the rod is part of the booster, not the pedal.
If you're getting normal brake operation when you press the pedal, I don't think your looking at a vacuum leak. It may be the return springs inside the booster are damaged or there is some other sort of mechanical defect where the rod passes through. But either way, I believe you are correct that it is your brake booster.
Check out this link on the operation of brake boosters, might help your troubleshooting:
http://auto.howstuffworks.com/power-brake2.htm