Swisher connected with a Dan Wheeler (4-4) fastball with one out in the ninth to give New York its fourth win in five games.
The homer made a winner of Mariano Rivera (2-2), who pitched a perfect ninth.
Evan Longoria and Jason Bartlett each homered for Tampa Bay, which has lost seven straight.
The Yankees remained nine games ahead of Boston in the AL East, while Tampa Bay fell 8 1/2 games behind the Red Sox in the wild-card race.
Copyright 2009 by United Press International.