Many people do not picture New Orleans when they think of a romantic getaway but that is absolutely wrong. With it's Southern charm and laid back attitude you can find plenty of things to do besides party on Bourbon Street.
I would recommend staying in the Garden District. There you'll find plenty of reasonably priced accommodations (or you can splurge too). A majority of the lodging were once previous older majestic homes and more often than not you will get a southern continental breakfast to boot! I've even stayed in an old orphanage turned bed and breakfast. Most places have a lovely courtyard to relax in with either a pool or waterfall and full of tropical vegetation.
While staying there take the trolley that runs directly down the middle of the Garden District- it practically takes you anywhere you want to go. I recommend a visit to the old plantations, the French quarter for some Jazz and Blues, take a day trip to some of the sleepy neighboring parishes and if you are adventuresome visit some of the old Voodoo shops! You are sure to find plenty of romantic atmosphere and charm in New Orleans.