The main problem is that while the Crown Vic and Grand Marquis have the same basic bodies, the Town Car has always had a completely different body - different contours on the body sides. - You could swap a Grand Marquis front door onto a Crown Vic, and it would fit fine, but you couldn't do that with a Town Car door - body lines on the outside wouldn't line up*. As Patrick said, you could turn a Crown Vic or Grand Marquis wagon into a Lincoln, but it would be A LOT of work. *Size and shape of the door window frames isn't the same, either.
The frame issue could be solved by using the Ford/Merc window frames and glass. Again, a LOT of work.