You could always just run your video through a script and crop the black borders off as a faster way of removing them, other than that I'm not sure of any particular way of doing that other than the way you've already stated.
Set your project settings to match your video's resolution and AR. If the video's aspect ratio is off and not the project, right click on it and uncheck "maintain aspect ratio" or manually change the AR on the second tab. And by the way, stick with pan/crop to zoom, unless you really need track motion as well.