Old posts viewed fine in the browser so its outputting correctly. I wouldn't assume that vBulletin did any conversion between formats and so I can only assume that the issue was the database table collation was wrong and that's what broke the conversion to IPS.
Whatever the issue, given the data used to output correctly it should be possible to script something to either export it correctly, or convert the raw data to the correct format. I'm not sure why it is likely to be a "very tough operation"?