Boy that MS-DTC, huh? It can give much trouble, no doubt.
If you can’t communicate between two parties and keep failing with errors such as “The flowed transaction could not be unmarshaled”, try the following:
- Use DNS instead of IP addresses
- Disable Firewall
- Remove security and enable the options in the MS-DTC security (in the Component Services shell)
- Try DTCPing to troubleshoot the case
If everything seems to be set properly and you’re still experiencing connectivity issues, you should try reinstalling MS-DTC.
This is specifically necessary if the OS was created from an image, the MS-DTC relies on a unique ID, and if you use the same image, this could cause the problems.
I strongly recommend you follow these instructions for removing MS-DTC and reinstalling it. In some cases, I had to do this complete wipe and install to solve the case.