Debugging Path objects with Visual Studio in C++

By
-
Login to Follow
-
Industry
  • Film & VFX
Subject
  • SDK
  • C++
  • Programming
Products
  • 3ds Max
Skill Level
  • Advanced
Duration
5 min

Have you ever needed to display the contents of a MaxSDK::Util::Path, while debugging though source code of a 3ds Max plug-in? If you tried to display the Path in a Visual Studio watch window, you probably noticed the class hides its data in a private implementation, an object of type PathImpl, not available in the SDK. Since Path objects are pretty important, and commonly encountered, the issue can present a frequent difficulty in debugging.

One solution is to update the code being debugged, by adding a call to Path::GetCStr(), which provides the raw string buffer. But obviously, that is not ideal. It requires a recompile, repeating the current test steps, and then removing the call later on. There's a better solution.

Simply use the following in a Visual Studio watch window:

(wchar_t*)(8+(uint64_t)(myPath.mImpl.mPtr))

To understand this, consider an example. Imagine you are implementing a custom render element, subclass of IRenderElement, which has the following method:

SetPBBitmap(PBBitmap *&b)

While debugging, you want to know the bitmap path which 3ds Max passes in. First, let's do it the hard way, by adding the following line:

const MCHAR* path = b->bi.GetPathEx().GetCStr();

Recompile with this change, then set a breakpoint on it, and trace into the disassembly. Putting on your assembly code reading glasses, you'll discover it returns the address of the memory location 8 bytes after the beginning of the PathImpl object. It's easy to fetch that same address directly, by casting the PathImpl pointer to a 64-bit int, adding 8, and casting back to a wchar_t pointer. In this example, the final entry in the watch window is:

(wchar_t*)(8+(uint64_t)(b->bi.mFilename.mImpl.mPtr))

Verify this is the same pointer as the path returned by GetCStr(). It works!

Caveat - this solution is tested in 3ds Max 2019 only. It exploits knowledge of the PathImpl memory layout, obtained though assembly debugging. It is not guaranteed to work in future releases. After all, potential changes to memory layout is the whole point of the private implementation pattern. But, this can save you some trouble in the current release.

Thank you for reading!

Michaelson Britt

 

 

 

 

 

 

 

 

Posted By
Tags
  • 3ds Max
  • SDK
  • C++
  • Programming
0 Comments
To post a comment please login or register
*Save $66 per month on Autodesk's Suggested Retail Price (SRP) when purchasing 1 year term 3ds Max or Maya subscription.