This seems to be the right section.  I have a question about Maya’s accuracy / precision.  Based on observation it seems to be kinda casual.

Lets say I have the default preferences set to centimeters and I create a platonic polygon tetrahedron and in the creation options I specify a side length of 1.0.

What I get in my version of maya 2012 x64 on vista is .9997xxx in the creation node and when I apply a measuring tool from any two vertices.

I can go into the creation node and change the side length to 1.0 but it remains .9997xxx when measured with the measuring tool although it now says 1.0 in the creation node.  I also went into the channel box settings and changed the precision from the default 3 to 5 and that did not seem to have any effect either.

This begs the question: Is there some documented limit to maya’s accuracy and precision?

For me this question is just out of curiousity; however, I know of several people that are trying to model things to an accuracy of 1/1000th to 1/10,000th of an inch accuracy and having very little luck because Maya just seems to round things off very casually and inconsistently.

I don’t usually model with this sort of accuracy, but I tried this out myself.  I’m using x64 on Windows 7.  When I create the platonic tetrahedron with a side length of one, the creation node says 1 but my measure tool has shown different values at different times ... once it shows .999972, and once it shows .999913. 

I wonder where this inaccuracy is - in the actual length of the edge, or in the tolerance of snapping the measure tool to the vertex?