Inside Sabertooth
Learn how Sabertooth uses 3ds Max to create 3D interactive projects, including HBO Go’s Game of Thrones interactive experience
  • 1/3
You are here: Forum Home / Autodesk® Maya® / MEL / Help - Converting double precision floats to single precision?
  RSS 2.0 ATOM  

Help - Converting double precision floats to single precision?
Rate this thread
 
53259
 
Permlink of this thread  
avatar
  • Total Posts: 1
  • Joined: 01 November 2008 12:16 AM

Hi.

Is there anyway to change the precision of a float value manually?
I’m trying to export the normals of a polygon, and fwrite writes out in
double precision by default, but I don’t need that precision.

...
float $norm[] = `polyNormalPerVertex -q -xyz`;
fwrite($fp$norm[0]);
...

thanks in advance.



Replies: 0
avatar
  • Scott D.
  • Posted: 21 April 2011 12:00 PM

You could try the “trunc” command, if you just need the whole number.  For example:

trunc 2.82;
// Result: 2 //
trunc -2.82;
// Result: -2 //

Also look at “ceil” and “floor”, but these also just return whole numbers.

If you really need to convert something like 2.82 to 2.8, by just chopping off the last number without rounding you could do some tricks by making the number a string.

string $num 2.82;
string $tokens[];
tokenize $num "." $tokens;
//startString is a built in Maya mel script, only available in Mel
$firstNumber startString ($tokens[1]1);
string $singlePrecision = ($tokens[0] "." $firstNumber);
// Result: 2.8 //


Replies: 0