Whenever software crashes, it is a bad thing. I think we can all agree on that. What a lot of people may not realize is that what causes the crash can be influenced by the environment they run the software in. While we always have to push harder to improve the software stability on our side, there are things customers must think about if they want to maximize stability with any release or really with any software that is as complex as 3ds Max.

Measuring Stability

First of all, let me tell you how we measure stability of a release, we have various tools at our disposal:

1. Customers send us CIP data packets (optional). These are able to answer important questions like “did the session crash or exit cleanly” and “how long did the session run”? We take the tens of thousands of these packages we receive monthly and we process them to generate a stability and a mean-time between failure (MTBF) metric. Please keep sending us CIP data, because it is what all Autodesk products use to review their quality and this is done at the most senior level of the company. It gets a lot of attention.
2. Customers also send us CER reports. These are anonymous crash incidents that we mine for statistical groupings that represent common crashes. The bigger the bucket, the more important it is to fix. The number of CERs is basically meaningless because it is influenced by too many other factors, but it is critical for actually fixing crashing software. Each product team has to set a high goal for resolving crashing bugs post release. Service Packs and Hot Fixes are used to meet the goals we’ve set for the release.
3. We survey the beta software before we release it. Beta customers tell us what frequency of crashes they are seeing and how they think the stability compares to prior releases.
4. We look at bugs found by beta and our QA group and determine how we’re doing compared to prior releases and our shipping goals.
5. We look at bugs reported on the forum and elsewhere to see if there are any legacy issues we need to deal with.

3ds Max 2010 Stability Compared to 3ds Max 2009

Now, let’s talk about how 3ds Max 2010 looks based on our CIP data, our dev manager, Claude Robillard offers these insights:

On the CIP side, we are seeing a pretty large improvement from release to release with 3ds Max 2010. Some of the data points are things that only we will see internally and others are things that our customers should be able to feel.

Here are the main two points:

1. Compared to the 3ds Max 2009 at the same time last year, we are seeing total crash rate reduction of somewhere between 20% and 50% for 3ds Max 2010. We are working to improve the CIP data so that we can more accurately pin it down, but we know for a fact that it falls in that range.
2. Fixing some problems with unclean exits has also helped with our MTBF value. It’s doubled from release to release, but we’re not sure if this is related to the customer experience or not. We know for sure that it improved, but we’ll need more data to prove the trend.

Last point on how customers perceive the new release. The initial crash rate of 3ds Max 2010 is 50% lower than the initial crash rate for 3ds Max 2009. This means that “as shipped” and before we released a Service Pack or Hot Fix.

3ds Max 2010 CER Data

What happens with the CER data you submit? Chris Johnson in QA gives us a summary of the most recent data:

For 3ds Max 2010 we have 124765 crashes that customers care to report to us. (As of an hour ago). These crashes land in 12698 different buckets. A bucket is a place where the application crashed. So on the surface it looks like max crashes in 12698 different places. That is not entirely true. The CER system does NOT group 32 bit buckets and 64 bit buckets together. Rather it keeps them separate. So the same crasher could land in two different buckets (32 bit or 64 bit). This of course will appear to inflate the number of places where max crashes.

Of these 12698 buckets, I have looked at, and assessed 182. They are generally the top buckets that get hit by customers. If I could, I would try to assess every bucket, and see what the problem is with it. But that is impractical. But it is helpful to look at what has been analyzed, and see what type of categories the buckets can be grouped into.

I took a snapshot of ALL the CER's (All 124000+) and looked for broad patterns. Here is what I found:
• Out of Memory - 8%
• Mental Ray - 5%
• Reference System - 7%
• .NET and WPF - 16%

Now the numbers are really rough, but they give us an idea of the relative numbers (not absolute) of our CER reports. Now some of the buckets can fall into both categories. For instance we have buckets where mental ray runs out of memory (quite high on the bucket lists btw).

Now what has been fixed? Generating a new CER report with all the data an hour or so after the first one shows we now have roughly 41% reports fixed.

That means of all the random bugs submitted from users all over the planet we’ve been able to fix a huge percentage of our most commonly perceived bugs with our Service Packs and Hot Fixes. Remember, we focus on the top buckets – these are the ones you’re most likely to experience. Since so many things outside of our control can crash 3ds Max, fixing 100% is impossible with something like several hundred thousand users trying to use our software in every imaginable environment – many of which we explicitly say we don’t support.

How can you improve your stability?

1. Easiest, submit your CERs!!! I don’t know how many times I ask if a customer who’s complaining about stability and they have never bothered to submit a CER to us. I’m sorry, I don’t have much respect for people who complain but don’t want to press a button to help us fix the problem. Every CER counts because we use it to drive a statistical analysis that says what we’ll work on next. We review these weekly. BTW, we now have enough CER data for multiple releases to know if one release is “more stable” than another release. 3ds Max 2010 is as stable if not more stable than prior releases based on this measurement. Every release since we introduced CER has been generally an improved release because we are mandated to fix a large proportion of the CERs. With 3ds Max 2010, we have actually fixed the highest number of CER crashes than any prior release. You'd experience this stability if you're working with our Service Packs or Hot Fixes.
2. Get more RAM! Easily the most common problem with stability. If you’re not using 8Gb RAM and 64bit OS, then you’re going to run into memory limits. 2Gb is definitely not enough, 4Gb is barely enough. Those using 32b OS, need to use the /3Gb memory extension. Move to 64b - it is important. Also, too many people set their page file setting too low. 4Gb at least.
3. 3ds Max 2010 uses the video GPU resources more than ever - so it isn’t surprising to hear that things might look/work differently WRT 3ds Max 2009. We get blamed for problems that are actually in the video driver. Make sure you’re using a recent DX10 Game or Quadro board (preferably one certified by us on our web page where we list all the qualified hw) along with the most recent video driver. The fact that we’re trying to optimize for GPUs means that some GPUs/drivers may be more sensitive to our code. Think about it. If you use the minimum amount of GPU resources then it works everywhere and on anything - it just is slow. Instead, we’re trying to take full advantage of it and this exposes us to more GPU dependencies. This is going to be more true in the future, so you might have to be more careful in buying your GPU and listen to our recommendations more.
4. Third-parties can cause complexities that affect us. Not all plug-ins are created equally - some have larger QA teams than others. Because of the open nature of our plug-in architecture, it is possible for plug-ins to affect our stability. The people that use “clean” installs of 3ds Max do report the fewest problems. We know how important plug-ins are to some of your workflows, so we're not suggesting you do without them. But you should see if a third-party might be contributing to the instability. Our CER reports tell us which plug-in is causing problems and we then generally try to work with the plug-in vendor to resolve it.
5. Log your bugs: http://usa.autodesk.com/adsk/servlet/item?siteID=123112&id=5600504&linkID=9241177

Yes, there are bugs in everyone’s software, including ours. Some software is less complex than other software and is less impacted by environmental factors. I think with every release there is somebody on a forum somewhere that says that this particular release is the “worst ever”. It probably is for that person, but what did they change in addition to 3ds Max? Perhaps the old plug-in they were using has been recompiled for the new version of 3ds Max - but it introduced a problem. Perhaps they are working with much more complex data than they used to?

When we tell you that 3ds Max 2010 is the most stable release we’ve ever shipped it is because that is what the data is telling us. This doesn’t help you when you’re the person experiencing instability. But hopefully, if you follow some of the suggestions described here you can improve your experience and help us make the software even better than before. I hope this has helped.