As for the shoot-out, wouldn’t that be nice? But I seem to recall at least one big EDA vendors End-User License Agreement expressly prohibiting publication of benchmark results, no matter if the benchmark ended in a win for them, a loss or a draw. Wonder if that’s still the case.

Some circuits do require accurate simulators to get the correct behavior and noise characteristics, but who is prepared to pay the extra for a simulator guaranteed to do it right?

What is the need for high accuracy in solvers? It’s really a case of how much precision is needed for the task at hand, no? If you’re designing a power grid, can you actually get your job done with 2 digits of precision? There are many design groups who’d say not. There are those who want SPICE to deliver more than the customary 6 digits (I know you can use a .OPTIONS card to print more significant digits if you wish.) Frankly, solvers that deliver fewer than 6 digits, or worse that vary the results depending on some unrelated parameter (number of threads is one that bugs me), are playing a shell game, just moving the error around.

I look at it this way: if you’re trying to determine a parameter that’s varying in the nth digit, you need to have tools accurate to the (n+2)th digit in order to have a chance of getting 1% accurate results. It’s not uncommon for n to be 6 today. Hence, solvers accurate to 8 significant figures are required. That’s non-trivial, and compute intensive. There’s some very interesting innovation in this area that should see the light of day some time before next DAC. Watch this space!

It is not that EDA players sweep the limitations under the carpet. More often, EDA falls into the trap of “averaging”. For example, if you take two test cases and if a tool shows +50% in one test case and -50% in another, the tool vendor could claim as average accuracy of 0%. While a tool that consistently gets +0.5% will appear bad. Another trap is that EDA vendors are making a general purpose solution that works for all cases. This is because the more applications they can run, they can attract a bigger market. Often times, even the most innovative companies fall into this trap. Focusing on a specific problem and solving to perfection may be the right approach but the tool is losing out on other markets. Today EDA is dominated by the big ones - C, S, M and M. These companies own probably 90% of the market. Any new player has to challenge this might in order to succeed. And the market is shrinking anyway. Big companies kill innovation through many means, popular are roadmap and lawsuit (You must be well aware of #2). Even for the customers there is no incentive for promoting innovation. Imagine, if you have a tool that provides orders of magnitude higher accuracy, do you think customers will start using this right away? Today, you need to read various SPICE netlists, must have ADE integration, support whole bunch of features before a customer even looks at the tool (believe me, this is based on personal experience). Even after you cross this hurdle, the customer will be saying, OK, we dont have any budget, because the ___(fill in your best EDA vendors’ name) sales guy tool the whole budget. This is even before the competitive tools play their roadmap/lawsuit card. The key thing is even brilliant algorithmic expert gets dragged into all these issues rather than providing highest accuracy or precision. It is no wonder that most EDA companies aspire their exit to be acquisition.
One last comment: You have discussed the need for precision/accuracy during extraction. But you havent explained the need for higher accuracy in solvers.