Change to using the Havok 3 physics solver when you are using only rigid body simulations. it is dramatically faster. I tested with your file and didn’t notice much if any slowdown compared to the Havok 1 engine.
You might consider increasing the Deactivator settings.
It keeps things from wiggling when they are supposed to be stationary.
I also noticed you said you use 100 for mass...mass is in kg...220lbs dominos?
As annoying as simulating reality can be...try to match the real world as much
as possible, it makes things more predictable. That way, when you want to tweak
reality you at starting from more “stable” settings.