Science applied to fictional turrets

This weekend, I randomly found myself reading this blog entry by Azual Skoll at The Altruist espousing what the author claimed was a "scientific" refutation of the claims made here by Ripard Teg. It's very easy for people to (whether by innocent error or malicious intent) to make flawed analysis of things, and because we live in a world where people are not expected to have a sound understanding of probability and statistics, there is often not the scrutiny applied to these analyses that there otherwise should be, and when I ready Azual's entry, my "bad science" alarm went off immediately at the lack of any words like "likelihood" or "expectation value" so I decided to check for myself.

Before I make my criticism, I want to make something very clear: there is much that is laudable about Azual's post: there is a clear statement of methodology and the construction of hypotheses designed to test a premise that are clear, cogent, and repeatable. All of the basic foundations of his analysis are present and correct. Azual's post is not bad science by any measure that you see around you in the real world. No person reading his post should ever for one moment think "this is a bad way of doing things". It isn't. His opinions are clearly separated from the data which he collects and analyses and at no time does he try and state that one is the other. This is rarely found in today's world, and should be held up as a good example of others. It was, for me, lacking one crucial ingredient, and this will hopefully become clear as I fill in the approach I took in checking the analysis

Theoretical Analysis

There is a glaring preface I need to make to this. I have approached this purely from a mathematical treatment. I haven't even logged into Eve in order to test these numbers - this is a spherical cow analysis designed to isolate the important behaviour without extraneous variables. Weaknesses in the analysis can therefore come from three sources:

1. Mathematical error - It's been a long time since I left university. Even if I did spend 9 years of my life there in labs and in front of analyses, I'm rusty. Even if I wasn't, don't take what I say as gospel. That's bad science. Check for yourself.
2. Exclusion of "real world" considerations that have material bearing on the behaviour of the model - as the people who pioneered QED found out: your theory may produce some of the most precise agreement with experimental measurements that has ever been, but if it claims that all of your particles have zero mass when they clearly don't, then there's something missing
3. I got a formula wrong in my spreadsheet. Never discount human error.

I also have a pre-conceived opinion that larger guns do disproportionately well against smaller targets (especially as compared to missiles) and whilst I have tried to keep this analysis free from bias, such things can always colour the approach I take.

Expectation values and DPS

What I found missing from Azual's analysis was any discussion of expected DPS. Firing guns at someone is a probabilistic event which I am modelling as a single event independent of gun grouping. As per evelopedia I am defining the applied DPS, based on a base DPS D_o as follows

where x is a random number generated in the range [0,1[
Pcrit is the probability of a wrecking shot
Phit is the probability of the shot hitting.
Note that this is explicitly for the condition

 The opposite case I'll mention later. The question then becomes, given the model defined above, what is the expectation value of D that we can expect? I'm going to define this as the sum of two sub-terms, W (for wrecking) and S (for standard), and talk about each one separately.

Random numbers and binning

I'm going to assume that the random number generator is perfect in terms of quality, but that it distributes numbers in bins of width k and treat this as a finite discrete series of data rather than a continuum. This simply is an acknowledgement that numbers are generated to a finite number of decimal places, and the change this makes when comparing to a continuous distribution. If we convert the second case (S) where the random number lies in the "normal hit" range, then we note the following:

• First value of x: Pcrit + k
• Last value of x: Phit
• for each bin, i

where Nb is the number of bins in the range we care about. This allows us to derive the expectation value term S.

which if we expand out Nb

Finally, in the limit that k tends to 0 (which is almost true) we can write S as:

The derivation of W in the same condition is trivial, and not included here. We will merely  state it as

allowing us to state the expectation value for the DPS as follows

This gives us a baseline estimate for comparison of different turret types for instances where the probability to hit is greater than the wrecking hit chance. In the case where this is not true, it is trivial to see that S=0 and W only applies (up to the point where x=Phit and is 0 otherwise).

Evaluation

Having constructed a workable formula, let's apply the situation as tested by Azual

it should be noted that I got the raw DPS figures from pyfa and the fits provided in Ripard's post. Everything else was sourced from Azual's analysis. The actual value of the DPS figure is less important than the values relative to one another (because of resists etc that will change the applied DPS).
At first blush, this seems like a solid confirmation of Azual's analysis. Everything is as you would expect. What caught my eye, though, is that Azual made the choice to substitute the Uranium in Ripard's fit for Iron. Without wanting to put words in Ripard's mouth, a reason to choose Uranium is that the optimal range is the same as a Ferox with spike. Of course this might not have any effect, but this needs to be established.

As you can see the picture is quite different. The larger guns are performing significantly better against a small mwd target (even with tracking computers) than the large guns at the 60-90km range, but the signature of an mwd cormorant is large, so this is not unexpected. The results at 300m/s (and 90 sig radius) are as follows

Iron Naga - 300m/s

Uranium Naga - 300m/s

Again, the naga massively outperforms the non-TC'd ferox when equivalent range ammo is loaded.

Conclusion to the analysis

From this short analysis, it would seem to be the case that the increase in DPS afforded by using shorter range ammunition much more than compensates for the loss in DPS caused by the difference in sig radius in the absence of tracking computers. Elimination of tracking as a factor shows that even for a good percentage of the stated engagement envelope, the increased DPS matters more than the difference in sig radius. Further extensions to the analysis would be to examine how the dps profiles change with further changes in ammo type as the range decreases.

Conclusion to the criticism

As I took great pains to point out at the start, there was nothing WRONG with the analysis Azual performed, but it lacked a theoretical underpinning to establish the validity of some of the choices made, and in the case of ammunition choice, this would have had an affect, though not on all of his conclusions, and as it's a quiet sunday evening, I'm indulging my pedantry :)