(no subject)

[ny_quant]

Добавление ко вчерашнему:



the data in Figure 2 shows a decrease in infection rates after countries eased national lockdowns with >99% statistical significance""

Comments

[ny-quant.livejournal.com]

An alternative process is commonly used:

1. Compute from the observations the observed value t_obs of the test statistic T.
2. Calculate the p-value. This is the probability, under the null hypothesis, of sampling a test statistic at least as extreme as that which was observed.
3. Reject the null hypothesis, in favor of the alternative hypothesis, if and only if the p-value is less than (or equal to) the significance level (the selected probability) threshold

What exactly is null hypothesis? That we got our "good fit" by accident, right? No distributional assumptions are required.

Случайно наткнулся на экономиста, который решил написать серию заметок типа введения в статистику.

https://hroniki-paisano.livejournal.com/118867.html
https://hroniki-paisano.livejournal.com/119264.html

Интересно, что он тоже не заморачивается спецификацией нулевой гипотезы.

[misha-b.livejournal.com]

We need a model for point two above: "This is the probability, under the null hypothesis, of sampling a test statistic at least as extreme as that which was observed."

For example what is 99% in your original post refers to? I have no idea, but here is one possibility: assume that the point has equal probability to be above and below the line and assume that the points are sampled iid. Under this model the picture shown in the graph is very unlikely and we can reject this hypothesis. However, iid is an key assumption here. If these data are not iid, this conclusion in general cannot be made.

For example, imagine that the rates are perfectly correlated between the states. Then either all of them will be below the line or all of them will be above the line.

[ny-quant.livejournal.com]

I certainly agree that one needs to state what hypothesis we are testing, i.e. what we want to accept in the end, or in the worst case reject. I'm sure the author of this note did have a very well-defined hypothesis but this little detail didn't get into condensed summary that was available to me.

I thought your original complaint was that the model that we hope to reject based on low p-value is not stated.

[misha-b.livejournal.com]

That's right, the issue I had was using a very suggestive number, like 99% without explaining the model. It is quite possible that the author had a very reasonable model. But I have seen this misused or abused many times, so I am by default skeptical.

[misha-b.livejournal.com]

99% is not the worst actually -- sometimes people have 99.9999% or something like that and you immediately suspect something is fishy.

[ny-quant.livejournal.com]

Я прочитал вашу статью. Во-первых, я, конечно очень слабо разбираюсь в теме: Decision Trees and Ensemble Methods единственное что я хоть как-то понимаю. Во-вторых, мне очень трудно привязать ваши результаты к какой-либо конкретной задаче. В моём маленьком мире очень мало полезных фич (по крайней мере я так думаю) и очень много шума. Было бы хорошо иметь перед глазами какой-то простой конкретный пример когда действительно получается омега-shaped curve.

[misha-b.livejournal.com]

Sorry, forgot to answer this. We did not want to call the curve omega-shaped or W-shaped because the second part typically does not rise. It is "cursive-u" shaped in most examples we looked at.

You can take a look at a simple synthetic model with linear regression, where the features of this curve can be seen: https://arxiv.org/abs/1903.07571