[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [nm-wg] treatment of lost packets when measuring delay



Loukik,
Two points should be considered:
[] one of the (many) advantages of percentile-based statistics is that you can do it either among the entire data set (counting the losses as infinite delay) *or* among any subset (e.g. those with finite delays) if you have reason to believe that that subset has significance.

[] note that, even apart from any debate about how to treat losses, the distribution of delays is very often heavy-tailed. Thus even to talk about means and (especially) standard deviations carries implicit assumptions about the mathematical nature of the distribution. Thus, even though the non-math-majors among us are naturally drawn to averages etc (the statistics we've been taught since third grade or so), we should understand that in the land of heavy-tailed distributions, these are suspect.

By the way, I agree with Stas's note. But I offer these two points in addition.
Regards,
-- Guy

--On Thursday, April 01, 2004 11:38:02 +0100 Loukik Kudarimoti <loukik.kudarimoti@dante.org.uk> wrote:

stanislav shalunov wrote:

Loukik Kudarimoti <loukik.kudarimoti@dante.org.uk> writes:



During the TPM workshop, we realized that there is a need to come to a
common understanding of OWD data representation ( esp. treatment of
packet loss ).


For what it's worth, the OWAMP specification is written so that send
times of all packets are known with fair precision to the receiver
(despite the element of pseudo-randomness in the timings).  Then, if a
packet does not arrive within a specified timeout, it is considered
lost; the send timestamp of such a packet is known and reported.


My concern is more towards the representation.

When interpreting the results, lost packets simply have infinite
delay, don't they?  This makes certain statistics meaningless (such as
mean delay), but if a value of an estimator becomes undefined because
of the presence of a small number of infinite values, the estimator is
not robust, and, therefore, should probably be avoided anyway.
Percentiles in general do not suffer from this problem.  (Harmonic
average works fine with infinite numbers, too, if one wanted to insist
on using non-robust---but more robust than mean---averaging
mechanisms.)



If ten packets were sent between times t1 and t2 and 1 was *lost*
(referred to as infinite delay), we report that *1 packet between times
t1 and t2 was lost* and 9 packets have an average (arithmetic mean )
value v1, min value v2, max value v3 and 95 %ile v4 (extensible to
include other types of aggregations as well).

A full report ( with no aggregations ) can also be provided. Now in such
a report, whether we show packet loss as infinite delay or report it as
packet loss still needs to be discussed.

Regards,
Loukik.