[nm-wg] latest request schema requirements doc

["Leese, MJ (Mark)" <M.J.Leese@dl.ac.uk>]

Hey Folks,

A new version of the request schema requirements document is now available
here: http://gridmon.dl.ac.uk/nmwg/request_schema_requirements-01d.pdf.
There are a few changes, mainly summarising our discussions of the
"outstanding issues" section made during phone calls on the 10th and 31st
August (described briefly below). Please say if there's anything I've
misreported, or there are any mistakes.

In terms of getting the requirements into a stable state, there are still
outstanding issues. Some are fairly minor, and some are not so minor, e.g.:
* using transaction IDs to help with tracking batches of results
* harmonisation of the request and response/publication schemas

I'll let you know what the next step is once it's been agreed. I say this
because the future of the current schemas isn't clear right now. During the
NM-WG session at GGF12 in Brussels last week Dan and Martin presented a new
approach that most people liked the look of (myself included). It was agreed
that they should press ahead with this work, but as a consequence it's not
obvious where this leaves the current schema work.

Things will hopefully be clearer to you when you've seen the notes from last
week. As I see it (and please feel free to say if you disagree) the two
choices are:

1. Complete the current work, allowing people to press ahead with some
implementations, *BUT* in the knowledge that an "upgrade" is planned.

2. Stop work on the current schemas as they stand and work on the new
approach which (I'm told :) will produce "normalized" schemas that in turn
provide an extensible framework. The current schemas would then be
refactored into the new framework.

Hopefully I've explained that properly ;-) Paul and I will circulate the
notes as soon as we can, but it was a seven hour meeting, so it may take a
while :-(

Cheers,

Mark.


1. Interim Communication, that between a requestor making a request and the
final results being returned, e.g. requestor and responder negotiating over
the duration of a requested iperf test:

We were hoping that the proposed WSRF standards may provide solutions in
this area, and we still are. However, I think the first thing we need to do
is debate exactly what we actually need, e.g. negotiation of test
parameters, then see what WSRF (and others) will provide, producing our own
solutions as appropriate. The point was amended accordingly.



2. Requiring the request schema to support more complex messages, combining
requests for both raw measurements and statistical summaries, e.g. "Give me
the last five raw values of this characteristic and the average over the
last day":

This isn't possible with the current schema, based on the current
requirements. To support this now would be a fairly radical departure from
where we are at the moment, so we'll stabilise on what we have, and add this
later. This *could* be addressed by the "Simplified Request/Response
Framework" work proposed by Dan and Martin.

So, this issue disappears from "Outstanding Issues" and re-appears in the
"Desirable Requirements" section.



3. Sharing of components between request and publication schemas:

Comparison of the schemas has started, but this is really an ongoing issue.
Point updated to say that work has begun.



4. Using wildcards for characteristic names, e.g. "path.delay.*" would match
against "path.delay.oneWay" and "path.delay.roundTrip":

The original suggestion was for "path.delay.*" to be interpreted as "give
me/measure for me all the delay characteristics that you can". This is the
logical interpretation of a wildcard in relation to a received request. This
was deemed to make sense for historic data, but not for future tests (it's
unlikely you would want two or more delay metrics for the same route and
time).

However it was also suggested that an implied behaviour could be defined,
where "path.delay.*" is interpreted as "tell me what delay characteristics
you have/can measure".

We decided that this second, ad-hoc capability discovery use made most sense
to us, even if not very logical in terms of interpreting the schema. "*"
would give all characteristics a node had, or is capable of measuring. This
was thought preferable to adding a capability discovery flag. This can of
course be changed later if:
a. actual use shows we would like to make wildcard requests for data
b. we/others (e.g. WSRF) develop methods for capability discovery 

So, this disappears from "Outstanding Issues" and reappears as a hard
requirement.



5. Specifying pre-defined set of tags for test (tool) parameters, e.g.
<tcpBufferSize>:

Still to be defined, so no change.



6. Are two lists of parameters specific tags required, with one for each end
of a possible test path (source and destination)?:

This referred to how we would specify test parameters when we wished to use
a different value at each end of a test path, e.g. port 5001 at the source
end, and 5002 at the destination. The original suggestion was to define
separate "src" and "dest" tags for test parameters that could be different.
An alternative is to use one sets of tags, which can be made specific to an
endpoint using an attribute. Both are shown below:

  <protocol>TCP</protocol>		<!-- common to source & dest -->
  <sourcePort>5001</sourcePort>	<!-- specific to source end -->
  <destPort>5002</destPort>		<!-- specific to dest end -->

An alternative is to use one sets of tags, which can be made specific to an
endpoint using an attribute:

  <protocol>TCP</protocol>
  <port endpoint=source>5001</port>
  <port endpoint=dest>5002</port>

We actually decided to go for the second option. If source and destination
are different, then we require two pairs of tags in both cases. But if they
are the same then the attribute version requires just one tag. Since in most
cases we'd expect both ends to be using the same values, it's easier to use
an attribute for those rarer, special cases.

Unless it proves difficult or messy during schema implementation, it should
only be possible to use the endpoint attribute with parameters that can be
different at each end, like port. This saves the system receiving a request
having to trawl through a malformed requests deciding whether...

  <protocol endpoint=source>TCP</protocol>
  <protocol endpoint=dest>UDP</protocol>

...is valid.

So, this disappears from "Outstanding Issues" and reappears as a hard
requirement.



7. Specify source parameters as parameter specific tags, and destination
parameters as a parameter list, and vice-versa:

Think there may have been some confusion here, so the point has been left as
it is.



8. Seeking advice on the mechanisms for allowing custom tags to be added to
messages:

This would allow implementers (perhaps within a single administrative
domain) to add their own specific tags to requests. Perhaps someone (just
within their domain) would to be able to request that a copy of any results
be sent to a specified second system as well as the original requestor. This
could be achieved with additional <copyTo>a.b.c.d</copyTo> tags.

I don't think there was any real change here, other than to say that this
has serious implications for validating incoming request messages against
the schema. Someone will be actioned to investigate further, especially with
respect to any possible recommendations defined by WSRF.



9. Inclusion of field for transaction IDs for tracking requests and
responses:

No real change, but the issue is clarified by very briefly outlining the two
options.

Transaction IDs are not essential. Clients could be required to keep track
of the requests they make based on the details of those requests (who the
request was sent to, the requested characteristic, test end points, time
range etc.). If this information is parroted back to the requestor in
response messages, it can associate incoming responses with the
corresponding requests it made.

The other option is the receiving system to generate a ticket number when a
request is made, and send this back to the requestor. This would make it
easier for both requestor and responder to deal with multiple requests and
partial responses (batching of results) because transactions could be
tracked using just one field (the transaction ID) and not a set of fields.
This would however require the responder to be stateful.