General discussion / feedback from Leshan Team

[sbernard31]

Summary :

Leshan is a java implementation of LWM2M and until now it was strictly based on californium.
Recently we did massive changes to abstract transport Layer. (See : eclipse-leshan/leshan#1025)

Now to test this abstraction we are playing with java-coap (See : eclipse-leshan/leshan#1373)

I will use this general issue to ask questions and provides feedback about it.

Some questions :

1. In californium, there is a endpoint concept. If I simplify the concept a lot this is the socket where message are received. In californium a coap server can have several endpoints. If I understand java-coap correctly, 1 server is only associated to 1 socket right ?

2. is there a way to check if a message (request or response) is CON or not ? something like : if (!CON.equals(coapRequest.getType()))

3. is there a way to set a message (request or response) as CON ? something like : coapRequest.setConfirmable(true);

4. is there a way to execute code after the message( mainly needed for response) is sent ? Maybe a callback once the message is sent ?

5. Patch, IPatch and Fetch seems not implemented ? correct ? is it planned to implement it ? (I think we mainly need Ipatch and fetch for lwm2m composite operation)

6. If you are curious you can have a look at how we are using java-coap, See : https://github.com/eclipse/leshan/compare/5b0b66fa675da2ad9224d15e828036d0c06d0f40
   E.g : I created a kind of ResourceService to be able to add some Resource (see RegistrationResource), I don't know if this is a good idea or if this could give you some hint about changing java-coap API ? 🤷‍♂️

(more question is coming soon 😁 )

[szysas]

Some answers

1. In java-coap, socket is abstracted with CoapTransport interface. If you need to use multiple sockets, then it can be done by creating CoapTransport adapter that will proxy to multiple, underlying instances. Out of curiosity, what is the use case when you want to receive from multiple sockets?
2. At the moment there is no way to check it if incoming request or response was NON. If needed this info can be easily added to TransportContext (lets create separate ticket).
3. By default all outgoing requests are CON, to change it to NON one need to set it in TransportContext, for example:
   https://github.com/open-coap/java-coap/blob/master/coap-core/src/test/java/com/mbed/coap/server/messaging/ExchangeFilterTest.java#L105
4. Not sure if I get you point, but you could wrap CoapTransport class and overwrite sendPacke method, there you could add callback to CompletableFuture.
5. Yes, it is planned.
6. Thanks for link, I will have a look.

[sbernard31]

thx 🙏

1. I don't know if I really need to have several socket by coapserver 😬. What I know for sure is that 1 LWM2M server should have several socket open with different protocol. n socket for coap, n socket for coaps, n socket for coap+tcp etc etc... This is just different way for a client to connect/communicate to a server. With Californium I create 1 coap-server with several endpoints. And as I have Californium design/API in mind, I tried to do the same with java-coap... But maybe I should just create several coapserver ?

2. Being able to check if a message (request/response) is a CON ? #29 I created the ticket but no urgency to work on it. Maybe better to first identify all problems :) (unless you're sure you want to add it to java-coap)

3. I will look at this.

4. I will try it.

5. Good to know.

6. ⚠️ this draft code not so clean. (do not hesitate to ask question if you see something not clear)

[szysas]

1. I think the only way would be to create multiple CoapServers, or on runtime client can create the one that is needed.

[sbernard31]

Some more questions about observe. I think this will be one of the big part.
Considering observe, a LWM2M server acts as a CoAP client (it sends request and receive notifications)

o1. I find API to send the observe request but didn't find how to get notification (I guess I should use ObservationConsumer but I'm not sure how)

o2. Leshan server based on Californium is able to persists "Observation", so if the server reboots no need to resent a new Observe request to new handle notification. Do you think this is doable with java-coap ? (if yes, could you share some hint)

[sbernard31]

I added a TokenGenerator to my experimentation, for now I go with a "not so java-coap" spirit way. (See eclipse-leshan/leshan@914f0d4). I guess it's OK for our POC.

But just by curiosity, reading #24 (comment), I understand there is maybe a more elegant way to do with filter ?

I try to investigate a bit but I'm not sure to find how I should add a filter to my client which I created like this CoapClient coapClient = CoapClientBuilder.clientFor(destination.getIdentity().getPeerAddress(), coapServer);

I guess that I should customize server.clientService() ? so maybe I need to create CoapClient without using Builder ?
Or maybe there is a way to customize clientService when I create coapServer but looking at the builder I didn't find anything.

I see there is an MIDSupplier but not a Token one.

I also maybe find a typo in coap server should inboundService be called outboundService ?

[szysas]

o1. To receive notifications, you could something like this:

        CoapClient client = CoapClientBuilder
                .newBuilder(new InetSocketAddress("localhost", 5683))
                .build();


        Function<CoapResponse, Boolean> consumer = coapResponse -> {
            System.out.println(coapResponse);
            return true; // return false to terminate observation
        };
        // send request to observe resource, observations will be handled in consumer callback
        CompletableFuture<CoapResponse> resp3 = client.observe("/sensors/temperature", consumer);

Note that this sets separate callback per observation relation.
I'm thinking to simplify it and set one observation consumer in CoapClientBuilder, what do you think?

o2. Currently that is not possible. How important is that feature for integration with Leshan?

[szysas]

Related to TokenGenerator, yes, it would be more 'elegant' and testable to create a filter. However I noticed that there is a missing method in builder class to add addition filters. I will try to improve it.

[szysas]

I also maybe find a type in coap server should inboundService be called outboundService ?

You're correct, thanks for pointing this out.

[sbernard31]

o1. I will try to play with your example then I let you know if 1 observation consumer seems better to me.

o2. It seems that users like observe feature but me not that much 😁 ... I try to rather encourage to use LWM2M Send Opeation which is a most simple alternative.
Why I don't like it ? because we faced lot of issues with observe until now. If you're curious you could have a look at : https://github.com/eclipse/leshan/wiki/LWM2M-Observe
So don't worry, I will be very surprised if java-coap fully support our needs out of the box concerning observe.

Currently that is not possible.

Maybe, it makes sense that some implementation support to persist observe relation and some others don't ? 🤔
So we could have a coap(s) support based on californium with persisted observation and coap(s)/coap(s)-tcp support based on java-coap without it ?
I need to check if I can modify Leshan in that way.

How important is that feature for integration with Leshan?

For some use case this could be very important. Without going too deep in the details. I guess when you create a LWM2M server based on Leshan and you manage lot of clients in production with lot of Observe Relation. When you reboot your server, this will be an issue to resend all observe requests. (For sure at Sierra, we need it but at least at short/mid term we will still use Californium and Scandium for coap/coaps in production)

[szysas]

I could not agree more with your finding about observe.

Related to storing observation relations, it comes down not only to store Token->UriPath but also IP address... which is very fragile device association because those will change (NAT).
Anyway, at least for that reason we would need to have single a single observation consumer.

[sbernard31]

Thats seems to confirm that implementers don't like so much "Observe" where users like it 😅

Related to storing observation relations, it comes down not only to store Token->UriPath but also IP address...

Ideally even more than a IP address, we find that to really make it work, it's better to store a kind of peer identify which are not the same depending of transport used or/and use cases.
(coap => peer address / dtls => could be a connection Id or a psk identity or a public key .... / coap+tcp => a tcp connection ?)

About o2) I just realize that I talk only about persisting observations but actually this is more than that...
Most of people who use Leshan in a production server want to be able to launch it in a cluster and so observation relation must be shared between all instance of the cluster. Currently in Californium this is achieve with a ObservationStore interface.

Let me know, if this is something you would "like" we explore together (first just by talking about it) or if you are too traumatized by "observe" feature 😁

[sbernard31]

In com.mbed.coap.packet.Code, should it be ?

     C201_CREATED(2, 01, 201),
-    C202_DELETED(2, 02, 200),
+    C202_DELETED(2, 02, 202),
-    C203_VALID(2, 03, 200),
+    C203_VALID(2, 03, 203),
-    C204_CHANGED(2, 04, 200),
+    C204_CHANGED(2, 04, 204),
-    C205_CONTENT(2, 05, 200),
+    C205_CONTENT(2, 05, 205),

[szysas]

No, third parameter is HTTP status code.

[sbernard31]

No, third parameter is HTTP status code.

🤔 So I probably totally missed the purpose of this. Could you give me some hints ? 🙏

I was thinking this was just a way to get the code as int but not as it is encoded in CoAP (only 1 byte)

[szysas]

Yes, it is used to translate (proxy) with HTTP

[sbernard31]

I have a first working version of observe with java-coap at LWM2M server side.
If you're curious, the first draft looks like this.

Without support of cluster, I don't need "single observation consumer" (more than that if you look at the code I think I could not implement if with 1 single observation consumer because too much information is missing in coapresponse)

[szysas]

Please create separate ticket about observation consumer and let's discuss more there. For start, what context information would you need?

[sbernard31]

Just to be sure this ticket should be about :

I'm thinking to simplify it and set one observation consumer in CoapClientBuilder

and ? or ?

persist observation / observation in a cluster

[szysas]

About storing observation relations.

[sbernard31]

Please create separate ticket about observation consumer and let's discuss more there. For start, what context information would you need?

This is done : #36

[sbernard31]

(After an huge refactoring in our integrations tests eclipse-leshan/leshan#1425 😫 which will allow to reuse most of our tests with java-coap, I'm back to work on this again)

I'm currently using coapRequest.options().getUriQueryMap().
It works well when all part of the query are key value (e.g. field1=value1&field2=value2&field3=value3)
But doesn't work if some key has no value (e.g. field1=value1&field2&field3)

This is needed in LWM2M for Q option of Register operation. (see : LWM2M-v1.1.1@transport§Table: 6.4.3.-1 Operation to Method and URI Mapping (Registration Interface))

Just in case I double check if this is valid in CoAP, I think it is the RFC says :

The query serves to further parameterize the resource. It consists
of a sequence of arguments separated by an ampersand character
(U+0026 AMPERSAND "&"). An argument is often in the form of a
"key=value" pair.

(source : https://datatracker.ietf.org/doc/html/rfc7252#section-6.1)

8. For each Uri-Query Option in the request, append a single
   character U+003F QUESTION MARK (?) (first option) or U+0026
   AMPERSAND (&) (subsequent options) followed by the option's
   value to |resource name|, after converting any character that is
   not either in the "unreserved" set, in the "sub-delims" set
   (except U+0026 AMPERSAND (&)), a U+003A COLON (:), a U+0040
   COMMERCIAL AT (@), a U+002F SOLIDUS (/), or a U+003F QUESTION
   MARK (?) character to its percent-encoded form.

(source : https://datatracker.ietf.org/doc/html/rfc7252#section-6.5)

Let me know, if you want to adapt java-coap code and if I should create a dedicated issue. 🙂

[szysas]

Sounds like a missing feature so if you could create a separate ticket for this. That should not be too difficult to support. Thanks.

[sbernard31]

For some integrations tests, I need to configure CoAP Timeout.
With californium the code looks like :

// configure retransmission, with this configuration a request without ACK should timeout in
// ~200*5ms
configuration.set(CoapConfig.ACK_TIMEOUT, 200, TimeUnit.MILLISECONDS) //
                                .set(CoapConfig.ACK_INIT_RANDOM, 1f) //
                                .set(CoapConfig.ACK_TIMEOUT_SCALE, 1f) //
                                .set(CoapConfig.MAX_RETRANSMIT, 4);

In java-coap, I guess this can be done like this :

CoapServer.builder().timeout(new CoapTimeout(ackTimeout, maxRetransmit));

But there is no way to change ACK_RANDOM_FACTOR which should be what replaces ACK_INIT_RANDOM and ACK_TIMEOUT_SCALE.

For now I fallback with CoapServer.builder().timeout(new SingleTimeout(1000)); in my tests.

If this is something you want to add, I can create a new issue.

[szysas]

Adding parameter for ACK_RANDOM_FACTOR should be trivial thing to do, so if it helps you, please create a separate ticket.

By the way, there was API change on how to set retransmission strategy (it was inspired by one of our discussion in this area). Now it is for example:

CoapServer.builder()
   .retransmission(RetransmissionBackOff.ofExponential(Duration.ofMillis(100), 4))

[sbernard31]

Not really a feature request but just a user feedback about CoapRequest API.

CoapRequest can be modified using some "setter".

Some of this setters are grouped as // --- MODIFIERS ---
other are grouped under // --- OPTIONS MODIFIERS ---

Modifiers does not modify the instance but create (clone) a new CoapRequest from the original with modified field and returns it.
Option Modifiers modify directly the instance and return it.
(not sure I'm so clear 😅)

Maybe there are good reason to do that but from a user point of view, I think this is a bit missleading.

My first try with the API looks like this and it took me some time to find that some 'setter' returns new instances.

// Create CoAP request from LWM2M Read Request
coapRequest = CoapRequest.get(getAddress(), getURI(request.getPath()));
coapRequest.token(tokenGenerator.createToken());  // this is the missleading part. 
if (request.getContentFormat() != null)
    coapRequest.options().setAccept(request.getContentFormat().getCode());

[sbernard31]

I didn't find the way to get token on CoapResponse, is there an API for this ?

I also see there is a way to get PeerAddress on CoapRequest but not on CoapResponse ? (This could be needed for observation but this more or less related to #36)

[sbernard31]

When we handle a message (mainly a request) is there a way to know local address (CoapTransport.getLocalSocketAddress()) which received it ?

[szysas]

Not really a feature request but just a user feedback about CoapRequest API.

CoapRequest can be modified using some "setter".

Some of this setters are grouped as // --- MODIFIERS --- other are grouped under // --- OPTIONS MODIFIERS ---

Modifiers does not modify the instance but create (clone) a new CoapRequest from the original with modified field and returns it. Option Modifiers modify directly the instance and return it. (not sure I'm so clear 😅)

Maybe there are good reason to do that but from a user point of view, I think this is a bit missleading.

My first try with the API looks like this and it took me some time to find that some 'setter' returns new instances.

// Create CoAP request from LWM2M Read Request
coapRequest = CoapRequest.get(getAddress(), getURI(request.getPath()));
coapRequest.token(tokenGenerator.createToken());  // this is the missleading part. 
if (request.getContentFormat() != null)
    coapRequest.options().setAccept(request.getContentFormat().getCode());

I agree, it is misleading. The idea is to have immutable CoapRequest and CoapResponse, so that if it's modified down the pipeline, there is no surprises. Currently header options are not immutable so that is one problem.

Maybe one solution would be to have dedicated builder class for CoapRequest and CoapResponse.

[szysas]

I didn't find the way to get token on CoapResponse, is there an API for this ?

I also see there is a way to get PeerAddress on CoapRequest but not on CoapResponse ? (This could be needed for observation but this more or less related to #36)

Yes, there is no API to get token from coap-response. Observation are wrapped around SeparateResponse class will provide those metadata.

[szysas]

When we handle a message (mainly a request) is there a way to know local address (CoapTransport.getLocalSocketAddress()) which received it ?

No, there is no such an API. What's the use case behind it?

[sbernard31]

The idea is to have immutable CoapRequest and CoapResponse, so that if it's modified down the pipeline, there is no surprises.

It makes sense.

Maybe one solution would be to have dedicated builder class for CoapRequest and CoapResponse.

Yes, when I run into this kind of problem, I usually end up using a builder. (Even if I feel that the builder also has drawbacks but I haven't found a better solution).

[sbernard31]

Yes, there is no API to get token from coap-response. Observation are wrapped around SeparateResponse class will provide those metadata

I'm not sure I get you. Are you saying that I can get an instance of SeparateResponse when I handle notification ?
Here is the code where I would prefer to find Token in CoapResponse : https://github.com/eclipse/leshan/blob/java-coap/leshan-tl-javacoap-server/src/main/java/org/eclipse/leshan/transport/javacoap/endpoint/JavaCoapServerEndpoint.java#L195-L275

I share that, just in case that you think this should be obviously added.
For now I have a workaround and If you are not so sure maybe better to look at that after #36.
(When we will have a better vision of what is really needed for observe support)

[sbernard31]

When we handle a message (mainly a request) is there a way to know local address (CoapTransport.getLocalSocketAddress()) which received it ?

No, there is no such an API. What's the use case behind it?

I suspected that you were going to ask the question 🙂 because that sounds like a strange feature request.

This is not strictly needed but this could make my code a bit simple. (so no urgency to take a decision about this)
When a client register to a LWM2M server, I need to store to which endpoint (server socket address) he connects to.
You could say there is only 1 socket address by coap-server in java-coap, so this is obvious, but :

• when you are using wildcard port, we only know port used once we start the coap server
• not sure we want to go in that way but could helps to support several socket by CoapServer.

Here is the trick I used because of wildcard port :

• https://github.com/eclipse/leshan/blob/5b70e01823ce38a75dddd3871ac0f9fd3b92421e/leshan-tl-javacoap-server/src/main/java/org/eclipse/leshan/transport/javacoap/endpoint/EndpointUriProvider.java#L16-L41
• https://github.com/eclipse/leshan/blob/5b70e01823ce38a75dddd3871ac0f9fd3b92421e/leshan-tl-javacoap-server/src/main/java/org/eclipse/leshan/transport/javacoap/endpoint/JavaCoapServerEndpointsProvider.java#L51-L74
• https://github.com/eclipse/leshan/blob/5b70e01823ce38a75dddd3871ac0f9fd3b92421e/leshan-tl-javacoap-server/src/main/java/org/eclipse/leshan/transport/javacoap/resource/RegistrationResource.java#L167-L174

(Not sure this is so clear 😬 )

[szysas]

One way of passing such information would be through TransportContext, we could put it there and then each received message will contain it, for example:
https://github.com/open-coap/java-coap/blob/transport-decorator/coap-core/src/test/java/com/mbed/coap/transport/udp/CoapTransportWithContextTest.java#L33-L40

In RegistrationResource class you could retried it via:

request.getTransContext().get(SERVER_SOCKET)

Note that SERVER_SOCKET is a context key, and could be defined like:

TransportContext.Key<InetSocketAddress> SERVER_SOCKET = new TransportContext.Key<>(null);

Still it is worth to note that currently, CoapServer does not support reading/writing from/to multiple sockets... so I'm not sure if my above suggestion will be of any help.

[sbernard31]

Still it is worth to note that currently, CoapServer does not support reading/writing from/to multiple sockets... so I'm not sure if my above suggestion will be of any help.

Yep I get it, I think this could help even if multiple sockets in not supported. This way I could remove the EndpointUriProvider class which exist just to be able to know which port is really used when we use wildcard port 0. (See code sample at #27 (comment))

[sbernard31]

Maybe you missed (#27 (comment)) ? Any opinions about it ?

[sbernard31]

I didn't look too much (so sorry if there is obvious API) :

1. What is the API to send notification.
2. Is there a way to send ACK + Separated Response.
3. For a CON Separated Response is there a way to know when the response was Acknowledge.

[sbernard31]

This use case is currently not supported :

RouterService.RouteBuilder routerBuilder = new RouterService.RouteBuilder();
routerBuilder //
        .any("/", new RootResource() //
        .any("/myResource", new MyResource() //
        .any("/*", new Default());
router = routerBuilder.build();

Do you think it make sense to support it ?

[szysas]

Maybe you missed (#27 (comment)) ? Any opinions about it ?

For that, my suggestion from comment: #27 (comment) will solve the issue. So server bind address will be passed in CoapRequest's transport context.

[szysas]

1. What is the API to send notification.

That's not obvious at all. Best it to look into ObservableResourceService class or https://github.com/open-coap/java-coap/blob/master/coap-core/src/test/java/com/mbed/coap/client/CoapClientTest.java#L95.

2. Is there a way to send ACK + Separated Response.

No

3. For a CON Separated Response is there a way to know when the response was Acknowledge.

No

[szysas]

This use case is currently not supported :

RouterService.RouteBuilder routerBuilder = new RouterService.RouteBuilder();
routerBuilder //
        .any("/", new RootResource() //
        .any("/myResource", new MyResource() //
        .any("/*", new Default());
router = routerBuilder.build();

Do you think it make sense to support it ?

Yes, it make sense.. I'm surprise that currently this does not work.

[sbernard31]

Maybe you missed (#27 (comment)) ? Any opinions about it ?

For that, my suggestion from comment: #27 (comment) will solve the issue. So server bind address will be passed in CoapRequest's transport context.

I think there is maybe a confusion because I was not asking about bind address but about getting Token on CoapResponse/Notificaiton thread.

I guess mixing several thread of discussion in same issue could easily lead to this kind of confusion 🙂 .
Maybe I should only use this thread to ask question and if there is not simple answer, we directly move in a dedicated issue ?
(Let me know if you want to change the way we do)

[sbernard31]

Should I create issue to discuss about :
2. Is there a way to send ACK + Separated Response.
3. For a CON Separated Response is there a way to know when the response was Acknowledge.

?

[sbernard31]

Some wording: I call observe passive cancel when client send a RST message and observe active cancel when it send a GET request with observe option set to 1. (https://www.rfc-editor.org/rfc/rfc7641#section-3.6)

Some context: I started to try to implement observe at LWM2M client side (CoAP server side)

I try something base on code you shared previously. (#27 (comment))

Here is a little description about how I implement it for now. (If you have better idea I take it)
Currently, I store all observe relation established.
When LWM2M Object Tree is modified, I execute a fake Get Request without (observe option) to get the current state.
You can have a look at how it looks like at : eclipse-leshan/leshan@10ab042

my problems/questions :

o1) I would like to remove the observe relation when is it removed in java-coap stack. (E.g. on passive cancel). Is there a way to do that ? Maybe I didn't use the right design ?
o2) Is observe active cancel implemented ? or does it exists an API to implement it ?

[szysas]

Hi, there is a major refactoring coming related to observation handling (inbound and outbound). It is mainly related to this issue: #36, and it should simplify use case that you have described.
It would be great if you could have a look at least at usage diff: https://github.com/open-coap/java-coap/tree/observation-handlers#coap-client-complete-usage, and give feedback, especially about naming (those are always the hardest ;))

[sbernard31]

Just give you some news about java-coap integration in Leshan.

I get something that I consider as testable/usable with :

• server : Add an EndpointProvider at Server side for CoAP based on java-coap(fork) eclipse-leshan/leshan#1382
• client : Add an EndpointProvider at Client side for CoAP based on java-coap(fork) eclipse-leshan/leshan#1443
• bootstrap server : not yet implemented

(I plan to integrate it in master soon)

There is still some issues or point which could be improved, I will take time to report it later. (If you can not wait 😅, you can have a look at the code, there is some TODO )

My next step will be to try to provide a coap+tcp support based on java-coap.

Out of topic question, I see in README :

netty: UDP and DTLS only

What is used for DTLS in that case ? I understood that DTLS support is only provided by mbedtls.

[szysas]

Great 🤞

Yes, for DTLS there is only integration done with mbedtls. Main reason is the support of DTLS with CID.

[sbernard31]

Yes, for DTLS there is only integration done with mbedtls. Main reason is the support of DTLS with CID.

AFAIK, the only pure java DTLS 1.2 implementation which support CID is Scandium from Californium project. (So with same problem than Californium to me)

I also understood that there is a DTLS implementaiton in JDK (probably without CID support), did you test it ?

Do you know if DTLS 1.3 is usable with your mbedtls binding ?

Common issue about using native binding in Java is when application crash or when debugging is needed. Did you face this kind of issue with mbedtls binding ?

Sometime I tell myself that Java world need an open source project dedicated to a pure Java DTLS implementation. 🤔

[szysas]

Yes, there is DTLS in JDK but without CID and is not obvious how to use it.
Mbedtls does not support yet DTLS 1.3, I hope they will add it at some point.
About native binding, well yes, if there is crashing it is not so easy to debug. JNA helps a lot in this area so it does not happen very often. In case of mbedtls binding I did not see any crashing in long time.