pkg: libcoap is partially broken and outdated

[nmeum]

I wrote an application for the RIOT board pba-d-01-kw2x which makes use of the libcoap pkg. The application compiles fine on native but when I compile it with BOARD=pba-d-01-kw2x the following error shows up:

Building application "feucht" for "pba-d-01-kw2x" with MCU "kw2xd".                                                    

rm -Rf /root/feucht/bin/pkg/pba-d-01-kw2x/libcoap          
mkdir -p /root/feucht/bin/pkg/pba-d-01-kw2x/libcoap        
/root/RIOT/dist/tools/git/git-cache clone "https://github.com/obgm/libcoap" "ef41ce5d02d64cec0751882ae8fd95f6c32bc018" "/root/feucht/bin/pkg/pba-d-01-kw2x/libcoap"                                                                            
git-cache: cloning from cache.                             
Cloning into '/root/feucht/bin/pkg/pba-d-01-kw2x/libcoap'...                                                           
done.                                                      
From /root/.gitcache                                       
 * [new tag]           b70b2821e7f700818bb2d5a631669f84b00de9e2/bsd-licensed -> bsd-licensed                           
 * [new tag]           b70b2821e7f700818bb2d5a631669f84b00de9e2/coap-lwip -> coap-lwip                                 
 * [new tag]           b70b2821e7f700818bb2d5a631669f84b00de9e2/coap-tinydtls -> coap-tinydtls                         
 * [new tag]           b70b2821e7f700818bb2d5a631669f84b00de9e2/v4.1.1 -> v4.1.1                                       
 * [new tag]           b70b2821e7f700818bb2d5a631669f84b00de9e2/v4.1.2 -> v4.1.2                                       
 * [new tag]           b70b2821e7f700818bb2d5a631669f84b00de9e2/v4_1_1 -> v4_1_1                                       
touch /root/feucht/bin/pkg/pba-d-01-kw2x/libcoap/.git-downloaded                                                       
git -C /root/feucht/bin/pkg/pba-d-01-kw2x/libcoap checkout -f ef41ce5d02d64cec0751882ae8fd95f6c32bc018                 
HEAD is now at ef41ce5d... improved install target         
git -C /root/feucht/bin/pkg/pba-d-01-kw2x/libcoap am --ignore-whitespace "/root/RIOT/pkg/libcoap"/patches/*.patch      
Applying: Add RIOT Makefile                                
Applying: Add config.h                                     
Applying: Remove two example programs in root              
Applying: Eliminate some compiler warnings and errors      
Applying: fix old-style function declarations              
Applying: debug: do not misuse NDEBUG                      
Applying: fix option parsing with debug enabled            
Applying: fix debug out in PDU parsing                     
Applying: delete unused function check_opt_size            
touch /root/feucht/bin/pkg/pba-d-01-kw2x/libcoap/.git-patched                                                          
"make" -C /root/RIOT/pkg/libcoap                           
"make" -C /root/feucht/bin/pkg/pba-d-01-kw2x/libcoap       
In file included from /root/feucht/bin/pkg/pba-d-01-kw2x/libcoap/net.h:47:0,                                           
                 from /root/feucht/bin/pkg/pba-d-01-kw2x/libcoap/async.h:19,                                           
                 from /root/feucht/bin/pkg/pba-d-01-kw2x/libcoap/async.c:22:                                           
/root/feucht/bin/pkg/pba-d-01-kw2x/libcoap/coap_time.h: In function 'coap_ticks_impl':                                 
/root/feucht/bin/pkg/pba-d-01-kw2x/libcoap/coap_time.h:129:3: error: implicit declaration of function 'gettimeofday' [-Werror=implicit-function-declaration]                                                                                   
   gettimeofday(&tv, NULL);                                
   ^
cc1: all warnings being treated as errors                  
/root/RIOT/Makefile.base:81: recipe for target '/root/feucht/bin/pba-d-01-kw2x/libcoap/async.o' failed                 
make[2]: *** [/root/feucht/bin/pba-d-01-kw2x/libcoap/async.o] Error 1                                                  
Makefile:9: recipe for target 'all' failed                 
make[1]: *** [all] Error 2                                 
../RIOT/Makefile.include:355: recipe for target '/root/feucht/bin/pba-d-01-kw2x/libcoap.a' failed                      
make: *** [/root/feucht/bin/pba-d-01-kw2x/libcoap.a] Error 2                                                           

This error seems to be caused by the fact that #define HAVE_SYS_TIME_H 1 in set in config.h but gettimeofday(3) is not defined in that header for this platform. If you remove that macro from config.h and recompile the application the error message changes to the following:

Building application "feucht" for "pba-d-01-kw2x" with MCU "kw2xd".

make[1]: Nothing to be done for 'prepare'.
"make" -C /root/RIOT/pkg/libcoap
"make" -C /root/feucht/bin/pkg/pba-d-01-kw2x/libcoap
In file included from /root/feucht/bin/pkg/pba-d-01-kw2x/libcoap/net.h:47:0,
                 from /root/feucht/bin/pkg/pba-d-01-kw2x/libcoap/async.h:19,
                 from /root/feucht/bin/pkg/pba-d-01-kw2x/libcoap/async.c:22:
/root/feucht/bin/pkg/pba-d-01-kw2x/libcoap/coap_time.h: In function 'coap_ticks_impl':
/root/feucht/bin/pkg/pba-d-01-kw2x/libcoap/coap_time.h:133:2: error: #error "clock not implemented"
 #error "clock not implemented"
  ^
/root/RIOT/Makefile.base:81: recipe for target '/root/feucht/bin/pba-d-01-kw2x/libcoap/async.o' failed
make[2]: *** [/root/feucht/bin/pba-d-01-kw2x/libcoap/async.o] Error 1
Makefile:9: recipe for target 'all' failed
make[1]: *** [all] Error 2
../RIOT/Makefile.include:355: recipe for target '/root/feucht/bin/pba-d-01-kw2x/libcoap.a' failed
make: *** [/root/feucht/bin/pba-d-01-kw2x/libcoap.a] Error 2

The clock is needed for the macro coap_ticks which could trivially be implemented using xtimer, however, when doing so and adding USEMODULE += xtimer to the application Makefile the error message changes to the following:

Building application "feucht" for "pba-d-01-kw2x" with MCU "kw2xd".

make[1]: Nothing to be done for 'prepare'.
"make" -C /root/RIOT/pkg/libcoap
"make" -C /root/feucht/bin/pkg/pba-d-01-kw2x/libcoap
In file included from /root/feucht/bin/pkg/pba-d-01-kw2x/libcoap/resource.h:18:0,
                 from /root/feucht/bin/pkg/pba-d-01-kw2x/libcoap/net.c:42:
/root/feucht/bin/pkg/pba-d-01-kw2x/libcoap/t_list.h:99:1: error: conflicting types for 'list_remove'
 list_remove(list_t the_list, void *item) {
 ^
In file included from /root/RIOT/core/include/mutex.h:27:0,
                 from /root/RIOT/sys/include/xtimer.h:35,
                 from /root/feucht/bin/pkg/pba-d-01-kw2x/libcoap/coap_time.h:17,
                 from /root/feucht/bin/pkg/pba-d-01-kw2x/libcoap/net.h:47,
                 from /root/feucht/bin/pkg/pba-d-01-kw2x/libcoap/async.h:19,
                 from /root/feucht/bin/pkg/pba-d-01-kw2x/libcoap/net.c:41:
/root/RIOT/core/include/list.h:86:28: note: previous definition of 'list_remove' was here
 static inline list_node_t *list_remove(list_node_t *list, list_node_t *node)
                            ^
In file included from /root/feucht/bin/pkg/pba-d-01-kw2x/libcoap/resource.h:18:0,
                 from /root/feucht/bin/pkg/pba-d-01-kw2x/libcoap/net.c:42:
/root/feucht/bin/pkg/pba-d-01-kw2x/libcoap/t_list.h:105:1: error: conflicting types for 'list_add'
 list_add(list_t the_list, void *item) {
 ^
In file included from /root/RIOT/core/include/mutex.h:27:0,
                 from /root/RIOT/sys/include/xtimer.h:35,
                 from /root/feucht/bin/pkg/pba-d-01-kw2x/libcoap/coap_time.h:17,
                 from /root/feucht/bin/pkg/pba-d-01-kw2x/libcoap/net.h:47,
                 from /root/feucht/bin/pkg/pba-d-01-kw2x/libcoap/async.h:19,
                 from /root/feucht/bin/pkg/pba-d-01-kw2x/libcoap/net.c:41:
/root/RIOT/core/include/list.h:53:20: note: previous definition of 'list_add' was here
 static inline void list_add(list_node_t *node, list_node_t *new_node)
                    ^
/root/RIOT/Makefile.base:81: recipe for target '/root/feucht/bin/pba-d-01-kw2x/libcoap/net.o' failed
make[2]: *** [/root/feucht/bin/pba-d-01-kw2x/libcoap/net.o] Error 1
Makefile:9: recipe for target 'all' failed
make[1]: *** [all] Error 2
../RIOT/Makefile.include:355: recipe for target '/root/feucht/bin/pba-d-01-kw2x/libcoap.a' failed
make: *** [/root/feucht/bin/pba-d-01-kw2x/libcoap.a] Error 2

There seems to be name clash with the linked list implementation used by libcoap and the one from list.h required by xtimer. This issue could be fixed by upgrading the bundled libcoap to 4.1.2. Doing so would probably require an update of RIOTs posix_sockets module since libcoap 4.1.2 makes extensive use of setsockopt(3) with options that are not supported by RIOT currently.

[kYc0o]

I gave a quick try for this issue and, though I have problems, those are not related to what is described here but:

BOARD=pba-d-01-kw2x make
Warning: no PORT set!
Building application "tests_pkg_libcoap" for "pba-d-01-kw2x" with MCU "kinetis".

rm -Rf /Users/facosta/git/RIOT2/RIOT/tests/pkg_libcoap/bin/pkg/pba-d-01-kw2x/libcoap
mkdir -p /Users/facosta/git/RIOT2/RIOT/tests/pkg_libcoap/bin/pkg/pba-d-01-kw2x/libcoap
/Users/facosta/git/RIOT2/RIOT/dist/tools/git/git-cache clone "https://github.com/obgm/libcoap" "ef41ce5d02d64cec0751882ae8fd95f6c32bc018" "/Users/facosta/git/RIOT2/RIOT/tests/pkg_libcoap/bin/pkg/pba-d-01-kw2x/libcoap"
Cloning into '/Users/facosta/git/RIOT2/RIOT/tests/pkg_libcoap/bin/pkg/pba-d-01-kw2x/libcoap'...
remote: Counting objects: 5441, done.
remote: Total 5441 (delta 0), reused 0 (delta 0), pack-reused 5441
Receiving objects: 100% (5441/5441), 1.97 MiB | 1.23 MiB/s, done.
Resolving deltas: 100% (3828/3828), done.
HEAD is now at ef41ce5... improved install target
touch /Users/facosta/git/RIOT2/RIOT/tests/pkg_libcoap/bin/pkg/pba-d-01-kw2x/libcoap/.git-downloaded
git -C /Users/facosta/git/RIOT2/RIOT/tests/pkg_libcoap/bin/pkg/pba-d-01-kw2x/libcoap checkout -f ef41ce5d02d64cec0751882ae8fd95f6c32bc018
HEAD is now at ef41ce5... improved install target
git -C /Users/facosta/git/RIOT2/RIOT/tests/pkg_libcoap/bin/pkg/pba-d-01-kw2x/libcoap am --ignore-whitespace "/Users/facosta/git/RIOT2/RIOT/pkg/libcoap"/patches/*.patch
Applying: Add RIOT Makefile
Applying: Add config.h
Applying: Remove two example programs in root
Applying: Eliminate some compiler warnings and errors
error: patch failed: net.h:132
error: net.h: patch does not apply
Patch failed at 0004 Eliminate some compiler warnings and errors
The copy of the patch that failed is found in: .git/rebase-apply/patch
When you have resolved this problem, run "git am --continue".
If you prefer to skip this patch, run "git am --skip" instead.
To restore the original branch and stop patching, run "git am --abort".
make[1]: *** [/Users/facosta/git/RIOT2/RIOT/tests/pkg_libcoap/bin/pkg/pba-d-01-kw2x/libcoap/.git-patched] Error 128
make: [pkg-prepare] Error 2 (ignored)
"/Applications/Xcode.app/Contents/Developer/usr/bin/make" -C /Users/facosta/git/RIOT2/RIOT/pkg/libcoap
git -C /Users/facosta/git/RIOT2/RIOT/tests/pkg_libcoap/bin/pkg/pba-d-01-kw2x/libcoap checkout -f ef41ce5d02d64cec0751882ae8fd95f6c32bc018
Warning: you are leaving 3 commits behind, not connected to
any of your branches:

  5d0a745 Remove two example programs in root
  8824a21 Add config.h
  42405b4 Add RIOT Makefile

If you want to keep them by creating a new branch, this may be a good time
to do so with:

 git branch <new-branch-name> 5d0a745

HEAD is now at ef41ce5... improved install target
git -C /Users/facosta/git/RIOT2/RIOT/tests/pkg_libcoap/bin/pkg/pba-d-01-kw2x/libcoap am --ignore-whitespace "/Users/facosta/git/RIOT2/RIOT/pkg/libcoap"/patches/*.patch
fatal: previous rebase directory .git/rebase-apply still exists but mbox given.
make[1]: *** [/Users/facosta/git/RIOT2/RIOT/tests/pkg_libcoap/bin/pkg/pba-d-01-kw2x/libcoap/.git-patched] Error 128
make: *** [/Users/facosta/git/RIOT2/RIOT/tests/pkg_libcoap/bin/pba-d-01-kw2x/libcoap.a] Error 2

Is actually libcoap still supported?

[nmeum]

@kYc0o what you are seeing there is probably related to #8289.

[kYc0o]

@nmeum thanks for the hint!

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. If you want me to ignore this issue, please mark it with the "State: don't stale" label. Thank you for your contributions.

[miri64]

Can someone confirm that this was fixed in #8396?

[miri64]

Ping @nmeum for example? The PR I referenced there does not update libcoap.

[aabadie]

libcoap has evolved quite a bit since the current version that is used in RIOT (https://github.com/obgm/libcoap/tree/ef41ce5d02d64cec0751882ae8fd95f6c32bc018, this commit is from 2014!). The latest release of libcoap is from 2019.

Is there someone willing to work on an update ? @chrysn (I see that you are one of the contributors ;) ) ?

[miri64]

With #12975 now merged this should also be a lot easier (from my experience way back when). But maybe it can also be ported with just using sock, now.