WARN [mc_pos_control] Offboard activation failed with error: Activation Failed

[lovettchris]

Trying to get AirSim to work with latest PX4 master bits. It works fine with 1.10.1, but not with master.

To Reproduce
Steps to reproduce the behavior:

1. Clone this version of AirSim.
2. make px4_sitl_default none
3. Launch this AirSim Neighborhood binary (1.3gb) or the AirSim Blocks binary (153mb) (built with the above AirSim bits already).
4. wait for message "Got GPS Home Location"
5. cd AirSim/PythonClient/multirotor
6. python box.py

Expected behavior
The takeoff works fine, but when it tries to request offboard control we see these errors in the PX4 window:

WARN  [mc_pos_control] Offboard activation failed with error: Activation Failed
WARN  [mc_pos_control] Position-Ctrl activation failed with error: Activation Failed
WARN  [mc_pos_control] Altitude-Ctrl activation failed with error: Activation Failed

This message seems to boil down to FlightTaskOffboard::updateInitialize and I can't see why this would be failing.

Log Files and Screenshots
This is the log that works perfectly on 1.10.1:
https://logs.px4.io/plot_app?log=a28b64a2-58fa-4871-ab60-7c230941fd8e
So you should be able to replay this log on master bits and get the same flight.

This log shows the problem happening:
https://logs.px4.io/plot_app?log=7de2e3fe-7dee-44d8-ad1e-5947ec948b53

Add screenshots to help explain your problem.

Drone (please complete the following information):
make px4_sitl none

Additional context
I'm also finding that it can sometimes take a very long time (forever) for it to get happy with the AirSim simulated GPS, and commander check shows the following sometimes:

INFO  [HealthFlags] GPS:                EN      ERR

[julianoes]

FYI @Jaeyoung-Lim @MaEtUgR

[MaEtUgR]

The issue I've seen last is microsoft/AirSim#2477 There it's also mentioned that the sensor bitmask interface that allows publishing different sensors at different timings is likely the cause. I don't know the exact details but @TSC21 might know where it's standing.

[Jaeyoung-Lim]

@lovettchris Out of curiosity, does this mean that mission mode is functional with the branch you mentioned?

[lovettchris]

@MaEtUgR , this checkin adds the bitmasks on HIL_SENSOR so the code in AirSim is now the same as the jMavSim code.

[lovettchris]

@Jaeyoung-Lim , yes AirSim now works well with PX4 v1.10.1 in SITL with QGroundControl missions, but this issue points out that things don't work in PX4 master branch. You can test it with the binaries I provided:

• AirSim Neighborhood binary (1.3gb)
• AirSim Blocks binary (153mb)
  

[lovettchris]

Oh, this is interesting, on v1.11.0-beta I am also getting these warnings, but the AirSim flight is working normally.

WARN  [mc_pos_control] Offboard activation failed with error: Activation Failed
WARN  [mc_pos_control] Position-Ctrl activation failed with error: Activation Failed
WARN  [mc_pos_control] Altitude-Ctrl activation failed with error: Activation Failed

So perhaps these warnings are a red herring, and there may be some other problem with master branch that is not understood yet...

[lovettchris]

On master offboard control definitely doesn't work, but also master HITL also doesn't work, the drone connects but refuses to arm. So I wonder if some breaking changes have happened in the mavlink protocol ?

[lovettchris]

Indeed I see lots of changes in mavlink_receiver.cpp to do with how it is processing HIL_SENSOR data. It's interesting that mavlink_receiver.cpp, MavlinkReceiver::handle_message_hil_sensor doesn't seem to care about fields_updated... AirSim is not generating termperature so I wonder if that is putting the _px4_baro into a bad state...

[lovettchris]

Hey, ok, I updated AirSim mavlink headers and fixed some other bugs in AirSim and now I can get it to fly with PX4 master bits. But the flight was kinda wonky and I got lots of warnings like this which is new:

The log is here:
https://logs.px4.io/plot_app?log=d27f2f38-415c-435e-a8ac-8d636492f6c5

See AirSim Pull Request
microsoft/AirSim#2649

[lovettchris]

But then sometimes it does this in the middle of the offboard flight, and the drone crashes:

WARN  [mc_pos_control] Offboard activation failed with error: Activation Failed
WARN  [mc_pos_control] Position-Ctrl activation failed with error: Activation Failed
WARN  [mc_pos_control] Altitude-Ctrl activation failed with error: Activation Failed

[lovettchris]

I'm also getting a lot of these:

WARN  [commander] Failsafe enabled: no RC and no offboard
WARN  [commander] Failsafe enabled: no RC and no offboard
WARN  [commander] Failsafe enabled: no RC and no offboard

Despite the fact that I'm setting these parameters:

pxh> param set COM_OBL_ACT 1
pxh> param set NAV_RCL_ACT 0
pxh> param set NAV_DLL_ACT 0

Previous versions allow me to fly without RC so long as I set the above parameters (which AirSim does by default).

[lovettchris]

I updated my AirSim binaries for testing this:

• AirSim Neighborhood binary (1.3gb)
• AirSim Blocks binary (153mb)

(built with the above AirSim PR #2649 already).

[lovettchris]

Oh, wow, this one is interesting, the flight completed and the PX4 took over the landing process:

[commander] Landing at current position

Then it went bonkers during landing, drone flipped and crashed.

See https://logs.px4.io/plot_app?log=7e05515c-7518-4a50-aabe-c3ab5c298886

[Jaeyoung-Lim]

@lovettchris It could be related to the message in your screenshot. I will try to reproduce it

ERROR [sensors] Accel #0 fail. TIMEOUT!

[stale]

This issue has been automatically marked as stale because it has not had recent activity. Thank you for your contributions.

[julianoes]

Should be fixed with #17725.