Compass calibration

[arthurbenemann]

Starting to implement compass calibration. This is a WIP

[arthurbenemann]

Calibration is working on the Desktop app, bellow is a system output converging to good offsets in less than 50 samples.

It'm still working on getting the libraries to compile with the Android Project, but I expect to have this done by the end of the day.

Listening on udp:127.0.1.1:14550
IMU[-307, 147, 884]     1   Center:{0; -0; -0}  Radii:{929.7; (NaN); (NaN)}      Eigenvector{-0.46; 0.23; 0.86}
IMU[-315, 148, 884]     2   Center:{-0; -0.01; -0}  Radii:{891.37; (NaN); (NaN)}         Eigenvector{0.01; -0.23; -0.97}
IMU[-235, 148, 896]     3   Center:{0; 0; -0}   Radii:{657.67; (NaN); (NaN)}         Eigenvector{-0.23; -0.47; 0.85}
IMU[173, 261, 527]  4   Center:{-0; -0; -0.04}  Radii:{329.18; 892.25; (NaN)}        Eigenvector{0.71; 0.7; -0.09}
IMU[173, 261, 527]  5   Center:{-0; -0; -0.04}  Radii:{329.18; 892.25; (NaN)}        Eigenvector{0.71; 0.7; -0.09}
IMU[173, 261, 527]  6   Center:{-0; -0; -0.04}  Radii:{329.18; 892.25; (NaN)}        Eigenvector{0.71; 0.7; -0.09}
IMU[105, 226, 297]  7   Center:{0.09; -0; -0.2} Radii:{252.9; 1,434.89; (NaN)}       Eigenvector{-0.26; -0.95; -0.15}
IMU[105, 226, 297]  8   Center:{0.09; -0; -0.2} Radii:{252.9; 1,434.89; (NaN)}       Eigenvector{-0.26; -0.95; -0.15}
IMU[-81, 239, 171]  9   Center:{0.07; -0.03; -0.23} Radii:{216.71; 993.86; (NaN)}        Eigenvector{-0.19; -0.98; 0.02}
IMU[-81, 239, 171]  10  Center:{0.07; -0.03; -0.23} Radii:{216.71; 993.86; (NaN)}        Eigenvector{-0.19; -0.98; 0.02}
IMU[-81, 239, 171]  11  Center:{0.07; -0.03; -0.23} Radii:{216.71; 993.86; (NaN)}        Eigenvector{-0.19; -0.98; 0.02}
IMU[-291, 281, 185]     12  Center:{282.29; 56.06; 328.72}  Radii:{163.53; 775.28; (NaN)}        Eigenvector{-0.02; -0.98; 0.22}
IMU[-291, 281, 185]     13  Center:{282.29; 56.06; 328.72}  Radii:{163.53; 775.28; (NaN)}        Eigenvector{-0.02; -0.98; 0.22}
IMU[-429, 336, 298]     14  Center:{-899.47; 2,295.78; 4,670.69}    Radii:{3,335.8; (NaN); (NaN)}        Eigenvector{-0.12; 0.45; 0.88}
IMU[-429, 336, 298]     15  Center:{-899.47; 2,295.78; 4,670.69}    Radii:{3,335.8; (NaN); (NaN)}        Eigenvector{-0.12; 0.45; 0.88}
IMU[-429, 336, 298]     16  Center:{-899.47; 2,295.78; 4,670.69}    Radii:{3,335.8; (NaN); (NaN)}        Eigenvector{-0.12; 0.45; 0.88}
IMU[-486, 374, 524]     17  Center:{-179.09; 288.86; 530.51}    Radii:{156.56; 336.73; 482.53}       Eigenvector{0.03; 0.99; 0.15}
IMU[-486, 374, 524]     18  Center:{-179.09; 288.86; 530.51}    Radii:{156.56; 336.73; 482.53}       Eigenvector{0.03; 0.99; 0.15}
IMU[-446, 362, 688]     19  Center:{-168.64; 282.34; 509.53}    Radii:{268.34; 336.14; 429.37}       Eigenvector{0.1; -0.9; -0.43}
IMU[-446, 362, 688]     20  Center:{-168.63; 282.27; 509.51}    Radii:{268.93; 336.15; 429.63}       Eigenvector{0.1; -0.9; -0.43}
IMU[-446, 362, 688]     21  Center:{-168.63; 282.24; 509.51}    Radii:{269.13; 336.15; 429.72}       Eigenvector{0.1; -0.9; -0.43}
IMU[-400, 393, 685]     22  Center:{-181.7; 258.18; 524.14} Radii:{229.04; 342.65; 401.4}        Eigenvector{0.01; -0.97; -0.25}
IMU[-400, 393, 685]     23  Center:{-182.11; 257.62; 524.54}    Radii:{227.76; 342.95; 400.96}       Eigenvector{0.01; -0.97; -0.25}
IMU[-247, 482, 522]     24  Center:{-184.11; 251.96; 526.94}    Radii:{230.01; 345.33; 396.48}       Eigenvector{0.01; -0.97; -0.23}
IMU[-247, 482, 522]     25  Center:{-184.13; 251.89; 526.97}    Radii:{230.04; 345.36; 396.43}       Eigenvector{0.01; -0.97; -0.23}
IMU[-247, 482, 522]     26  Center:{-184.14; 251.87; 526.98}    Radii:{230.05; 345.37; 396.42}       Eigenvector{0.01; -0.97; -0.23}
IMU[8, 426, 543]    27  Center:{-167.08; 277.47; 517.4} Radii:{196.69; 339.06; 420.92}       Eigenvector{-0.13; -0.95; -0.27}
IMU[8, 426, 543]    28  Center:{-165.73; 280.08; 515.43}    Radii:{193.35; 338.64; 426}      Eigenvector{-0.14; -0.95; -0.27}
IMU[135, 198, 715]  29  Center:{-173.28; 227.34; 535.82}    Radii:{245.73; 358.3; 381.92}        Eigenvector{-0.16; -0.96; -0.23}
IMU[135, 198, 715]  30  Center:{-173.65; 223.68; 535.92}    Radii:{249.68; 359.84; 380.87}       Eigenvector{-0.16; -0.96; -0.23}
IMU[135, 198, 715]  31  Center:{-173.78; 222.34; 535.93}    Radii:{251.14; 360.4; 380.52}        Eigenvector{-0.16; -0.96; -0.23}
IMU[-29, -53, 863]  32  Center:{-207.68; 27.83; 499.87} Radii:{402.66; 437.8; 460.74}        Eigenvector{-0.06; -0.25; 0.97}
IMU[-29, -53, 863]  33  Center:{-208.54; 22.54; 498.67} Radii:{403.91; 439.8; 466.35}        Eigenvector{-0.06; -0.25; 0.97}
IMU[-405, -69, 845]     34  Center:{-207.4; 25.73; 499.76}  Radii:{403.22; 438.47; 462.76}       Eigenvector{-0.07; -0.25; 0.97}
IMU[-405, -69, 845]     35  Center:{-207.38; 25.81; 499.78} Radii:{403.21; 438.44; 462.67}       Eigenvector{-0.07; -0.25; 0.97}

[arthurbenemann]

I just hit a bottleneck, the 65k method limit on the dalvik VM. Here is the error message

[2014-10-08 14:51:42 - Dex Loader] Unable to execute dex: method ID not in [0, 0xffff]: 65536
[2014-10-08 14:51:42 - Android] Conversion to Dalvik format failed: Unable to execute dex: method ID not in [0, 0xffff]: 65536

So I found the root of the problem, after reading this post. By running this script with master, we can get which libraries are creating too many methods. The culprit is google-play-services accounting for ~38k methods, droidplanner and it's internal libraries account for 4k methods. Here is a summarized output of the script:

Read in 60591 method IDs.
<root>: 60591
    : 3
    android: 7311
        support: 4933
        util: 47
        view: 639
        webkit: 51
        widget: 365
    com: 45136
        MAVLink: 919
            Messages: 911
                ardupilotmega: 807
                enums: 37
        cocoahero: 246
        ftdi: 336
        geeksville: 1962
        getpebble: 100
        google: 37760
            ads: 124
            android: 18660
                gms: 18660
                    actions: 1
                    ads: 336
                    analytics: 583
                    appindexing: 7
                    appstate: 121
                    auth: 23
                    cast: 453
                    common: 633
                    drive: 1709
                    dynamic: 213
                    games: 4258
                    gcm: 16
                    identity: 75
                    internal: 4651
                    location: 192
                    maps: 2168
                    panorama: 9
                    plus: 704
                    security: 12
                    tagmanager: 976
                    wallet: 714
                    wearable: 796
            common: 13649
            protobuf: 5310
            thirdparty: 17
        hoho: 143
            android: 143
                usbserial: 143
        jakewharton: 97
        mapbox: 1317
        nineoldandroids: 719
        sothree: 215
        squareup: 1322
    dalvik: 2
    ellipsoidFit: 3
    it: 806
    java: 1884
    javax: 22
    junit: 2
    org: 4369
        apache: 40
        droidplanner: 3948
            android: 2974
                activities: 232
                communication: 181
                dialogs: 61
                fragments: 647
                    calibration: 161
                    helpers: 96
                    mode: 58
                gcs: 8
                graphic: 38
                helpers: 65
                maps: 286
                notifications: 74
                proxy: 180
                utils: 204
                widgets: 976
            core: 955
                MAVLink: 151
                drone: 256
                firmware: 6
                gcs: 71
                helpers: 162
                mission: 200
                model: 44
                parameters: 28
                polygon: 11
                srtm: 26
        json: 49
        xmlpull: 16
        zeromq: 316
    sun: 7
    uk: 153
    zmq: 893

I wish we could just ignore this PR, but we are going to reach the limit again in a very short future. And the calibration method included here is impressive ( I'll post a video very soon, but it only runs on the desktop droidplanner).

@DroidPlanner/dev @geeksville @ne0fhyk Anybody knows a way out of this problem? Here are the possible solutions I see:

• Try to trim down the google-play-services package
• Make all calibration stuff an external app
• Move to the external services library structure
• Stop using Google Maps

[kellyschrock]

I'll do some more investigation as time permits, but I vote for stripping out unused packages in the google play services library. https://gist.github.com/dmarcato/d7c91b94214acd936e42

[Thalek]

How often does calibration have to be performed? If not very often, perhaps calibration should be an external app (and calibration added to the user's checklist as a reminder). Can the software call an external app, and then return when the external app is closed?

[kellyschrock]

Sure, it can, but will there be any issues sharing the MAVLink connection
between the two apps, things like that?

On Wed, Oct 8, 2014 at 5:47 PM, Thalek [email protected] wrote:

How often does calibration have to be performed? If not very often,
perhaps calibration should be an external app (and calibration added to the
user's checklist as a reminder). Can the software call an external app, and
then return when the external app is closed?

—
Reply to this email directly or view it on GitHub
#1147 (comment)
.

[arthurbenemann]

Here are the results of trying the algorithm with DroidPlanner, and using the algorithm on MP.

   x_mp      x_dp       y_mp      y_dp     z_mp      z_dp          
   202.000   196.750    40.000    45.040   515.000   518.610
   203.000   199.890    51.000    44.960   517.000   518.380
   201.000   214.310    60.000    50.000   518.000   525.750
   207.000   209.300    57.000    52.800   517.000   520.430
   207.000   207.500    50.000    51.420   518.000   525.290

[arthurbenemann]

Here is a quick demo of the desktop app generating the calibration parameters: https://www.youtube.com/watch?v=3EYVZ1QbleI

[m4gr3d]

@arthurbenemann @kellyschrock there's several gradle scripts that allow to strip the methods count automatically at build time: https://gist.github.com/dmarcato/d7c91b94214acd936e42.

The official google solution is to separate the app in multiple dex file. That method can't automatically be used in Eclipse, and requires a separate build step.

[arthurbenemann]

After a chat with @ne0fhyk, we decided that the easiest thing to do was to extract and strip down the math library we where using. I did this on be53f2d

[arthurbenemann]

@ne0fhyk Could you setup the graddle build for this branch?

[m4gr3d]

@arthurbenemann UI issues have been fixed. A couple of items remained before completion of this PR:

• Improve the scatter plot rendering: As pointed by @jason4short, the current plot is hard to understand and map to the autopilot positioning.
• Apply mag offsets to the values received from the autopilot.

[arthurbenemann]

• The Desktop branch must be updated

[arthurbenemann]

• The ElipsoidFit library should be included in the project when you "import Android projects" on eclipse.

[arthurbenemann]

I'm merging this so it doesn't diverge too much from master, and for us to start testing. The fitness function may need some improvement before releasing it publicly.