Use lowercase resource and owner names when storing assets on disk #130

nkoenig · 2020-11-09T19:30:49Z

This will store models and worlds using lowercase name and owner.

I've done the following tests:

Run SubT worlds with an existing cache that has capitalization. Result: nothing new is downloaded, and the worlds run.
Run SubT world with some models missing from the cache. Result: new models are downloaded to all lowercase directories.
Run worlds with random capitalization in model URIs. Result: utilizes existing cache, or downloads new resources.
Download a collection, such as ign fuel download -v 4 -j 8 -u "https://fuel.ignitionrobotics.org/OpenRobotics/collections/SubT Tech Repo". Result: This will download all the models using lowercase.
If there is a model is two places, such as OpenRobotics/models/X1 Config 1 and openrobotics/x1 config 1, then the lower case version will be used.

I think this won't break anyone, and Ignition Gazebo should still use an existing cache even if the cache has capitalization.

Signed-off-by: Nate Koenig [email protected]

Signed-off-by: Nate Koenig <[email protected]>

Signed-off-by: John Shepherd <[email protected]>

JShep1

LGTM pending CI, I had to revert a version number on tests 82ba7a1 since I was failing locally, but all tests are passing for me after that.

Signed-off-by: Nate Koenig <[email protected]>

codecov · 2020-11-30T16:49:59Z

Codecov Report

Merging #130 (b4183da) into ign-fuel-tools3 (d82571c) will increase coverage by 0.01%.
The diff coverage is 100.00%.

@@                 Coverage Diff                 @@
##           ign-fuel-tools3     #130      +/-   ##
===================================================
+ Coverage            86.50%   86.51%   +0.01%     
===================================================
  Files                   19       19              
  Lines                 2223     2225       +2     
===================================================
+ Hits                  1923     1925       +2     
  Misses                 300      300

Impacted Files	Coverage Δ
src/ModelIdentifier.cc	`92.12% <100.00%> (+0.09%)`	⬆️
src/WorldIdentifier.cc	`100.00% <100.00%> (ø)`

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update d82571c...b4183da. Read the comment docs.

Signed-off-by: Nate Koenig <[email protected]>

* Use lowercase resource and owner names when storing assets on disk (#130) * Use lowercase resource and owner names when storing assets on disk Signed-off-by: Nate Koenig <[email protected]> * Remove added diff text from merge Signed-off-by: John Shepherd <[email protected]> * change model version in test back Signed-off-by: John Shepherd <[email protected]> * Fix test Signed-off-by: Nate Koenig <[email protected]> * Fix windows Signed-off-by: Nate Koenig <[email protected]> * Fix more windows tests Signed-off-by: Nate Koenig <[email protected]> * Testing another windows fix Signed-off-by: Nate Koenig <[email protected]> Co-authored-by: Nate Koenig <[email protected]> Co-authored-by: John Shepherd <[email protected]> * Prepare for 3.5.0 release (#135) * Prepare for 3.5.0 release Signed-off-by: Nate Koenig <[email protected]> * Update Changelog.md Co-authored-by: Louise Poubel <[email protected]> Co-authored-by: Nate Koenig <[email protected]> Co-authored-by: Louise Poubel <[email protected]> * Download model dependencies (#123) Signed-off-by: John Shepherd <[email protected]> Co-authored-by: Nate Koenig <[email protected]> * Prepare for 4.3.0 release (#139) Signed-off-by: Nate Koenig <[email protected]> Co-authored-by: Nate Koenig <[email protected]> * Set keep alive on (#141) Signed-off-by: Nate Koenig <[email protected]> Co-authored-by: Nate Koenig <[email protected]> Co-authored-by: Nate Koenig <[email protected]> Co-authored-by: John Shepherd <[email protected]> Co-authored-by: Louise Poubel <[email protected]>

azeey · 2020-12-10T18:57:25Z

I think this is causing the INTEGRATION_sdf_include and INTEGRATION_log_systems tests to fail in ign-gazebo3

As of gazebosim/gz-fuel-tools#130, fuel paths use all lowercase letters. This fixes some tests that had uppercase letters. Signed-off-by: Addisu Z. Taddese <[email protected]>

As of gazebosim/gz-fuel-tools#130, fuel paths use all lowercase letters. This fixes some tests that had uppercase letters. Signed-off-by: Addisu Z. Taddese <[email protected]> Signed-off-by: Guillaume Doisy <[email protected]>

* Use lowercase resource and owner names when storing assets on disk (#130) * Use lowercase resource and owner names when storing assets on disk Signed-off-by: Nate Koenig <[email protected]> * Remove added diff text from merge Signed-off-by: John Shepherd <[email protected]> * change model version in test back Signed-off-by: John Shepherd <[email protected]> * Fix test Signed-off-by: Nate Koenig <[email protected]> * Fix windows Signed-off-by: Nate Koenig <[email protected]> * Fix more windows tests Signed-off-by: Nate Koenig <[email protected]> * Testing another windows fix Signed-off-by: Nate Koenig <[email protected]> Co-authored-by: Nate Koenig <[email protected]> Co-authored-by: John Shepherd <[email protected]> * Prepare for 3.5.0 release (#135) * Prepare for 3.5.0 release Signed-off-by: Nate Koenig <[email protected]> * Update Changelog.md Co-authored-by: Louise Poubel <[email protected]> Co-authored-by: Nate Koenig <[email protected]> Co-authored-by: Louise Poubel <[email protected]> * Download model dependencies (#123) Signed-off-by: John Shepherd <[email protected]> Co-authored-by: Nate Koenig <[email protected]> * Prepare for 4.3.0 release (#139) Signed-off-by: Nate Koenig <[email protected]> Co-authored-by: Nate Koenig <[email protected]> * Set keep alive on (#141) Signed-off-by: Nate Koenig <[email protected]> Co-authored-by: Nate Koenig <[email protected]> * Support editing/patching model files (#140) * Support editing/patching model files Signed-off-by: Nate Koenig <[email protected]> * Added documentation Signed-off-by: Nate Koenig <[email protected]> * Set keep alive on Signed-off-by: Nate Koenig <[email protected]> * Documentation and tests Signed-off-by: Nate Koenig <[email protected]> * One more test and exit catch missing -u Signed-off-by: Nate Koenig <[email protected]> Co-authored-by: Nate Koenig <[email protected]> * Prepare for 5.1.0 release (#145) Signed-off-by: Nate Koenig <[email protected]> Co-authored-by: Nate Koenig <[email protected]> * Fix light map URI (#146) Signed-off-by: Ian Chen <[email protected]> * Bump to 5.1.1 (#147) Signed-off-by: Louise Poubel <[email protected]> Co-authored-by: Nate Koenig <[email protected]> Co-authored-by: Nate Koenig <[email protected]> Co-authored-by: John Shepherd <[email protected]> Co-authored-by: Ian Chen <[email protected]>

* Initial commit Signed-off-by: Martiño Crespo <[email protected]> * Added plugin files Signed-off-by: Martiño Crespo <[email protected]> * Initial .sdf file Signed-off-by: Martiño Crespo <[email protected]> * Pipeline for getting sensor contacts Signed-off-by: Martiño Crespo <[email protected]> * Added visualization of postion and forces Signed-off-by: Martiño Crespo <[email protected]> * Added plugin for Transform Control Signed-off-by: Martiño Crespo <[email protected]> * Added initial interpolation Signed-off-by: Martiño Crespo <[email protected]> * Code check Signed-off-by: Martiño Crespo <[email protected]> * Update info after specified milliseconds Signed-off-by: Martiño Crespo <[email protected]> * Set marker lifetime for better performance Signed-off-by: Martiño Crespo <[email protected]> * Added Depth Camera Signed-off-by: Martiño Crespo <[email protected]> * Added callback and unpacking to Depth Camera messages Signed-off-by: Martiño Crespo <[email protected]> * Compute and visualize normal forces Signed-off-by: Martiño Crespo <[email protected]> * PR Feedback 1 Signed-off-by: Martiño Crespo <[email protected]> * Update sdf example to a more realistic environment Signed-off-by: Martiño Crespo <[email protected]> * Allow moving Depth Camera from model origin Signed-off-by: Martiño Crespo <[email protected]> * Make markers dimensions available as parameters Signed-off-by: Martiño Crespo <[email protected]> * Visualize sensor as a marker instead of <visual> and filter out normal forces outside sensor Signed-off-by: Martiño Crespo <[email protected]> * PR Feedback 2 Signed-off-by: Martiño Crespo <[email protected]> * PR Feedback 3 Signed-off-by: Martiño Crespo <[email protected]> * Minor fixes for sdf, sensor marker and profiler Signed-off-by: Martiño Crespo <[email protected]> * PR Feedback 4 Signed-off-by: Martiño Crespo <[email protected]> * fix mac warning Signed-off-by: Mabel Zhang <[email protected]> * Helper function to set component data (gazebosim#436) Signed-off-by: Louise Poubel <[email protected]> * Remove unneeded if statement (gazebosim#432) Signed-off-by: John Shepherd <[email protected]> * Fixes flaky RecordAndPlayback test in INTEGRATION_log_system (gazebosim#463) The flakiness comes from two sources: 1. Poses recorded by the LogRecorder are published by the SceneBroadcaster system throttled at 60 Hz. The throttle mechanism uses real-time instead of sim-time which causes a variance in the number of recorded poses from run to run. However, the expected number of recorded poses was calculated with the assumption that the simulation would run with a 1.0 RTF. If the CPU load is high, there could be a mismatch between the expected and the actual number of recorded poses, which causes the test to fail. This can be checked by running the test with `cpulimit -l 20 -f bin/INTEGRATION_log_system -- --gtest_filter="*RecordAndPlayback"` 2. An attempt is made to match up played back poses with the closest timestamp in the recorded file. These poses are again published by the SceneBroadcaster, so they are subject to the same kind of timing issues as the recorded poses. The solution in this patch for the first issue is to determine the expected number of recorded poses by counting them in a separate system that mimics the throttling mechanism of ign-transport. For the second issue, a testing system is added to the playback server bypassing the SceSceneBroadcaster altogether. Signed-off-by: Addisu Z. Taddese <[email protected]> * Make PeerTracker test more robust (gazebosim#452) Signed-off-by: Louise Poubel <[email protected]> Co-authored-by: Michael Carroll <[email protected]> * Clarify how sim time is interpreted in a System's step (gazebosim#467) Signed-off-by: Ashton Larkin <[email protected]> * 3 ➡️ 4: fixes for gazebosim#463 (gazebosim#469) Signed-off-by: Louise Poubel <[email protected]> * Link link tutrial (gazebosim#472) Signed-off-by: Louise Poubel <[email protected]> * Switch to async state service request (gazebosim#461) Signed-off-by: Ian Chen <[email protected]> * Use a std::promise/std::future to avoid busy waiting the step ack messages in NetworkManagerPrimary (gazebosim#470) Signed-off-by: Ivan Santiago Paunovic <[email protected]> Signed-off-by: Louise Poubel <[email protected]> Co-authored-by: Louise Poubel <[email protected]> * Fix tests that use Fuel paths with uppercase letters (gazebosim#480) As of gazebosim/gz-fuel-tools#130, fuel paths use all lowercase letters. This fixes some tests that had uppercase letters. Signed-off-by: Addisu Z. Taddese <[email protected]> * 4.1.0 (gazebosim#485) Signed-off-by: Ashton Larkin <[email protected]> * Update key event handling (gazebosim#466) Signed-off-by: John Shepherd <[email protected]> * Fix slot in Plotting plugin (gazebosim#490) Signed-off-by: Alejandro Hernández <[email protected]> * Tape Measure Plugin (gazebosim#456) Signed-off-by: John Shepherd <[email protected]> * Satisfy make codecheck (gazebosim#491) Signed-off-by: Louise Poubel <[email protected]> * Move deselect and preview termination to render thread (gazebosim#493) Signed-off-by: John Shepherd <[email protected]> * Fix codecheck (gazebosim#499) Signed-off-by: Nate Koenig <[email protected]> Co-authored-by: Nate Koenig <[email protected]> * Logical Audio Sensor Plugin (gazebosim#401) Signed-off-by: Ashton Larkin <[email protected]> * Codecheck - initialize primitive in LogicalAudio component (gazebosim#502) Signed-off-by: Ashton Larkin <[email protected]> * 3 to 4: fix codecheck for ign-gazebo4 Signed-off-by: Ashton Larkin <[email protected]> * 3 to 4: resolve codecheck warnings Signed-off-by: Ashton Larkin <[email protected]> * add frame_id and child_frame_id attribute support for DiffDrive (gazebosim#361) Add configuration of the odom frame_id and child_frame_id fields from sdf attributes <frame_id> and <child_frame_id> Signed-off-by: Guillaume <[email protected]> Signed-off-by: Guillaume Doisy <[email protected]> * Require ign-gui 4.1.0 (gazebosim#505) Signed-off-by: Louise Poubel <[email protected]> * Fix shadow artifacts by disabling double sided rendering (gazebosim#446) * read double sided sdf param Signed-off-by: Ian Chen <[email protected]> * update migration Signed-off-by: Ian Chen <[email protected]> * Make the tunnels example world more interesting (gazebosim#462) Signed-off-by: Louise Poubel <[email protected]> * add double sided msg to sdf conversion Signed-off-by: Ian Chen <[email protected]> Co-authored-by: Louise Poubel <[email protected]> * Add ability to record video based on sim time (gazebosim#414) * add ability to record video from gui camera using sim time Signed-off-by: Ian Chen <[email protected]> * add msg Signed-off-by: Ian Chen <[email protected]> * use QueryBoolText Signed-off-by: Ian Chen <[email protected]> Co-authored-by: Nate Koenig <[email protected]> Co-authored-by: Louise Poubel <[email protected]> * Add lockstep mode to video recording (gazebosim#419) Signed-off-by: Ian Chen <[email protected]> Signed-off-by: Louise Poubel <[email protected]> Co-authored-by: Nate Koenig <[email protected]> Co-authored-by: Louise Poubel <[email protected]> * Kinetic energy monitor plugin (gazebosim#492) Signed-off-by: Gonzalo de Pedro <[email protected]> Signed-off-by: Nate Koenig <[email protected]> Co-authored-by: Nate Koenig <[email protected]> Co-authored-by: Louise Poubel <[email protected]> * Disable right click menu when using measuring tool (gazebosim#458) Signed-off-by: John Shepherd <[email protected]> Co-authored-by: Louise Poubel <[email protected]> * Bump to 3.6.0 (gazebosim#524) Signed-off-by: Louise Poubel <[email protected]> * Don't make docs on macOS (gazebosim#528) add comment about doxygen bug Signed-off-by: Louise Poubel <[email protected]> Signed-off-by: Steve Peters <[email protected]> Co-authored-by: Louise Poubel <[email protected]> * Updates to ardupilot migration tutorial (gazebosim#525) Signed-off-by: Louise Poubel <[email protected]> * Update gtest to 1.10.0 for Windows compilation (ign-gazebo3) (gazebosim#506) * Compile new gtest with c++11 * Use INSTANTIATE_TEST_SUITE_P instead of deprecated -INSTANTIATE_TEST_CASE_P Signed-off-by: Jose Luis Rivero <[email protected]> * Apply suggestions from code review Signed-off-by: Michael Carroll <[email protected]> Co-authored-by: Louise Poubel <[email protected]> * change nullptr to a int ptr for qt 5.15.2 bug (gazebosim#527) See: https://bugreports.qt.io/browse/QTBUG-89114 Signed-off-by: acxz <[email protected]> Co-authored-by: Louise Poubel <[email protected]> Signed-off-by: Louise Poubel <[email protected]> * Generate valid topics everywhere (support names with spaces) (gazebosim#522) Signed-off-by: Louise Poubel <[email protected]> * One more tutorial version bump Signed-off-by: Michael Carroll <[email protected]> * Fix bad merge Signed-off-by: Michael Carroll <[email protected]> * change nullptr to a int ptr for qt 5.15.2 bug (gazebosim#527) See: https://bugreports.qt.io/browse/QTBUG-89114 Signed-off-by: acxz <[email protected]> Co-authored-by: Louise Poubel <[email protected]> Signed-off-by: Louise Poubel <[email protected]> * Generate valid topics everywhere (support names with spaces) (gazebosim#522) Signed-off-by: Louise Poubel <[email protected]> * Change deprecated test case->suite Signed-off-by: Michael Carroll <[email protected]> * 3 ➡️ 4 (gazebosim#533) * Clarify how sim time is interpreted in a System's step (gazebosim#467) * add frame_id and child_frame_id attribute support for DiffDrive (gazebosim#361) * Add ability to record video based on sim time (gazebosim#414) * add ability to record video from gui camera using sim time * add msg * use QueryBoolText * Add lockstep mode to video recording (gazebosim#419) * Disable right click menu when using measuring tool (gazebosim#458) * Bump to 3.6.0 (gazebosim#524) * Don't make docs on macOS (gazebosim#528) * Updates to ardupilot migration tutorial (gazebosim#525) * Update gtest to 1.10.0 for Windows compilation (ign-gazebo3) (gazebosim#506) * Compile new gtest with c++11 * Use INSTANTIATE_TEST_SUITE_P instead of deprecated -INSTANTIATE_TEST_CASE_P * Apply suggestions from code review * One more tutorial version bump * Fix bad merge * change nullptr to a int ptr for qt 5.15.2 bug (gazebosim#527) * Generate valid topics everywhere (support names with spaces) (gazebosim#522) * Change deprecated test case->suite Co-authored-by: Ashton Larkin <[email protected]> Co-authored-by: G.Doisy <[email protected]> Co-authored-by: Ian Chen <[email protected]> Co-authored-by: Nate Koenig <[email protected]> Co-authored-by: Louise Poubel <[email protected]> Co-authored-by: John Shepherd <[email protected]> Co-authored-by: Steve Peters <[email protected]> Co-authored-by: Jose Luis Rivero <[email protected]> Co-authored-by: Akash Patel <[email protected]> * Add support for topic statistics on breadcrumb deployments (gazebosim#532) * Add support for topic statistics on breadcrumb deployments Signed-off-by: Nate Koenig <[email protected]> * Require version 9.1 of ignition transport Signed-off-by: Nate Koenig <[email protected]> * Move to after mutex Signed-off-by: Nate Koenig <[email protected]> Co-authored-by: Nate Koenig <[email protected]> Co-authored-by: Carlos Agüero <[email protected]> Co-authored-by: Martiño Crespo <[email protected]> Co-authored-by: Louise Poubel <[email protected]> Co-authored-by: Mabel Zhang <[email protected]> Co-authored-by: John Shepherd <[email protected]> Co-authored-by: Addisu Z. Taddese <[email protected]> Co-authored-by: Michael Carroll <[email protected]> Co-authored-by: Ashton Larkin <[email protected]> Co-authored-by: Ian Chen <[email protected]> Co-authored-by: Ivan Santiago Paunovic <[email protected]> Co-authored-by: Alejandro Hernández Cordero <[email protected]> Co-authored-by: Nate Koenig <[email protected]> Co-authored-by: Nate Koenig <[email protected]> Co-authored-by: G.Doisy <[email protected]> Co-authored-by: Gonzo <[email protected]> Co-authored-by: Steve Peters <[email protected]> Co-authored-by: Jose Luis Rivero <[email protected]> Co-authored-by: Akash Patel <[email protected]> Co-authored-by: Carlos Agüero <[email protected]>

Use lowercase resource and owner names when storing assets on disk

9fedd07

Signed-off-by: Nate Koenig <[email protected]>

github-actions bot added the 📜 blueprint Ignition Blueprint label Nov 9, 2020

chapulina requested a review from JShep1 November 9, 2020 19:51

chapulina assigned nkoenig Nov 16, 2020

John Shepherd added 3 commits November 20, 2020 15:59

Merge branch 'ign-fuel-tools3' into nkoenig/lowercase

a0882d9

Remove added diff text from merge

817089f

Signed-off-by: John Shepherd <[email protected]>

change model version in test back

82ba7a1

Signed-off-by: John Shepherd <[email protected]>

JShep1 approved these changes Nov 24, 2020

View reviewed changes

Fix test

0b31817

Signed-off-by: Nate Koenig <[email protected]>

Nate Koenig added 3 commits November 30, 2020 08:59

Fix windows

e4c059e

Signed-off-by: Nate Koenig <[email protected]>

Fix more windows tests

3a47e3b

Signed-off-by: Nate Koenig <[email protected]>

Testing another windows fix

b4183da

Signed-off-by: Nate Koenig <[email protected]>

nkoenig merged commit 4f7f5b6 into ign-fuel-tools3 Nov 30, 2020

nkoenig deleted the nkoenig/lowercase branch November 30, 2020 17:57

nkoenig mentioned this pull request Dec 8, 2020

Playback of logs containing Freyja config 2 - upper/lower case problem osrf/subt#689

Closed

azeey mentioned this pull request Dec 10, 2020

Fix tests that use Fuel paths with uppercase letters gazebosim/gz-sim#480

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Use lowercase resource and owner names when storing assets on disk #130

Use lowercase resource and owner names when storing assets on disk #130

nkoenig commented Nov 9, 2020

JShep1 left a comment •

edited

Loading

codecov bot commented Nov 30, 2020 •

edited

Loading

azeey commented Dec 10, 2020

Use lowercase resource and owner names when storing assets on disk #130

Use lowercase resource and owner names when storing assets on disk #130

Conversation

nkoenig commented Nov 9, 2020

JShep1 left a comment • edited Loading

Choose a reason for hiding this comment

codecov bot commented Nov 30, 2020 • edited Loading

Codecov Report

azeey commented Dec 10, 2020

JShep1 left a comment •

edited

Loading

codecov bot commented Nov 30, 2020 •

edited

Loading