archive.extracted: don't try to cache local sources #38285
Conversation
This will keep us from trying to cache file when we already have it locally, which will help significantly with larger archives.
terminalmage
force-pushed
the
archive-extracted-local-source
branch
from
4d15dab
to
e774080
Compare
|
@terminalmage Looks like there are some tests that need to be looked at here. |
rallytime
added
the
Pending-Discussion
|
Yeah I only checked the integration tests, forgot to look at unit tests. This is likely just due to the logic changes since we're not caching the file any more. I'll get these taken care of. |
|
Tests are updated |
terminalmage
force-pushed
the
archive-extracted-local-source
branch
from
d23b3fb
to
60311e3
Compare
|
I've got a fix for the lint failures, waiting for the rest of the tests to complete before I push the lint fixes though. |
terminalmage
force-pushed
the
archive-extracted-local-source
branch
from
60311e3
to
d018447
Compare
|
@terminalmage Hrm. It looks like some of these related tests are still failing on Ubuntu 14. |
This fixes failing tests for Ubuntu 14
* upstream/develop: (35 commits) Pylint fix Add space around equality operator Fix wrong function names Remove unused import Pylint fix Allow the creation of an internal interface for an openvswitch port Add NM_CONTROLLED to redhat configured interfaces Remove support for client_acl and client_acl_blacklist Remove enable/disable funcs from apache_* states Remove PidfileMixin as it is deprecated for removal in Nitrogen Remove doc markup references from 2016.11 branch archive.extracted: don't try to cache local sources (saltstack#38285) ZMQ: add an option for zmq.BACKLOG to salt master (zmq_backlog) allowing states from other saltenvs to be includes in top.sls Add --ssh-option to salt-ssh Fix influxdb_database.present state Don't check the doc/conf.py file for doc markup refs Update tests to reflect change in cache behavior archive.extracted: don't try to cache local sources (2016.3 branch) Add a unit test to search for new doc markup refs ...
| @@ -1304,7 +1315,7 @@ def extracted(name, | |||
| ret['changes']['directories_created'] = [name] | |||
| ret['changes']['extracted_files'] = files | |||
| ret['comment'] = '{0} extracted to {1}'.format(source_match, name) | |||
| if not keep: | |||
| if not source_is_local and not keep: | |||
There was a problem hiding this comment.
Am I reading this right that the default behavior now is to remove the local cached copy of a remove archive? It looks like the docs say the default for keep is supposed to be True but here it looks like the default is False.
I noticed a large increase in network activity and time when I updated some minions recently and tracked down the source to archive.extracted. The source files are no longer being stored in the minion cache and are being downloaded to the minions each time highstate is run.
Could this be the cause of the change in behavior?
This will keep us from trying to cache file when we already have it locally,
which will help significantly with larger archives.