Adding options field to ext_management_system #15398

enoodle · 2017-06-19T17:52:24Z

This will add the serialzed column options to ExtManagementSystem.
There is also a migration for ManageIQ::Providers::Kubernetes::ContainerManager options that where saved in CustomAttributes and general settings but should be there. This takes inspiration from #13826 and #14106 but the options details will be added in a later stage in the appropriate provider modules.

RFE BZ: https://bugzilla.redhat.com/show_bug.cgi?id=1462835

enoodle · 2017-06-19T17:52:38Z

cc @simon3z @blomquisg

simon3z · 2017-06-19T21:54:48Z

db/migrate/20170619161514_add_options_to_ext_management_system.rb

+      ExtManagementSystem.where(:type => "ManageIQ::Providers::Kubernetes::ContainerManager").each do |cp|
+        migrated_options = {}
+        migrated_options[:image_inspector_repository] = Settings.ems.ems_kubernetes.image_inspector_repository
+        migrated_options[:image_inspector_registry] = Settings.ems.ems_kubernetes.image_inspector_registry


@enoodle we don't want to move the Settings values in the options.

simon3z · 2017-06-19T21:55:55Z

db/migrate/20170619161514_add_options_to_ext_management_system.rb

+          :resource_type => cp.type,
+          :resource_id => cp.id,
+          :section => ManageIQ::Providers::Kubernetes::ContainerManager::Scanning::Job::ATTRIBUTE_SECTION,
+          :name    => ManageIQ::Providers::Kubernetes::ContainerManager::Scanning::Job::PROXY_ENV_VARIABLES).each do |ca|


@enoodle I don't think we want to rely on the Kubernetes module dependency here for this migration. cc @blomquisg

enoodle · 2017-06-20T07:26:25Z

I am not sure I understand, if we dont migrate these kubernetes options and not the ones from the settings, what should we migrate? On Jun 20, 2017 12:56 AM, "Federico Simoncelli" <[email protected]> wrote: *@simon3z* commented on this pull request. ------------------------------ In db/migrate/20170619161514_add_options_to_ext_management_system.rb <#15398 (comment)>:

+ serialize :serialized_value

+ end + + def up + add_column :ext_management_systems, :options, :text + + say_with_time("Migrating Kubernetes provider options") do + ExtManagementSystem.where(:type => "ManageIQ::Providers::Kubernetes::ContainerManager").each do |cp| + migrated_options = {} + migrated_options[:image_inspector_repository] = Settings.ems.ems_kubernetes.image_inspector_repository + migrated_options[:image_inspector_registry] = Settings.ems.ems_kubernetes.image_inspector_registry + CustomAttribute.where( + :resource_type => cp.type, + :resource_id => cp.id, + :section => ManageIQ::Providers::Kubernetes::ContainerManager::Scanning::Job::ATTRIBUTE_SECTION, + :name => ManageIQ::Providers::Kubernetes::ContainerManager::Scanning::Job::PROXY_ENV_VARIABLES).each do |ca| @enoodle <https://github.com/enoodle> I don't think we want to reply on the Kubernetes module dependency here for this migration. cc @blomquisg <https://github.com/blomquisg> — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#15398 (review)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AC-ogKxtxMuepzazq7RdtTYwaxaFyXH5ks5sFu5vgaJpZM4N-kbK> .

simon3z · 2017-06-20T07:59:31Z

I am not sure I understand, if we dont migrate these kubernetes options and
not the ones from the settings, what should we migrate?

@enoodle I don't want you to prepopulate the values in the db for the Settings ones... otherwise we can't change the defaults going forward.

Read more carefully the comment about the kubernetes options, it's about the module dependency.

simon3z · 2017-06-20T09:29:58Z

Current implementation adding a generic options json column to ext_management_system LGTM 👍.

Of course I am also considering the downside of this approach. The table authentications just recently added the same options json column so I assume we reached the same conclusion on the need for a more flexible field for special best-of-breed options (of course it must be used with care).

For completeness I want to mention that since most of the times (I'd say always?) these are also global settings, e.g.:

:ems:
  :ems_kubernetes:
    :miq_namespace: management-infra
    :image_inspector_registry: docker.io
    :image_inspector_repository: openshift/image-inspector

It's worth mentioning that this could be achieved also using ExtManagementSystem specific settings in the settings_changes table:

id	resource_type	resource_id	key	value
100	ExtManagementSystem	1	/ems/ems_kubernetes/image_inspector_registry	mydockerregistry.io
101	ExtManagementSystem	2	/ems/ems_kubernetes/image_inspector_registry	myotherdockerregistry.io

I am not sure of the implications of this second approach though.

cc @blomquisg

simon3z · 2017-06-20T10:50:35Z

db/migrate/20170619161514_add_options_to_ext_management_system.rb

+          :resource_type => cp.type,
+          :resource_id   => cp.id,
+          :section       => ManageIQ::Providers::Kubernetes::ContainerManager::Scanning::Job::ATTRIBUTE_SECTION,
+          :name          => ManageIQ::Providers::Kubernetes::ContainerManager::Scanning::Job::PROXY_ENV_VARIABLES


@enoodle as mentioned before I am not sure if we can afford this dependency to the Kubernetes module. cc @blomquisg You may have to hard-code the values here.

enoodle · 2017-06-20T15:47:42Z

I have added deletion of the old CustomAttribute's with the settings and also made sure that the migration is happening on Kubernetes and Openshift Providers.

simon3z · 2017-06-20T15:49:49Z

db/migrate/20170619161514_add_options_to_ext_management_system.rb

+          :name          => %w(no_proxy http_proxy https_proxy)
+        ).each do |ca|
+          migrated_options["image_inspector_#{ca.name}".to_sym] = ca.value
+          ca.delete


@enoodle isn't it risky to delete this before having it persisted in the new options? (cp.update)

simon3z · 2017-06-21T10:00:43Z

@blomquisg @Fryguy can you review?

miq-bot · 2017-06-22T17:37:14Z

This pull request is not mergeable. Please rebase and repush.

Fryguy · 2017-06-22T17:46:02Z

Database migrations have now been moved to the https://github.com/ManageIQ/manageiq-schema repo. Please see http://talk.manageiq.org/t/new-split-repo-manageiq-schema/2478 for instructions on how to transfer your database migrations. If this PR contains only migrations, I will leave it open for a short time during the transition, after which I will close this if it has not been moved over.

miq-bot · 2017-06-25T08:47:58Z

Checked commit enoodle@ad37b0e with ruby 2.2.6, rubocop 0.47.1, and haml-lint 0.20.0
1 file checked, 0 offenses detected
Everything looks fine. 🏆

moolitayer · 2017-07-25T17:45:53Z

cc @Loicavenel

miq-bot added the sql migration label Jun 19, 2017

simon3z reviewed Jun 19, 2017

View reviewed changes

enoodle force-pushed the add_options_to_ems branch from e42b21c to 28e7df3 Compare June 20, 2017 08:53

enoodle force-pushed the add_options_to_ems branch 2 times, most recently from 5ce53e8 to fca068a Compare June 20, 2017 10:22

simon3z reviewed Jun 20, 2017

View reviewed changes

enoodle force-pushed the add_options_to_ems branch from fca068a to bbcdbfe Compare June 20, 2017 15:01

enoodle mentioned this pull request Jun 20, 2017

Using the new options for image-scanning options ManageIQ/manageiq-providers-kubernetes#45

Merged

simon3z reviewed Jun 20, 2017

View reviewed changes

enoodle force-pushed the add_options_to_ems branch from 2fef4cc to ae9a16d Compare June 20, 2017 16:06

chessbyte assigned Fryguy Jun 20, 2017

simon3z mentioned this pull request Jun 22, 2017

Adding 'visible' field to custom attributes #14611

Closed

miq-bot added the unmergeable label Jun 22, 2017

This was referenced Jun 23, 2017

Containers: redact username and password from custom attributes values when displaying ManageIQ/manageiq-ui-classic#844

Merged

Don't redact custom attributes ManageIQ/manageiq-ui-classic#1596

Merged

adding options field to ext_management_system

ad37b0e

enoodle force-pushed the add_options_to_ems branch from ae9a16d to ad37b0e Compare June 25, 2017 08:45

miq-bot removed the unmergeable label Jun 25, 2017

enoodle mentioned this pull request Jun 25, 2017

Add options to ems ManageIQ/manageiq-schema#23

Merged

enoodle mentioned this pull request Jul 5, 2017

Add the ability to edit options settings for providers ManageIQ/manageiq-ui-classic#1652

Merged

Fryguy merged commit 56f639e into ManageIQ:master Jul 25, 2017

Fryguy added core enhancement and removed sql migration labels Jul 25, 2017

Fryguy added this to the Sprint 66 Ending Aug 7, 2017 milestone Jul 25, 2017

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Adding options field to ext_management_system #15398

Adding options field to ext_management_system #15398

enoodle commented Jun 19, 2017

enoodle commented Jun 19, 2017

simon3z Jun 19, 2017

simon3z Jun 19, 2017 •

edited

Loading

enoodle commented Jun 20, 2017 via email

simon3z commented Jun 20, 2017

simon3z commented Jun 20, 2017

simon3z Jun 20, 2017

enoodle commented Jun 20, 2017

simon3z Jun 20, 2017

simon3z commented Jun 21, 2017

miq-bot commented Jun 22, 2017

Fryguy commented Jun 22, 2017

miq-bot commented Jun 25, 2017

moolitayer commented Jul 25, 2017

Adding options field to ext_management_system #15398

Adding options field to ext_management_system #15398

Conversation

enoodle commented Jun 19, 2017

enoodle commented Jun 19, 2017

simon3z Jun 19, 2017

Choose a reason for hiding this comment

simon3z Jun 19, 2017 • edited Loading

Choose a reason for hiding this comment

enoodle commented Jun 20, 2017 via email

simon3z commented Jun 20, 2017

simon3z commented Jun 20, 2017

simon3z Jun 20, 2017

Choose a reason for hiding this comment

enoodle commented Jun 20, 2017

simon3z Jun 20, 2017

Choose a reason for hiding this comment

simon3z commented Jun 21, 2017

miq-bot commented Jun 22, 2017

Fryguy commented Jun 22, 2017

miq-bot commented Jun 25, 2017

moolitayer commented Jul 25, 2017

simon3z Jun 19, 2017 •

edited

Loading