Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fix transformation host selection #379

Merged
merged 5 commits into from
Jul 30, 2018
Merged

Fix transformation host selection #379

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
83b3926
Fix transformation host selection
fabiendupont Jul 28, 2018
1040d2f
Separate ems_max_runners logic into a method
fabiendupont Jul 29, 2018
717b0f5
Add a separate method for max runners per host
fabiendupont Jul 29, 2018
f7a8760
Make default value more flexible for max runners
fabiendupont Jul 30, 2018
70f5df7
Trim the class structure as it is only a utility class
fabiendupont Jul 30, 2018
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
44 changes: 30 additions & 14 deletions ...nt/automate/ManageIQ/Transformation/TransformationHosts/Common.class/__methods__/utils.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,18 +4,26 @@ module Transformation
module TransformationHosts
module Common
class Utils
def initialize(handle = $evm)
@debug = true
@handle = handle
end

def main
end
DEFAULT_EMS_MAX_RUNNERS = 10
DEFAULT_HOST_MAX_RUNNERS = 10

def self.get_runners_count_by_host(host, handle = $evm)
handle.vmdb(:service_template_transformation_plan_task).where(:state => 'active').select { |task| task.get_option(:transformation_host_id) == host.id }.size
end

def self.host_max_runners(host, factory_config, max_runners = DEFAULT_HOST_MAX_RUNNERS, handle = $evm)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@fdupont-redhat Is this just a Utility class which only has class methods that you want to embed into other classes. We don't need to have an initialize or the call at the bottom if you are going to just call it from other methods.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I noticed this as well as there is an empty main method defined. I'm good with including those changes here if appropriate but that refactoring could be in a separate PR as well since it is unrelated.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, it's just a utility class. It is meant to be added as an embedded method. Should I remove it ? It's a bit out of scope for this PR. Maybe another cleanup PR ?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, so I removed the unneeded parts: initialize, main and call at end of file.

if host.custom_get('Max Transformation Runners')
handle.log(:info, "Using max transformation runners from host custom attribute: #{host.custom_get('Max Transformation Runners')}")
host.custom_get('Max Transformation Runners').to_i
elsif factory_config['transformation_host_max_runners']
handle.log(:info, "Using max transformation runners from factory config: #{factory_config['transformation_host_max_runners']}")
factory_config['transformation_host_max_runners'].to_i
else
handle.log(:info, "Using default max transformation runners: #{max_runners}")
max_runners
end
end

def self.transformation_hosts(ems, factory_config)
thosts = []
ems.hosts.each do |host|
Expand All @@ -26,7 +34,7 @@ def self.transformation_hosts(ems, factory_config)
:host => host,
:runners => {
:current => get_runners_count_by_host(host),
:maximum => host.custom_get('Max Transformation Runners') || factory_config['transformation_host_max_runners'] || 10
:maximum => host_max_runners(host, factory_config)
}
}
end
Expand All @@ -41,11 +49,23 @@ def self.get_runners_count_by_ems(ems, factory_config)
transformation_hosts(ems, factory_config).inject(0) { |sum, thost| sum + thost[:runners][:current] }
end

def self.ems_max_runners(ems, factory_config, max_runners = DEFAULT_EMS_MAX_RUNNERS, handle = $evm)
if ems.custom_get('Max Transformation Runners')
handle.log(:info, "Using max transformation runners from EMS custom attribute: #{ems.custom_get('Max Transformation Runners')}")
ems.custom_get('Max Transformation Runners').to_i
elsif factory_config['ems_max_runners']
handle.log(:info, "Using max transformation runners from factory config: #{factory_config['ems_max_runners']}")
factory_config['ems_max_runners'].to_i
else
handle.log(:info, "Using default max transformation runners: #{max_runners}")
max_runners
end
end

def self.get_transformation_host(task, factory_config, handle = $evm)
ems = handle.vmdb(:ext_management_system).find_by(:id => task.get_option(:destination_ems_id))
ems_max_runners = ems.custom_get('Max Transformation Runners').to_i || factory_config['ems_max_runners'] || 1
ems_cur_runners = get_runners_count_by_ems(ems, factory_config)
transformation_host_hash = ems_cur_runners < ems_max_runners ? eligible_transformation_hosts(ems, factory_config).first : {}
transformation_host_hash = ems_cur_runners < ems_max_runners(ems, factory_config) ? eligible_transformation_hosts(ems, factory_config).first : {}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Were you planning to change this call to ems_max_runners to pass 1 as the third parameter to override max_runners? That would maintain the previous logic.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nope. It was 1 just because I didn't know what could be a reasonable value. Now that RHV perf team has said that 10 should be the default per host, I'm setting it to 10 at the EMS level. This would allow 1 conversion host per EMS by default. Then user can override it using custom attribute.

return transformation_host_hash[:type], transformation_host_hash[:host], transformation_host_hash[:transformation_method]
end

Expand Down Expand Up @@ -118,7 +138,3 @@ def self.virtv2vwrapper_options_vmwarews2rhevm_ssh(task)
end
end
end

if $PROGRAM_NAME == __FILE__
ManageIQ::Automate::Transformation::TransformationHosts::Common::Utils.new.main
end
2 changes: 1 addition & 1 deletion ...omate/ManageIQ/Transformation/TransformationHosts/Common.class/__methods__/vmtransform.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ def main
@handle.log(:info, "Transformation - Started On: #{start_timestamp}")

destination_ems = @handle.vmdb(:ext_management_system).find_by(:id => task.get_option(:destination_ems_id))
max_runners = destination_ems.custom_get('Max Transformation Runners').to_i || factory_config['max_transformation_runners_by_ems'] || 10
max_runners = Transformation::TransformationHosts::Common::Utils.ems_max_runners(destination_ems, factory_config)
if Transformation::TransformationHosts::Common::Utils.get_runners_count_by_ems(destination_ems, factory_config) >= max_runners
@handle.log(:info, "Too many transformations running [#{max_runners}]. Retrying.")
else
Expand Down