Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Discrete resource tracking for VirtualMachines #8356

Closed
PieterL75 opened this issue Jan 14, 2022 · 17 comments
Closed

Discrete resource tracking for VirtualMachines #8356

PieterL75 opened this issue Jan 14, 2022 · 17 comments
Assignees
Labels
status: accepted This issue has been accepted for implementation type: feature Introduction of new functionality to the application
Milestone

Comments

@PieterL75
Copy link
Contributor

NetBox version

v3.1.5

Feature type

Change to existing functionality

Proposed functionality

Device have the option to document inventory items. (cpu, disks, blades, modules, ...)
For virtual machines,this option is not available.

I would like to see the possibility to add inventory items to virtual-machines

Use case

VM can have additional 'hardware' like disks.
The inventory items on a VM will allow us to document how many disks and what size the disks has.
By default we have no CDROM attached to the VM. with the inventory we could add a CDROM to document that exception

Database changes

extend the dcim/inventory-items model to be attached to virtualization/virtual-machines

External dependencies

No response

@PieterL75 PieterL75 added the type: feature Introduction of new functionality to the application label Jan 14, 2022
@jk-onnet
Copy link

#1887 #4783 and others have similar intend approach, according to NetBox maintainer, this might push NetBox to asset management tools or virtual orchestration tools. But I would like to have this feature to be accepted.

In v3.2 milestone, I think the inventory items in device has a huge change of core fields. As long as NetBox grows, the custom field (reference to object or to network interfaces) and potentially inventory item in virtualization will make this more friendly to users and a better source of truth for automation.

In our use case, we developed a custom script to provision our virtual machine to document the network. We would like to pass and document other possible proposed values of virtual machine. Our alternative approach is using custom field (json) but this is hard for our script users to audit or track the value based on the json field instead of object based field. A similar inventory item model as device to virtual machine can be introduced.

@PieterL75
Copy link
Contributor Author

I see that the request were closed with the remarks 'NB primary function is to manage network infrastructure, not end hosts.'
I think that NB evolved and is more than that.
In my case, we try do document the physical world of our datacenter. That does not only include the network gear, but also all servers, appliances, etc.
If I have cable coming out of my switch, then I want it to go to a device that exists in Netbox, and not only a 'is connected' checkbox with some remarks on what other CMDB that owns the server.
And because the IPAM is 100% in Netbox, we have almost no other choice then to add the VM's to Netbox also.

I feel that Netbox was not intended to do that (Devices are mainly network focused and adding Servers quickly hits limitations), but step by step it could become a 100% physical world tool

@dkirhlarov
Copy link

NetBox version 3.1.0

We are using VM's with CEPH store or local store in our own OpenNebula clouds. For now we have to use custom field for separate store types. We would like to use VM types or inventory items for VMs like as Device.

@jeremystretch jeremystretch added the status: under review Further discussion is needed to determine this issue's scope and/or implementation label Apr 6, 2022
@jeremystretch
Copy link
Member

I appreciate the use case here, but I'm not sure it makes sense to repurpose the existing inventory item model, which supports fields like part_id and manufacturer, as well as component mapping. It might be better to introduce a separate model e.g. VMResource to model VM-specific items.

@DanSheps
Copy link
Member

DanSheps commented Apr 8, 2022

I think a new model would be appropriate.

I guess the question is what fields do we want to capture? The problem with VM resources is they are sometimes all over the map with what you might actually want to capture.

For example,

  • Disk you might want to associate it with a parent on the cluster, which I don't think we can capture here unless we dive deep into it and come out with multiple models.
  • CPU you care about speed, which could just simply be tied to disk using a generic "value".
  • CD/DVD drives get a little more complex as you don't need speed but you might want to track whether it is a physical passthrough or a ISO mounted by the hypervisor
  • NIC, again you might want to associate a vswitch or something.

I think, if we wanted to get a decent model we might want to revisit the VM model all together and see if we want to do things like modelling vswitches/sans and stuff as well. In the meantime, perhaps a few freeform text fields would be all that is needed but that does make automatic migrations difficult.

@themurman
Copy link

My specific use case is passing pcie nics through from the host to a vm running a software firewall/router appliance. If we are documenting the network, I would argue that we have to go inside the hosts to have complete documentation.

@MalfuncEddie
Copy link

My use case is has 2 parts:
Vmware vm:
I would write a scritp that read out the API. For each disk on the vm create a Virtual disk with size and spec (HDD. SSD ...)

vmware vm with direct attache lun or physical server:

Well this is a bit more complex. I could read out the storage subsystem and gather all the disks and create them in a "virtual disk/lun device" The tricky part would be mapping it to the correct server (hate to do this manual)

There is also the question what "storage" you want?
What do you do with a LUN that is replicated? or thin provisioned or has a snapshot?

I would suggest splitting up storage into a backend layer and a front end layer
backend:

  • 3par/ceph/storwize look of "how much raw storage do I have"

frontend:

  • What Luns are defined (and are the replicated or snapshotted) and to what device (if any) are they mapped.

The underlying use case is a question I got that mgmt wants to know how much storage/cpu/... a certain department (tenant) is using so a replicated LUN needs to have a checkmark "replicated" so it ha a factor times 2.

@jose-pr
Copy link

jose-pr commented Aug 9, 2022

Would like to be able to document passthrough devices, such as hddd, nic, gpu.

@github-actions
Copy link
Contributor

github-actions bot commented Oct 8, 2022

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. NetBox is governed by a small group of core maintainers which means not all opened issues may receive direct feedback. Do not attempt to circumvent this process by "bumping" the issue; doing so will result in its immediate closure and you may be barred from participating in any future discussions. Please see our contributing guide.

@github-actions github-actions bot added the pending closure Requires immediate attention to avoid being closed for inactivity label Oct 8, 2022
@arjenvri
Copy link

Could we possibly reduce the scope initially for this to being able to add multiple disks? The current field ‘disk space’ seems less useful as @PieterL75 outlined since many vms have multiple disks

@pycolas
Copy link

pycolas commented Dec 6, 2022

I upvote this idea (possibility to add inventory items to virtual-machines).
I don't see why an inventory item shall be a physical object.

I was thinking of using inventory items to list certificates and licences attached to a device (physical or virtual), but it's not possible for VM.
Maybe it's not the perfect match, but here it would help removing spreadsheets completely.

@jeremystretch
Copy link
Member

I don't see why an inventory item shall be a physical object.

@pycolas the InventoryItem model was constructed specifically to represent physical resources; hence the presence of e.g. the serial number and manufacturer fields. These wouldn't make sense in the context of a virtual resource, so we'd need to use a separate model, much like we do for interfaces belong to VMs vs. devices.

I'm tagging this as needs milestone with the understanding that it will entail the introduction of a new model suitable for representing virtual items.

@jeremystretch jeremystretch added needs milestone Awaiting prioritization for inclusion with a future NetBox release and removed status: under review Further discussion is needed to determine this issue's scope and/or implementation pending closure Requires immediate attention to avoid being closed for inactivity labels Jan 24, 2023
@thefreakquency
Copy link

We use Inventory items to track individual Software license keys. Having this for VM would be much appreciated.

@pdenessen
Copy link

Hi, I also go the question to add the used licenses to the system. Inventory items would indeed a nice solutions but it lacks, as already mentioned, the use of the VMs. Hope this feature can be added soon.

@stebr1
Copy link

stebr1 commented Aug 1, 2023

I also will appreciate to have the disks details for each VMs

@jeremystretch jeremystretch changed the title Inventory items for VirtualMachines Discrete resource tracking for VirtualMachines Oct 13, 2023
@jeremystretch jeremystretch added status: accepted This issue has been accepted for implementation and removed needs milestone Awaiting prioritization for inclusion with a future NetBox release labels Oct 13, 2023
@jeremystretch jeremystretch added this to the v3.7 milestone Oct 13, 2023
@jeremystretch
Copy link
Member

In an effort to progress work on this FR, I'm going to narrow its scope as suggested above to virtual disks. We can introduce a new VirtualDisk model with name and size fields, instances of which must be assigned to a specific virtual machine. We'll retain the existing disk field on VirtualMachine for now for backward compatibility, however I'd expect it to be removed at some point in the future (possibly in v4.0).

I'd be open to introducing a similar change for modeling discrete virtual CPUs as well, however there doesn't seem to be any demand for it, and it doesn't seem to me like it would bring much value.

@DanSheps
Copy link
Member

One other thing I am seeing demand for would be virtual switches/etc (including their relation to the specific host or cluster's uplinks).

We may want to look at that after this FR is completed (as a new FR of course).

jeremystretch added a commit that referenced this issue Nov 17, 2023
* 8356 add virtual disk model

* 8356 add supplemental forms

* 8356 add menu

* 8356 cleanup views

* 8356 virtual machine tab

* 8356 migrations

* 8356 vm disk tables

* 8356 cleanup

* 8356 graphql

* 8356 graphql

* 8356 add components button

* 8356 bulk add on virtualmachine

* 8356 bulk add fixes

* 8356 api tests

* 8356 news tests add rename

* 8356 VirtualDiskCreateForm

* 8356 fix test

* 8356 add todo to remove disk from vm

* 8356 review changes

* 8356 fix test

* 8356 deprecate disk field

* 8356 review changes

* 8356 fix test

* 8356 fix test

* Simplify view actions

* 8356 review changes

* 8356 split trans tag

* 8356 add total virtual disk size to api

* 8356 add virtual disk list to virtual machine detail view

* 8356 move virtual disk size to property

* 8356 revert property

* Tweak display of deprecated disk field

* 8356 render single disk field

* 8356 update serializer

* 8356 model property

* 8356 fix test

* 8356 review changes

* Revert disk space annotation

* Use existing disk field to store aggregate virtual disk size

* Introduce abstract ComponentModel for VM components

* Add search index for VirtualDisk

* Misc cleanup

---------

Co-authored-by: Jeremy Stretch <[email protected]>
@github-actions github-actions bot locked as resolved and limited conversation to collaborators Feb 16, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
status: accepted This issue has been accepted for implementation type: feature Introduction of new functionality to the application
Projects
None yet
Development

No branches or pull requests