Total Parameter added to original model

[keyuchen21]

Is there anyway we can calculate the total parameter added to original model?

[lenglaender]

Hi @km5ar,
to get a summary of the currently added adapters, you can use the adapter_summary() function like this:

print(model.adapter_summary())

For an example script & output, see here: #371
You can then sum up the "#Param" column values to get the total number of parameters added.

Alternatively, if you want a function that directly returns the total number of added parameters, you could write a new function based on the implementation of adapter_summary(). This would need just very few changes; the relevant part in the code is here:

adapters/src/adapters/model_mixin.py

Lines 1033 to 1050 in 35fd1c2

	for name, config_name in self.adapters_config.adapters.items():
	if config_name in self.adapters_config.config_map:
	config = self.adapters_config.config_map.get(config_name, None)
	else:
	config = ADAPTER_CONFIG_MAP.get(config_name, None)
	if isinstance(config, str):
	config = ADAPTER_CONFIG_MAP[config]
	row = {"name": name, "architecture": config.get("architecture", None) or "bottleneck"}
	weights = self.get_adapter(name)
	row["active"] = self.active_adapters is not None and name in self.active_adapters.flatten()
	# count parameters
	no_params = 0
	train = True
	for _, module_dict in weights.items():
	for _, module in module_dict.items():
	no_params += sum(p.numel() for p in module.parameters())
	train &= all(p.requires_grad for p in module.parameters())
	row["#param"] = no_params