ADD: Repository methods for repo CLI and other features

[ramirezfranciscof]

Methods added and modifications:

• Added has_objects as new abstract and adapted has_object
• Added delete_objects as new abstract and adapted delete_object
• Added list_objects method
• While modifying the inherited classes I removed some of the docstrings
  since these should be anyways inherited from the parent class and were
  just adding visual noise and maintenance costs.

[ramirezfranciscof]

Created from #4965, @chrisjsewell I modified the type returned so let me know if this now works for you too or there is something else that would need adapting. In the meantime I'll try to add a couple of tests for these new features.

[codecov]

Codecov Report

Merging #5156 (c2c64b1) into develop (2c5cc6f) will increase coverage by 0.02%.
The diff coverage is 95.72%.

@@             Coverage Diff             @@
##           develop    #5156      +/-   ##
===========================================
+ Coverage    80.91%   80.93%   +0.02%     
===========================================
  Files          536      536              
  Lines        37019    37056      +37     
===========================================
+ Hits         29952    29987      +35     
- Misses        7067     7069       +2     

Flag	Coverage Δ
django	75.78% <95.72%> (+0.03%)	⬆️
sqlalchemy	74.93% <95.72%> (+0.02%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
aiida/backends/general/migrations/utils.py	86.39% <72.73%> (-1.11%)	⬇️
aiida/repository/backend/abstract.py	98.42% <100.00%> (+2.58%)	⬆️
aiida/repository/backend/disk_object_store.py	96.50% <100.00%> (+0.34%)	⬆️
aiida/repository/backend/sandbox.py	100.00% <100.00%> (ø)

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 2c5cc6f...c2c64b1. Read the comment docs.

[chrisjsewell]

Yep cheers, another thing from #5145 is that I think there should be a key_format property, e.g.

for the DiskObjectStoreRepositoryBackend it would be:

@property
def key_format(self):
	return self.container.hash_type

and for SandboxRepositoryBackend it would be:

@property
def key_format(self):
	return 'uuid4'

The reason this is necessary is that, when migrating between backends (e.g. archive -> main), one needs to know if the key_format are equal. If not, then you would need to re-compute all the Node.repository_metadata before importing them, otherwise they will not match with the repository.

[chrisjsewell]

FYI, another thing I want to do, is to move Profile.get_repository to Backend.get_repository (see #5154)

[chrisjsewell]

Suggested change

	"""Return iterable that yeilds all available objects by key.
	"""Return iterable that yields all available objects by key.

[chrisjsewell]

If you use VS Code, I certainly recommend https://marketplace.visualstudio.com/items?itemName=streetsidesoftware.code-spell-checker 😉

[ramirezfranciscof]

If I'm developing inside a docker container, will this extension automatically install inside of it when I attach through the remote explorer or will I have to manually re-install it every time I shutdown and re-start the container?

[chrisjsewell]

I know with the new vs code devcontainers, you can specify which extensions to load, see e.g. https://github.com/chrisjsewell/aiida-install/blob/56fe6d7398594c8813890652af99c521a77f249c/.devcontainer/devcontainer.json#L41

[ramirezfranciscof]

When you are using these you need to create the container through the vscode right? I'm not sure this would work for me because I need to initialize mine using docker-compose (I have 3 containers running for this: rabbitmq, postgres, and the work container).

[chrisjsewell]

I was actually playing around with that a bit but found, at least on my macbook, it was a bit slow a prone to the container dying:

.devcontainer/devcontainer.json

// For format details, see https://aka.ms/devcontainer.json. For config options, see the README at:
// https://github.com/microsoft/vscode-dev-containers/tree/v0.187.0/containers/docker-existing-dockerfile
// https://github.com/microsoft/vscode-dev-containers/tree/main/containers/python-3-postgres
{
    "name": "AiiDA development environment",
    "dockerComposeFile": "docker-compose.yml",
    "service": "main",
    "workspaceFolder": "/workspace",

	// Set *default* container specific settings.json values on container create.
	"settings": {
        "editor.rulers": [100],
        "python.pythonPath": "/usr/local/bin/python",
		"python.defaultInterpreterPath": "/usr/local/bin/python",
        "python.languageServer": "Pylance",
		"python.linting.enabled": true,
		"python.linting.pylintEnabled": true,
        "python.linting.pylintPath": "/usr/local/py-utils/bin/pylint",
        "python.formatting.provider": "yapf",
        "python.formatting.yapfPath": "/usr/local/py-utils/bin/yapf",
    },
	
	// Add the IDs of extensions you want installed when the container is created.
	"extensions": [
		"ms-python.python",
		"ms-python.vscode-pylance",
		"ms-azuretools.vscode-docker",
		"streetsidesoftware.code-spell-checker",
		"chrisjsewell.aiida-explore-vscode"
	],
}

.devcontainer/docker-compose.yml

version: '3.4'

services:

  main:
    image: 'mcr.microsoft.com/vscode/devcontainers/python:3.8'
    volumes:
      - ..:/workspace:cached
    # Overrides default command so things don't shut down after the process ends.
    command: sleep infinity

  postgres:
    image: postgres:12.3
    environment:
      POSTGRES_USER: pguser
      POSTGRES_PASSWORD: password
    restart: unless-stopped

  rmq:
    image: rabbitmq:3.8.3-management
    environment:
      RABBITMQ_DEFAULT_USER: guest
      RABBITMQ_DEFAULT_PASS: guest
    restart: unless-stopped

[ramirezfranciscof]

Uhm, very interesting! Thanks, I will give this a try =D

[ramirezfranciscof]

I added the tests I could think of so this is now ready for another round of review. I also added the key_format property: if I understood correctly you only need the property defined here, not any feature using it, correct @chrisjsewell ?

[chrisjsewell]

All good thanks @ramirezfranciscof 👌