[Request For Comment] : about repositories storage strategy

[ghost]

I've recently added RSpec unit tests to test the plugin and I discovered caveats in the repositories storage configuration strategy.

I explain :

You have 2 modes to store repositories in Gitolite :

• hierarchical : repositories will be stored in Gitolite into a hierarchy that mirrors the project hierarchy.
• flat : repositories will be stored in Gitolite directly under repository/, regardless of the number and identity of any parents that they may have.

On the top of that you can you decide if repositories identifiers should be unique in Redmine with the :unique_repo_identifier parameter.

When :unique_repo_identifier is set to true, repository identifiers must be unique across all projects (and cannot match the identifiers of projects either).

When set to false, then repository identifiers do not have to be unique across projects (although they still cannot have the same value as project identifiers).

This leads to a configuration matrix which looks like that :

• hierarchical / unique : true
• hierarchical / unique : false
• flat / unique : true
• flat / unique : false

The last case is the worst case and should not be used. It leads to buggy behavior like repositories not updated after commiting.

The problem is that you can change any of this parameter at any time and here is the caveat : switching from one case in the matrix to another can lead to the same buggy behavior.

I wrote a complete (I hope) tests suite to test the configuration matrix and switches between each one and could identify several bugs due to the switches.

These bugs cannot be solved as they are not really bugs but more side effects of the configuration matrix and the possibility of switching at any time.

But we can do things to avoid the worst cases to happen :

• add documentation in the wiki about this
• only allow the 2 truly working mode :
  • hierarchical / unique : false
  • flat / unique : true

This means that the button for 'unique_repo_identifier' parameter within Redmine interface will be removed. It will be set with the hierarchical / flat button.

• once the storage strategy is set, disallow switching from one to other.

I'd like to have your opinion about this 'issue' and the fix I propose.

Thank you!

Edit (12/06/2014) :

Two examples of side effects :

• unable to download_git_revision with hierarchial repositories #210
• missing project directory with hierarchical repositories... #217

[JaroslavHron]

Hi, thanks for the nice job developing this plugin.

I agree that these 2 options make the most sense.

What we use in some cases is the same repo for 2 projects/subprojects - that woks with the option 1 (hierarchical / unique : false)

I assume this would't work with the (unique : true) setting - or would it?
May be then one would need a choice - create repo or add existing repo
when connecting project with the rpository....

JH

[oliverguenther]

What is the use case for flat repositories? From my understanding, only with the flat hierarchy are repositories required to be uniquely named. What is the advantage with flat storage?

In an environment with a large number of users, the maintainer of some project X may be unable to assure that a sub-project X/Y is globally unique, whereas in the hierarchical form, only the top-level projects (e.g., X itself) have to be unique.

Why not just stick with hierarchical repositories at all times?

[n-rodriguez]

@j-h

I assume this would't work with the (unique : true) setting - or would it?

It won't work.

May be then one would need a choice - create repo or add existing repo
when connecting project with the rpository....

Why not, have to think about it ;)

@oliverguenther

From my understanding, only with the flat hierarchy are repositories required to be uniquely named

You're right

What is the advantage with flat storage?

When you don't want to expose your project hierarchy throught Git (http or ssh) addresses.

Why not just stick with hierarchical repositories at all times?

That's the purpose of this RFC : take a decision about the storage strategy :)
I didn't want to take such a decision alone as it could break a lot of configurations based on it.

[oliverguenther]

Thanks Nicolas, the point of exposed hierarchies did not occur to me. For my/our use case, this exposure is negligible. Of course there is also the issue of maintaining compatibility as you provided multiple variants.

[n-rodriguez]

Any other suggestion?

I think this could be done for the next release (0.8).

[rchady]

There is one more possible thing to consider here. Would it be possible to decouple the storage from the access path? What I mean by this is to store the repositories hierarchically, but have a flat access URL. This is actually something I wanted as it would allow you to move the repositories around on the backend without impacting the http/ssh URL.

I'm not sure it can be done fully in the plugin though since the SSH access happens outside the plugin. It was just a wish I had so I could move repositories without having to update the URL everywhere.

Beyond doing something like that, I think there is nothing wrong with only offering the 2 combinations.

[dgeo]

My thoughts about it:

• of course hierarchical/not unique and flat/unique are the only two ways to go, and changing from one to another shoud be avoided (and may require manual changes)
• One thing that must (in my mind at least) be forbidden is changing the access url's of an existing repo.
• I've been surprised that the url does not seems to be always derived from url (or root_url) field of the database, and seems computed (in missing project directory with hierarchical repositories... #217) – note that I didn't read the code (ruby skills missing here) to confirm that. It may help avoiding breakage after a switch, or upgrading from earlier versions (as I did)
• don't forget sub-projects repositories while testing :)

[n-rodriguez]

@rchady :

There is one more possible thing to consider here. Would it be possible to decouple the storage from the access path? What I mean by this is to store the repositories hierarchically, but have a flat access URL. This is actually something I wanted as it would allow you to move the repositories around on the backend without impacting the http/ssh URL.

This would be the best, but as you said, the SSH access happens outside the plugin and Gitolite must have the full repository path to access it.

@dgeo :

One thing that must (in my mind at least) be forbidden is changing the access url's of an existing repo.

The reason given above have 2 consequences :

• in hierarchical mode, if you move a project from a hierarchy to another you must move projects' repositories, else Gitolite will create a new (empty) repository and will let orphan repos behind.
• Hence you must update access url.

To avoid this... use flat mode ;)

[n-rodriguez]

The issue has been fixed in v0.8, I close it.

Thank you all for your contribution!