Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

GitLab Integration #3327

Merged
merged 43 commits into from
Dec 7, 2017
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
43 commits
Select commit Hold shift + click to select a range
ef9c792
Add GitLab repo sync and webhook support
Jan 9, 2016
cf5c47c
oauth: gitlab: use unicode whenever format is used
BenoitKnecht Jun 16, 2016
2356d84
oauth: gitlab: prevent duplicate entries after sync
BenoitKnecht Jun 20, 2016
7113e94
oauth: gitlab: avoid KeyError when owner is missing
BenoitKnecht Jun 20, 2016
5d7d6c1
oauth: gitlab: setup_webhook must return a tuple
BenoitKnecht Jun 20, 2016
f9df35e
oauth: gitlab: set a default avatar if none is returned
BenoitKnecht Jun 20, 2016
e4adb98
oauth: gitlab: use SSH url if repo is private
BenoitKnecht Jun 20, 2016
da7471f
Added tests for the GitLabService
Aug 5, 2016
aec6941
Added some documentation for the Gitlab integration
Aug 5, 2016
9152bb6
Merge pull request #1 from galileo-press/sync-gitlab-repos
saily Aug 5, 2016
299a2b5
Improved url_pattern of the GitLabService to support private reposito…
Aug 5, 2016
8a10848
Fixed linting error
Aug 5, 2016
97a4fb1
Simplified exception handling
Aug 10, 2016
dff4d7d
Use PRODUCTION_DOMAIN as webhook id
Aug 10, 2016
4fbe6bb
Make imports python3 compatible
Aug 10, 2016
50285fe
Use the GitLab project ID, not the PRODUCTION_DOMAIN
Aug 10, 2016
e288bf3
Merge pull request #2 from galileo-press/sync-gitlab-repos
saily Oct 5, 2016
6c8d1c3
Update urls as mentioned in comments
Oct 19, 2016
0cb1ed5
Update webhook docs as mentioned in comments
saily Oct 19, 2016
435e870
Merge branch 'sync-gitlab-repos' of https://github.com/saily/readthed…
humitos Nov 27, 2017
c33b691
Style changes
humitos Nov 27, 2017
be5ccd0
New interface for paginated results
humitos Nov 27, 2017
7e38e70
Improve gitlab integration's code
humitos Nov 27, 2017
90e4ba1
Fix sync orgs and webhook create
humitos Nov 28, 2017
1e8a8a8
Fix webhook URL
humitos Nov 28, 2017
2fa1c51
Update webhook workflow for GitLab
humitos Nov 28, 2017
39c9899
Merge branch 'master' of github.com:rtfd/readthedocs.org into humitos…
humitos Nov 28, 2017
c2f16d6
Better yapf setting to fulfill PEP257 D202
humitos Nov 28, 2017
e77efa4
Lint fix
humitos Nov 28, 2017
b1ff48c
Remove default avatars for GitLab
humitos Nov 28, 2017
4db7339
Do not use REGEX to get owner and repo
humitos Nov 28, 2017
5e5a6c8
Merge branch 'master' of github.com:rtfd/readthedocs.org into humitos…
humitos Nov 28, 2017
b8e1b25
Fix GitLab tests
humitos Nov 28, 2017
ad6ba74
Mock `is_owned_by` properly
humitos Nov 28, 2017
66f8ea2
Added button to Import Project dashboard
humitos Nov 28, 2017
237531c
Feedback on docs
humitos Nov 28, 2017
4e6f4a6
Fix isort settings
humitos Nov 28, 2017
0ce1702
Fix lint
humitos Nov 29, 2017
1e668e1
Styles / Linting
humitos Dec 4, 2017
4eec51e
Context for GitLab on templates
humitos Dec 4, 2017
1e5200e
Merge branch 'master' of github.com:rtfd/readthedocs.org into humitos…
humitos Dec 4, 2017
12eae4b
More style
humitos Dec 5, 2017
3d276dd
Test for get_gitlab_url
humitos Dec 6, 2017
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 2 additions & 3 deletions docs/features.rst
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,8 @@ Read the Docs features

This will serve as a list of all of the features that Read the Docs currently has. Some features are important enough to have their own page in the docs, others will simply be listed here.

GitHub and Bitbucket Integration
--------------------------------
GitHub, Bitbucket and GitLab Integration
----------------------------------------

We now support linking by default in the sidebar. It links to the page on your host, which should help people quickly update typos and send pull requests to contribute to project documentation.

Expand Down Expand Up @@ -66,4 +66,3 @@ Alternate Domains
-----------------

We provide support for CNAMEs, subdomains, and a shorturl for your project as well. This is outlined in the :doc:`alternate_domains` section.

6 changes: 2 additions & 4 deletions docs/getting_started.rst
Original file line number Diff line number Diff line change
Expand Up @@ -88,15 +88,13 @@ Then in your ``conf.py``:
Sign Up and Connect an External Account
---------------------------------------

.. TODO Update this with GitLab support later

If you are going to import a repository from GitHub or Bitbucket, you should
If you are going to import a repository from GitHub, Bitbucket or GitLab, you should
connect your account to your provider first. Connecting your account allows for
easier importing and enables Read the Docs to configure your repository webhooks
automatically.

To connect your account, got to your *Settings* dashboard and select *Connected
Services*. From here, you'll be able to connect to your GitHub or Bitbucket
Services*. From here, you'll be able to connect to your GitHub, Bitbucket or GitLab
account. This process will ask you to authorize a connection to Read the Docs,
that allows us to read information about and clone your repositories.

Expand Down
2 changes: 1 addition & 1 deletion docs/webhooks.rst
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ Webhooks

The primary method that Read the Docs uses to detect changes to your
documentation is through the use of *webhooks*. Webhooks are configured with
your repository provider, such as GitHub or Bitbucket, and with each commit,
your repository provider, such as GitHub, Bitbucket or GitLab, and with each commit,
merge, or other change to your repository, Read the Docs is notified. When we
receive a webhook notification, we determine if the change is related to an
active version for your project, and if it is, a build is triggered for that
Expand Down
44 changes: 41 additions & 3 deletions readthedocs/builds/models.py
Original file line number Diff line number Diff line change
Expand Up @@ -21,8 +21,8 @@

from readthedocs.core.utils import broadcast
from readthedocs.projects.constants import (
BITBUCKET_REGEXS, BITBUCKET_URL, GITHUB_REGEXS, GITHUB_URL, PRIVACY_CHOICES,
PRIVATE)
BITBUCKET_REGEXS, BITBUCKET_URL, GITHUB_REGEXS, GITHUB_URL, GITLAB_REGEXS,
GITLAB_URL, PRIVACY_CHOICES, PRIVATE)
from readthedocs.projects.models import APIProject, Project

from .constants import (
Expand Down Expand Up @@ -113,7 +113,7 @@ def commit_name(self):
Return the branch name, the tag name or the revision identifier.

The result could be used as ref in a git repo, e.g. for linking to
GitHub or Bitbucket.
GitHub, Bitbucket or GitLab.
"""
# LATEST is special as it is usually a branch but does not contain the
# name in verbose_name.
Expand Down Expand Up @@ -295,6 +295,44 @@ def get_github_url(
action=action_string,
)

def get_gitlab_url(
self, docroot, filename, source_suffix='.rst', action='view'):
repo_url = self.project.repo
if 'gitlab' not in repo_url:
return ''

if not docroot:
return ''
else:
if docroot[0] != '/':
docroot = '/{}'.format(docroot)
if docroot[-1] != '/':
docroot = '{}/'.format(docroot)

if action == 'view':
action_string = 'blob'
elif action == 'edit':
action_string = 'edit'

for regex in GITLAB_REGEXS:
match = regex.search(repo_url)
if match:
user, repo = match.groups()
break
else:
return ''
repo = repo.rstrip('/')

return GITLAB_URL.format(
user=user,
repo=repo,
version=self.commit_name,
docroot=docroot,
path=filename,
source_suffix=source_suffix,
action=action_string,
)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should test this method

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done!


def get_bitbucket_url(self, docroot, filename, source_suffix='.rst'):
repo_url = self.project.repo
if 'bitbucket' not in repo_url:
Expand Down
12 changes: 12 additions & 0 deletions readthedocs/doc_builder/backends/sphinx.py
Original file line number Diff line number Diff line change
Expand Up @@ -82,6 +82,11 @@ def get_config_params(self):
bitbucket_version_is_editable = (self.version.type == 'branch')
display_bitbucket = bitbucket_user is not None

gitlab_user, gitlab_repo = version_utils.get_gitlab_username_repo(
url=self.project.repo)
gitlab_version_is_editable = (self.version.type == 'branch')
display_gitlab = gitlab_user is not None

# Avoid hitting database and API if using Docker build environment
if getattr(settings, 'DONT_HIT_API', False):
versions = self.project.active_versions()
Expand Down Expand Up @@ -119,6 +124,13 @@ def get_config_params(self):
'bitbucket_version': remote_version,
'bitbucket_version_is_editable': bitbucket_version_is_editable,
'display_bitbucket': display_bitbucket,

# GitLab
'gitlab_user': gitlab_user,
'gitlab_repo': gitlab_repo,
'gitlab_version': remote_version,
'gitlab_version_is_editable': gitlab_version_is_editable,
'display_gitlab': display_gitlab,
}

finalize_sphinx_context_data.send(
Expand Down
4 changes: 4 additions & 0 deletions readthedocs/doc_builder/templates/doc_builder/conf.py.tmpl
Original file line number Diff line number Diff line change
Expand Up @@ -105,6 +105,10 @@ context = {
'bitbucket_repo': '{{ bitbucket_repo }}',
'bitbucket_version': '{{ bitbucket_version }}',
'display_bitbucket': {{ display_bitbucket }},
'gitlab_user': '{{ gitlab_user }}',
'gitlab_repo': '{{ gitlab_repo }}',
'gitlab_version': '{{ gitlab_version }}',
'display_gitlab': {{ display_gitlab }},
'READTHEDOCS': True,
'using_theme': (html_theme == "default"),
'new_theme': (html_theme == "sphinx_rtd_theme"),
Expand Down
16 changes: 16 additions & 0 deletions readthedocs/integrations/models.py
Original file line number Diff line number Diff line change
Expand Up @@ -306,6 +306,22 @@ def can_sync(self):
return False


class GitLabWebhook(Integration):

integration_type_id = Integration.GITLAB_WEBHOOK
has_sync = True

class Meta(object):
proxy = True

@property
def can_sync(self):
try:
return all((k in self.provider_data) for k in ['id', 'url'])
except (ValueError, TypeError):
return False


class GenericAPIWebhook(Integration):

integration_type_id = Integration.API_WEBHOOK
Expand Down
8 changes: 8 additions & 0 deletions readthedocs/projects/constants.py
Original file line number Diff line number Diff line change
Expand Up @@ -291,9 +291,17 @@
re.compile('bitbucket.org/(.+)/(.+)/'),
re.compile('bitbucket.org/(.+)/(.+)'),
]
GITLAB_REGEXS = [
re.compile('gitlab.com/(.+)/(.+)(?:\.git){1}'),
re.compile('gitlab.com/(.+)/(.+)'),
re.compile('gitlab.com:(.+)/(.+).git'),
]
GITHUB_URL = (
'https://github.com/{user}/{repo}/'
'{action}/{version}{docroot}{path}{source_suffix}')
BITBUCKET_URL = (
'https://bitbucket.org/{user}/{repo}/'
'src/{version}{docroot}{path}{source_suffix}')
GITLAB_URL = (
'https://gitlab.com/{user}/{repo}/'
'{action}/{version}{docroot}{path}{source_suffix}')
4 changes: 2 additions & 2 deletions readthedocs/projects/views/base.py
Original file line number Diff line number Diff line change
Expand Up @@ -35,15 +35,15 @@ def get_context_data(self, **kwargs):
onboard = {}
project = self.get_object()

# TODO: gitlab

# Show for the first few builds, return last build state
if project.builds.count() <= 5:
onboard['build'] = project.get_latest_build(finished=False)
if 'github' in project.repo:
onboard['provider'] = 'github'
elif 'bitbucket' in project.repo:
onboard['provider'] = 'bitbucket'
elif 'gitlab' in project.repo:
onboard['provider'] = 'gitlab'
context['onboard'] = onboard

return context
Expand Down
12 changes: 11 additions & 1 deletion readthedocs/restapi/templates/restapi/footer.html
Original file line number Diff line number Diff line change
Expand Up @@ -93,6 +93,16 @@
<a href="{{ bitbucket_url }}">Edit</a>
</dd>
</dl>
{% elif gitlab_edit_url %}
<dl>
<dt>On GitLab</dt>
<dd>
<a href="{{ gitlab_view_url }}">View</a>
</dd>
<dd>
<a href="{{ gitlab_edit_url }}">Edit</a>
</dd>
</dl>
{% endif %}
{% endblock %}

Expand All @@ -113,7 +123,7 @@

<hr/>
{% block footer %}
Free document hosting provided by <a href="https://readthedocs.org">Read the Docs</a>.
Free document hosting provided by <a href="https://readthedocs.org">Read the Docs</a>.
{% endblock %}

{% if not new_theme %}
Expand Down