Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add OAuth2 to GoogleDrive connector #2

Merged
merged 27 commits into from
Sep 29, 2020
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
553ae03
Initialized new Git repository: googledrive
Oct 22, 2019
dd35a7d
Initial commit
alexbourret Nov 7, 2019
8bd38e4
Adding README and implementing plugin dev kit's suggestions
alexbourret Nov 8, 2019
c59b5c5
Merge branch 'master' of github.com:dataiku/dss-plugin-googledrive
alexbourret Nov 8, 2019
1c5f582
Merge branch 'master' into feature/dss60-fs-to-googledrive
alexbourret Nov 8, 2019
56a6f2b
'Last modified' added to folder view
alexbourret Nov 14, 2019
2caa437
Cleaning + fix browsing to parent directory in folder mode
alexbourret Nov 15, 2019
6ea0ab2
NoDir mode removed for now
alexbourret Nov 15, 2019
7c989c9
Checking constraints on file paths, using password field instead of t…
alexbourret Nov 15, 2019
3266326
Cleaning
alexbourret Nov 15, 2019
9eefb61
misc improvements
JoachimZ Nov 17, 2019
1690dd5
Fix broken python3 import
alexbourret Nov 18, 2019
f545c4e
Minimalist readme
alexbourret Nov 18, 2019
b7af780
Merging
alexbourret Nov 18, 2019
83d55b2
mall tweaks
JoachimZ Nov 20, 2019
dbb786e
Add OAuth2
alexbourret Feb 7, 2020
fbc2aed
Merge master
alexbourret Feb 7, 2020
1ba236b
Merge branch 'master' into feature/dss70-oauth-for-googledrive
alexbourret Feb 7, 2020
f54b412
Fix link to online doc, plugin version in Makefile
alexbourret Feb 7, 2020
c280a55
Skeleton for unit tests
alexbourret Feb 14, 2020
13a80ad
Refactored
alexbourret Feb 17, 2020
54aef69
Cleaning
alexbourret Feb 26, 2020
ed9edda
Fix for CVE-2020-11078
alexbourret May 20, 2020
f287ddb
Implementing comments
alexbourret Sep 18, 2020
92d0072
Fix for google documents
alexbourret Sep 21, 2020
3adb301
Fix for ImportError messages
alexbourret Sep 21, 2020
8da37c7
Merge branch 'master' into feature/dss70-oauth-for-googledrive
alexbourret Sep 29, 2020
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
236 changes: 229 additions & 7 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,11 +1,233 @@
.journal
.mainlock
# DSS specific stuff
dist/
.wlock
.ts
.git
*.pyc

# Byte-compiled / optimized / DLL files
__pycache__/
*.py[cod]
*$py.class

# C extensions
*.so

# Distribution / packaging
.Python
build/
develop-eggs/
dist/
downloads/
eggs/
.eggs/
lib/
lib64/
parts/
sdist/
var/
wheels/
pip-wheel-metadata/
share/python-wheels/
*.egg-info/
.installed.cfg
*.egg
MANIFEST
.DS_Store

# PyInstaller
# Usually these files are written by a python script from a template
# before PyInstaller builds the exe, so as to inject date/other infos into it.
*.manifest
*.spec

# Installer logs
pip-log.txt
pip-delete-this-directory.txt

# Unit test / coverage reports
htmlcov/
.tox/
.nox/
.coverage
.htmlcov
.coverage.*
.cache
nosetests.xml
coverage.xml
unit.xml
*.cover
*.py,cover
.hypothesis/
.pytest_cache/
cover/

# Translations
*.mo
*.pot

# Django stuff:
*.log
local_settings.py
db.sqlite3
db.sqlite3-journal

# Flask stuff:
instance/
.webassets-cache

# Scrapy stuff:
.scrapy

# Sphinx documentation
docs/_build/

# PyBuilder
target/

# Jupyter Notebook
.ipynb_checkpoints

# IPython
profile_default/
ipython_config.py

# pyenv
# For a library or package, you might want to ignore these files since the code is
# intended to run in multiple environments; otherwise, check them in:
.python-version

# pipenv
# According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control.
# However, in case of collaboration, if having platform-specific dependencies or dependencies
# having no cross-platform support, pipenv may install dependencies that don't work, or not
# install all needed dependencies.
Pipfile.lock

# PEP 582; used by e.g. github.com/David-OConnor/pyflow
__pypackages__/

# Celery stuff
celerybeat-schedule
celerybeat.pid

# SageMath parsed files
*.sage.py

# Environments
.env
.venv
env/
venv/
ENV/
env.bak/
venv.bak/

# Spyder project settings
.spyderproject
.spyproject

# Rope project settings
.ropeproject

# mkdocs documentation
/site

# mypy
.mypy_cache/
.dmypy.json
dmypy.json

# Pyre type checker
.pyre/

# pytype static type analyzer
.pytype/

# mac stuff
.DS_Store
.idea
dist/

# History files
.Rhistory
.Rapp.history

# Session Data files
.RData

# User-specific files
.Ruserdata

# Example code in package build process
*-Ex.R

# Output files from R CMD build
/*.tar.gz

# Output files from R CMD check
/*.Rcheck/

# RStudio files
.Rproj.user/

# produced vignettes
vignettes/*.html
vignettes/*.pdf

# OAuth2 token, see https://github.com/hadley/httr/releases/tag/v0.3
.httr-oauth

# knitr and R markdown default cache directories
*_cache/
/cache/

# Temporary files created by R markdown
*.utf8.md
*.knit.md

# R Environment Variables
.Renviron

# pkgdown site
docs/

# translation temp files
po/*~

# Compiled class file
*.class

# Log file
*.log

# BlueJ files
*.ctxt

# Mobile Tools for Java (J2ME)
.mtj.tmp/

# Package Files #
*.jar
*.war
*.nar
*.ear
*.zip
*.tar.gz
*.rar

# virtual machine crash logs, see http://www.java.com/en/download/help/error_hotspot.xml
hs_err_pid*

.gradle
**/build/
!src/**/build/

# Ignore Gradle GUI config
gradle-app.setting

# Avoid ignoring Gradle wrapper jar file (.jar files are usually ignored)
!gradle-wrapper.jar

# Cache of project
.gradletasknamecache

# # Work around https://youtrack.jetbrains.com/issue/IDEA-116898
# gradle/wrapper/gradle-wrapper.properties
setup.cfg
.vscode
47 changes: 42 additions & 5 deletions Makefile
Original file line number Diff line number Diff line change
@@ -1,8 +1,45 @@
PLUGIN_VERSION=1.0.1
PLUGIN_ID=googledrive
# Public variable to be set by the user in the Makefile
TARGET_DSS_VERSION=8.0

# evaluate additional variable
plugin_id=`cat plugin.json | python -c "import sys, json; print(str(json.load(sys.stdin)['id']).replace('/',''))"`
plugin_version=`cat plugin.json | python -c "import sys, json; print(str(json.load(sys.stdin)['version']).replace('/',''))"`
archive_file_name="dss-plugin-${plugin_id}-${plugin_version}.zip"
remote_url=`git config --get remote.origin.url`
last_commit_id=`git rev-parse HEAD`


plugin:
cat plugin.json|json_pp > /dev/null
@echo "[START] Archiving plugin to dist/ folder..."
@cat plugin.json | json_pp > /dev/null
@rm -rf dist
@mkdir dist
@echo "{\"remote_url\":\"${remote_url}\",\"last_commit_id\":\"${last_commit_id}\"}" > release_info.json
@git archive -v -9 --format zip -o dist/${archive_file_name} HEAD
@zip -u dist/${archive_file_name} release_info.json
@rm release_info.json
@echo "[SUCCESS] Archiving plugin to dist/ folder: Done!"

unit-tests:
@echo "[START] Running unit tests..."
@( \
python3 -m venv env/; \
source env/bin/activate; \
pip3 install --upgrade pip; \
pip install --no-cache-dir -r tests/python/requirements.txt; \
pip install --no-cache-dir -r code-env/python/spec/requirements.txt; \
export PYTHONPATH="$(PYTHONPATH):$(PWD)/python-lib"; \
pytest -o junit_family=xunit2 --junitxml=unit.xml tests/python/unit || true; \
deactivate; \
)
@echo "[SUCCESS] Running unit tests: Done!"

integration-tests:
@echo "[START] Running integration tests..."
# TODO add integration tests
@echo "[SUCCESS] Running integration tests: Done!"

tests: unit-tests integration-tests

dist-clean:
rm -rf dist
mkdir dist
zip --exclude "*.pyc" -r dist/dss-plugin-${PLUGIN_ID}-${PLUGIN_VERSION}.zip plugin.json python-lib custom-recipes python-fs-providers parameter-sets code-env
14 changes: 4 additions & 10 deletions code-env/python/spec/requirements.txt
Original file line number Diff line number Diff line change
@@ -1,11 +1,5 @@
cachetools==3.1.1
google-api-python-client==1.7.11
google-auth==1.6.3
google-auth-httplib2==0.0.3
httplib2==0.14.0
google-api-python-client==1.12.1
google-auth==1.21.2
google-auth-httplib2==0.0.4
httplib2==0.18.0
oauth2client==4.1.3
pyasn1==0.4.7
pyasn1-modules==0.2.7
rsa==4.0
six==1.12.0
uritemplate==3.0.0
4 changes: 2 additions & 2 deletions parameter-sets/googledrive-set-id/parameter-set.json
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
{
"meta" : {
"label": "Google Drive connection",
"label": "Google Drive Token",
"description": "",
"icon": "icon-file-alt"
"icon": "icon-file"
},
"defaultDefinableInline": true,
"defaultDefinableAtProjectLevel": true,
Expand Down
24 changes: 24 additions & 0 deletions parameter-sets/oauth-credentials/parameter-set.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
{
"meta": {
"label": "Google Single Sign On",
"description": "",
"icon": "icon-file"
},
"defaultDefinableInline": true,
"defaultDefinableAtProjectLevel": true,
"pluginParams": [
],
"params": [
{
"name": "access_token",
"type": "CREDENTIAL_REQUEST",
"credentialRequestSettings": {
"type": "OAUTH2",
"oauth2Flow": "authorization_code",
"authorizationEndpoint": "https://accounts.google.com/o/oauth2/v2/auth",
"tokenEndpoint": "https://oauth2.googleapis.com/token",
"scope": "https://www.googleapis.com/auth/drive"
}
}
]
}
6 changes: 3 additions & 3 deletions plugin.json
Original file line number Diff line number Diff line change
@@ -1,14 +1,14 @@
{
"id": "googledrive",
"version": "1.0.1",
"version": "1.1.0",
"meta": {
"label": "Google Drive",
"description": "Read and write data from/to your Google Drive account",
"author": "Dataiku (Alex Bourret)",
"icon": "icon-file-alt",
"icon": "icon-file",
"tags": [ "Connector", "Google", "Cloud"],
"url": "https://www.dataiku.com/product/plugins/google-drive/",
"licenseInfo": "Apache Software License",
"supportLevel": "NOT_SUPPORTED"
alexcombessie marked this conversation as resolved.
Show resolved Hide resolved
}
}
}
Loading