Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Attempt to finish work on #45 #68

Merged
merged 46 commits into from
Feb 24, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
46 commits
Select commit Hold shift + click to select a range
f7a6c65
add new 'replace' functionality
BurnzZ Aug 6, 2020
a7de341
update 'sub' with tests and docs to differentiate it with 'replace'
BurnzZ Aug 6, 2020
2752e99
Add new 'split' functionality
renancunha Aug 6, 2020
8cff274
Add new 'find' functionality
renancunha Aug 6, 2020
2cd633e
Merge pull request #46 from scrapinghub/feature-replace
BurnzZ Aug 7, 2020
313dd67
add new 'format' functionality
BurnzZ Aug 7, 2020
931cf33
Merge pull request #47 from scrapinghub/feature-split
renancunha Aug 7, 2020
9e78d4e
Merge pull request #49 from scrapinghub/feat-format
BurnzZ Aug 12, 2020
b4fbef0
Add new 'append' functionality
BurnzZ Apr 9, 2020
a59a0d2
Add new 'extend' functionality
BurnzZ Aug 10, 2020
ceedfa3
Merge pull request #50 from scrapinghub/feat-append-extend
BurnzZ Aug 13, 2020
9af200b
Merge branch 'master' into feature-find
renancunha Aug 14, 2020
29099de
Merge pull request #48 from scrapinghub/feature-find
renancunha Aug 17, 2020
875126a
add new 'str' functionality
renancunha Aug 24, 2020
50431fd
Merge pull request #54 from scrapinghub/feature-str
renancunha Aug 27, 2020
da9be87
flak8 linting
akshayphilar Aug 28, 2020
aa6a082
renamed str to string
akshayphilar Aug 28, 2020
f6d606d
fixing join function
akshayphilar Aug 28, 2020
da72822
fixing join function
akshayphilar Aug 28, 2020
8d6347d
Merge pull request #57 from scrapinghub/fix-join
akshayphilar Aug 28, 2020
55937a3
added urljoin pipe function
akshayphilar Aug 29, 2020
8d8c9cb
Merge pull request #58 from scrapinghub/add-urljoin
akshayphilar Aug 29, 2020
01597cd
updated setup.py
akshayphilar Aug 29, 2020
ee59e09
added license
akshayphilar Aug 29, 2020
3893c19
Merge pull request #59 from scrapinghub/publish-pypi
akshayphilar Aug 29, 2020
57e6859
Added Identity function
Sep 4, 2020
55f129a
Update shublang/shublang.py
Sep 7, 2020
dec1205
Merge pull request #61 from scrapinghub/feature-identity
Sep 7, 2020
859b0d9
adding bumpversion config
akshayphilar Sep 7, 2020
a22ba9e
Bump version: 0.2.1 → 0.2.2
akshayphilar Sep 7, 2020
668bdee
add new 'map_value' feature
renancunha Sep 15, 2020
9c1c3b4
Added support for urlparse functions
sagararora19992 Sep 25, 2020
c625b58
fix typos
renancunha Sep 29, 2020
e10b4d2
improvements on map_value unit tests
renancunha Sep 29, 2020
1ec47c0
Merge pull request #64 from scrapinghub/feature-map-value
renancunha Oct 1, 2020
4553e14
Added urlparse method to return dict & added test cases
sagararora19992 Oct 2, 2020
02edc07
introduced unidecode while doing encoding-decoding in sanitize
sagararora19992 Oct 2, 2020
672b95b
Merge pull request #63 from scrapinghub/urlparse-support
sagararora19992 Oct 5, 2020
e74d195
added test case and re-order imports
sagararora19992 Oct 5, 2020
3e145b4
Merge pull request #65 from scrapinghub/unidecode-sanitize
sagararora19992 Oct 6, 2020
338e0bd
Refactor tox.ini to ease local runs
Gallaecio Feb 24, 2021
7813175
Create separate CircleCI jobs for separate Python version
Gallaecio Feb 24, 2021
ab729a9
Fix broken references in the CircleCI config file
Gallaecio Feb 24, 2021
dbda22b
Merge remote-tracking branch 'upstream/master' into add-tox-2
Gallaecio Feb 24, 2021
a2adc31
Remove unsupported Python 3.5 from CircleCI jobs
Gallaecio Feb 24, 2021
d78c59e
Fix broken references in the CircleCI config file
Gallaecio Feb 24, 2021
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions .bumpversion.cfg
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
[bumpversion]
current_version = 0.2.2
commit = True
tag = True

[bumpversion:file:setup.py]
54 changes: 21 additions & 33 deletions .circleci/config.yml
Original file line number Diff line number Diff line change
@@ -1,47 +1,35 @@
# Python CircleCI 2.0 configuration file
#
# Check https://circleci.com/docs/2.0/language-python/ for more details
#
version: 2
jobs:
toxify:
python3.6:
docker:
- image: themattrix/tox
- image: python:3.6
steps:
- checkout
- run:
name: Run tests in supported Python versions
command: |
pip install tox tox-pyenv
pyenv local 3.5.3 3.6.0 3.7.0 3.8.0
tox
build:
- run: |
pip install tox
tox
python3.7:
docker:
- image: circleci/python:3.6.1
- image: python:3.7
steps:
- checkout
# Download and cache dependencies
- restore_cache:
keys:
- v1-dependencies-{{ checksum "requirements.txt" }}
# fallback to using the latest cache if no exact match is found
- v1-dependencies-
- run:
name: install dependencies
command: |
python3 -m venv venv
. venv/bin/activate
pip install -r requirements.txt
pip install -r requirements-dev.txt
- save_cache:
paths:
- ./venv
key: v1-dependencies-{{ checksum "requirements.txt" }}
- run: |
pip install tox
tox
python3.8:
docker:
- image: python:3.8
steps:
- checkout
- run: |
pip install tox
tox

workflows:
version: 2

shublang:
jobs:
- toxify
- build
- python3.6
- python3.7
- python3.8
26 changes: 26 additions & 0 deletions LICENSE
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
Copyright 2020 Scrapinghub

Redistribution and use in source and binary forms, with or without modification,
are permitted provided that the following conditions are met:

1. Redistributions of source code must retain the above copyright notice,
this list of conditions and the following disclaimer.

2. Redistributions in binary form must reproduce the above copyright notice,
this list of conditions and the following disclaimer in the documentation and/or
other materials provided with the distribution.

3. Neither the name of the copyright holder nor the names of its contributors
may be used to endorse or promote products derived from this software without
specific prior written permission.

THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR
ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
(INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON
ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
1 change: 1 addition & 0 deletions requirements.txt
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@ py==1.8.0
pyparsing==2.4.2
pytest==5.2.1
six==1.12.0
unidecode==0.4.20
w3lib==1.21.0
wcwidth==0.1.7
zipp==0.6.0
Expand Down
11 changes: 6 additions & 5 deletions setup.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,13 +4,13 @@

setup(
name='shublang',
version='0.1.2',
version='0.2.2',
license='BSD',
author='Akshay Philar',
author_email='[email protected]',
description='Shublang - Data Extraction DSL',
author='Akshay',
author_email='[email protected]',
url="https://github.com/scrapinghub/shublang",
packages=find_packages(exclude=exclude),
#package_data={'shublang': ['*.py']},
include_package_data=True,
entry_points={
'console_scripts': [
Expand All @@ -24,17 +24,18 @@
'Operating System :: OS Independent',
'Programming Language :: Python',
'Programming Language :: Python :: 3',
'Programming Language :: Python :: 3.5',
'Programming Language :: Python :: 3.6',
'Programming Language :: Python :: 3.7',
'Programming Language :: Python :: 3.8',
],
python_requires='>=3.6',
install_requires=[
'pipe >= 1.5.0',
'jmespath >= 0.9.4',
'w3lib >= 1.21.0',
'parsel >= 1.5.2',
'dateparser >= 0.7.2',
'price-parser >= 0.3.2',
'unidecode >= 0.4.20'
]
)
Loading