Petl use fsspec (#494)

* use fsspec for handling remote protocols

use fsspec for handling remote protocols
Add RemoteSource for handling all remote sources with fsspec
Remove S3Source replaced by fsspec
Keep SMBSource as it isn't handled by fsspec
Refactored namespaces to remotes

* use compression from fsspec

remove codec implementation in petl

* update docs with changes for fsspec

* added another test case for fsspec

* make appveyor happy

* Apply suggestions from code review of #494

Ready to merge.

Co-authored-by: Alistair Miles <[email protected]>

Co-authored-by: Juarez Rudsatz <[email protected]>
Co-authored-by: Juarez Rudsatz <[email protected]>
Co-authored-by: Alistair Miles <[email protected]>

docs/changes.rst

@@ -1,6 +1,22 @@
 Changes
 =======
 
+Version 1.6.0
+-------------
+
+* Added class :class:`petl.io.remotes.RemoteSource` using package **fsspec**
+  for reading and writing files in remote servers by using the protocol in the
+  url for selecting the implementation.
+  By :user:`juarezr`, :issue:`494`.
+
+* Removed classes :class:`petl.io.source.s3.S3Source` as it's handled by fsspec
+  By :user:`juarezr`, :issue:`494`.
+
+* Removed classes :class:`petl.io.codec.xz.XZCodec`, :class:`petl.io.codec.xz.LZ4Codec`
+  and :class:`petl.io.codec.zstd.ZstandardCodec` as it's handled by fsspec.
+  By :user:`juarezr`, :issue:`494`.
+
+
 Version 1.5.0
 -------------
 

docs/io.rst

@@ -359,13 +359,8 @@ The behaviour of each source can usually be configured by passing arguments
 to the constructor, see the source code of the :mod:`petl.io.sources` module
 for full details.
 
-.. autoclass:: petl.io.sources.FileSource
-.. autoclass:: petl.io.sources.GzipSource
-.. autoclass:: petl.io.sources.BZ2Source
-.. autoclass:: petl.io.sources.ZipSource
 .. autoclass:: petl.io.sources.StdinSource
 .. autoclass:: petl.io.sources.StdoutSource
-.. autoclass:: petl.io.sources.URLSource
 .. autoclass:: petl.io.sources.MemorySource
 .. autoclass:: petl.io.sources.PopenSource
 
@@ -383,38 +378,34 @@ in :ref:`Extract <io_extract>` and :ref:`Load <io_load>`.
 It's possible to read and write just by prefixing the protocol (e.g: `s3://`)
 in the source path of the file.
 
-.. autoclass:: petl.io.source.s3.S3Source
-.. autoclass:: petl.io.source.smb.SMBSource
+.. autoclass:: petl.io.remotes.RemoteSource
+.. autoclass:: petl.io.remotes.SMBSource
 
-.. _io_codecs:
+.. _io_deprecated:
 
-Compression I/O helper classes
-------------------------------
+Deprecated I/O helper classes
+-----------------------------
 
-The following classes are helpers for decompressing (``from...()``) and 
-compressing (``to...()``) in functions transparently as a file-like source.
+The following helpers are deprecated and will be removed in a future version.
 
-There are no need to instantiate them. They are used in the mecanism described
-in :ref:`Extract <io_extract_codec>` and :ref:`Load <io_load_codec>`.
+It's functionality was replaced by helpers in :ref:`Remote helpers <io_remotes>`.
 
-It's possible to compress and decompress just by specifying the file extension
-(e.g: `.csv.xz`) in end of the source filename.
-
-.. autoclass:: petl.io.codec.xz.XZCodec
-.. autoclass:: petl.io.codec.zstd.ZstandardCodec
-.. autoclass:: petl.io.codec.lz4.LZ4Codec
+.. autoclass:: petl.io.sources.FileSource
+.. autoclass:: petl.io.sources.GzipSource
+.. autoclass:: petl.io.sources.BZ2Source
+.. autoclass:: petl.io.sources.ZipSource
+.. autoclass:: petl.io.sources.URLSource
 
 .. _io_custom_helpers:
 
 Custom I/O helper classes
 ------------------------------
 
 For creating custom helpers for :ref:`remote I/O <io_remotes>` or
-:ref:`compression <io_codecs>` use the following functions:
+`compression` use the following functions:
 
 .. autofunction:: petl.io.sources.register_reader
 .. autofunction:: petl.io.sources.register_writer
-.. autofunction:: petl.io.sources.register_codec
 
 See the source code of the classes in :mod:`petl.io.sources` module for
 more details.

petl/io/__init__.py

@@ -39,3 +39,7 @@
 from petl.io.avro import fromavro, toavro, appendavro
 
 from petl.io.sources import register_codec, register_reader, register_writer
+
+from petl.io.remotes import RemoteSource
+
+from petl.io.remotes import SMBSource