Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

"Revert" master back to 0.2 branch #68

Merged
merged 13 commits into from
Nov 30, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@ on:
push:
branches:
- master
- release-*
tags: '*'
pull_request:
jobs:
Expand Down Expand Up @@ -48,7 +49,7 @@ jobs:
- uses: actions/checkout@v2
- uses: julia-actions/setup-julia@latest
with:
version: '1'
version: '1.6'
- run: julia --project=docs -e '
using Pkg;
Pkg.develop(PackageSpec(; path=pwd()));
Expand Down
2 changes: 0 additions & 2 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,3 +1 @@
Manifest.toml
/docs/build
/docs/Manifest.toml
4 changes: 2 additions & 2 deletions Project.toml
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
name = "DataSets"
uuid = "c9661210-8a83-48f0-b833-72e62abce419"
authors = ["Chris Foster <[email protected]> and contributors"]
version = "0.2.6"
version = "0.2.10"

[deps]
AbstractTrees = "1520ce14-60c1-5f80-bbc7-55ef81b5835c"
Expand All @@ -15,7 +15,7 @@ TOML = "fa267f1f-6049-4f14-aa54-33bafae1ed76"
UUIDs = "cf7118a7-6976-5b1a-9a39-7adc72f591a4"

[compat]
AbstractTrees = "0.4"
AbstractTrees = "0.3,0.4"
ReplMaker = "0.2"
ResourceContexts = "0.1,0.2"
TOML = "1"
Expand Down
147 changes: 147 additions & 0 deletions docs/Manifest.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,147 @@
# This file is machine-generated - editing it directly is not advised

[[ArgTools]]
uuid = "0dad84c5-d112-42e6-8d28-ef12dabb789f"

[[Artifacts]]
uuid = "56f22d72-fd6d-98f1-02f0-08ddc0907c33"

[[Base64]]
uuid = "2a0f44e3-6c83-55bd-87e4-b1978d98bd5f"

[[Dates]]
deps = ["Printf"]
uuid = "ade2ca70-3891-5945-98fb-dc099432e06a"

[[DocStringExtensions]]
deps = ["LibGit2", "Markdown", "Pkg", "Test"]
git-tree-sha1 = "9d4f64f79012636741cf01133158a54b24924c32"
uuid = "ffbed154-4ef7-542d-bbb7-c09d3a79fcae"
version = "0.8.4"

[[Documenter]]
deps = ["Base64", "Dates", "DocStringExtensions", "IOCapture", "InteractiveUtils", "JSON", "LibGit2", "Logging", "Markdown", "REPL", "Test", "Unicode"]
git-tree-sha1 = "3ebb967819b284dc1e3c0422229b58a40a255649"
uuid = "e30172f5-a6a5-5a46-863b-614d45cd2de4"
version = "0.26.3"

[[Downloads]]
deps = ["ArgTools", "LibCURL", "NetworkOptions"]
uuid = "f43a241f-c20a-4ad4-852c-f6b1247861c6"

[[IOCapture]]
deps = ["Logging"]
git-tree-sha1 = "377252859f740c217b936cebcd918a44f9b53b59"
uuid = "b5f81e59-6552-4d32-b1f0-c071b021bf89"
version = "0.1.1"

[[InteractiveUtils]]
deps = ["Markdown"]
uuid = "b77e0a4c-d291-57a0-90e8-8db25a27a240"

[[JSON]]
deps = ["Dates", "Mmap", "Parsers", "Unicode"]
git-tree-sha1 = "81690084b6198a2e1da36fcfda16eeca9f9f24e4"
uuid = "682c06a0-de6a-54ab-a142-c8b1cf79cde6"
version = "0.21.1"

[[LibCURL]]
deps = ["LibCURL_jll", "MozillaCACerts_jll"]
uuid = "b27032c2-a3e7-50c8-80cd-2d36dbcbfd21"

[[LibCURL_jll]]
deps = ["Artifacts", "LibSSH2_jll", "Libdl", "MbedTLS_jll", "Zlib_jll", "nghttp2_jll"]
uuid = "deac9b47-8bc7-5906-a0fe-35ac56dc84c0"

[[LibGit2]]
deps = ["Base64", "NetworkOptions", "Printf", "SHA"]
uuid = "76f85450-5226-5b5a-8eaa-529ad045b433"

[[LibSSH2_jll]]
deps = ["Artifacts", "Libdl", "MbedTLS_jll"]
uuid = "29816b5a-b9ab-546f-933c-edad1886dfa8"

[[Libdl]]
uuid = "8f399da3-3557-5675-b5ff-fb832c97cbdb"

[[Logging]]
uuid = "56ddb016-857b-54e1-b83d-db4d58db5568"

[[Markdown]]
deps = ["Base64"]
uuid = "d6f4376e-aef5-505a-96c1-9c027394607a"

[[MbedTLS_jll]]
deps = ["Artifacts", "Libdl"]
uuid = "c8ffd9c3-330d-5841-b78e-0817d7145fa1"

[[Mmap]]
uuid = "a63ad114-7e13-5084-954f-fe012c677804"

[[MozillaCACerts_jll]]
uuid = "14a3606d-f60d-562e-9121-12d972cd8159"

[[NetworkOptions]]
uuid = "ca575930-c2e3-43a9-ace4-1e988b2c1908"

[[Parsers]]
deps = ["Dates"]
git-tree-sha1 = "c8abc88faa3f7a3950832ac5d6e690881590d6dc"
uuid = "69de0a69-1ddd-5017-9359-2bf0b02dc9f0"
version = "1.1.0"

[[Pkg]]
deps = ["Artifacts", "Dates", "Downloads", "LibGit2", "Libdl", "Logging", "Markdown", "Printf", "REPL", "Random", "SHA", "Serialization", "TOML", "Tar", "UUIDs", "p7zip_jll"]
uuid = "44cfe95a-1eb2-52ea-b672-e2afdf69b78f"

[[Printf]]
deps = ["Unicode"]
uuid = "de0858da-6303-5e67-8744-51eddeeeb8d7"

[[REPL]]
deps = ["InteractiveUtils", "Markdown", "Sockets", "Unicode"]
uuid = "3fa0cd96-eef1-5676-8a61-b3b8758bbffb"

[[Random]]
deps = ["Serialization"]
uuid = "9a3f8284-a2c9-5f02-9a11-845980a1fd5c"

[[SHA]]
uuid = "ea8e919c-243c-51af-8825-aaa63cd721ce"

[[Serialization]]
uuid = "9e88b42a-f829-5b0c-bbe9-9e923198166b"

[[Sockets]]
uuid = "6462fe0b-24de-5631-8697-dd941f90decc"

[[TOML]]
deps = ["Dates"]
uuid = "fa267f1f-6049-4f14-aa54-33bafae1ed76"

[[Tar]]
deps = ["ArgTools", "SHA"]
uuid = "a4e569a6-e804-4fa4-b0f3-eef7a1d5b13e"

[[Test]]
deps = ["InteractiveUtils", "Logging", "Random", "Serialization"]
uuid = "8dfed614-e22c-5e08-85e1-65c5234f0b40"

[[UUIDs]]
deps = ["Random", "SHA"]
uuid = "cf7118a7-6976-5b1a-9a39-7adc72f591a4"

[[Unicode]]
uuid = "4ec0a83e-493e-50e2-b9ac-8f72acf5a8f5"

[[Zlib_jll]]
deps = ["Libdl"]
uuid = "83775a58-1f1d-513f-b197-d71354ab007a"

[[nghttp2_jll]]
deps = ["Artifacts", "Libdl"]
uuid = "8e850ede-7688-5339-a07c-302acd2aaf8d"

[[p7zip_jll]]
deps = ["Artifacts", "Libdl"]
uuid = "3f19e933-33d8-53b3-aaab-bd5110c3b7a0"
3 changes: 0 additions & 3 deletions docs/Project.toml
Original file line number Diff line number Diff line change
@@ -1,5 +1,2 @@
[deps]
Documenter = "e30172f5-a6a5-5a46-863b-614d45cd2de4"

[compat]
Documenter = "0.27"
12 changes: 6 additions & 6 deletions docs/src/Data.toml
Original file line number Diff line number Diff line change
Expand Up @@ -13,11 +13,11 @@ uuid="b498f769-a7f6-4f67-8d74-40b770398f26"
[datasets.storage]
# The name of the storage driver.
driver="FileSystem"
# Data stored in FileSystem is either File (a file) or FileTree (a directory/folder)
type="File"
# Data stored in FileSystem is either Blob (a file) or BlobTree (a directory/folder)
type="Blob"
# Path with posix `/` separators.
# Relative paths are relative to the location of Data.toml
path="data/file.txt"
# Use @__DIR__ for paths relative to Data.toml
path="@__DIR__/data/file.txt"

# A second example
[[datasets]]
Expand All @@ -27,8 +27,8 @@ uuid="e7fd7080-e346-4a68-9ca9-98593a99266a"

[datasets.storage]
driver="FileSystem"
type="FileTree"
path="data/csvset"
type="BlobTree"
path="@__DIR__/data/csvset"

# Further datasets can be added as desired
# [[datasets]]
Expand Down
32 changes: 9 additions & 23 deletions docs/src/reference.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,18 +3,12 @@
## Using datasets

The primary mechanism for loading datasets is the `dataset` function, coupled
with `open()` to open the resulting `DataSet` as some Julia type.
with `open()` to open the resulting `DataSet` as some Julia type. In addition,
DataSets.jl provides two macros [`@datafunc`](@ref) and [`@datarun`](@ref) to
help in creating program entry points and running them.

```@docs
dataset
```

In addition, DataSets.jl provides two macros [`@datafunc`](@ref) and
[`@datarun`](@ref) to help in creating program entry points and running them.
Note that these APIs aren't fully formed and might be deprecated before
DataSets-1.0.

```@docs
@datafunc
@datarun
```
Expand Down Expand Up @@ -52,31 +46,23 @@ DataSets.ActiveDataProject
DataSets.TomlFileDataProject
```

### Modifying datasets

The metadata for a dataset may be updated using `config!`

```@docs
DataSets.config!
```

## Data Models for files and directories

DataSets provides some builtin data models [`File`](@ref) and
[`FileTree`](@ref) for accessin file- and directory-like data respectively. For
DataSets provides some builtin data models [`Blob`](@ref) and
[`BlobTree`](@ref) for accessin file- and directory-like data respectively. For
modifying these, the functions [`newfile`](@ref) and [`newdir`](@ref) can be
used.
used, together with `setindex!` for `BlobTree`.

```@docs
File
FileTree
Blob
BlobTree
newfile
newdir
```

## Storage Drivers

To add a new kind of data storage backend, call [`DataSets.add_storage_driver`](@ref)
To add a new kind of data storage backend, implement [`DataSets.add_storage_driver`](@ref)

```@docs
DataSets.add_storage_driver
Expand Down
Loading
Loading