!data/01_raw/* should not be in the default .gitignore

[grofte]

If I make a new Kedro project the lines

# keep also the example dataset
!data/01_raw/*

are in .gitignore file. At the same time you have written in the README that users should not delete any of the lines you have added to .gitignore. I feel like this is going to hurt a lot of newbies that accidentally add a big dataset to git and then struggle to get rid of it. Even for the example with spaceflights I think it's a terrible idea to add the data to git.

PS I have not installed the example.

PPS I could not find it in your codebase or I would have written a PR. At the same time it's not a bug, just a terrible choice.

[noklam]

The starters are not in kedro template, it is cloned via cookiecutter and has its own repository: https://github.com/kedro-org/kedro-starters/tree/main/spaceflights-pandas

[DimedS]

I agree, thank you @grofte , better to fix that

[datajoely]

Why don't we explicitly allowlist the files needed in 01_raw for the starter and remove the wildcard?

[grofte]

But you already have a bunch more files when you init the example compared to the just a plain init. Can't you have the !data/01_raw/specific_file only occur in a .gitignore that comes with the example? And then a different .gitignore for the plain init?

[grofte]

Why not put the data in a public cloud bucket and then have the pipeline pull it from there? Isn't one of the selling points of the Kedro catalog that it's easy to do something like that? And then you get to demo it in the tutorial. There's even three files so you could host them on AWS, GCP, and Azure.

[grofte]

Can you load from an URL? Because then you could put it in a Gist. The CSVs at least.