-
-
Notifications
You must be signed in to change notification settings - Fork 493
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
More detailed passthrough copy options #214
Comments
It would certainly be nice to be able to structure a project by having an input directory, an output directory and at the root level, project configuration (e.g. .eleventy.js, package.json, etc.). |
Interesting request! I see the benefit there. Maybe we should add an optional second argument to
OR, it may be better to create a new method instead. Maybe I’m leaning toward the latter, if the name is good. |
This repository is now using lodash style issue management for enhancements. This means enhancement issues will now be closed instead of leaving them open. The enhancement backlog can be found here: https://github.com/11ty/eleventy/issues?utf8=%E2%9C%93&q=label%3Aneeds-votes+sort%3Areactions-%2B1-desc+ Don’t forget to upvote the top comment with 👍! |
I'm not sure I entirely follow what you intend the second option to do. Act as the current function does, and change the existing one to match my expected behaviour? or the other way around?Changing the existing method isn't backwards compatible of course. My instinct is that the current behavior of preserving directory structure before the path you've specified is not a common need. If someone wants a directory structure to appear in their output, they can recreate it from the point they've specified as their source. But to have a file copied through to the root of the destination currently requires it to exist in the root of the project. I think I'd have more of a preference for the first option, as it more closely matches what I'm familiar with - such as with gulp or grunt. Both of those frequently have a source and destination directory. This gives you flexibility about how you want to structure your files. In gulp / grunt it's often done as:
In the short term, I've bypassed this method by doing my copying with gulp rather than eleventy. |
Oh, no I wouldn’t break backwards compatibility. Your instinct may be correct there. If it is, a new method might allow us to change the main path in the docs and de-emphasize the current method in order to minimize confusion moving forward. Maybe All of my projects rely on the current method’s behavior 🤷♂️ |
I certainly agree with @edwardhorsford's description of how he would expect that to behave. I would also tend to want to have the easier control over the resultant file structure that this delivers. When it comes to the implementation options suggested, I find the first idea, where an optional parameter dictates if the paths created by the pass through are relative to the root or not, a little hard to comprehend from looking at it. While tricky to name, I think a new method with the relative path preserving behaviour, might be the most easy to comprehend down the line. Naming it though... Tricky tricky!
|
Just throwing this out there.
// copy-pass-through.json
{
"docs/*.pdf": "/assets/pdf/",
"icons/favicon.ico": "/",
"icons/*": "/icons",
"colors/*.md": false
} And along with that, from eleventyConfig.addPassthroughCopy('string'); // gets treated as it currently does
eleventyConfig.addPassthroughCopy({ // object as (src glob): (dest)
'docs/*.pdf': '/assets/pdf/',
'icons/favicon.ico': '/',
'icons/*': '/icons',
'colors/*.md': false // don't output
}) Maybe this idea can help spark some other ideas to help us get a bit closer... This one is tricky, indeed. |
I like that @jevets! |
I needed something like this so I have a PR that has about half the work done: #452 Is anyone interested in collaborating on the rest of what is mentioned here? |
Big 👍 to this feature request. My preference is to update the existing method ( const inputDir = './src';
const outputDir = './public';
module.exports = eleventyConfig => {
// copy everything in `./src/_assets` to `./public/assets`
eleventyConfig.addPassthroughCopy(`${inputDir}/_assets`, 'assets');
return {
dir: {
input: inputDir,
output: outputDir,
passthroughFileCopy: true
}
}
}; |
I've been working on this and I'm struggling with this pattern:
The problem with this is that it makes it difficult to recursively copy directories. What should be do here:
Do we flatten the directory structure and put all files, no matter how deep in the directory Otherwise we look for the start of a regex and decide this is the base? But what if I wanted to copy all files in nested CSS directories, to a base CSS folder?
If I look for the start of a regex here I'm going to get the unwanted folders in the output :( No idea how to solve. Please HALP! |
It looks like this API will be very complex; thus, it should be specified clearly. We need a set of test cases of input globs and their expected output paths that we agree on. Here are some points I think should be handled by the API.
In general, I’m not sure if we should allow globs at all there. A mapping like The core of this feature request is this: It should be able to string an input path’s prefix from the output path when passthrough-copying files. That is:
|
I can see several meanings:
What about using some placeholders?
(not fully thought through the implications, just want to bring up ideas). @edwardhorsford I believe you brought it up first here. What would be your expectation? Do we have prior art? It reminds me on |
@kleinfreund I agree with what you're saying above. The leading slash was intended to imply relativity to the root of the designated output path (per glob), but I agree it's better to avoid that ambiguity entirely. Getting back to the original issue and @zachleat's initial solution idea, though... The real goal of this feature seems more about keeping things organized locally but modifying how and where some files/folders end up when built. i.e. keep a static One solution is to use an npm package script to handle the copying, but this doesn't work well during development ( I've just dealt with it and kept my static assets right in Recently I've been thinking it could be better to offer an // .eleventy.js
module.exports = eleventyConfig => {
// this could run on --watch, --serve, and (build)
eleventyConfig.afterBuild(() => {
// fs.copyFileSync(...)
})
} This way we could handle passthrough directory mapping and anything else we may need to do. Or if the existing Transforms could somehow be used to alter the paths after the fact. (Copy Passthroughs currently don't go through Transforms.) Just an idea, thinking out loud. |
I've updated the pull request with what I think is a passable solution: #452 Basically old API works. That means if there is no output path it's resolved automatically. For both these the output would be resolved to
If it's a glob, it's just going to copy everything to the specified output directory. That means both these resolve all files to
If it's not a glob and the key resolves to a directory or file, it will use the recursive copy method and you can rename the directory just as above but keeping the source directory structure. The only limitation here is you will get all files. Eg map
|
@zachleat going to run this on my site for a while and see how it goes. Would love to get some feedback from you. Also If we did that, it would mean with this example any nested
|
I think what's being proposed is not what I would have expected - and doesn't meet my particular needs. I'd like my folder structure preserved - I think what's being proposed flattens the structure. I'd like to provide two locations. Point I always assumed FWIW all my images are in nested folders by date. This keeps them nicely grouped - but also means I don't need to worry about file name collisions. It's also super helpful as I use cloudinary as my imageCDN - and my assets locations there mirror my local assets. |
@edwardhorsford You can still copy folders just not with globs. So if you want the entire js directory you'd just go: config.addPassthroughCopy({"src/site/static/js/": "/js"}); This works for a lot of use-cases. The one that is tricky is this... imagine the following folders all contain
And you want to copy all the It's easy to match files, but the tricky part in our situation is what do you want to remove? With all other cases we're assuming you want to remove everything before the matched filename, e.g: config.addPassthroughCopy({"src/site/static/js/": "/js"});
// Removes: src/site/static/js/ from matched paths and copies to /js
config.addPassthroughCopy({"src/site/static/js/file.js": "/my-script/"});
// Also removes: src/site/static/js/ from matched paths and copies to /js What do we do for globs? At the moment we are doing the same. Is it as simple as stripping everything before |
config.addPassthroughCopy({"/static/**/js" : "/js"});
// Remove: /static/ from matched paths and copy to /js ???
config.addPassthroughCopy({"/static/**/js/*.js" : "/js"});
// Also remove: /static/ from matched paths and copy to /js ???
config.addPassthroughCopy({"/static/**/components/**/js/*.js" : "/js"});
// Also remove: /static/ from matched paths and copy to /js ??? I don't know if the above is what people expect? Maybe it is as simple as removing everything before the first |
Okay to remove this one from the enhancement queue because the PR was merged? Or are there additional things documented left open here? |
There is a suggested change for how remap paths with globs containing |
@MadeByMike is this a globstar thing? https://github.com/mrmlnc/fast-glob#globstar |
@zachleat the globstar option is about how we want it to match files. I assume the default of The extra discussed is about what we should do with the copy after matching a glob. The question is, given the directory structure: @zachleat What do you think the users expectation is? Also what about this one... Given the directory structure: |
I think any time you’re changing the output directory, it should flatten the matches to the output directory. If they want to maintain directory structure, they should use the prior behavior with a An example: assuming input directory of // Pick one
eleventyConfig.addPassthroughCopy("src/**/file.txt"); // copy to _site/nested/file.txt
eleventyConfig.addPassthroughCopy("src/**/file.txt": "dest"); // copy to _site/dest/file.txt
eleventyConfig.addPassthroughCopy("src/**/file.txt": "dest/nested"); // copy to _site/dest/nested/file.txt I think there are plenty of options here to make it flexible enough—unless anyone has any other complaints here. I don’t think we need to get too elaborate. |
@MadeByMike I am using npm glob for recursive file search then fs.mkdir(path, { recursive: true }) and fs.copyFile to do the trick. See: #379 (comment) Also when using passthrough, make sure that it works with permalink settings too! |
Would be really nice being able to specify Relative paths are actually still quite broken. For an example, EleventyRenderPlugin for some reason completely ignores 11ty's input directory path and takes everything relative to where you run node. Oh wait, addPassthroughCopy does that too. Or is that intended to be this way? upd: Is it just me skipping over entire blocks of docs, or was this actually added recently? All good now 😂 |
I'm trying to use manual passthrough copy to copy some files over to my dist directory - but the outcome is not what I expected. I don't know if this is intentional, but it seems odd.
I've got a folder of files I'd like to be copied. They're at
src/copy-pass-through/
. I added them to the eleventy config witheleventyConfig.addPassthroughCopy("./src/copy-pass-through");
They get copied to
dist
, but the starting folder structure is preserved. So I getdist/src/copy-pass-through
I would expect the files to be copied relative to the starting location. So a file located in
src/copy-pass-through/images/foo.jpg
would get copied todist/images/foo.jpg
.The current behavior seems odd to me - if my files are in a folder, I'm forced to have that appear in dist. This doesn't support my own organisation. It means if I want a file to appear in the root of
dist
I'm forced to have it in the root of the project.The text was updated successfully, but these errors were encountered: