caddy: add all addons #14671
Comments
|
This does sound like a good idea to improve the usefulness of caddy, but it's quite non-trivial from what I can see. It looks like there's some code generation required (in https://github.com/caddyserver/caddydev/blob/master/caddybuild/build.go) to insert the custom directives into caddy. If that's the case, it would probably be easier to start with desired addons and do the code generation at build time. I expect this would need a bit more Nix code than currently exists for the package in order to support hashes for caddy and all addons, then modifying and building the source after this. I'm not great with the language just yet, so I'm not even sure of the feasibility of this. (Anything's possible, right?) |
|
I found a Dockerfile where someone installed caddy with the git addon. That is also the one i want to use. https://github.com/jumanjihouse/docker-caddy/blob/master/Dockerfile.build#L34 You could write the nix-expression like for a general program, but the go infrastructure is different. |
|
you can override build phases like described here https://nixos.org/nixpkgs/manual/#ssec-controlling-phases for example the preBuildPhases like here: nixpkgs/pkgs/top-level/go-packages.nix Line 642 in dfc6d42 then you can use if you are experienced with go, you can do that. i could also figure it out and create a PR. how you like :) |
|
we can also create an additional package of caddyext and use it as caddy buildInputs. but i see 2 problems:
|
|
If the source of the addons changes over time, I would try to convince upstream to change their ways and otherwise completely ignore this project, for a lack of professional software development practices. Sometimes "gratis" software is even too costly. |
|
@0xABAB you may misunderstood me. caddy has it's own "package manager" for addons. if i execute i asked the developer of caddyext if it can install from local directory. then we can clone the addons repository and checkout a specific version. Caddy is free software. |
|
in version 0.9 addons will be plugins. i havn't looked what that means for building from source. https://forum.caddyserver.com/t/caddy-0-9-beta-version-available/146?u=matt we should wait for 0.9 to integrate the fix for this issue into that. |
|
Any updates on this? |
|
It is now quite easy to add plugins in caddy: https://github.com/mholt/caddy/wiki/Plugging-in-Plugins-Yourself |
|
I managed to add the preConfigure = ''
substituteInPlace ./caddy/caddymain/run.go \
--replace '// This is where other plugins get plugged in (imported)' \
'_ "github.com/restic/caddy"'
'';
goDeps = ./deps.nix;And adding [
{
goPackagePath = "github.com/restic/caddy";
fetch = {
type = "git";
url = "https://github.com/restic/caddy";
rev = "v0.2.0";
sha256 = "0wjhbnm405vdpf3jwi9dzhz6nd5rhmxqibsa1nx2iyq43bc3p6sk";
};
}
{
goPackagePath = "github.com/restic/rest-server";
fetch = {
type = "git";
url = "https://github.com/restic/rest-server";
rev = "v0.9.3";
sha256 = "0f6i952iy4arnc96wpayvh9wa1mdw7vprwwbnjxwj09jvifqd0hp";
};
}
]So it is now really easy to add some plugins to
|
|
I think that 2 versions would be OK. Another option would be to enable each plugin like described for Chromium and Firefox here https://nixos.wiki/wiki/Chromium |
|
The above method for me isn't working. Has anybody had any luck or got a better approach? |
|
According to the https://github.com/caddyserver/caddy/wiki/Plugging-in-Plugins-Yourself The current derivation is at: https://github.com/NixOS/nixpkgs/blob/master/pkgs/servers/caddy/default.nix All we need to do is is something like: { caddy }:
caddy.overrideAttrs (attrs: {
preBuild = ''
cat << EOF > caddy/main.go
package main
import "github.com/caddyserver/caddy/caddy/caddymain"
import _ "github.com/techknowlogick/caddy-s3browser"
func main() {
caddymain.EnableTelemetry = false
caddymain.Run()
}
EOF
'';
})Which I override the You can see that I added in: However that dependency is not being brought into the Normally with |
|
There's a Instead it seems like it is needed to override the No documentation on how to override the |
|
I think what needs to be done is that the Because that's when Either that or the This does not work even if I update The |
|
Abandoned trying to do overrides. I just had to create my own Go package to do this. But in the future, caddy needs some work to be able to compile plugins more easily like how firefox/chrome does its plugins. |
|
@CMCDragonkai Hey there! I couldn't get overrides to work either, but I ended up with this and thought I'd share. I ran into some issues with nix caching old versions of the module after tweaking the derivation but leaving with pkgs; buildGoModule rec {
pname = "caddy";
version = "1.0.3";
goPackagePath = "github.com/caddyserver/caddy";
subPackages = [ "caddy" ];
src = fetchFromGitHub {
owner = "caddyserver";
repo = pname;
rev = "v${version}";
sha256 = "1n7i9w4vva5x5wry7gzkyfylk39x40ykv7ypf1ca3zbbk7w5x6mw";
};
modSha256 = "1xs5qf89sddj21vig4lmhdfkyccf2alrd39915spz845xhv3pl8w";
overrideModAttrs = (old: {
preBuild = ''
go mod edit -require=github.com/tarent/[email protected]
go mod edit -require=github.com/BTBurke/[email protected]+incompatible
'';
});
preBuild = ''
cat << EOF > caddy/main.go
package main
import (
"github.com/caddyserver/caddy/caddy/caddymain"
_ "github.com/BTBurke/caddy-jwt"
_ "github.com/tarent/loginsrv/caddy"
)
func main() {
caddymain.EnableTelemetry = false
caddymain.Run()
}
EOF
'';
meta = with stdenv.lib; {
homepage = https://caddyserver.com;
description = "Fast, cross-platform HTTP/2 web server with automatic HTTPS";
license = licenses.asl20;
maintainers = with maintainers; [ rushmorem fpletz zimbatm ];
};
} |
|
Hello, I'm a bot and I thank you in the name of the community for opening this issue. To help our human contributors focus on the most-relevant reports, I check up on old issues to see if they're still relevant. This issue has had no activity for 180 days, and so I marked it as stale, but you can rest assured it will never be closed by a non-human. The community would appreciate your effort in checking if the issue is still valid. If it isn't, please close it. If the issue persists, and you'd like to remove the stale label, you simply need to leave a comment. Your comment can be as simple as "still important to me". If you'd like it to get more attention, you can ask for help by searching for maintainers and people that previously touched related code and @ mention them in a comment. You can use Git blame or GitHub's web interface on the relevant files to find them. Lastly, you can always ask for help at our Discourse Forum or at #nixos' IRC channel. |
stale
bot
added
the
2.status: stale
|
this is still important |
stale
bot
removed
the
2.status: stale
|
I marked this as stale due to inactivity. → More info |
stale
bot
added
the
2.status: stale
|
this is still important |
|
Here's my take based on #86349 (comment):
final: prev: {
caddy = (prev.caddy.override {
buildGoModule = args: prev.buildGoModule (args // {
vendorSha256 = "sha256-445MYvi487ls6g6i30UTTK2/n2wbILgJEuwNUQE//ZE";
patches = [ ./0001-plugins.patch ];
});
});
}
diff --git a/cmd/caddy/main.go b/cmd/caddy/main.go
index 2383546..59e467e 100644
--- a/cmd/caddy/main.go
+++ b/cmd/caddy/main.go
@@ -31,6 +31,7 @@ import (
// plug in Caddy modules here
_ "github.com/caddyserver/caddy/v2/modules/standard"
+ _ "github.com/caddy-dns/duckdns"
)
func main() {
diff --git a/go.mod b/go.mod
index 6fe6902..68bfedd 100644
--- a/go.mod
+++ b/go.mod
@@ -6,6 +6,7 @@ require (
github.com/Masterminds/sprig/v3 v3.1.0
github.com/alecthomas/chroma v0.8.2
github.com/aryann/difflib v0.0.0-20170710044230-e206f873d14a
+ github.com/caddy-dns/duckdns v0.3.1
github.com/caddyserver/certmagic v0.13.1
github.com/dustin/go-humanize v1.0.1-0.20200219035652-afde56e7acac
github.com/go-chi/chi v4.1.2+incompatible
Not exactly pretty, but at least it works: |
stale
bot
removed
the
2.status: stale
|
I find #89268 (comment) works better than previous comment's because it is more caddy-version-agnostic. |
|
Just in case, somebody else who isn't too familiar with nix golang infrastructure is trying to build a current Caddy with plugins: I've updated the package file by @diamondburned in #89268 (comment). Uses fetchFromGitHub instead of fetchGit and declares the current maintainer. I also needed to set "runVend = true" to get all plugin dependencies pkgs/caddy.nix{ lib, fetchFromGitHub, buildGoModule, plugins ? [], vendorSha256 ? "" }:
with lib;
let imports = flip concatMapStrings plugins (pkg: "\t\t\t_ \"${pkg}\"\n");
main = ''
package main
import (
caddycmd "github.com/caddyserver/caddy/v2/cmd"
_ "github.com/caddyserver/caddy/v2/modules/standard"
${imports}
)
func main() {
caddycmd.Main()
}
'';
in buildGoModule rec {
pname = "caddy";
version = "2.4.6";
runVend = true;
subPackages = [ "cmd/caddy" ];
src = fetchFromGitHub {
owner = "caddyserver";
repo = "caddy";
rev = "v${version}";
sha256 = "sha256-xNCxzoNpXkj8WF9+kYJfO18ux8/OhxygkGjA49+Q4vY=";
};
inherit vendorSha256;
overrideModAttrs = (_: {
preBuild = "echo '${main}' > cmd/caddy/main.go";
postInstall = "cp go.sum go.mod $out/ && ls $out/";
});
postPatch = ''
echo '${main}' > cmd/caddy/main.go
cat cmd/caddy/main.go
'';
postConfigure = ''
cp vendor/go.sum ./
cp vendor/go.mod ./
'';
meta = {
homepage = https://caddyserver.com;
description = "Fast, cross-platform HTTP/2 web server with automatic HTTPS";
license = licenses.asl20;
maintainers = with maintainers; [ Br1ght0ne ];
};
}example.nixpkgs.callPackage ./pkgs/caddy.nix {
plugins = [
"github.com/mholt/caddy-webdav"
"github.com/greenpau/caddy-git"
"github.com/greenpau/caddy-auth-portal"
"github.com/greenpau/caddy-authorize"
];
vendorSha256 = "sha256-K2i9MlrR8PMDHHW7V+d5mRgtgb0H5okMz/TlEncvgIo=";
} |
|
@phaer So should I add this to my list of overlays? Also, how would I go about getting the |
|
how do i use plugin? |
|
@MROvaiz If you’ve never used overlays before it goes like this:
final: prev:
{
# Updated Caddy with support for including extra modules
#
# Source: https://github.com/NixOS/nixpkgs/pull/259275
caddy-custom = prev.callPackage ./caddy-custom {};
}
{
# … other settings …
# System package customization
nixpkgs.overlays = [
(import ./overlay)
];
# … other settings …
}
{
services.caddy = {
package = pkgs.caddy-custom.override {
externalPlugins = [
{name = "transform-encoder"; repo = "github.com/caddyserver/transform-encoder"; version = "58ebafa572d531b301fdbc6e2fd139766bac7e8d";}
{name = "connegmatcher"; repo = "github.com/mpilhlt/caddy-conneg"; version = "v0.1.4";}
] ++ (
# Caddy Layer4 modules
lib.lists.map (name: {
inherit name;
repo = "github.com/mholt/caddy-l4";
version = "f3a880d4c01c884f4a096ccceb6c6d1e2d1d983d";
}) ["layer4" "modules/l4proxy" "modules/l4tls" "modules/l4proxyprotocol"]
);
vendorHash = "sha256-7cRI65foALEsfYhvdGresq7oma/cIsnVtbq+Gan5DCU=";
};
# … other caddy settings …
};
}Hope this helps! |
|
@ntninja Thank you, it's working. |
|
PR @ #317881 |
|
@ntninja I've tried this, but it's not working for me. I recreated the problem in a flake for easier debugging: https://github.com/crabdancing/nixos-caddy-with-plugins It says: Does anyone have any ideas? |
Now it should work. Again this is a hack. Edit: this does not actually work if you use it in an overlay, but crabdancing made a nice flake (see comment below) which I'm using in my setup. If you are coming from Google, Ignore my response above and head over to her flake, or wait for the PR to be merged 👍 |
|
@airone01 Thanks so much! I'm very happy with my packaging for this now. :) It may be a hack, but it's the least hacky hack that we have for now. For anyone struggling with using caddy-l4 or other Caddy modules / Caddy plugins declaratively on NixOS, I wrote a flake you can use for it here. I hope it can help someone :) |
@crabdancing It’s probably that some module you’re using isn’t compatible with the older version of Caddy that was used in my guide, try repeating after doing the steps above for Caddy 2.8.4! |
|
(Also, I’ll try doing a write-up on how I think plugin support could be done in a NixPkgs-acceptable way later on. Just know I probably won’t have time to implement it myself in any reasonable timeframe.) |
|
It would be neat if we could use |
Is it possible? Considering the modules are used at compile time I mean. I'm a newbie so correct me if I'm wrong. |
|
@airone01 Yes, it should be possible. Unfortunately, it's nontrivial to even explain the problem and how it can be solved (very in-depth Nix). But since there is obviously interest, I’ll try sketching it out soon (can’t right now, we don’t have much time per day, sorry) – someone else will need to fumble with Nix derivations to actually make it work though. |
Background on derivationsBefore trying to fix this it is important to understand the two major kinds of derivations in Nix: In general derivations take some kind of input and run some command (typically a shell script) that produces some desired output. If you’ve ever written a derivation before, it was most-likely a so-called fixed-input derivation: You can specify/use some arbitrary inputs and some (shell) command(s); once all the required inputs are available the Nix program will then run the specified command(s) are run it in an isolated build environment with access only to the used inputs (and, in particular no network access!) – the so-called “sandbox” you might have seen mentioned. The idea here is for fixed-input derivations we expect the command(s) basically just be some arbitrary-complex transformation from the given inputs to whatever output the command(s) result in. Notably we do not need to specify what the output actually will be since we know (within some minor margin of error) that it will essentially just be some alternate representation of the input. The other major type of derivation is the so-called fixed-output derivation: Again you specify/use some arbitrary inputs and some (shell) command(s), but this time you also specify the hash of the resulting output. Unlike fixed-input derivations, fixed-output derivation are not run in a fully isolated environment and can access the network – there is no need to limit their inputs after all since we know there is only one possible output in may produce! Notably the The currently suggested approach for Caddy pluginsIf you look at the current suggestion for external modules in Caddy, you can see that the However this brings us to the actual problem in terms of NixPkgs packaging: All we did is add some references to these external plugins, we have not actually made their source code available locally at all! Since building Caddy itself uses a fixed-input derivation (see above) we’d now get errors as the Go tool attempts to download these dependencies from the internet – something a fixed-input derivation is not allowed to do. The reason why the approach still kinda works is because we haven’t just patched the There is a major inconvenience with this however: Since it’s a fixed-output derivation we had to specify the expected output hash of this step and this hash, the A potential solutionBasically, what we need to do is treat each Caddy plugin as its own separate Go project that we run It should be possible to do that last step as part of So in to total it would be something like this:
… maybe, there are probably several catches and pitfalls. You’ll need to experiment to actually get it to work. (You can also look at the Good luck! (@airone01 why 😄 ?) |
|
@ntninja I thought it was a "happy" emoji but it's a "laugh" emoji. My bad Also great explanation, it's really detailed and simple and really helps a Nix newbie like me to understand how it works behind the scene. Thanks for that! |
|
@ntninja Great explanation! Thanks, I've written many derivations before, but now it all makes sense to me. |
|
@airone01 Ah, I see 😁 @diamondburned Yes, it does. I just looked it over and it sadly cannot be used from inside NixPkgs directly: You’d have to run it ahead-of-time for every Caddy release (can be done on every Caddy update using a PR), as well as for every plugin you’d like to support (meaning every plugin needs to be also packaged in NixPkgs ahead-of-time, since users would be required to locally run that tool each time they add or update one that is not pregenerated in NixPkgs)*. In addition that tool does not appear to actually support merging dependency derivations, that code would still need to be written regardless. So based on this – while it certainly is a nicer solution than the current giant-vendor-directory approach – it seems the tool is mostly orthogonal to the problems the Caddy packaging has. * The blog post has a background on why – technically this could also be solved using import from derivation, but that’s forbidden for NixPkgs. |
and others
compile caddy with all addons listed on the download page.
source: Directive section on https://caddyserver.com/docs/caddyfile
a later improvement could be to select just the addons you want like vim_configurable.
@angus-g
The text was updated successfully, but these errors were encountered: