Container won't start #76
Comments
|
Hi can you send me the architecture of the machine running the container and your docker-compose file? |
|
Hey Alex, Belows is my compose file and system info extract. Let me know If I can provide any better info thanks :) Compose: networks: System Info: |
|
Thank you, try to run container with default docker-compose |
|
Same error occurs. Waiting for the container to restart the follow up error is: Config plugin initialization completed. If that helps |
|
It seems that database files was created but it's corrupted. |
|
Yeah I did this, Originally I tried various locations and the including root based on the default compose. The above error was after it created the new one and failed, I just left out the already posted errors. I will try using docker project instead of portainer |
|
Hello, |
|
Ok after taking a look, the hook is asynchroneous so my theory does not apply. The done method cannot be used in this case and this may be a real timeout. Perhaps we should try to increase the timeout period but this feels strange. The database initialization has nothing that take long. |
|
Oh, i just saw the second set of logs with (reading 'version') |
|
@julichan the second set of logs come because the container stops but has "always-restart" on so when it restarts the database already exists. I tried again making sure to delete the image from the cache but no luck. |
|
Okay so I have an unraid machine and figured out how to get compose running on it. Secrets saved. No issues using the exact same compose script. I did get a warning about kernel version but doesn't appear to impact it. So there must be something on Synology NAS side that is causing issues. |
|
I took the db that was created within unraid and put in the folder I was referencing on portainer and this times it started up correctly. Starting migrations... |
|
@Whalee110, thanks for the information, can you tell me what synology model and dsm version you have because it s working on my latest one but i havent tried on others. I know they have either kernel 3.10 or 4.x. docker isn't well supported with kernel 3.10 lately. Edit: you already provided this info: Can you get the kernel version? |
|
I looked around google a little bit and I don't see a problem aside from a performance issue in this case. Edit: according to this, we can disable the timeout. |
|
@julichan Yes, i think we can make a env variable. I'll make a PR. |
Fix for #76 Container won't start on Synology NAS
|
@Whalee110 ok, Should be fixed in 1.4.11. Change |
|
I just looked at your changes. If we keep the default to 10000, it won't change much. I'd set it to 120000 to feel very safe. This is what they fo with plugin fastify-mongodb. But anyway, we would need a test from @Whalee110 first to confirm this would fix things up. |
|
@julichan I think the kernel version is 4.4.302+ I removed everything and re-created the stack repulling image and the below is the output. The compose file used is: In portainer I had an enviroment variable FASTIFY_PLUGIN_TIMEOUT with value set to 0 |
|
@Whalee110 try to run with FASTIFY_PLUGIN_TIMEOUT=15000 or higher |
|
I ended up doing 15000, 30000, 100000 and 360000. No change same output. Are there any commands i can run to see more detail that can help? |
|
@Whalee110, did the value change the timing in which the error occured? If not, it would mean the parameter pluginTimeout doesn't work for hooks or because of the autoloader... There are only 2 awaited function at this point: await import and await migrationPlugin, neither should take so long without a reason. The await import could be because the initial script file is locked against read but it's only inside the container and there is no other usage case in neonlink code. For the second, there is only a set of queries to create new tables that should barely take a few ms to an already opened database. @AlexSciFier, We could add a log between the two to ensure the migrationplugin loaded from import to be aware which of the function is timing out but i'm not sure knowing that will be so helpful. Also, since the app work with an initialized database, i don't think the second function is timing out due to a hidden error but we could add logs in the foreach of the "initial" migrationplugin script. @Whalee110, do you have an anti virus on your dsm that could be locking files? This is far fetched reason though as the related files are so small. From what i've read, we could also try the following:
This is from the least to the most costly solutions i see but without knowing actual reason for the crash, it's too much for a bug. Would also monitoring the performance of the initialization code be possible/useful? Do you have any other idea @AlexSciFier ? |
|
It appeared to be a bit longer but that just could be normal variance as it wasn't huge. I don't believe I have any anti virus enabled - Later today I will go through and disable some of the "security" I put on it for now. Is it trying to connect to any external addresses? I do have a DNS level Ad block if that impacts it that I will also try disabling. I'll try doing it exclusively through SSH with docker run to see if it provides any additional information. |
|
@Whalee110, no need to go as far as disabling your dns level securities. Initialization steps doesn't connect anywhere. Only anti virus like app may affect it although i very much doubt it. |
|
@Whalee110 try to run container without binding volumes. If the container starts without errors, then the problem is in the file system and not in the application. |
|
Same error I was unsure about the umask 022 so I ran on both again to see, but it does look like it's just the version where the logs vary |
|
To be clear, you are running the container without this section? Because judging by your logs, the application is trying to open a database file that should not exist. |
|
I had to stop container, then chown the entire directory to the GID/UID of the user from the docker-compose.yml |
|
I manually create directories beforehand to ensure permissions are set, but regardless even with no mapped drives the same error occurs. On a Synology discord where I first got a sample of the compose they mentioned it needing an additional line so I will be waiting to test that once they provide the updated compose. @AlexSciFier Yeah sorry, those logs were comparing the startup of the device that is fine and the one that fails. |
|
Hello @Whalee110, I did not have time to test on my older server but here is my compose you may adapt to yours that works on my ds923+. Please take note that some feature don't work with synology docker. Try the following:
Dockerfile: docker-compose.yml |
|
@AlexSciFier, it seems good but i'd need to test gosu myself to have a good idea. However i tried many methods in the past and i have one conscen about this method. There are many phases in docker that result in different states: build, container creation and container running. Each state should have its own access right. I believe the root command should never be accessible beyond build because it leaves potential opening for root escalation by reusage of those tools in a non-root container. So i always override the dockerfile and dockerfile entrypoint when i see such tools in images. |
No, i added it after server code update Ok so, the actual fix for permission error, i think, is to leave as it is right now.
With this i don't need |
|
Not sure if related, but just now I also had problems with my container starting due to permission issues. Once I added in the user parameter and changed 1000 to my user ID it started up no problem. I usually use PUID or UID to set that but I guess not the case here. The discussion in this issue helped me figure that out so thanks! |
|
Apologies for no update, the forum I am with the person that said they had a fix for it has yet to respond. I've tried the compose provided by Julichan but no difference in outcome. |
|
It works! The person provided their code, I had to change it to 0 but it looks like using portainer environment option didn't apply it correctly. Apologies for missing that, I'm learning how to use docker compose currently so didn't realise :/ This is the compose that worked: |
|
Alright so my worries were well founded. We should increase the value to at least 120000 by default for now since it could occur during a db upgrade resulting to a loss of data. |
|
@Whalee110 in your compose file, you use a synology existing user (1026 is usually the first created user id). I imagine that is an user with administrative privileges. You'd better make sure neonlink is not accessible outside of your private network. |
|
@julichan oh I see, my presumption this is done to avoid "permissions issues" but I just copy it from others I see. If that is potentially security risk then I think maybe I should read up on compose a bit more to avoid more like it haha. Thank you for the info! |
|
@Whalee110, this is allowed because there are important uses cases. For exemple, if you have a an app that manage your documents in docker and u want your documents accessible from a user stand point of the file system, you'll use a volume that can be read/write by your host user. But in that case, you usually create a user on your host that is only allowed to do things in thag part of the file system for secuirity reasons. Limited host access will prevent ability to hack. |
@Whalee110 What version did you use? If it's 1.4.12 than you don't need PUID, PGID and UMASK variables. I removed it in this version. Now if you want to change user in docker you need to use default |
|
@AlexSciFier yeah that's the way I do in my compose this was just the one the person got working from a synology discord. I think it was 1.4.11 at the time I did it. @julichan Oh I see. I knew why you'd want certain users similar with the "no new priv" tag, but I was unsure as to when you'd want to make it what user. Your explanation helped alot there! ty I have also created a user that can only access docker folder for future instead of my admin account :) |
|
@Whalee110 If the problem is resolved, can I close the issue? |
|
Yup thank you for your help! |
I am attempting to deploy portainer stack with docker compose and it keeps failing to "initialise"
The error I get is below but I can't see any error in the compose.
Secrets saved.
Settings saved.
Config plugin initialization completed.
Database plugin initialization completed.
Life cycle plugin initialization completed.
Starting migrations...
Applying initial database script...
Attempt to apply initial...
FastifyError [Error]: A callback for 'onReady' hook timed out. You may have forgotten to call 'done' function or to resolve a Promise
at exit (/app/node_modules/fastify/lib/hooks.js:97:37)
at manageTimeout (/app/node_modules/fastify/lib/hooks.js:124:11)
at _encapsulateThreeParam (/app/node_modules/avvio/boot.js:562:7)
at Boot.timeoutCall (/app/node_modules/avvio/boot.js:458:5)
at Boot.callWithCbOrNextTick (/app/node_modules/avvio/boot.js:440:19)
at Task.release (/app/node_modules/fastq/queue.js:149:16)
at worked (/app/node_modules/fastq/queue.js:201:10)
at Boot.timeoutCb (/app/node_modules/avvio/boot.js:467:7)
at /app/node_modules/fastify/lib/hooks.js:146:9
at _encapsulateThreeParam (/app/node_modules/avvio/boot.js:562:7) {
code: 'FST_ERR_HOOK_TIMEOUT',
statusCode: 500,
cause: AvvioError [Error]: Plugin did not start in time: 'bound _encapsulateThreeParam'. You may have forgotten to call 'done' function or to resolve a Promise
at Timeout._onTimeout (/app/node_modules/avvio/boot.js:451:21)
at listOnTimeout (node:internal/timers:569:17)
at process.processTimers (node:internal/timers:512:7) {
code: 'AVV_ERR_READY_TIMEOUT',
fn: [Function: bound _encapsulateThreeParam]
}
}
The text was updated successfully, but these errors were encountered: