[FEAT] Update Smartctl database

[modem7]

As per the official website, there is a method to update the drive database.

It may be worthwhile putting it into the Dockerfile, so each time there is a release, it'll have the latest database.

https://www.smartmontools.org/wiki/Download#Updatethedrivedatabase

TLDR:
/usr/sbin/update-smart-drivedb

Example output:

sudo /usr/sbin/update-smart-drivedb
/var/lib/smartmontools/drivedb/drivedb.h updated from branches/RELEASE_7_2_DRIVEDB

[AnalogJ]

is that command not executed during the package installation? If so, yeah we should definitely do this.

The Dockerfile is build from scratch on ever release, so if the database update command is run as part of the package install, we should always have an up-to-date drive db.

[modem7]

Apparently not, just during the package build (and as we know, distros rarely have the latest versions of anything unless we use an edge version(https://www.smartmontools.org/wiki/Packages)).

With the command, regardless of the build version (minimum version caveats in link), it grabs the latest database as it's a static file that's just built when the package is.

So either build from source (waste of time) or just do the update command during image build and call it done n dusted.

Alternatively, for a middle ground solution... Implement it in the build + s6/entrypoint so it updates each startup (only downside I can see for this is that there would need to be logic in place to see if there is an Internet connection in case it's being run in an airgapped environment).

---

The only caveat really, is that it needs curl, wget or lynx.

[AnalogJ]

ah, thats unfortunate.

I'm a bit busy at the moment, but if you're willing to test and contribute a change to the Dockerfile to run /usr/sbin/update-smart-drivedb during build, I'd be happy to merge it :)

Lets stay away from running it during s6 init, until we start getting requests for it.

[modem7]

ah, thats unfortunate.

I'm a bit busy at the moment, but if you're willing to test and contribute a change to the Dockerfile to run /usr/sbin/update-smart-drivedb during build, I'd be happy to merge it :)

Lets stay away from running it during s6 init, until we start getting requests for it.

Sure thing, I was already looking at the dockerfile (hence the other issue I raised) to see if I could improve the caching behaviour.

I'll raise the PR, hopefully by end of week.

[adamantike]

This is a good idea to get updates for the drive database, but let's consider that as it is proposed, it would go against the idea of reproducible builds in #231.

[AnalogJ]

hm. I think there's a bit of a misunderstanding here @adamantike .

#231 specifically references reproducible builds for the golang binaries. While scrutiny has a number of users and contributors, its still a fairly new project that effectively requires root/privileged access to your machine to run smartctl.

The intention behind reproducible builds is to help users trust that the code in the repo is what is being packaged and distributed in each binary release.

While we could extend reproducible builds to cover the Docker images, like you noticed that only works if you can lock os packages & artifacts to specific versions, which update-smart-drivedb doesn't seem to allow.

TBH, I'm more interested in implementing #232 in-addition to reproducible Docker images -- with an almost empty base image, it would be actually reasonable for paranoid users to explore the filesystem and verify authenticity of the (few) packages that are installed

What are your thoughts?

[martadinata666]

Let me chime in, when essentially update-smart-drivedb is basically update drivedb.h file. There are two approach I can think of,

• Run `update-smart-drivedb on docker build
• Manually fetch drivedb.h from smartmontools mirror svn repo

Second approach likely more suit to distroless approach.

[LinusCDE]

Another solution might be to compile in the newest release for smartmontools instead of using the debian provided one.

I stumbled into an issue where my Samsung T7 would not get listed at all. I looked in the documentation and saw that the collector uses smartctl --scan and indeed it did not show up there either.

Releated to the issue I found this PR on GH for smartmont ools alongside this ticket discussion it. I tried to check whether using a debian:bookworm base instead of the current debian:bullseye would resolve this, but at least for the T7, the release still was not recent enough.

It seems to also boil down to a update of the database. For now I solved my problem the following way:

• Build smartmontools from source using these instructions
• Mount the generated binary into the container as /usr/local/bin/smartctl.

This approach seems to work fine for me and now scrutiny recognizes the T7 properly without any issues:

The docker-compose.yml for my collector now looks like this and it should work properly with auto-updates from watchtower:

version: '3'
# web and influxdb services excluded here

  collector:
    image: ghcr.io/analogj/scrutiny:master-collector
    restart: unless-stopped
    labels: [ 'com.centurylinklabs.watchtower.enable=true' ]
    cap_add:
      - SYS_RAWIO
    #  - SYS_ADMIN # optional (needed for nvme)
    environment:
      TZ: Europe/Berlin
      COLLECTOR_API_ENDPOINT: http://web:8080
      COLLECTOR_HOST_ID: <snip>
    volumes:
      - /run/udev:/run/udev:ro
      - ./smartmontools-7.4/smartctl:/usr/local/bin/smartctl
    devices:
      - /dev/sda
      - /dev/sdb
      - /dev/sdc
      - /dev/sdd

So if doing the db update is not ideal, maybe just building and installing a recent smartmontools release from source into the container would be a good long term solution?

When time I could probably look into adding the building and integration of it into the collector.Dockerfile.