Generated files depend on local file modification time

[igor-sirotin]

Related: #4191

TLDR;

I noted that even generating with nix-shell, I still might get ~13 files change on latest develop. It depends from which repository directory I change the files.

Details

1. I have 2 directories with status-go repo cloned.
2. I run make generate (with nix-shell, ofc) in directory 1 and push the changes
3. Pull the changes in directory 2
4. run make generate in directory 2.

Result:

On branch develop
Your branch is up to date with 'origin/develop'.

Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git restore <file>..." to discard changes in working directory)
        modified:   appdatabase/migrations/bindata.go
        modified:   appdatabase/migrationsprevnodecfg/bindata.go
        modified:   mailserver/migrations/bindata.go
        modified:   multiaccounts/migrations/bindata.go
        modified:   protocol/anonmetrics/migrations/migrations.go
        modified:   protocol/encryption/migrations/migrations.go
        modified:   protocol/migrations/migrations.go
        modified:   protocol/pushnotificationclient/migrations/migrations.go
        modified:   protocol/pushnotificationserver/migrations/migrations.go
        modified:   protocol/transport/migrations/migrations.go
        modified:   static/bindata.go
        modified:   t/bindata.go
        modified:   walletdatabase/migrations/bindata.go

Each file contain such change:

diff --git a/static/bindata.go b/static/bindata.go
index 8bc0f32d1..71beff84f 100644
--- a/static/bindata.go
+++ b/static/bindata.go
@@ -103,7 +103,7 @@ func emojisTxt() (*asset, error) {
                return nil, err
        }
 
-       info := bindataFileInfo{name: "emojis.txt", size: 28134, mode: os.FileMode(0644), modTime: time.Unix(1709295743, 0)}
+       info := bindataFileInfo{name: "emojis.txt", size: 28134, mode: os.FileMode(0644), modTime: time.Unix(1707305567, 0)}
        a := &asset{bytes: bytes, info: info, digest: [32]uint8{0xf5, 0x28, 0xc, 0x22, 0x34, 0xa1, 0xeb, 0x8, 0x8d, 0xef, 0x38, 0x1b, 0xd8, 0xc2, 0x1a, 0x6d, 0xa2, 0x62, 0xad, 0x43, 0xfc, 0x1c, 0x38, 0xda, 0x8c, 0x3f, 0x34, 0xa, 0x8c, 0x6f, 0x5d, 0xd8}}
        return a, nil
 }
@@ -123,7 +123,7 @@ func ConfigReadmeMd() (*asset, error) {
                return nil, err
        }
 
-       info := bindataFileInfo{name: "../config/README.md", size: 3031, mode: os.FileMode(0644), modTime: time.Unix(1709295743, 0)}
+       info := bindataFileInfo{name: "../config/README.md", size: 3031, mode: os.FileMode(0644), modTime: time.Unix(1707305567, 0)}
        a := &asset{bytes: bytes, info: info, digest: [32]uint8{0x15, 0x44, 0x2b, 0x13, 0x14, 0x34, 0xa, 0x66, 0x62, 0x1b, 0xc6, 0x4a, 0x2c, 0x7d, 0x4d, 0x89, 0xfb, 0xc9, 0x69, 0xe4, 0x18, 0x5f, 0x3, 0x98, 0x6d, 0x3c, 0x9e, 0xa8, 0xcd, 0x53, 0x5d, 0x75}}
        return a, nil
 }
@

So although I have the same nix-shell configuration, I get different output. Same happens if one directory is on my machine and the second directory is one another dev's machine.

Why does this change occur?

The only thing changed is the modTime. Which stands for "modified time".

This field is calculated here:

func assetReleaseCommon(w io.Writer, c *Config, asset *Asset, digest [sha256.Size]byte) error {
	fi, err := os.Stat(asset.Path)
	if err != nil {
		return err
	}

	mode := uint(fi.Mode())
	modTime := fi.ModTime().Unix()

It's taken from stat OS call.

For me it changed

• from 1709295743 (Fri Mar 01 2024 12:22:23)
• to 1707305567 (Wed Feb 07 2024 11:32:47)

This corresponds to the real value returned by stat:

➜  status-go git:(develop) ✗ stat -f "%m" ./static/emojis.txt
1707305567

And this value is the same for all generated files. No matter of when the file was actually modified.

E.g. static/emoji.txt was modified a long time ago:

commit 9930dbe4563bbf560eb7730f87607bf2ec1ddf79
Author: osmaczko <[email protected]>
Date:   Fri Mar 11 22:17:48 2022 +0100

    feat: add visual identity service (#2550)
~

Questions

1. Is this a bug of go-bindata?
   I suppose the migration should contain some real timestamps from git?

2. Should we bother about this?
   I don't understand if there're really any consequences.

I expect:

1. Timestamp to be different for each file
2. No change depending on who runs the migration.

And the fun part:
This will still be an issue, even if we move the generation to CI 🤔

[igor-sirotin]

cc @osmaczko @cammellos

[igor-sirotin]

Sorry for many letters guys 😄

[osmaczko]

And the fun part
This will still be an issue, even if we move the generation to CI 🤔

Why will it be an issue? If they are removed from the repository, it won't affect the diff; it will just generate a different modTime every CI run, which I believe is fine.

[igor-sirotin]

it will just generate a different modTime every CI run, which I believe is fine.

@osmaczko I wasn't sure about this part. If this value doesn't affect the code, then... why is it there for? 😄

[cammellos]

It doesn't impact the logic of the code, I believe it's just a timestamp to know when the files are generated, I believe we could in theory override modTime and set it to any arbitrary value, if we don't want that to be changed, but overall I think it's fine to keep it as it is

[osmaczko]

@osmaczko I wasn't sure about this part. If this value doesn't affect the code, then... why is it there for? 😄

Because #4191 has not been completed yet 🤔 Normally, this type of file is not under source control. If that were the case, you wouldn't even notice modTime changes.

[igor-sirotin]

Because #4191 has not been completed yet 🤔 Normally, this type of file is not under source control. If that were the case, you wouldn't even notice modTime changes.

@osmaczko I meant the part about "... which I believe is fine".