Q: Migrate / upgrade project from manifest.json / lock.json to Gopkg.toml / Gopkg.lock?

[marccarre]

Disclaimer: apologies in advance if I missed some doc/issue/PR answering this question already. The few searches I ran unfortunately did not yield anything.

Question:
If I have a project using the "old" format (pre #342) to define dependencies and constraints on them, i.e.:

• manifest.json
• lock.json

is there a way to have dep migrate or upgrade the project from these to the "new" (post #342) format, i.e.:

• Gopkg.toml
• Gopkg.lock

without having to do the following?

• dep init && dep ensure && dep prune
• manual translation of all constraints from JSON to TOML (:man_facepalming:)
• rm manifest.json; rm lock.json

[sdboyer]

hi! sorry you're running into this. no, there's really no automated path - this is why we warned pretty stridently against even committing these files until after we stabilized them. (now that we have, we guarantee backwards compatibility for Gopkg.lock and Gopkg.toml files)

however, when the format changed, very little of the semantics changed. so, you might be able to get a leg up by simply using a JSON to TOML converter (like this one, i guess?) to translate each of the files, then performing the necessary changes to the TOML result as described in #644.

[marccarre]

ACK. Many thanks for your reply & confirmation @sdboyer!

[marccarre]

The converter mentioned above failed on my lock.json and manifest.json, but the below (very naive) script worked and might be useful to someone else having the same needs. If so, here are the full steps to migrate your project:

1. Go to the project you want to migrate:

   $ cd /path/to/project
   

2. Initialise the latest version of dep:

   $ dep init
   

3. Run the below Python script:

   $ cat <<EOF | python
   import json
   from sys import argv
   from os.path import abspath, dirname, join
   
   if __name__ == '__main__':
       directory = dirname(abspath(__file__))
       with open(join(directory, 'lock.json'), 'r') as f:
           lock = json.load(f)
       with open(join(directory, 'Gopkg.toml'), 'w') as f:
           f.write('# Gopkg.toml\n')
           f.write('# Refer to https://github.com/golang/dep/blob/master/docs/Gopkg.toml.md\n')
           f.write('# for detailed Gopkg.toml documentation.\n\n')
           for project in lock['projects']:
               f.write('[[constraint]]\n')
               f.write('  name = "%s"\n' % project['name'])
               if 'version' in project.keys():
                   f.write('  version = "%s"\n' % project['version'])
               elif 'revision' in project.keys():
                   f.write('  revision = "%s"\n' % project['revision'])
               elif 'branch' in project.keys():
                   f.write('  branch = "%s"\n' % project['branch'])
               else:
                   raise RuntimeError('Could not process %s' % project)
               f.write('\n')
   EOF
   

   Note that this takes dependencies in your lock.json file and makes them contraints in your Gopkg.toml file. This is to avoid potential build errors by accidentally upgrading dependencies if you had "moving targets" (e.g. master) configured previously. You most likely want to do some housekeeping in your Gopkg.toml afterward running this script, e.g.: pin to fixed, released versions, remove unnecessary contraints, etc.

4. Let dep take over:

   $ dep ensure && dep prune
   

P.S.: I'm still interested in a better way to solve this problem, so please comment if you know one.