chore: modify script

Esri · Aug 24, 2023 · 3e6269a · 3e6269a
1 parent 0808954
commit 3e6269a
Show file tree

Hide file tree

Showing 5 changed files with 338 additions and 88 deletions.
diff --git a/README.md b/README.md
@@ -1,93 +1,33 @@
-# Koop
+# Koop Fork
 
-*Translate, query, & integrate any geospatial API on the web*
+This is a specialized fork of the Koop repositiory.
 
-Koop is a JavaScript toolkit for making requests to spatial APIs. It exposes a Node.js web server that faciliates on-the-fly transformations of geospatial data from one format to another and delivers it to clients by HTTP.  Koop allows you to keep your data in its native format, while making it accessible in any format required.  Out-of-the-box, Koop can translates your data into the GeoServices specification supported by ArcGIS products. It's plugin architecture supports output in other formats including vector-tile, WMS, and plain old GeoJSON. Learn more at [https://koopjs.github.io](https://koopjs.github.io).
+## Creating a release branch
+1. Make sure your master branch is synchronized with the Koop upstream.  This will require a rebase due to the changes made on this fork.
 
-Koop's plugin-architecture facilates custom deployments specific to your needs. "Provider" plugins to connect to novel data formats and translate to a common format (GeoJSON), while "output" plugins then transform that GeoJSON to other specifications. See a list of plugins already authored and maintained [here](https://koopjs.github.io/docs/available-plugins). If you have a novel datasource or require an as-yet unsupported output format, new plugins can be easily developed and integrated. See the [development section of the Koop docs](https://koopjs.github.io/docs/development).
-
-![lots of geojson into feature services](https://user-images.githubusercontent.com/7832202/28444721-43eb6ea6-6d8d-11e7-8d56-3af46fd5bf88.png)
-
-## Demo
-Want to see Koop in action? The repository ships with a demo that shows Koops support for GeoServices (ArcGIS). It leverages the file-geojson data provider and the GeoServices output-plugin:
+2. Branch off of synced master.  In this example we call the branch cdf-11.2:
 
 ```bash
-git clone https://github.com/koopjs/koop
-cd koop
-npm run demo
+> git checkout -b cdf-11.2
 ```
 
-Koop will start listening on port 8080. You should the following console logging noting the exposed file-geojson/GeoService routes:
-
+3. Run modification script.  This will result in file deletions, modifications, and the creation of a Github Action.
 ```bash
-2023-03-17T19:18:29.416Z info: [Geoservices] routes for [file-geojson] provider
-2023-03-17T19:18:29.416Z info: ROUTE | [GET, POST] | /file-geojson/rest/info
-2023-03-17T19:18:29.416Z info: ROUTE | [GET, POST] | /file-geojson/tokens/:method
-2023-03-17T19:18:29.417Z info: ROUTE | [GET, POST] | /file-geojson/tokens
-2023-03-17T19:18:29.417Z info: ROUTE | [GET, POST] | /file-geojson/rest/services/:id/FeatureServer/:layer/:method
-2023-03-17T19:18:29.417Z info: ROUTE | [GET, POST] | /file-geojson/rest/services/:id/FeatureServer/layers
-2023-03-17T19:18:29.417Z info: ROUTE | [GET, POST] | /file-geojson/rest/services/:id/FeatureServer/:layer
-2023-03-17T19:18:29.417Z info: ROUTE | [GET, POST] | /file-geojson/rest/services/:id/FeatureServer
-2023-03-17T19:18:29.417Z info: ROUTE | [GET, POST] | /file-geojson/rest/services/:id/FeatureServer*
+> node fork-scripts/modify-fork.js && npm install
 ```
 
-The following request will take the demo data from `demo/provider-data/line.geojson` and send it as geojson:
-
-```
-http://localhost:8080/file-geojson/rest/services/line/FeatureServer/0/query
-```
-
-Query for features with a specific property value:
-
-```
-http://localhost:8080/file-geojson/rest/services/line/FeatureServer/0/query?where=foo='bar'
+4. Run E2E tests and ensure they all pass
+```bash
+> npm run test:e2e
 ```
 
-Return data in a different coordinate system:
-
-```
-http://localhost:8080/file-geojson/rest/services/line/FeatureServer/0/query?outSR=3857
+5. Commit changes and push your branch
+```bash
+> git commit -m 'chore: prep for cdf release'
+> git push origin cdf-11.2
 ```
 
-## Koop Monorepo
-
-This repository is home of the Koop monorepo.  In contains a collection of packages that are shipped by default with every Koop instance.  [koop-core](https://github.com/koopjs/koop/packages/core) is the parent package and is used to generate a default configuration of Koop. References to the "Koop version" refer to the version of this package. The other packages in this monorepo are dependencies of core and include the Geoservices output-plugin and its dependencies, the default in-memory data cache, and a logger.  All other plugins (providers, outputs, etc) are in separate repositories.
-
-The Koop dependency graph is shown below.
-![Screen Shot 2022-11-30 at 1 03 46 PM](https://user-images.githubusercontent.com/4369192/204908289-82659cfe-fcf3-404a-aa70-79baf540f1b8.png)
-
-### Test Coverage
-Test coverages for each package are shown below. Coverage for some packages includes integration tests as opposed to true unit tests. Difference in coverage is shown below. Our goal is to have complete unit test coverage, and breakout integration/e2e tests separately.
-
-| package | integration + unit | unit |
-|---|---|---|
-|cache-memory|N/A|![coverage](./packages/cache-memory/coverage.svg)|
-|featureserver|![coverage](./packages/featureserver/coverage.svg)|![coverage](./packages/featureserver/coverage-unit.svg)|
-|koop-core|N/A|![coverage](./packages/core/coverage.svg)|
-|logger|N/A|![coverage](./packages/logger/coverage.svg)|
-|output-geoservices|N/A|![coverage](./packages/output-geoservices/coverage.svg)|
-|winnow|![coverage](./packages/winnow/coverage.svg)|![coverage](./packages/winnow/coverage-unit.svg)|
-
-## Contributing
-Pull requests are welcomed and encouraged. Please consider the following PR guidelines:
-1. Provide a clear description of what the PR is trying to solve.  Link to any existing issues
-2. Aim for clear, readable code.
-3. Use conventional commit messages. For convenience, you can add commits with `npm run commit`.
-3. Add unit tests and ensure any new code has 100% test coverage. You can do this by running `npm run test:cov` and then looking for your file in the `/coverage/index.html` output.
-4. Run `npm run lint:fix` and ensure you're not commiting lint
-5. If your new code requires a release, please run `npm run changeset:add` and commit the generated changeset file as a part of your PR.
-
-Esri welcomes contributions from anyone and everyone. Please see our [guidelines for contributing](https://github.com/Esri/contributing).
-
-## Issues
-Find a bug or want to request a new feature? Post it [here](https://github.com/koopjs/koop/issues).
-
-## Resources
-
-* [Koop Documentation](https://koopjs.github.io/)
-* [ArcGIS REST API Documentation](http://resources.arcgis.com/en/help/arcgis-rest-api/)
-* [ArcGIS for Developers](http://developers.arcgis.com)
-* [@esri](http://twitter.com/esri)
+6.  Consider adding branch protection rules on Github (to prevent deletions).
 
 ## License
 

diff --git a/fork-scripts/ci-tests.yml b/fork-scripts/ci-tests.yml
@@ -0,0 +1,36 @@
+name: CI tests
+
+on:
+  push:
+    paths:
+      - "./.github/**.yml"
+      - "**/packages/**.js"
+      - "**/packages/**/package.json"
+
+jobs:
+  pr-tests:
+    name: Install, lint, test
+    runs-on: ${{ matrix.os }}
+    strategy:
+      matrix:
+        node-version: [18.x]
+        os: [ubuntu-latest, windows-latest]
+        # See supported Node.js release schedule at https://nodejs.org/en/about/releases/
+
+    steps:
+    - uses: actions/checkout@v3
+    - uses: actions/setup-node@v3
+      with:
+        node-version: ${{ matrix.node-version }}
+
+    - name: Install npm 7
+      run: npm i -g npm@7 --registry=https://registry.npmjs.org
+
+    - name: Install
+      run: npm ci
+
+    - name: Lint
+      run: npm run lint:ci
+
+    - name: E2E tests
+      run: npm run test:e2e
diff --git a/fork-scripts/modify-fork.js b/fork-scripts/modify-fork.js
@@ -1,25 +1,33 @@
 const path = require('path');
+const { promisify } = require('util');
+const copyfiles = require('copyfiles');
+const copy = promisify(copyfiles);
 const deleteAsync = require('del');
 const findInFiles = require('find-in-files');
 const { modifyJsonFile } = require('modify-json-file');
 const replace = require('replace');
 const packages = require('../package.json').workspaces;
 
 async function execute() {
-  await deleteIrrelevantFiles();
+  await deleteNonEssentialFiles();
   await modifyPackageFiles();
   await replacePackageReferences();
   await replaceKoopCoreRefs();
+  return copyGithubAction();
 }
 
 async function modifyPackageFiles() {
-  for (let i = 0; i < packages.length; i++) {
-    const packageFile = require(path.join('..', packages[i], 'package.json'));
+  console.log('- modifying package.json files');
 
+  for (let i = 0; i < packages.length; i++) {
+    const filePath = path.join('../', packages[i], 'package.json');
+    const packageFile = require(filePath);
+    const name = packageFile.name.replace('@koopjs/', '');
+    const buildDest = `../../customdata/framework/${name}`;
     await modifyJsonFile(
-      `./${packages[i]}/package.json`,
+      path.join(packages[i], 'package.json'),
       {
-        name: packageFile.name.replace('@koopjs/', ''),
+        name,
         dependencies: stripKoopDeps(packageFile.dependencies),
         devDependencies: undefined,
         files: undefined,
@@ -29,6 +37,10 @@ async function modifyPackageFiles() {
         repository: undefined,
         directories: undefined,
         types: undefined,
+        scripts: {
+          clean: `shx rm -rf ${buildDest} && shx rm package-lock.json`,
+          build: `shx mkdir ${buildDest} && shx cp package.json ${buildDest} && shx cp -r ./src ${buildDest} && shx cp -r ./node_modules ${buildDest}`,
+        },
       },
       { ifFieldIsMissing: 'skip' },
     );
@@ -47,7 +59,8 @@ function stripKoopDeps(dependencies) {
     }, {});
 }
 
-async function deleteIrrelevantFiles() {
+async function deleteNonEssentialFiles() {
+  console.log('- deleting nonessential files');
   await deleteAsync([
     'packages/**/*.*',
     '!packages/**',
@@ -63,15 +76,23 @@ async function deleteIrrelevantFiles() {
     'packages/**/*.ts',
     'packages/**/*.spec.js',
   ]);
-  await deleteAsync(['packages/**/test']);
+
+  await deleteAsync(['.github/workflows/*.yml']);
   return deleteAsync(['packages/**/benchmark']);
 }
 
+async function copyGithubAction() {
+  return copy(['fork-scripts/ci-tests.yml', '.github/workflows'], { up: true});
+}
+
 async function replacePackageReferences() {
+  console.log(
+    '- replacing references to monorepo packages with relative paths',
+  );
   for (let i = 0; i < packages.length; i++) {
     const found = await findInFiles.find(
       /@koopjs\/[a-z-]+/,
-      packages[i],
+      path.join(packages[i]),
       '.js$',
     );
 
@@ -103,11 +124,12 @@ async function replacePackageReferences() {
 }
 
 async function replaceKoopCoreRefs() {
+  console.log('- replacing koop-core references in demo and e2e test files');
   const directories = ['test', 'demo'];
   for (let i = 0; i < directories.length; i++) {
     const found = await findInFiles.find(
       '@koopjs/koop-core',
-      directories[i],
+      path.join(directories[i]),
       '.js$',
     );