How to properly use the index.export() function?

[lostdesign]

Hi there, I am trying to export the index to persist it locally in a searchindex.jsonfile.

I've tried to use it as follows:

const saveSearchIndex = async (index) => {
  let exportedIndex = []

  index.export((key, data) => {
    exportedIndex.push({ key, data })
  })

  fs.writeFileSync(searchIndexPath, JSON.stringify(exportedIndex))
}

Then I read it's an async function - so the code above obviously doesn't work. However, I am unsure how to progress at this point. The only solution I can think of is constantly writing to the file and appending whatever the export gives.

Why can I not just export the index as is in one go and writing it to a json file? Can anyone give me any other hint on what I can do here or should I really just write to file inside the export function?

[lostdesign]

So I tried to export the data again and came up with this solution, it's rather hacky but at least I am able to export the index, persist it and re-import it once needed again.

import { Document } from 'flexsearch'
import * as fs from 'fs'

const index = new Document({
  document: {
    id: 'date',
    index: ['content']
  },
  tokenize: 'forward'
})

index.add({"date": "2021-11-05", "content": "i"})
index.add({"date": "2021-11-02", "content": "m"})

const searchIndexPath = './search-index.json'

const exportIndex = () => {
  fs.writeFileSync(searchIndexPath, '')
  index.export((key, data) => {
	// create json objects with the key,data
    fs.appendFileSync(searchIndexPath, `{"${key}":${data === undefined ? "null" : data}},`)
  })
}

const importIndex = () => {
  const file = fs.readFileSync(searchIndexPath)
  // in order to parse it properly, an array has to be wrapped around the json
  console.log(JSON.parse(`[${file.slice(0, -1)}]`))
}

Probably could do better in the file write part, where first a broken json is written, then fetched and parsed with string literals to be able to consume it properly. Have not tried, but maybe you can expect the length of the exported objects/times the callback export is called (currently is 6 times), have a local counter to write a better json as that is expectable, but that doesnt feel any better than just writing a broken json to begin with.

I'd love a first party thought on this, since the docs only say "Here is how you do it with localstorage but you probably should not use it".

Any objections against having the prior export "dump" as a secondary option next to this new, callback based export?

[lostdesign]

Another option I thought of, was writing lines into a text file to then later read each line on its own, getting the key,data back from it. However, you'd need a good separator to do that.

Or well, could just write the data, without key and match them later on the import, where the keys are hardcoded and point to the array values.

[lostdesign]

For anyone wondering, this seems to be the best solution proposed on stackoverflow. I went ahead and fixed OP's code which then also became my solution: https://stackoverflow.com/questions/69760524/flexsearch-export-and-import-document-index-issue/69853828#69853828