Unable to rebuild/update index: IOException in lucene

[icidel]

This error occurs at Eclipse startup even if I clear my .m2/cache folder.

Eclipse m2e version 1.17.2
Eclipse Spring Tool Suite 4.10.0.RELEASE
Maven : Embedded (3.6.3)
OS: Windows 10

Special configuration: my Maven settings.xml depends on two Nexus repositories.

Unable to update index for [...]
java.io.IOException: Resetting to invalid mark
at java.base/java.io.BufferedInputStream.reset(BufferedInputStream.java:446)
at org.apache.maven.index.updater.IndexDataReader.(IndexDataReader.java:69)
at org.apache.maven.index.updater.DefaultIndexUpdater.unpackIndexData(DefaultIndexUpdater.java:385)
at org.apache.maven.index.updater.DefaultIndexUpdater.loadIndexDirectory(DefaultIndexUpdater.java:202)
at org.apache.maven.index.updater.DefaultIndexUpdater.access$300(DefaultIndexUpdater.java:76)
at org.apache.maven.index.updater.DefaultIndexUpdater$LuceneIndexAdaptor.setIndexFile(DefaultIndexUpdater.java:518)
at org.apache.maven.index.updater.DefaultIndexUpdater.fetchAndUpdateIndex(DefaultIndexUpdater.java:736)
at org.apache.maven.index.updater.DefaultIndexUpdater.fetchAndUpdateIndex(DefaultIndexUpdater.java:161)
at org.eclipse.m2e.core.internal.index.nexus.NexusIndexManager.updateRemoteIndex(NexusIndexManager.java:1139)
at org.eclipse.m2e.core.internal.index.nexus.NexusIndexManager.updateIndex(NexusIndexManager.java:1083)
at org.eclipse.m2e.core.internal.index.nexus.NexusIndexManager.lambda$0(NexusIndexManager.java:657)
at org.eclipse.m2e.core.internal.index.nexus.IndexUpdaterJob.run(IndexUpdaterJob.java:74)
at org.eclipse.core.internal.jobs.Worker.run(Worker.java:63)

If I try to update manually aftewards for any of the two repositories I get this exception (even if I close all projects before tryning to rebuild the index !) :
org.apache.lucene.store.LockObtainFailedException: Lock held by this virtual machine: C:\Users[...].m2\repository.cache\m2e\1.17.2\fea3490659764ccdda7e438192661907\min\write.lock
at org.apache.lucene.store.NativeFSLockFactory.obtainFSLock(NativeFSLockFactory.java:127)
at org.apache.lucene.store.FSLockFactory.obtainLock(FSLockFactory.java:41)
at org.apache.lucene.store.BaseDirectory.obtainLock(BaseDirectory.java:45)
at org.apache.lucene.index.IndexWriter.(IndexWriter.java:776)
at org.apache.maven.index.context.NexusIndexWriter.(NexusIndexWriter.java:55)
at org.apache.maven.index.context.DefaultIndexingContext.openAndWarmup(DefaultIndexingContext.java:531)
at org.apache.maven.index.context.DefaultIndexingContext.prepareCleanIndex(DefaultIndexingContext.java:287)
at org.apache.maven.index.context.DefaultIndexingContext.prepareIndex(DefaultIndexingContext.java:265)
at org.apache.maven.index.context.DefaultIndexingContext.(DefaultIndexingContext.java:162)
at org.apache.maven.index.context.DefaultIndexingContext.(DefaultIndexingContext.java:193)
at org.apache.maven.index.DefaultNexusIndexer.addIndexingContextForced(DefaultNexusIndexer.java:155)
at org.eclipse.m2e.core.internal.index.nexus.NexusIndexManager.updateRemoteIndex(NexusIndexManager.java:1135)
at org.eclipse.m2e.core.internal.index.nexus.NexusIndexManager.updateIndex(NexusIndexManager.java:1086)
at org.eclipse.m2e.core.internal.index.nexus.NexusIndex.updateIndex(NexusIndex.java:147)
at org.eclipse.m2e.core.ui.internal.views.MavenRepositoryView$5$1.run(MavenRepositoryView.java:375)
at org.eclipse.core.internal.jobs.Worker.run(Worker.java:63)

[icidel]

My Eclipse installation has two org.apache.lucene.core OSGi plugins:

• 6.1.0 referenced by Mylin and m2e-wtp.jpa
• 8.4.1 referenced by Eclipse Help (4.3.200)

None of them are referenced by m2e.maven.indexer which uses an embedded Lucene library (5.5.5).
Maybe replacing this embedded libraey by the appropriate OSGi plugin/package could help (avoid Classloading issues...) ?

[mickaelistria]

Isn't it a duplicate of #169 ?

[HannesWell]

Just from locking at the code I think this is not exactly a duplicate of #169, because getIndexer().addIndexingContextForced() in NexusIndexManager.updateRemoteIndex() creates a new DefaultIndexingContext which in turn creates a new IndexWriter that creates the overlapping FileLock.
The problem is probably that there is already another IndexingContext for the repository.
A few lines below that already existing context is removed together with the just created 'temporary' cacheCtx context in order to create a fully new Context. I think a solution could be to remove the already existing context before creating the cacheCtx context by moving the line

getIndexer().removeIndexingContext(context, true); // nuke workspace index files

somewhere above

IndexingContext cacheCtx = getIndexer().addIndexingContextForced(id, id, null, directory, null, null,
    getIndexers(details));

But I cannot say at the moment if this would have other negative implications.

However there is a lot of deprecated code in this class. Maybe this would be an opportunity to replace that code with the intended replacement.

[kmcmurtrieca]

Obvious bug here: https://github.com/apache/maven-indexer/blob/c1d5fc594edc0397a4d3c7827a9c435054f1357c/indexer-core/src/main/java/org/apache/maven/index/updater/IndexDataReader.java#L68

That code block should be

is.reset();
data = new BufferedInputStream( is, 1024 * 8 );

[mickaelistria]

@kmcmurtrieca Please report this issue to the relevant project, ie maven-indexer.

[laeubi]

I have proposd a change in MAVEN Indexer: apache/maven-indexer#142

[laeubi]

Seems I'm not the only one apache/maven-indexer#33 but no reaction from the Maven Indexer team for more than 2 years 👎

[mickaelistria]

Next release will remove the indexer (see #510 ), so we'll basically never fix this one.