Handle Deleting of Files Properly w/ GraphQL, etc

[bradennapier]

Description

If you delete or rename markdown pages right now you can not get rid of them and errors pop up even if you restart gatsby develop. Only way to fix it appears to be to delete everything and start over (.cache and public folders).

Perhaps cleaning the cache should be done in some way? It still thinks these posts exist many hours after they are completely removed.

I had assumed maybe there was a gatsby clean or gatsby purge but I didn't see it so it would appear this would break forever for anyone that did't know they had to manually delete those folders on every change?

Steps to reproduce

1. Make Markdown File in gatsby-advanced-starter but any should work as the gatsby node config doesnt re-run and/or runs in cache (dont know really with the magic that happens)

2. Delete it

3.It still exists on reloads in the graphql database

...

I get these errors because it is thinking a bunch of files exist when I simply renamed or deleted them.

[KyleAMathews]

Are you on the latest version of gatsby? I spent nearly a day on this very issue a few weeks ago :-)

[bradennapier]

I believe I am unless it came out like a day ago. Will be awesome if that is it - one moment.

[bradennapier]

Nope.. reinstalled gatsby completely and on version [email protected] and still the items are returned from graphql queries there is rendering and I deleted that file about 8 hours ago. Only fix is to delete .cache and public manually.

I will do that now though in case it will only handle from nwo on rathr than files I deleted previously.

[bradennapier]

oof - just noticed tons of errors - yall need a gatsby clean to clear the cache and public or something. Looks like updating versions made it break since it was reading cache which was now invalid?

[KyleAMathews]

Those errors are React errors... it's not necessarily something Gatsby is doing. I'll look at gatsby-advanced-starter and see if I can reproduce this really quick.

[KyleAMathews]

We aim towards the cache being 100% accurate. A cache you have to clean all the time is useless.

Would love it if you could help me figure out why this is happening so we can solve it!

[KyleAMathews]

I just installed the starter and ran yarn why gatsby and it said the version of Gatsby is 1.9.178 — which means it doesn't have any of the new fixes for handling deleting files.

Can you double-check what version you're on?

[KyleAMathews]

I'm upgrading packages now and then will test if I can make the cache invalid by deleting files.

[bradennapier]

I updated to the package shown in the post above. The errors were react but they were a result of the query returning data about files that no longer exist. Removed cache manually now and going to try more, unfortunately my ISP appears to be on the fritz.

[KyleAMathews]

I've replicated the bug — looking into why it's happening.

[bradennapier]

Great! Also happens when I move them into different folders as well. Seems it just needs to run a new query on the overall file contents when certain things occur.

[KyleAMathews]

Just put up a PR which should fix this. I'm feeling a bit gun-shy given I thought I'd fixed this once or twice before 😅 But I improved the tests + manually tested this a bunch so I think I've wrapped my head around the problem and the solution is correct.

Will merge and release once tests pass.

[KyleAMathews]

Thanks for reporting the issue! And sorry for the frustration!

[bradennapier]

Sweet! Thank you for working on this so quickly! It's a bit hard to work out the magic Gatsby does but overall everything is pretty well put together! It definitely manages the details very well!

[bradennapier]

Sorry to say now deleting files crashes Gatsby

 DONE  Compiled successfully in 808ms                                                                                                                                                                                                                                   02:32:37

file deleted at /exchange_services.md
error UNHANDLED REJECTION


  TypeError: Cannot read property 'type' of undefined

  - page-query-runner.js:127
    /[gatsby]/src/internal-plugins/query-runner/page-query-runner.js:127:47

  - Array.reduce

  - page-query-runner.js:124 findDirtyIds
   [gatsby]/src/internal-plugins/query-runner/page-query-runner.js:124:13

[m-allanson]

@bradennapier I'm not able to replicate that error, deleting a markdown file gives me output like:

info file deleted at /Users/m-allanson/gatsby/examples/using-remark/src/pages/2017-03-21---copy-linked-files-intercepting-local-links/index.md

 WAIT  Compiling...                                                                                             14:18:17

 DONE  Compiled successfully in 349ms                                                                           14:18:18

 WAIT  Compiling...                                                                                             14:18:18

 DONE  Compiled successfully in 121ms

Could you try out the Using remark example and see if you still get that unhandled rejection error? You'll need to upgrade it to use the latest version of Gatsby.