Backlinks don't update immediately

[seriesof]

Experiencing some strange behaviour with backlinks. They don't update immediately (and hence can't be renamed until they do), and the wiki-graph-find-backlings claims the file is not being linked to.

Expected behaviour

Create link, follow link, write something and save. Rename via wiki-page-rename and have both file and links to it be updated

Observed behaviour

Create link, follow link, write something and save. Rename via wiki-page-rename. The file changes name but the links are not updated, and the parent links now point to an empty page.

Existing wiki-files eventually get backlinks over time. I have observed from a minute to an hour for this to happen, not sure what is causing it (or rather what is stopping the backlings for updating in the first place.) Both index.md and the linked files are saved. Wiki-graph-related in the index-file shows that it links to the new non-connected page, but going to the page and pressing wiki-graph-related does not show any connection back to the index.

Another possibly related weirdness: wiki-graph-find-backlinks often shows strange result once the backlings are finally connected. For example showing a page being linked to by another page that isn't actually linking to it. Or multiple entries of index, or multiple entries of itself, or a combination. This behaviour is not consistent though (and it works properly more often than not). Sometimes these entries are normal for all files, sometimes not. I have not found any pattern.

vimrc contains:

let g:wiki_root = '/home/.../doc/wiki'
let g:wiki_write_on_nav = v:true

Tried with an empty vimrc with only these entries, to no avail.

My wiki is very plain, just an index.md that links to a couple pages in an hierarchical fashion. Nothing complex. Outside of these backlink issues, the software act as expected so the installation itself seems fine. The journal is working.

System:

• Vim: 9.1
• Distro: Fedora 39

Output of vim --version:

Huge version without GUI.  Features included (+) or not (-):
+acl               +file_in_path      +mouse_urxvt       -tag_any_white
+arabic            +find_in_path      +mouse_xterm       -tcl
+autocmd           +float             +multi_byte        +termguicolors
+autochdir         +folding           +multi_lang        +terminal
-autoservername    -footer            -mzscheme          +terminfo
-balloon_eval      +fork()            +netbeans_intg     +termresponse
+balloon_eval_term +gettext           +num64             +textobjects
-browse            -hangul_input      +packages          +textprop
++builtin_terms    +iconv             +path_extra        +timers
+byte_offset       +insert_expand     +perl/dyn          +title
+channel           +ipv6              +persistent_undo   -toolbar
+cindent           +job               +popupwin          +user_commands
-clientserver      +jumplist          +postscript        +vartabs
-clipboard         +keymap            +printer           +vertsplit
+cmdline_compl     +lambda            +profile           +vim9script
+cmdline_hist      +langmap           -python            +viminfo
+cmdline_info      +libcall           +python3/dyn       +virtualedit
+comments          +linebreak         +quickfix          +visual
+conceal           +lispindent        +reltime           +visualextra
+cryptv            +listcmds          +rightleft         +vreplace
+cscope            +localmap          +ruby/dyn          +wildignore
+cursorbind        +lua/dyn           +scrollbind        +wildmenu
+cursorshape       +menu              +signs             +windows
+dialog_con        +mksession         +smartindent       +writebackup
+diff              +modify_fname      +sodium            -X11
+digraphs          +mouse             -sound             +xattr
-dnd               -mouseshape        +spell             -xfontset
-ebcdic            +mouse_dec         +startuptime       -xim
+emacs_tags        +mouse_gpm         +statusline        -xpm
+eval              -mouse_jsbterm     -sun_workshop      -xsmp
+ex_extra          +mouse_netterm     +syntax            -xterm_clipboard
+extra_search      +mouse_sgr         +tag_binary        -xterm_save
-farsi             -mouse_sysmouse    -tag_old_static    
   system vimrc file: "/etc/vimrc"
     user vimrc file: "$HOME/.vimrc"
 2nd user vimrc file: "~/.vim/vimrc"
      user exrc file: "$HOME/.exrc"
       defaults file: "$VIMRUNTIME/defaults.vim"
  fall-back for $VIM: "/usr/share/vim"
Compilation: gcc -c -I. -Iproto -DHAVE_CONFIG_H -O2 -flto=auto -ffat-lto-objects -fexceptions -g -grecord-gcc-switches -pipe -Wall -Wno-complain-wrong-lang -Werror=format-security -Wp,-D_GLIBCXX_ASSERTIONS -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -fstack-protector-strong -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -fno-omit-frame-pointer -mno-omit-leaf-frame-pointer -D_GNU_SOURCE -D_FILE_OFFSET_BITS=64 -DSYS_VIMRC_FILE=/etc/vimrc -D_REENTRANT -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=1 
Linking: gcc -Wl,--enable-new-dtags -Wl,-z,relro -Wl,--as-needed -Wl,-z,now -specs=/usr/lib/rpm/redhat/redhat-hardened-ld -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -Wl,--build-id=sha1 -Wl,-z,relro -Wl,--as-needed -Wl,-z,now -specs=/usr/lib/rpm/redhat/redhat-hardened-ld -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -Wl,--build-id=sha1 -specs=/usr/lib/rpm/redhat/redhat-package-notes -L/usr/local/lib -o vim -lm -lselinux -lncurses -lsodium -lacl -lattr -lgpm 

[lervag]

Experiencing some strange behaviour with backlinks. They don't update immediately (and hence can't be renamed until they do), and the wiki-graph-find-backlings claims the file is not being linked to.

I don't have this problem, as far as I know. One possible problem here is the caching mechanism. You could try turning it off with let g:wiki_cache_persistent = v:false.

Expected behaviour

Create link, follow link, write something and save. Rename via wiki-page-rename and have both file and links to it be updated

Observed behaviour

Create link, follow link, write something and save. Rename via wiki-page-rename. The file changes name but the links are not updated, and the parent links now point to an empty page.

Does it work if you do this:

1. Create link
2. save file (added step)
3. follow link
4. write something and save
5. Rename via wiki-page-rename

Another possibly related weirdness: wiki-graph-find-backlinks often shows strange result once the backlings are finally connected. For example showing a page being linked to by another page that isn't actually linking to it. Or multiple entries of index, or multiple entries of itself, or a combination. This behaviour is not consistent though (and it works properly more often than not). Sometimes these entries are normal for all files, sometimes not. I have not found any pattern.

Yes, that is weird - and again, I can't reproduce it. It seems this happens sometimes on your end? If you are able to make a reproducible example, then it would be very much easier to address.

My wiki is very plain, just an index.md that links to a couple pages in an hierarchical fashion. Nothing complex. Outside of these backlink issues, the software act as expected so the installation itself seems fine. The journal is working.

I think that it should be possible to find a good solution to this. Either there is something strange in your config - probably not, if you were able to properly test with a minimal example. Or there is a bug, which is not impossible!

[seriesof]

I thought the 'let g:wiki_cache_persistent = v:false' fixed it just now, because suddenly the backlink worked instantly first try, and i could rename the file and everything updated correctly. But then i created another link and it didn't work. And now it doesn't work at all, just as before. So i guess it was just a coincidence. It really seem quite random to me.

Would love to be able to present you a reproducible example but the pattern seem very erratic unfortunately, i have tried so many things to see if i can trigger some consistent behaviour, but not been able to find anything.
I did try to save before following the link also, but no results from that either.

[lervag]

I thought the 'let g:wiki_cache_persistent = v:false' fixed it just now, because suddenly the backlink worked instantly first try, and i could rename the file and everything updated correctly. But then i created another link and it didn't work. And now it doesn't work at all, just as before. So i guess it was just a coincidence. It really seem quite random to me.

Ok; does this mean that your listed steps to reproduce this only fails sometimes? I.e. this:

Create link, follow link, write something and save. Rename via wiki-page-rename …

If these steps always fails for you, then you do have a reproducible example that we can work with.

I did try to save before following the link also, but no results from that either.

"no results" - does that mean that it fails or that it is erratic?

[lervag]

Notice, my next suggested course of action is to create a proper minimal example. I.e., something like this:

1. Create a temporary file init.vim e.g. in your $HOME. Use the following, but ensure that the path to the wiki.vim plugin is correct and that the wiki_root path is correct.

   set nocompatible
   set runtimepath^=~/.vim/plugged/wiki.vim
   filetype plugin indent on
   syntax enable
   
   let g:wiki_root = '/home/.../doc/wiki'
   let g:wiki_write_on_nav = v:true

2. Start Vim with this config and with all other config disabled:

   cd $HOME
   vim --clean -u init.vim

3. Now open your wiki with :WikiIndex.

4. Create a link in the index.

5. Follow the link.

6. Add some text to the new page and save with :w.

7. Rename file with :WikiPageRename.

[seriesof]

Sorry if my language was vague.

Yes, my listed steps only fail sometimes, not consistently.
It will very likely fail, but not always.
"No results" just meant that it acts as before.

To phrase it differently:

• Backlinks very rarely works immediately (it has happened a few times only)
• Backlinks rarely works within a few minutes (though slightly more likely)
• Backlinks usually works within an hour or two of creating the link.

It never always works. It never always fails. It is likely to fail. The chance to fail seem to decrease with time, though without any obvious pattern or predictability.

I followed your steps precisely to create a proper minimal example and it is still not working. However, it seemed to me to increase the chance of backlinks working by a lot - i was suddenly able to create three links in a row and immediately rename them and have the links updated, but then it just stopped working again. I tried to create a new directory and start over, and the first link also worked immediately, then nothing worked immediately after that. I even tried to create yet another directory and start over to see if 'it always works once the first time in a new directory' could be a predictable behavior we could latch on to, but nope! Not even that was predictable.

This increase in chance of immediately working i thought i observed from the minimal example could however just be the randomness making me assign meaning when there is none, but i mention it in case it helps. I understand that all this makes it hard for you to pinpoint where the issue might be.

[lervag]

I followed your steps precisely to create a proper minimal example and it is still not working.

Ok, then let's try a new variant. The goal is to create a simple and reproducible setup that reproduces your problem in an isolated manner.

0. Create a test directory:

   mkdir /home/USER/wiki-test
   cd /home/USER/wiki-test
   touch index.md

1. Add some initial content to index.md. Like "Hello world".

2. Create init.vim. Use the following, but as before, ensure that the path to the wiki.vim plugin is correct.

   set nocompatible
   set runtimepath^=~/.vim/plugged/wiki.vim
   filetype plugin indent on
   syntax enable
   
   let g:wiki_root = '/home/USER/wiki-test'
   let g:wiki_write_on_nav = v:true
   let g:wiki_cache_root = '/home/USER/wiki-test'
   let g:wiki_cache_persistent = v:true

3. Start Vim with this config and with all other config disabled:

   cd /home/USER/wiki-test
   vim --clean -u init.vim

4. Now open your wiki with :WikiIndex. It should open the new index file.

5. Create a link in the index.

6. Follow the link.

7. Add some text to the new page and save with :w.

8. Rename file with :WikiPageRename.

Now, if it still fails:

• What is the contents of the cache file(s) generated in the test directyory?
• Does it work if you change the g:wiki_cache_persistent to v:false?

[seriesof]

I tried this setup multiple times:
The links i create from index.md now seem to work (8/8) using this setup (i can rename and update them immediately when generated from index.md.)
None of the links created within another link has worked (0/8).

There is a possibility that this is the same behavior as the first 'minimal setup' you proposed, and i just failed to see how it consistently worked in index but not from any other pages. In my original setup however, i could not get the links to work even when created from index.md.

Wiki_cache_persistent set to v:false does not seem to make any difference.

This is the output of the json files from the latest attempt:

capture:json

{"uname":["Linux"],"__validate":"cache_v2"}

links-in....json:

{
  "/home/.../wiki-test/renamedlink1.md": [
    {
      "filename_from": "/home/.../wiki-test/index.md",
      "lnum": 3,
      "col": 1,
      "content": "[testlink1](testlink1.md)",
      "filename_to": "/home/.../wiki-test/testlink1.md",
      "text": "testlink1",
      "anchor": "#"
    },
    {
      "filename_from": "/home/.../wiki-test/index.md",
      "lnum": 3,
      "col": 1,
      "content": "[testlink1](renamedlink1.md)",
      "filename_to": "/home/.../wiki-test/renamedlink1.md",
      "text": "testlink1",
      "anchor": "#"
    }
  ],
  "/home/.../wiki-test/index.md": [],
  "__validate": "cache_v2",
  "/home/.../wiki-test/renamedlink2.md": [
    {
      "filename_from": "/home/.../wiki-test/index.md",
      "lnum": 6,
      "col": 1,
      "content": "[testlink2](testlink2.md)",
      "filename_to": "/home/.../wiki-test/testlink2.md",
      "text": "testlink2",
      "anchor": "#"
    },
    {
      "filename_from": "/home/.../wiki-test/index.md",
      "lnum": 6,
      "col": 1,
      "content": "[testlink2](renamedlink2.md)",
      "filename_to": "/home/.../wiki-test/renamedlink2.md",
      "text": "testlink2",
      "anchor": "#"
    }
  ],
  "/home/.../wiki-test/testlink3.md": []
}

links-out:

{
  "/home/.../wiki-test/renamedlink1.md": {
    "ftime": 1714133513,
    "links": []
  },
  "/home/.../wiki-test/index.md": {
    "ftime": 1714133573,
    "links": [
      {
        "filename_from": "/home/.../wiki-test/index.md",
        "lnum": 3,
        "col": 1,
        "filename_to": "/home/.../wiki-test/renamedlink1.md",
        "anchor": "#",
        "text": "testlink1",
        "content": "[testlink1](renamedlink1.md)"
      },
      {
        "filename_from": "/home/.../wiki-test/index.md",
        "lnum": 6,
        "col": 1,
        "filename_to": "/home/.../wiki-test/renamedlink2.md",
        "anchor": "#",
        "text": "testlink2",
        "content": "[testlink2](renamedlink2.md)"
      }
    ]
  }
}

tags.json:

{
  "/home/.../wiki-test/renamedlink1.md": {
    "ftime": 1714133587,
    "tags": []
  },
  "/home/.../wiki-test/index.md": {
    "ftime": 1714133573,
    "tags": []
  },
  "/home/.../wiki-test/renamedlink3.md": {
    "ftime": 1714133590,
    "tags": []
  },
  "/home/.../wiki-test/renamedlink2.md": {
    "ftime": 1714133554,
    "tags": []
  }
}

In this last attempt, the testlink3.md/renamedlink3.md (which was not created from index but from within renamedlink2.md) didnt work, while testlink1.md/renamedlink1.md and testlink2.md/renamedlink2.md worked correctly, as they were created in the index.md.

[lervag]

Ok, please slow down. You are doing much more than I'm asking and it becomes confusing and unclear to me.

None of the links created within another link has worked (0/8).

Can you be very specific about what you are doing now - what are the specific steps to reproduce this?

[seriesof]

I'll be specific. These are the precise steps i did:

0. Create a test directory:

   mkdir /home/USER/wiki-test
   cd /home/USER/wiki-test
   touch index.md

1. Add some initial content to index.md. Like "Hello world".

2. Create init.vim. Use the following, but as before, ensure that the path to the wiki.vim plugin is correct.

   set nocompatible
   set runtimepath^=~/.vim/plugged/wiki.vim
   filetype plugin indent on
   syntax enable
   
   let g:wiki_root = '/home/USER/wiki-test'
   let g:wiki_write_on_nav = v:true
   let g:wiki_cache_root = '/home/USER/wiki-test'
   let g:wiki_cache_persistent = v:true

3. Start Vim with this config and with all other config disabled:

   cd /home/USER/wiki-test
   vim --clean -u init.vim

4. Now open your wiki with :WikiIndex. It should open the new index file.

5. Create a link in the index.

6. Follow the link.

7. Add some text to the new page and save with :w.

8. Rename file with :WikiPageRename.

---

So far, this is what you asked for in your previous post. Using only these steps, the backlink worked.

But since i was not convinced the issue was solved, i added another link using these steps:

9. Create a link from within the page i created at step 5.

10. Follow the link

11. Add some text to the new page and save with :w.

12. Rename file with :WikiPageRename

---

The backlink did not work now. To see if the backlinks worked when created in index.md again, i did these steps:

13. Create a link in the index.

14. Follow the link

15. Add some text to the new page and save with :w.

16. Rename file with :WikiPageRename

The backlink worked now.

I did these exact steps 8 times from scratch to help eliminate chance.

I guess I should have given you the result using only the steps you provided without adding my own steps to the mix. Sorry about that!

Hope this is more clear.

[lervag]

Hope this is more clear.

Thanks, yes, much better! I still have questions, so please be patient with me!

I'll be specific. These are the precise steps i did: …

So far, this is what you asked for in your previous post. Using only these steps, the backlink worked.

Great, thanks.

But since i was not convinced the issue was solved, i added another link using these steps:

9. Create a link from within the page i created at step 5.

And this is after step 8, right? I know I'm being very pedantic, but: can you give me the list of precise steps you are doing after step 8 until and including step 9? I assume it is something like this:

1. :WikiIndex → to verify that the link in the index was changed.
2. Follow the changed linked to get back to the new page.
3. Create a new link from this page. (this is step 9)

Here I'm after the commands and mappings you are using in Vim with total precision.

10. Follow the link

11. Add some text to the new page and save with :w.

12. Rename file with :WikiPageRename

The backlink did not work now.

Can you explain what this means, exactly?

I guess I should have given you the result using only the steps you provided without adding my own steps to the mix. Sorry about that!

Yes, or, the reason I was blunt about this is because you are sort of moving too fast and not providing the details regarding i) exactly what you are doing, and ii) exactly what is happening.

Again, I know I am pedantic, but it is because I am trying the best I can to reach a position from where I can understand what is happening.

[lervag]

But ok, I can finally reproduce. Thanks!

[lervag]

I can reproduce this also with Neovim. It seems that, to reproduce, I need to make two "layers" of new pages. I.e., first create one new page. Renaming this one is fine. But if I add one more, then it won't work.

[lervag]

I think I have what I need to look into this now. I'll see what I can do about it - I think it should be fixed this weekend.

[lervag]

Ok, I've pushed a first attempt at fixing this. Can you please update and test?

The problem was related to the caching. The graph feature is very dependent on caching to work well - we need to avoid doing more work than necessary. However, the cache prevented us from picking up the change in a file after you added a new link. To fix it, I needed to mark the old page as "changed" so that it will be parsed again while renaming the page.