Questions: Converting from Obsidian

[thw26]

I've been trying to switch off of Obsidian for a while, because the CLI is just that great and Obsidian keeps freezing when larger files (>3000 words) are open. This is the only offline software I've found that comes close without requiring having to learn emacs or waiting for a zim update (kudos!).

I've read some issues that have been opened with other folks looking to roll off of Obsidian and have noted that you have not used it. I'm going to attempt to explain what I can with Obsidian for what I think is relevant to my questions.

Some Background

When I compare my usage of Obsidian to wiki.vim, there are a few "costs" to switching:

1. Database of links.
   • Whether you have a flat file structure or if you have a hierarchy of files, Obsidian tracks all of the file names within your "vault" (their term for essentially a wiki root; they have an fzf type of file viewer to find and open files within that root).
   • Let's say I have a file foo.md , and I want to link to or edit the file foo.md. No matter what my current working directory is, if I tell Obsidian to open the file foo or to link to it, then Obsidian will open/link to foo.md.¹
   • Thus if I create a link in /A/bar.md or /B/bar.md to foo.md (without specifying the directory) Obsidian automatically goes to foo.md. For simplicity's sake, I would call this one file of name foo.md per hierarchy. These links are written in a wiki link syntax of [[foo]].
   • If there are multiple foo.md files within the wiki root (e.g., index.md), Obsidian appends the fewest number of directories required to differentiate the two.
   • The obvious difference is that wiki.vim would see these as being two different foo.md files, one under A and one under B unless I append a path in the file link.

2. Being able to tab complete filenames as you type.

   • This really helps to speed things up when filenames are long or if you aren't quite sure where a filename is located.
   • This would, in my opinion, even let me obviate the need for the one file per hierarchy paradigm noted above—as then I wouldn't have to remember where I stored something!
   • Because I have multiple files of the same name (i.e., not just index.md), this prevents me from using a flat hierarchy.
   • Obsidian handles this by opening an fzf-type of file finder within the [[]] when typing.

3. Links to Headings

   • If foo.md has a heading # Xyz, you can link to it with [[foo#Xyz]]
   • This is nice for not having to repeat content or specifying a part of a file to link to

4. Seamless automatic link updating (related to the above)

   • This is probably made possible by the database of filenames in the wiki root.
   • When I tried to use the wiki.vim link update function, I must have done something wrong as I lost data.

Question (or Issue)

The reason I am opening this issue up is to ask if:

1. Obsidian-style relative links (one file of name foo.md per hierarchy) are possible;
2. if tab completing filenames within wiki.vim could be possible in the future (inline fzf and complete?);
3. and if links to headings could be possible in the future.

(And help with the automatic link updating! haha)

I saw the 0.6 update and was wondering if the link_url_parser might be able to accomplish the relative-style linking.

I ask if the other two are possible because I can either then learn vim script or know if this is something I have to be okay losing, and thus update my notes.

Thank you for making this software publicly available!

Footnotes

1. Obsidian defaults to markdown and hides the file's extension by default as it assumes all are markdown; plugins are required for other files, except pictures and PDFs can be viewed within the program. ↩

[lervag]

This is the only offline software I've found that comes close without requiring having to learn emacs or waiting for a zim update (kudos!).

Thanks! :)

I've read some issues that have been opened with other folks looking to roll off of Obsidian and have noted that you have not used it.

Correct.

When I compare my usage of Obsidian to wiki.vim, there are a few "costs" to switching:

1. Database of links

This is interesting. Thanks for the explanations, it makes it easier to understand some of the more special Obsidian ideas.

wiki.vim is clearly much "simpler". There is no database; links are just resolved based on deterministic and usually quite simple rules. I get the impression that you already understand these things well.

2. Being able to tab complete filenames as you type.

wiki.vim does complete filenames. But it works by looking at targets in the path relative to the current (or rooted to g:wiki_root if you prepend a /).

• Because I have multiple files of the same name (i.e., not just index.md), this prevents me from using a flat hierarchy.

I've found that hierarchies are mostly "bloat". They do not help me categorize; indeed, often they do the opposite. This is because, for instance, many topics are not uniquely defined by a single category. I also found I sometimes used a "wrong" category. By using hierarchies, I ended up spending a lot of time fixing problems caused by the implied categories. Instead, today, I am basically 100% flat. For concepts that have similar name, I just use a longer name to provide context. For instance, "Pandas" may refer to the python package Pandas, but also the animal. In this case, I have a file called "python Pandas" and one called simply "Pandas".

If you did not already read it, then I believe you may find my thoughts on how I write notes interesting, see :help wiki-intro-guide.

3. Links to Headings

This also works in wiki.vim.

4. Seamless automatic link updating (related to the above)
   • …
   • When I tried to use the wiki.vim link update function, I must have done something wrong as I lost data.

It is unclear what you mean here. wiki.vim provides a command WikiPageRename to rename a page, which should also update all links that target the page. This sounds like what you refer to, but I'm not sure.

Question (or Issue)

1. Obsidian-style relative links (one file of name foo.md per hierarchy) are possible;

No, they are not possible now. We could discuss this further, I might be willing to consider trying to implement it or sOmething similar.

2. if tab completing filenames within wiki.vim could be possible in the future (inline fzf and complete?);

Completion works fine already, please see :help wiki-completion. However, it will not with the Obsidian link concept, you need to use relative paths in general.

If I were to consider the more "volatile" link concept from Obsidian, then I believe it would also be prudent to consider a similar type of completion support. But the first step would be to make the links work; corresponding completion would need to be implemented afterwards.

3. and if links to headings could be possible in the future.

This is already possible.

I saw the 0.6 update and was wondering if the link_url_parser might be able to accomplish the relative-style linking.

The most relevant option right now is the g:wiki_resolver option. It allows to specify a function that is used to resolve wiki-schemed link URLs.

I believe things are going in a more generalized direction that can be useful to you and others. The link_url_parser is one such step, but this is specifically for configuring the journal index links.

There are currently two recent and more specific issues opened that address similar concerns:

• Making wiki.vim support multi-filetype links #296: More generalized filetype options (perhaps not that relevant, but still).
• Feature request: apply wiki resolver to all supported link types #297: More general wiki resolver. This is relevant, because it is about how we resolve link targets.

I ask if the other two are possible because I can either then learn vim script or know if this is something I have to be okay losing, and thus update my notes.

First: everything is possible :)

But I'm quite limited on my time, so even if I were to address these things, it may take a while and I won't make any promises on when I deliver. Help from the community is always appreciated! If this means you will have to update notes or not, that I don't know.

My first priorities of wiki.vim are to make things customizable and robust.

Thank you for making this software publicly available!

Glad you find it useful/interesting!

[lervag]

I'll close this issue. Feel free to continue the discussion, ask more questions, or to open other issues.

[thw26]

Thank you for taking the time to respond!

wiki.vim is clearly much "simpler". There is no database; links are just resolved based on deterministic and usually quite simple rules.

This was my understanding of how wiki.vim works. No complaints thus far, just bouncing ideas off really.

wiki.vim does complete filenames. But it works by looking at targets in the path relative to the current (or rooted to g:wiki_root if you prepend a /).
…
Completion works fine already, please see :help wiki-completion. However, it will not with the Obsidian link concept, you need to use relative paths in general.
If I were to consider the more "volatile" link concept from Obsidian, then I believe it would also be prudent to consider a similar type of completion support. But the first step would be to make the links work; corresponding completion would need to be implemented afterwards.

I will review the docs this week hopefully and probably attempt another hard switch off from Obsidian. If I understand right, linking in the volatile manner would require traversing a directory to find all possible matching endpoints, whereas wiki.vim only finds the next dir/file down, repeating, until a target file is reached, correct?

It is unclear what you mean here. wiki.vim provides a command WikiPageRename to rename a page, which should also update all links that target the page. This sounds like what you refer to, but I'm not sure.

This command is what I tried using. I'm trying to recall exactly what happened—this was a while ago now—but at the very least I ran the command and the command did not update all links that target the page. (I may have lost data, too, but it may be completely unrelated to the use of the command and/or may have happened at the same time as my failed use of the command.)

3. and if links to headings could be possible in the future.
   This is already possible.

Thanks for correcting me! I will have to play with it some more.

I've found that hierarchies are mostly "bloat". They do not help me categorize; indeed, often they do the opposite. This is because, for instance, many topics are not uniquely defined by a single category. I also found I sometimes used a "wrong" category. By using hierarchies, I ended up spending a lot of time fixing problems caused by the implied categories. Instead, today, I am basically 100% flat. For concepts that have similar name, I just use a longer name to provide context. For instance, "Pandas" may refer to the python package Pandas, but also the animal. In this case, I have a file called "python Pandas" and one called simply "Pandas".

I have migrated towards an as-flat-as-possible system by using as few directories as possible, aiming for 2 or less, though I can think of a few clunky places where I am using 4 (e.g., a number of my files are quotes with reference info; those go in a quotes directory and then get tagged based on content), and also by including tags in my markdown files (not as easy for ODT and ODS docs!).

If you did not already read it, then I believe you may find my thoughts on how I write notes interesting, see :help wiki-intro-guide.

I must confess, I skipped to the commands and syntax in my first go-around. I will give your thoughts a read perhaps in the next week.

But I'm quite limited on my time, so even if I were to address these things, it may take a while and I won't make any promises on when I deliver. Help from the community is always appreciated!

I fully understand! No demands here. :)

My hope is to be able to contribute! I will see if I can start learning vimscript by reviewing this plugin's codebase and playing around with the code sometime within the next month, time permitting.

I will look over #296, #297, and the others branching off from them, too, to make sure I am familiar with what you're saying. I will make sure to ask questions as they arise!

[lervag]

…, whereas wiki.vim only finds the next dir/file down, repeating, until a target file is reached, correct?

I'm not fully sure I understand properly. wiki.vim will resolve the link as if it is relative to the current file. So, a link to ../My note would find the note My note.md in the parent directory.

It is unclear what you mean here. wiki.vim provides a command WikiPageRename to rename a page, which should also update all links that target the page. This sounds like what you refer to, but I'm not sure.

This command is what I tried using. I'm trying to recall exactly what happened—this was a while ago now—but at the very least I ran the command and the command did not update all links that target the page.

Based on the info you've given me on Obsidian, I may be less surprised that this does not work as you expect it to. The underlying assumption on how the links should work is different in wiki.vim, thus the mechanism for collecting incoming links will not necessarily work quite as you expect if you come from Obsidian.

I have migrated towards an as-flat-as-possible system by using as few directories as possible, aiming for 2 or less, though I can think of a few clunky places where I am using 4 (e.g., a number of my files are quotes with reference info; those go in a quotes directory and then get tagged based on content), and also by including tags in my markdown files (not as easy for ODT and ODS docs!).

Do I understand that you would say that /file.md is at level 0, /other/file.md is level 1, and /yet/another/file.tex is level 2 here? I would still call this very non flat. When I say flat, I mean generally void of subdirectories. I.e., in my wiki, >99% of the files are in the root directory (if I ignore my journal, which has its own directory).

If you did not already read it, then I believe you may find my thoughts on how I write notes interesting, see :help wiki-intro-guide.

I must confess, I skipped to the commands and syntax in my first go-around.

The wiki-intro-guide is not about commands and syntax, it is "an essay" about how my process for taking and maintaining notes work. wiki.vim is the tool I use, but I believe the essay is still relevant if you use any other tool.

My hope is to be able to contribute! I will see if I can start learning vimscript by reviewing this plugin's codebase and playing around with the code sometime within the next month, time permitting.

That would be more than welcome! I am currently trying to rewrite the core handling of links and URLs. I've already gotten quite far and I'm now working with URL handling, which is the last part of the puzzle for me. I believe, when this is finished, it should be quite easy to implement more link resolve concepts to mimic Obsidian behaviour, for instance. And things like that would be something I would be very happy to have contributions for.

[thw26]

I'm not fully sure I understand properly. wiki.vim will resolve the link as if it is relative to the current file. So, a link to ../My note would find the note My note.md in the parent directory.

I think WikiFzfPages covers most of I'm trying to communicate.

Based on the info you've given me on Obsidian, I may be less surprised that this does not work as you expect it to. The underlying assumption on how the links should work is different in wiki.vim, thus the mechanism for collecting incoming links will not necessarily work quite as you expect if you come from Obsidian.

Okay. I will have to play with this and see if I can figure out what I need to do differently.

Do I understand that you would say that /file.md is at level 0, /other/file.md is level 1, and /yet/another/file.tex is level 2 here? I would still call this very non flat. When I say flat, I mean generally void of subdirectories. I.e., in my wiki, >99% of the files are in the root directory (if I ignore my journal, which has its own directory).

As my notetaking has evolved, it has moved from a hyper-nested format to using directories for organizing types of information and projects, not actual content. So I have at present several hundred files which are simply quotes by various authors with metadata and some comments on the quote; all these quotes get shoved in a Quotes subdirectory. I have several hundred files which contain outlines of books; these outlines all get shoved in an Outlines subdirectory. I essentially imagine each of these subdirectories as its own root directory. For instance, I have an outline of a book where I have each chapter in its own file (e.g., author_book_chapter_1.md, author_book_chapter_2.md); rather than each file getting shoved in a subdir by the book's name (i.e., author_book/), it is left in the Outlines directory because its content type is an outline and it is backlinked to the parent file.

I have some files which belong to projects (e.g., Class_1) where its files are stored in the subdir (e.g., Class_1_Lectures, Class_1_Student_Notes), but this belonged to a former manner of my organizing and I've been trying to think of a way to flatten this hierarchy.

The wiki-intro-guide is not about commands and syntax, it is "an essay" about how my process for taking and maintaining notes work. wiki.vim is the tool I use, but I believe the essay is still relevant if you use any other tool.

Yes! That's what I meant. :) I didn't read your essay; I skipped the essay to read how to use the commands. But I will go back and read it and perhaps learn a thing or two!

That would be more than welcome! I am currently trying to rewrite the core handling of links and URLs. I've already gotten quite far and I'm now working with URL handling, which is the last part of the puzzle for me. I believe, when this is finished, it should be quite easy to implement more link resolve concepts to mimic Obsidian behaviour, for instance. And things like that would be something I would be very happy to have contributions for.

I'm in the middle of a move, but I can squeeze things in here and there. Hopefully I can jump in sooner rather than later.

[lervag]

Based on the info you've given me on Obsidian, I may be less surprised that this does not work as you expect it to. The underlying assumption on how the links should work is different in wiki.vim, thus the mechanism for collecting incoming links will not necessarily work quite as you expect if you come from Obsidian.

Okay. I will have to play with this and see if I can figure out what I need to do differently.

:thumbs_up:

As my notetaking has evolved, it has moved from a hyper-nested format to using directories for organizing types of information and projects, …

Even though I prefer the flat hierarchies, I don't disagree that some things can be useful to hide away in subdirectories. wiki.vim should work equally well regardless - or, at least I intend for it to do!

I have some files which belong to projects (e.g., Class_1) where its files are stored in the subdir (e.g., Class_1_Lectures, Class_1_Student_Notes), but this belonged to a former manner of my organizing and I've been trying to think of a way to flatten this hierarchy.

This is an example where I think it is easy to categorize wrongly. You should distinguis between content that is specific to the class itself, e.g. information about the class content, educator/lecturer, the fellow students, groups, perhaps assignements and so on. But the actual topics is better to write in dedicated notes outside of the class notes, because several classes will for sure deal with the same matter at hand.

The wiki-intro-guide is not about commands and syntax, it is "an essay" about how my process for taking and maintaining notes work. wiki.vim is the tool I use, but I believe the essay is still relevant if you use any other tool.

Yes! That's what I meant. :) I didn't read your essay; I skipped the essay to read how to use the commands. But I will go back and read it and perhaps learn a thing or two!

Cool. :)

That would be more than welcome! I am currently trying to rewrite the core handling of links and URLs. I've already gotten quite far and I'm now working with URL handling, which is the last part of the puzzle for me. I believe, when this is finished, it should be quite easy to implement more link resolve concepts to mimic Obsidian behaviour, for instance. And things like that would be something I would be very happy to have contributions for.

I'm in the middle of a move, but I can squeeze things in here and there. Hopefully I can jump in sooner rather than later.

Notice that I'm currently in the middle of a major refactoring of wiki.vim and its link and url handling. I'm quite close to finishing this. I believe it will make the code much easier to reason about and to further improve and expand. See e.g. #302 and #304.

If you do have some spare time, then I would be happy to have someone test the work in #304. I would also be happy to get feedback on the new option(s) and on which parts of the docs should be improved to clarify important concepts.

[thw26]

This is an example where I think it is easy to categorize wrongly. You should distinguis between content that is specific to the class itself, e.g. information about the class content, educator/lecturer, the fellow students, groups, perhaps assignements and so on. But the actual topics is better to write in dedicated notes outside of the class notes, because several classes will for sure deal with the same matter at hand.

All good points! I'll have to munch on that as I will have to reformat several other project directories as well.

Notice that I'm currently in the middle of a major refactoring of wiki.vim and its link and url handling. I'm quite close to finishing this. I believe it will make the code much easier to reason about and to further improve and expand. See e.g. #302 and #304.
If you do have some spare time, then I would be happy to have someone test the work in #304. I would also be happy to get feedback on the new option(s) and on which parts of the docs should be improved to clarify important concepts.

I'll put that on my next to-do!

[lervag]

This is an example where I think it is easy to categorize wrongly. You should distinguis between content that is specific to the class itself, e.g. information about the class content, educator/lecturer, the fellow students, groups, perhaps assignements and so on. But the actual topics is better to write in dedicated notes outside of the class notes, because several classes will for sure deal with the same matter at hand.

All good points! I'll have to munch on that as I will have to reformat several other project directories as well.

I've done my share of "refactoring" on my notes. :)

Notice that I'm currently in the middle of a major refactoring of wiki.vim and its link and url handling. I'm quite close to finishing this. I believe it will make the code much easier to reason about and to further improve and expand. See e.g. #302 and #304.
If you do have some spare time, then I would be happy to have someone test the work in #304. I would also be happy to get feedback on the new option(s) and on which parts of the docs should be improved to clarify important concepts.

I'll put that on my next to-do!

Heh, too late. #304 is merged already :)

But if you want to contribute I will be glad to help find things to do. I think a good start would be to read some of the main code and try and understand it. The autoload/wiki/link.vim and autoload/wiki/url.vim are really the most essential parts. Even though it is Vimscript, which is esoteric, I will still claim that the code should be quite easy to grasp. But it is not (nor will it ever be) perfect, and I will be happy to get suggestions for improving things!

[thw26]

Ah, well, all the better then!

I've just discovered some Neovim plugins I was unaware of, most pertinently https://github.com/renerocksai/telekasten.nvim and https://github.com/epwalsh/obsidian.nvim. Are you familiar with either of these? They seem to implement some of what I had brought up in my first post.

[lervag]

I'm aware of both, but I've not spent any time with them and would not claim to be familiar, no.

[lervag]

They both seem like quite mature and popular projects, so they are probably quite good. If you are already invested in Obsidian, then it is probably a good idea to look closer at obsidian.nvim!

[thw26]

What I've seen so far as I've played with them is that this plugin and the other two each appear to have some functionality I enjoy. I will have to play around with them more.

Your FZF implementation is better ;)

[thw26]

From obsidian.nvim, the main feature it contributes (which I was looking for) is found in this file: autocompletion of wiki filename inline (haven't gotten the last part working smoothly yet)

https://github.com/epwalsh/obsidian.nvim/blob/main/lua/cmp_obsidian.lua

But I think this functionality could be covered by FZF. If the user had the option either to open the file or insert a link to the file, I think that would cover the use case I'm imagining quite nicely. I would think that should be feasible as it is just extending the existing FZF functionality. I might dub it something like :WikiPagesLinkInsert instead of :WikiPagesFollow. Coupled with the existing omnifunction completion

Otherwise it looks like there's a lot of crossover between the projects, each of which are aiming at the same kind of thing with their own twists here or there (e.g., Telekasten has easy insertion and viewing of images).

[lervag]

From obsidian.nvim, the main feature it contributes (which I was looking for) is found in this file: autocompletion of wiki filename inline (haven't gotten the last part working smoothly yet)

Well, wiki.vim also has completion of wiki filenames inline. It is provided as an omnicomplete function and works with cmp.nvim through the omni extension.

The exact method of extracting the relevant files and paths in the expected format may differ from Obsidian, but one of the main points of #304 is to allow the user to implement their own resolver for the wiki scheme.

I don't mind adding and/or improving features, but I want to have specific suggestions with clear specifications (as much as possible). I prefer to implement things after I know how to describe it in the docs. :)

But I think this functionality could be covered by FZF.

Ah, well; I don't think I agree. But perhaps I don't fully understand what you want here. Feel free to open an issue where you try and explain it more concretely. The best explanations are the ones where I understand what you do and what you expect to happen. Such as: "I write [[foo and expect a completion menu to pop up automatically and show completion candidates that begin with the letters foo". (I believe this exact thing should work as expected today.)

If the user had the option either to open the file or insert a link to the file, I think that would cover the use case I'm imagining quite nicely. I would think that should be feasible as it is just extending the existing FZF functionality.

I think I don't agree - the current use of FZF is as a backend to navigation features where we ask to open a specific page with :WikiPages. The command opens a selection engine such as FZF to choose from all pages in the wiki. This is not quite the same as a completion menu.

By the way, I believe there do exist plugins or wrappers that will use FZF for completion. I don't remember where I saw that, but perhaps you find it if you search?

I might dub it something like :WikiPagesLinkInsert instead of :WikiPagesFollow. Coupled with the existing omnifunction completion

Hmm... ok; we could consider something like :WikiPagesInsertLink. A command that inserts a link at the current cursor location after selecting one with the same backend as :WikiPages. If we are still talking about the same thing, then I propose to open a new feature request for this where we can agree on the specs.

Otherwise it looks like there's a lot of crossover between the projects, each of which are aiming at the same kind of thing with their own twists here or there (e.g., Telekasten has easy insertion and viewing of images).

My goal with wiki.vim is to make it robust, flexible and powerful. I want to keep it "small" in the sense that it should not become a filetype plugin. Things that are not related to links and writing notes in a relatively agnostic manner should most likely not be part of the plugin. (I am aware there are a couple of features that actually might be unsuitable given that statement.)