WikiJournalIndex: only wiki text files with any naming in the journal folder

[artkpv]

I want to thank contributors for this plugin and for their work! I use this tool often.

Description

In short, :WikiJournalIndex doesn't create links for files like 2022-12-02-2020_some_name_here.md. I wish :WikiJournalIndex would create links in the current buffer to existing files which is named arbitrary. I create many files in my diary with nvim --nofork $HOME/mydir/notes/$(date +%Y)/$(date +%Y-%m-%d-%H%M).md -- -c WikiEnable -c 'cd ~/mydir/notes'.

Steps:

1. My config as below. I open file ./2022/дневник.md. It contains files like: 2022-12-01-1330_my_name_of_file.md, 2022-03-03.md, etc., as well as images, PDF files.
2. Run :WikiJournalIndex.
3. Result:

# 2022

## 01.Январь

[2022-01-01](/2022/2022-01-01)    // This is wrong link! No such file. There though files like "./2022-01-01-2230.md"
[2022-01-01](/2022/2022-01-01)
[2022-01-01](/2022/2022-01-01)
[2022-01-01](/2022/2022-01-01)
[2022-01-02](/2022/2022-01-02)
[2022-01-02](/2022/2022-01-02)
[2022-01-03](/2022/2022-01-03)
[2022-01-03](/2022/2022-01-03)
[2022-01-03](/2022/2022-01-03)
[2022-01-03](/2022/2022-01-03)
[2022-01-04](/2022/2022-01-04)
[2022-01-04](/2022/2022-01-04)
...

Expected: correct links to existing files.

Minimal working example

/path/to/minimal-working-example-wiki
├── index.wiki
├── 2022
│   └── 2017-11-24-2359_name_here.md
       ├── дневник.md

Config

set nocompatible

let g:wiki_root = '~/mydir/notes/'

let g:wiki_link_extension = ''
let g:wiki_filetypes = ['md', 'wiki']
let g:wiki_link_target_type = 'md'

let g:wiki_month_names = [ '01.Январь', '02.Февраль', '03.Март', '04.Апрель', '05.Май', '06.Июнь', '07.Июль', '08.Август', '09.Сентябрь', '10.Октябрь', '11.Ноябрь', '12.Декабрь' ]

let g:wiki_toc_title = 'Содержание'

let g:wiki_journal = {
    \ 'index_use_journal_scheme': v:false,
    \ 'name': '2022',
    \ 'frequency': 'daily',
    \ 'date_format': {
    \   'daily' : '%Y-%m-%d',
    \ },
    \}

let g:wiki_template_title_week = '# %(year). %(week) неделя'
let g:wiki_template_title_month = '# %(year). %(month-name) месяц'

function! MyWikiBufferInit()
  try
    let $TYPESET_PATH = expand("%:p")
    if $TYPESET_PATH !~# '\v\.(index.md|index.wiki)$'
        "set guifont=Liberation\ Mono:h13
    endif
  catch
  endtry
  set laststatus=0
  set foldcolumn=4
  au! BufWritePost ~/mydir/notes/* !git add "%";git commit -m "Auto commit of %:t." "%"

  nmap <leader>mh :s_\v^((#+) +)?(.*)_#\2 \3_e<CR> :noh<CR>
  nmap <leader>mH :s_\v^#(#* )? *(.*)_\1\2_e<CR> :noh<CR>
  set foldlevel=1
  "CocDisable
endfunction

augroup MyWikiAutocmds
  autocmd!
  autocmd User WikiBufferInitialized call MyWikiBufferInit()
augroup END

let g:wiki_map_text_to_link = 'WikiMapTextToLink'

function WikiMapTextToLink(text) abort
  if a:text !~# '\v\.(md|wiki|txt)$'
    let t = substitute(tolower(a:text), '\v[ \t;:.*?\)\(]+', '_', 'g')
    if getcwd() !~# '\v\/0$|\/0\/'
        return [strftime('%Y-%m-%d-%H-%M') . '_' . t . '.md', a:text]
    else
        return [strftime('%Y%m%d%H%M') . '_' .  t . '.md', a:text]
    endif

  endif
  return [a:text, a:text]
endfunction

let g:wiki_viewer = {'_' : 'xdg-open'}

[lervag]

I want to thank contributors for this plugin and for their work! I use this tool often.

Thanks, I'm glad to hear it!

In short, :WikiJournalIndex doesn't create links for files like 2022-12-02-2020_some_name_here.md. I wish :WikiJournalIndex would create links in the current buffer to existing files which is named arbitrary. I create many files in my diary with nvim --nofork $HOME/mydir/notes/$(date +%Y)/$(date +%Y-%m-%d-%H%M).md -- -c WikiEnable -c 'cd ~/mydir/notes'.

I see. This is not supported right now. :WikiJournalIndex scans the journal directory for files that match the specified date format (g:wiki_journal.date_format). The main code for gathering the files is here:

wiki.vim/autoload/wiki/journal.vim

Lines 223 to 236 in 8aded59

	function! wiki#journal#get_all_nodes(frq, ...) abort " {{{1
	let l:rx = wiki#date#format_to_regex(g:wiki_journal.date_format[a:frq])
	let l:nodes = filter(map(
	\ glob(wiki#paths#s(b:wiki.root_journal . '/**'), 1, 1),
	\ { _, x -> s:path_to_node(x) }),
	\ { _, x -> x =~# l:rx })
	return a:0 > 0
	\ ? uniq(sort(a:1 + l:nodes))
	\ : l:nodes
	endfunction
	" }}}1

Your idea seems to be that you use the specified date_format only for the start of the name, but that we should allow random strings after. Nothing is impossible, of course. The main difficulty here is that we use the date_format settings to parse between a journal filename and the date "names", e.g. from 2023-01-01.md to "1. Январь 2023".

I may consider looking into this and see if it is possible to generalize in a manner that does not break anything in an unexpected way while allowing what you are asking for.

But first: I am curious, do you really find the journal index useful? I personally just search for journal entries with Telescope these days (or FZF, CtrlP, and similar, in earlier days), and at some time a long time ago I realized I never actually used the journal index. I've kept the feature, but I'm really curious how people use it and if it really is useful to anyone...

[artkpv]

Thanks for the comment!

But first: I am curious, do you really find the journal index useful? I personally just search for journal entries with Telescope these days (or FZF, CtrlP, and similar, in earlier days), and at some time a long time ago I realized I never actually used the journal index. I've kept the feature, but I'm really curious how people use it and if it really is useful to anyone...

I keep my dairy to track many things. This is anything that changes as opposed to permanent knowledge, i.e. something like E = mc^2 would go to my wiki files, and my tasks, emotion/mental tracking, projects, book reviews, etc. etc. everything that changes goes into the diary. Actually it is many diaries per year, so parent directories for diaries are years 2022/, 2023/ and for that never changes (ideas) goes into 0/ dir. So the diary becomes pretty messy and has many files (e.g. 750 files for 2022). Headers for notes helps to organize this as well as file names. I use ctrlp.vim for fast switches, and ctrlsf.vim for text search. But to have the bird's eye view it helps to create an index note (per year). Also it helps to be able to move from current note to the next / previous as ordered by date (time) to quickly skim some period of time. Also this structure fits well with version control as the diary doesn't need it as much as the ideas section as our views change on them. Hope it clears things!

[lervag]

Thanks; seems like a nice way to do things. It is one of the things I've realized over the years: most of us do things differently, and that's ok. We shouldn't force solutions; instead, we should aim for our tools to be flexible and useful in different types of workflows.

Ok, so, it would make my life simpler if we can design a much more minimal example. I've tried the following:

❯ tree
.
├── test.vim
└── wiki
    ├── diary
    │   ├── 2017-11-24.md
    │   ├── 2022-12-31.md
    │   ├── 2023-01-01_book-review.md
    │   ├── 2023-01-01.md
    │   └── 2023-01-02.md
    └── index.md

where test.vim has

set nocompatible
set runtimepath^=~/.local/plugged/wiki.vim
filetype plugin indent on
syntax enable

nnoremap q :qall!<cr>

let g:wiki_cache_persistent = 0

let g:wiki_filetypes = ['md']
let g:wiki_write_on_nav = 1
let g:wiki_root = fnamemodify('wiki', ':p')
let g:wiki_link_extension = ''
let g:wiki_link_target_type = 'md'
let g:wiki_journal = {
    \ 'name': 'diary',
    \ 'index_use_journal_scheme': v:false,
    \}

runtime plugin/wiki.vim

silent edit wiki/index.md
WikiJournalIndex

I believe this includes all the important configuration from your initial example. When I run nvim --clean -u test.vim, it will open the outer index.md and fill it with this:

# 2017

## November

[2017-11-24](/diary/2017-11-24)

# 2022

## December

[2022-12-31](/diary/2022-12-31)

# 2023

## January

[2023-01-01](/diary/2023-01-01)
[2023-01-01](/diary/2023-01-01)
[2023-01-02](/diary/2023-01-02)

Here the expected output is similar, except we want something like this at the end:

[2023-01-01](/diary/2023-01-01)
[2023-01-01](/diary/2023-01-01_book-review)
[2023-01-02](/diary/2023-01-02)

This raises some questions both on how to format the link text and on how to parse the filename. The current implementation assumes all diary entries are just dated entries and so it will use the current ISO date for the entry titles. I believe we could do something like this:

1. Either of:

   • Add a "title" format key to the date_format to capture text (e.g. %s). Thus, you would want to say something like date_format.daily = '%Y-%m-%d%s'.
   • Capture the string after the date_format until the extension and assume it is a form of title string. Benefit is we don't need an option.

2. Add a new option g:wiki_journal.index_title_parser: a function that takes an input string from the filename and outputs something you use in the journal index link titles.

So, with 1. and 2. above, we could have e.g.

let g:wiki_journal = {
    \ 'name': 'diary',
    \ 'index_title_parser': { x -> substitute(x, '[_-]', ' ', 'g') },
    \ 'index_use_journal_scheme': v:false,
    \}

and if things are implemented correctly, we would get

[2023-01-01](/diary/2023-01-01)
[2023-01-01 book review](/diary/2023-01-01_book-review)
[2023-01-02](/diary/2023-01-02)

[artkpv]

Thanks for this detailed answer. This project is exceptional for how issues are handled I think. Sorry for this long answer.

It is one of the things I've realized over the years: most of us do things differently, and that's ok. We shouldn't force solutions; instead, we should aim for our tools to be flexible and useful in different types of workflows.

That's good point. I can't agree more with this. The devil in details as always.

Here the expected output is similar, except we want something like this at the end:

[2023-01-01](/diary/2023-01-01)
[2023-01-01](/diary/2023-01-01_book-review)
[2023-01-02](/diary/2023-01-02)

Can we have relative links everywhere option? I mean here it would try to generate relative to current file links. Why: We decouple this file from wiki.vim as only wiki.vim knows where the root dir '/' is. Examples: 1) I use Markor (markdown editor) on my Android device which supports markdown rendering but it doesn't know anything about root directories of the wiki. 2) To render markdown on my Linux I use Firefox plugin 'Markdown Viewer' which also doesn't know about root dirs.

This raises some questions both on how to format the link text and on how to parse the filename.

Can it be more versatile? We have this WikiLinkExtractHeader command. Can we use it here?

The current implementation assumes all diary entries are just dated entries and so it will use the current ISO date for the entry titles

Yes, that couples vim.wiki plugin with the generated wiki or diary. I'm not sure about extracting title via both options presented as it seems to duplicate functionality in WikiLinkExtractHeader.

---

Another functionality that doesn't work for me know is WikiJournalNext and WikiJournalPrev. Both of them moves to non-existent notes: 1) open existing file
2023-01-02-1337.md; 2) Run WikiJournalNext ; 3) Buffer opens for non-existent file named 2023-01-02.md .

[lervag]

Thanks for this detailed answer. This project is exceptional for how issues are handled I think.

Thanks for the kind words!

Here the expected output is similar, except we want something like this at the end:

[2023-01-01](/diary/2023-01-01)
[2023-01-01](/diary/2023-01-01_book-review)
[2023-01-02](/diary/2023-01-02)

Can we have relative links everywhere option?

I think that's a valid request, but I think it is out of "scope" for the current issue. I've opened #265 which I will follow up after resolving this particular issue.

Another functionality that doesn't work for me know is WikiJournalNext and WikiJournalPrev. Both of them moves to non-existent notes: 1) open existing file 2023-01-02-1337.md; 2) Run WikiJournalNext ; 3) Buffer opens for non-existent file named 2023-01-02.md .

Yes, you are right - but again, I think it will be easier to first resolve the present issue. Can you open a new issue for this, preferably with a simple example for how to reproduce?

This raises some questions both on how to format the link text and on how to parse the filename.

Can it be more versatile? We have this WikiLinkExtractHeader command. Can we use it here?

Good idea; perhaps adding a choice to either collect title from first header or from filename is sufficient here?

---

I propose that we separate the journal index-specific options from g:wiki_journal into g:wiki_journal_index and that we implement something that would match the following documentation:

g:wiki_journal_index
  A dictionary for configuring the behaviour of |WikiJournalIndex|. Available
  options are:

    use_journal_scheme
      Whether to use the `journal:` scheme on the links.

    link_title_method
      Method for specifying link text. Can be one of:

        basename:     use basename of dictionary file as title
        custom_func:  allo
        first_header: use first header in file (as with |WikiLinkExtractHeader|)

    link_title_custom_func
      A function with a single input argument (a filename) that is used with link_title_method = `'custom_func'` to specify the link text.

  Default:

    let g:wiki_journal_index = {
          \ 'use_journal_scheme': v:true,
          \ 'title_parser': 'basename'
          \}

In addition to adding the above options, the implementation should fix the issue described earlier and captured by the minimal example in my previous post.

Let me know if you think I've missed something now or confirm that this sounds good. After that, I'll look into getting this fixed and implemented.

[artkpv]

Yes, you are right - but again, I think it will be easier to first resolve the present issue. Can you open a new issue for this, preferably with a simple example for how to reproduce?

Please, see #266

Good idea; perhaps adding a choice to either collect title from first header or from filename is sufficient here?

I actually only make the title for a link from the first header. I've never used filename. I'm not sure about other users.

Let me know if you think I've missed something now or confirm that this sounds good. After that, I'll look into getting this fixed and implemented.

Looks good to me. Thank you

[lervag]

I've pushed an update that I believe may resolve this issue for you. Please update and read the releated docs :help wiki_journal_index. I think this should be what you want, more or less:

let g:wiki_journal_index = {
      \ 'use_journal_scheme': v:false,
      \ 'link_text_parser': { b, d, p -> wiki#toc#get_page_title(p) }
      \}

I would appreciate comments and feedback on this. I believe it is a flexible implementation that should make it quite easy to customize according to your own liking. For instance, you could do this to prepend the date:

let g:wiki_journal_index = {
      \ 'use_journal_scheme': v:false,
      \ 'link_text_parser': { b, d, p ->
      \   d . ': ' . wiki#toc#get_page_title(p) }
      \}

You can also use full functions:

let g:wiki_journal_index = {
      \ 'use_journal_scheme': v:false,
      \}
function! g:wiki_journal_index.link_text_parser(base, date, path)
  let l:title = wiki#toc#get_page_title(a:path)
  if empty(l:title)
    return a:base
  else
    return a:date . ': ' . l:title
  endif
endfunction

[lervag]

Good idea; perhaps adding a choice to either collect title from first header or from filename is sufficient here?

I actually only make the title for a link from the first header. I've never used filename. I'm not sure about other users.

I always have a single page per date, which makes it natural for me to use the date as the link title. But I agree that this is certainly a specific workflow and I hope the recent changes may improve the behaviour for other workflows!