Purpose of completion files and customizing completion

[rhelder]

Thanks again for your help with my issue the other day - sorry to be back again so soon! This time I have a question about what I think is expected behavior - TL;DR, based on what I understand from the documentation and from my testing, it is VimTeX's parsing of .fls files and/or of \usepackage statements that does the heavy lifting when it comes to determining what commands to suggest during completion. If that's true, my question is what role the completion files in autoload/vimtex/complete are playing, and whether or not it is possible to customize what commands are suggested during completion.

Ok, now for the long-winded part. What I'm asking about is expected behavior (as far as I know), and there's no difference in behavior between my full configuration and a minimal configuration - but just to hold everything constant, I used the following minimal.vim:

set nocompatible
set runtimepath^=~/.config/nvim/vim-plug/vimtex
set runtimepath+=~/.config/nvim/vim-plug/vimtex/after
filetype plugin indent on
syntax enable

And here's a minimal tex file (minimal1.tex):

\documentclass{article}

\usepackage{biblatex}

\setlength{\bibitemsep}{0pt} \DeclareLanguageMapping{latin}{american}

\begin{document}
Hello, world
\end{document}

While typing (after having typed \usepackage{biblatex} and used :VimtexReloadState), \bibitemsep was suggested during completion, and \DeclareLanguageMapping was not. This is what I expected, since \bibitemsep is in biblatex's completion file in autoload/vimtex/complete, and \DelareLanguageMapping is not.

However, suppose I really wanted \DeclareLanguageMapping to be suggested during completion. I expected (in my ignorance) that appending DeclareLanguageMapping to biblatex's completion file in autoload/vimtex/complete would cause \DeclareLanguageMapping to appear in the list of possible commands. As it is (even after :VimtexReloadState or compilation), adding \DeclareLanguageMapping to the biblatex completion file causes no change in what is suggested during completion.

I also tried this out with the following minimal.sty file:

\ProvidesPackage{minimal}
\NewDocumentCommand{\texttitle}{m}{\emph{#1}}
\NewCommandCopy{\define}{\emph}
\endinput

and the following minimal2.tex file:

\documentclass{article}

\usepackage{minimal}

\begin{document}

My favorite book by Kant is the \texttitle{Groundwork of the Metaphysics of Morals}.
He begins the first section by discussing the \define{good will},
which he says is the only thing in the world that is unconditionally good.

\end{document}

Without any completion file at all, VimTeX was able to recognize \texttitle and suggest it during completion, but not \define. This is not what I expected, but I realized that it made sense, since VimTeX looks for new commands either after compilation or upon initialization (or after :VimtexReloadState) - even without a completion file - and since there are good reasons for not wanting to include copies of commands among the commands that are suggested during completion. What I did expect (in my ignorance) is that if I drafted the following completion file

texttitle
define

and added it to autoload/vimtex/complete, that not only \texttitle but also \define would be suggested during completion. As it is, drafting and placing the completion file made no difference. (This is the same behavior as I observed with my actual, non-minimal .sty files).

So what it looks like on my end (and also from what I can tell from going back over the documentation) is that it's really VimTeX's parsing of .fls files and/or \usepackage statements that does (all?) the work when determining what commands to suggest during completion.

Hence my initial question: what role do the completion files in autoload/vimtex/complete play, since they don't have any visible effect on the behavior described above? And if one can't customize completion by modifying the completion files, do you have any suggestions for how completion could be customized?

(I did try to see for myself by having a look at complete.vim, but I'm very much a novice at vimscript, and I'm not confident that I followed completely. I also had a look at #650 and #770, which were illuminating but still left me wondering).

I hope that I didn't miss anything either in the documentation or online - I looked very hard and couldn't find what I was looking for. Thanks for your help, and for your work on this excellent plugin.

[lervag]

Thanks again for your help with my issue the other day - sorry to be back again so soon!

My pleasure, and no problem! Although, that's a lot of text to "parse" :)

based on what I understand from the documentation and from my testing, it is VimTeX's parsing of .fls files and/or of \usepackage statements that does the heavy lifting when it comes to determining what commands to suggest during completion.

Sort of. The .fls file and \usepackages are used to recognize which packages are loaded for a given LaTeX project. And command completion will load completion candidates from the files under autoload/vimtex/complete/*. These files are named after the specific packages (or classes) and list the available commands (and possibly a symbol that is also shown in the completion menu).

If that's true, my question is what role the completion files in autoload/vimtex/complete are playing, and whether or not it is possible to customize what commands are suggested during completion.

See above; customizing is harder, but you could potentially propose a PR with new/updated candidates for a given package.

Ok, now for the long-winded part. …

I believe the main cause of confusion is due to caching. After initiating completion, VimTeX will cache the candidates. You can delete the cache with :VimtexClearCache pkgcomplete or just delete the file under ~/.cache/vimtex/pkgcomplete.json and then restart VimTeX.

Without any completion file at all, VimTeX was able to recognize \texttitle and suggest it during completion, but not \define.

Yes; if there is no completion file present, then VimTeX will parse the actual package file. However, it does not recognize \NewCommandCopy. Thus, that may be a potential feature request. If you think that this should be supported, please open a short feature request with the above minimal example.

This is not what I expected, but … and added it to autoload/vimtex/complete, that not only \texttitle but also \define would be suggested during completion.

Again, I believe this is due to the caching. However, I think it should actually work, so that may also be a bug.

Hence my initial question: what role do the completion files in autoload/vimtex/complete play, since they don't have any visible effect on the behavior described above?

The relevant part of the code is here:

vimtex/autoload/vimtex/complete.vim

Lines 736 to 746 in 8d82185

	function! s:load_from_package(pkg, type) abort " {{{1
	let s:pkg_cache = get(s:, 'pkg_cache',
	\ vimtex#cache#open('pkgcomplete', {'default': {}}))
	let l:current = s:pkg_cache.get(a:pkg)
	let l:pkg_file = s:complete_dir . '/' . a:pkg
	if filereadable(l:pkg_file)
	if !has_key(l:current, 'candidates')
	let s:pkg_cache.modified = 1
	let l:current.candidates
	\ = s:_load_candidates_from_complete_file(a:pkg, l:pkg_file)

You will find this specific function called from s:completer_cmd.gather_candidates on line 310 in the same file.

And if one can't customize completion by modifying the completion files, do you have any suggestions for how completion could be customized?

In fact, I don't think there is any way to customize this right now. I prefer not to add options unless necessary. Perhaps the easiest thing here is to add potential missing completion candidates to the right package files?

I hope that I didn't miss anything either in the documentation or online - I looked very hard and couldn't find what I was looking for.

I think you already did a very good job! Thanks for actually spending time trying to understand things first! I hope my answer is clear and that it does actually answer your questions properly.

I propose this: First, if relevant, feel free to open the feature request I hinted at. Next, also feel free to open an issue with the second minimal example where you add the autoload/complete/minimal file and you still don't see the candidates. It may be a bug, and I will be glad to look into it - but it is much easier for me with shorter, more precise issues. I don't have much time for maintaining VimTeX these days, and so it really helps when I don't need to read a lot of text to get into a specific issue.

Thanks for your help, and for your work on this excellent plugin.

Glad you find it useful!

[rhelder]

Yes, you hit the nail on the head -- the thing that was thwarting my testing was caching. In my testing, I allowed VimTeX to parse minimal.sty, and then I put the completion file minimal into autoload/vimtex/complete. It seems that, once VimTeX had already parsed the package file, it was unwilling to circle back and parse the completion file. If, however, I delete the cache and close VimTeX (or close VimTeX and delete the cache) and put the minimal completion file in autoload/vimtex/complete (thus not allowing VimTeX to initiate completion during this time, or else VimTeX would parse the package file, re-create the cache, and not parse the completion file), and then open VimTeX, \define shows up in the completion menu! (I can't tell you how satisfying that feels after tinkering with no success all day yesterday).

So I don't see any need for a bug or feature request. If someone knows to delete the cache, close VimTeX, and not reinitialize VimTeX until they've made the changes they want to make in the autoload/vimtex/complete directory, then they can customize completion easily. And anyone who has the same questions that I had can reference our conversation here, unless you think it's worth putting a note about this in the documentation. So I will go ahead and mark the issue as closed.

I don't have much time for maintaining VimTeX these days, and so it really helps when I don't need to read a lot of text to get into a specific issue.

Thanks for your generosity with your time -- I had hoped not only to get some clarity on my problem but also to learn something new about how VimTeX works, and I got both, so I'm doubly grateful!

[lervag]

Yes, you hit the nail on the head -- the thing that was thwarting my testing was caching.

I guess there's an advantage to having written most of the code :)

So I don't see any need for a bug or feature request. If someone knows to delete the cache, close VimTeX, and not reinitialize VimTeX until they've made the changes they want to make in the autoload/vimtex/complete directory, then they can customize completion easily.

Cool; but please feel free to open issues if you have thoughts on how to improve things! And if you should find missing candidates in the existing completion files, feel free to open a PR that adds these.

And anyone who has the same questions that I had can reference our conversation here, unless you think it's worth putting a note about this in the documentation. So I will go ahead and mark the issue as closed.

I think that sounds good.

Thanks for your generosity with your time -- I had hoped not only to get some clarity on my problem but also to learn something new about how VimTeX works, and I got both, so I'm doubly grateful!

My pleasure!