main file detection strictness

[shuber2]

Description

Issue.
Commit e5de00e adds more strict mainfile detection and with commit fa5cbe4 an error is emitted when mainfile detection fails.

I have a use case where I produce lecture note slides; each lecture unit is a beamer presentation. The preamble for each lecture-xx.tex is almost the same and hence hence each file starts with an \input{slides.inc} to reduce code duplication. In particular, the \documentclass line is contained in the slides.inc.

This now breaks the mainfile detection, because a \documentclass line became necessary. (Actually, having a \documentclass is not sufficient: It cannot be certain documentclass variants and the document-environment is also necessary.)

Proposal.
I would like to propose two things:

1. Add the required properties for the mainfile detection logic into the documentation (:help). I reverse engineered the properties through git bisect and the corresponding vimscript code.
2. Maybe rethink whether the existence of \begin{document}…\end{document} would not be a sufficient property for a file to be a mainfile.

Steps to reproduce

No response

Expected behavior

No response

Actual behavior

No response

Do you use a latexmkrc file?

Yes

VimtexInfo

System info:
  OS: Debian GNU/Linux 11 (bullseye)
  Vim version: NVIM v0.7.0
  Has clientserver: true
  Servername: /tmp/nvimqvrkzB/0

VimTeX project: foo
  base: foo.tex
  root: /tmp
  tex: /tmp/foo.tex
  main parser: fallback current file
  document class: 
  compiler: latexmk
    engine: -pdf
    options:
      -verbose
      -file-line-error
      -synctex=1
      -interaction=nonstopmode
    callback: 1
    continuous: 1
    executable: latexmk
  viewer: General
  qf method: LaTeX logfile

[lervag]

Issue. Commit e5de00e adds more strict mainfile detection and with commit fa5cbe4 an error is emitted when mainfile detection fails.
...
This now breaks the mainfile detection, because a \documentclass line became necessary. (Actually, having a \documentclass is not sufficient: It cannot be certain documentclass variants and the document-environment is also necessary.)

I'm sorry for breaking your workflow, but I hope you can understand how these changes are actually an improvement. From VimTeX perspective, it is important to clearly specify the requirements that allow a robust "understanding" of a given project. Before the said commit, VimTeX did work in many cases, but this was coincidental. The changes were made to ease the maintenance burden for myself and to help users avoid situations that are difficult to handle from VimTeX and that may cause unexpected consequences.

Proposal. I would like to propose two things:

1. Add the required properties for the mainfile detection logic into the documentation (:help). I reverse engineered the properties through git bisect and the corresponding vimscript code.

The error message should already point in the right direction:

vimtex/autoload/vimtex/compiler.vim

Lines 191 to 198 in 06c47ed

	if !b:vimtex.is_compileable()
	call vimtex#log#error(
	\ 'Compilation error due to failed mainfile detection!',
	\ 'Please ensure that VimTeX can locate the proper main .tex file.',
	\ 'Read ":help vimtex-multi-file" for more info.'
	\)
	return
	endif

If you read :help vimte-multi-file, it will explain how VimTeX locates the main file for a given LaTeX project. However, I agree it can be improved to better explain the relevant mechanisms. I'm pushing an update where I attempt to explain it better asap.

2. Maybe rethink whether the existence of \begin{document}…\end{document} would not be a sufficient property for a file to be a mainfile.

I've thought this through quite a lot, actually. The existence of \begin{document}…\end{document} is actually very clearly not sufficient. Simple proof:

% main.tex
\documentclass{minimal}
\input{content}

% content.tex
\begin{document}
Hello world
\end{document}

I believe it would be quite safe to say that the \documentclass is a good identifier for the main file. But writing a main file detection algorithm becomes much harder if we do not also enforce \begin{document}. I can go into the details, if you want?

[lervag]

While improving the docs, I realize there may be a relatively simple solution that have escaped me. I'll look into it and see if I may relax the strictness.

[shuber2]

Wow, I am just amazed by your elaborate and quick response. Thank you very much and thank you for vimtex, which has been great for me for many years.

You minimal counter example of my proposal is of course right. Interestingly – and ironically – it is just the complementary example of my now broken workflow:

% slides.inc --- the common preamble for all slides
\documentclass{beamer}

% lecture-42.tex --- one of many main files
\input{slides.inc}
\begin{document}
\end{document}

Knowing about this two symmetric counter examples, I totally understand if either one will just not work. But I am quite curious about your new idea.

Regarding documentation: From the implementation of s:file_is_main(file) I conclude that two conditions must be given for file to qualify for a main file:

• At least one line needs to start with \documentclass that does not contain {subfiles} or {standalone}
• At least one line needs to start with \begin{document}

(In either cases there can be some whitespace between tokens.) This I did not find in the documentation. Did I overlook it?

[lervag]

Wow, I am just amazed by your elaborate and quick response. Thank you very much and thank you for vimtex, which has been great for me for many years.

Thank you for the kind words and feedback!

You minimal counter example of my proposal is of course right. Interestingly – and ironically – it is just the complementary example of my now broken workflow:

I'll have to admit I mixed that up. I intended to give an example precisely like the one you provided. In fact, the one I provided actually works.

Regarding documentation: From the implementation of s:file_is_main(file) I conclude that two conditions must be given …

… This I did not find in the documentation. Did I overlook it?

No, you were correct. I pushed an update that I believe should clarify:

vimtex/doc/vimtex.txt

Lines 450 to 462 in 0bbd6b2

	Recursive search~
	If no other method provides an appropriate candidate, then the recursive
	search detects the main LaTeX file by searching for a file in the current
	and parent directories that
	1. includes the present file (directly or indirectly),
	2. has a `\documentclass` line near the top, and
	3. contains `\begin{document}` (or includes a file that does).
	As long as a project follows these rules, the recursive search should work.
	Notice, though, that the first point is important and is where the recursive
	nature is implemented. There must be a "straight" inclusive line from the
	main file to the present file. If not, then rule 1 is violated.

Of course, this may probably still be improved, and suggestions are more than welcome.

But I am quite curious about your new idea.

Yes, so: the s:file_is_main(file) will look at the top lines of file for \documentclass. Thus, that is very strict. However, the search for \begin{document} is performed in a much looser manner and will also consider lines from included files by use of vimtex#parser#preamble:

vimtex/autoload/vimtex/state.vim

Lines 490 to 495 in 0bbd6b2

	" A main file contains `\begin{document}`
	let l:lines = vimtex#parser#preamble(a:file, {
	\ 'inclusive' : 1,
	\ 'root' : fnamemodify(a:file, ':p:h'),
	\})
	call filter(l:lines, 'v:val =~# ''^\s*\\begin\s*{document}''')

I was thinking that I could use that same function instead of readfile for the search for \documentclass as well. It may even trivially solve your use case. But I often find that these things are slightly more complicated that I first think, so I will need to check the existing tests and possible add some more before I make the change.

[lervag]

@shuber2 I've implemented the said fix and I think it works. Please see @2558. Let's continue the discussion there. (If you happen to have some large and complex projects (e.g. books or similar) in LaTeX, it would be useful if you could test whether the PR introduces a lag during startup, as I mention in the PR description.)

[lervag]

The PR #2558 is now merged, which means mainfile detection for cases like this will now work as expected:

% preamble.tex
\documentclass{minimal}

% main.tex
\input{preamble}
\begin{document}
\end{document}