Folding interaction with commented command definitions

[ji-cerny]

Description

When using foldmethod=expr, folding (in particular in the preamble) breaks when the file contains lines like

%\newcommand{\Pt}{\tilde{\mathbb{P}}}

vimtex#fold#level() function returns s1 for such lines, since they match the regular expression for the fold-end marker %.*}}}, as defined in autoload/vimtex/fold/markers.vim

While debugging this, I also found that there is probably a typo in autoload/vimtex/fold/markers.vim in function s:folder_level(), where self should be used instead of s:self. Otherwise the vimtex#fold#level() actually finishes with error and returns -1

Steps to reproduce

No response

Expected behavior

No response

Actual behavior

No response

Do you use a latexmkrc file?

No

VimtexInfo

System info:
  OS: Linux 6.1.15-200.fc37.x86_64
  Vim version: VIM 9.0 (1-1367)
  Has clientserver: true
  Servername: VIM

VimTeX project: gproc
  base: bla.tex
  root: /home/xxx/foo
  tex: /home/xxx/foo/bla.tex
  main parser: current file verified
  document class: scrbook
  packages: amsbsy amsfonts amsgen amsmath amsopn amssymb amstext amsthm atbegshi atbegshi-ltx atveryend atveryend-ltx auxhook babel bigintcalc bitset bookmark calc caption caption3 changebar color enumitem epstopdf-base etexcmds etoolbox expl3 fontenc footmisc geometry gettitlestring graphics graphicx hycolor hyperref ifmtarg iftex ifthen ifvtex infwarerr inputenc intcalc keyval kvdefinekeys kvoptions kvsetkeys letltxmacro ltxcmds makeidx marginnote mathtools mhsetup nameref pdfescape pdftexcmds pgf pgfcomp-version-0-65 pgfcomp-version-1-18 pgfcore pgffor pgfkeys pgfmath pgfrcs pgfsys refcount rerunfilecheck scrbase scrkbase scrlayer scrlayer-scrpage scrlfile scrlfile-hook scrlogo tikz tocbasic trig typearea uniquecounter url verbatim xcolor xifthen xparse xparse-2020-10-01 xpatch
  compiler: latexmk
    engine: -pdf
    options:
      -verbose
      -file-line-error
      -synctex=1
      -interaction=nonstopmode
    callback: 1
    continuous: 1
    executable: latexmk
    job: 
      jobid: process 32563 dead
      output: /tmp/vN3cqSx/0
      cmd: max_print_line=2000 latexmk -verbose -file-line-error -synctex=1 -interaction=nonstopmode -pdf 'bla.tex'
      pid: 32563
  viewer: General
  qf method: LaTeX logfile

[lervag]

While debugging this, I also found that there is probably a typo in autoload/vimtex/fold/markers.vim in function s:folder_level(), where self should be used instead of s:self. Otherwise the vimtex#fold#level() actually finishes with error and returns -1

Thanks, that is fixed now!

When using foldmethod=expr, folding (in particular in the preamble) breaks when the file contains lines like

%\newcommand{\Pt}{\tilde{\mathbb{P}}}

vimtex#fold#level() function returns s1 for such lines, since they match the regular expression for the fold-end marker %.*}}}, as defined in autoload/vimtex/fold/markers.vim

Ah, yes. I would not really call this a bug, though; rather an unintended side effect of the marker folding.

The question here is how to treat this particular case? Now we recognize the markers as long as they are inside a comment. The open/close markers are configurable, see :help g:vimtex_fold_types and the <markers> type description.

Do you have any suggestion that makes more sense than the current?

[ji-cerny]

Ah, yes. I would not really call this a bug, though; rather an unintended side effect of the marker folding.

It looks like a bug, if one does not understand what is going on under the hood. For me VimTeX expression folding of preamble was simply not working. In essentially all my TeX files the preamble stopped at seemingly random location. It was hard to find the reason for this, since I did not expect that marker and expression folding are mixed in VimTeX, in pure vim they are separate folding methods. (Also, previously, I used marker folding with markers '%<<<' and '%>>>', and forgot that standard vim foldmarks are '{{{, }}}'. )

The question here is how to treat this particular case? Now we recognize the markers as long as they are inside a comment. The open/close markers are configurable, see :help g:vimtex_fold_types and the <markers> type description.

I know, it is great how configurable the folding is.

Do you have any suggestion that makes more sense than the current?

It is not an easy question. It is probably difficult to fix it to make everybody happy. I do not know how frequently the mixed folding (expressions/markers) is used. Otherwise I would say, one should follow the vim way and make expression and marker folding separate folding methods.
Other possibility would be to use different markers. I know that vim documentation discourages this, but '}}}' occurs too often in the TeX code, and it happens regularly that one wants to comment out part of the code. It does not need to be in the preamble, commented out parts of long formulas are much more frequent (especially in work-in-progress files) and have exactly the same effect (at least I believe, I have equation folding off).

[lervag]

Ah, yes. I would not really call this a bug, though; rather an unintended side effect of the marker folding.

It looks like a bug, if one does not understand what is going on under the hood.

Yes, I can understand that!

Do you have any suggestion that makes more sense than the current?

It is not an easy question.

Agreed. :)

It is probably difficult to fix it to make everybody happy. I do not know how frequently the mixed folding (expressions/markers) is used. Otherwise I would say, one should follow the vim way and make expression and marker folding separate folding methods. Other possibility would be to use different markers. I know that vim documentation discourages this, but '}}}' occurs too often in the TeX code, and it happens regularly that one wants to comment out part of the code. It does not need to be in the preamble, commented out parts of long formulas are much more frequent (especially in work-in-progress files) and have exactly the same effect (at least I believe, I have equation folding off).

Yes, so, we have three options:

1. Keep the current version.
2. Disable marker folding by default.
3. Change the default markers.

I tend towards option 1, because VimTeX has become quite stable the last 5 years and there's not been many complaints on this part (I believe you may be the first). So, in your case, you can easily fix the problem by disabling marker folding on your end or change the markers.

If, instead, I went for option 2 or 3, I risk breaking the expected behaviour for current users of VimTeX. I don't mind breaking things if I have a strong opinion or believe that it is for a greater good.

Unless you have a convincing argument for changing this or a better option, I'll leave this as "won't fix".