Gvim freezes after failed compilation

[Svalorzen]

Description

This is happening on a fairly large project, which was compiling fine. I have just added a new chapter, copy pasted from a previous latex file; my plan was to go through the errors until it was integrated.

Unfortunately, it seems that one way or another, if I try to now compile the project, gvim ends up hanging (EDIT: I didn't think of it at first, but the same happens in vim in a terminal, it's not only related to gvim):

• If I try to use continuous compilation and waiting a bit, the compilation ends with the error "VimTex: Compilation failed!" and the cursor remains blinking at the end of the error message. However, gvim does not respond anymore after that, and must be killed with SIGTERM.
• If I compile using a single shot, it is even worse. There is no error at the end, and gvim freezes completely. SIGTERM does not kill it, and I had to use SIGKILL to get rid of it.

I am trying to pinpoint what is causing the error and generate a MWE, but I'm not having much success. Sometimes I remove something and I get the freeze, but after restarting gvim I do not get it anymore (only getting a bunch of compilation errors).

I do have a hunch that it might be related to too many errors; as I'm trying to create an MWE a "successful" compilation hanged for a couple of seconds, before producing ~1300 lines of errors. I'm not sure how to verify that this is indeed the cause, and if so, what I could do to avoid the crash.

Steps to reproduce

No response

Expected behavior

No response

Actual behavior

No response

Do you use a latexmkrc file?

No

VimtexInfo

info:
  OS: Ubuntu 22.04.1 LTS
  Vim version: VIM 9.0 (1-749)
  Has clientserver: true
  Servername: GVIM2

VimTeX project: crash
  base: crash.tex
  root: /home/svalorzen/Tests/latex
  tex: /home/svalorzen/Tests/latex/crash.tex
  main parser: current file verified
  document class: book
  packages: amsmath amssymb etoolbox fancyhdr geometry graphicx subcaption titlesec xcolor xstring
  source files:
    crash.tex
    config/headers.tex
    mats.tex
  compiler: latexmk
    engine: -pdf
    options:
      -verbose
      -file-line-error
      -synctex=1
      -interaction=nonstopmode
    build_dir: build
    callback: 1
    continuous: 1
    executable: latexmk
    job: 
      jobid: process 13031 run
      output: /tmp/vYXjcyw/0
      cmd: max_print_line=2000 latexmk -verbose -file-line-error -synctex=1 -interaction=nonstopmode -pdf -outdir=build -pvc -view=none -e '$compiling_cmd = ($compiling_cmd ? $compiling_cmd . " ; " : "") . "echo vimtex_compiler_callback_compiling"' -e '$success_cmd = ($success_cmd ? $success_cmd . " ; " : "") . "echo vimtex_compiler_callback_success"' -e '$failure_cmd = ($failure_cmd ? $failure_cmd . " ; " : "") . "echo vimtex_compiler_callback_failure"' 'crash.tex'
      pid: 13031
  viewer: General
  qf method: LaTeX logfile

[lervag]

If I try to use continuous compilation and waiting a bit, the compilation ends with the error "VimTex: Compilation failed!" and the cursor remains blinking at the end of the error message. However, gvim does not respond anymore after that, and must be killed with SIGTERM.

Just curious, but did you try to wait to see if something happens after a (perhaps long) time?

If I compile using a single shot, it is even worse.

I would expect this to be the same problem, so I'll "ignore it" for now.

I am trying to pinpoint what is causing the error and generate a MWE, but I'm not having much success. Sometimes I remove something and I get the freeze, but after restarting gvim I do not get it anymore (only getting a bunch of compilation errors).

Can you reprocuce this problem with the same LaTeX files but with a minimal configuration? E.g., create test.vim like this:

set nocompatible

" UPDATE PATHS HERE:
set runtimepath^=~/.local/plugged/vimtex
set runtimepath+=~/.local/plugged/vimtex/after

filetype plugin indent on
syntax enable

" Add relevant VimTeX configuration here, if any:
" let g:vimtex_...

Then do cd /home/svalorzen/Tests/latex and vim -u test.vim crash.tex. Does it still freeze? If it does freeze, it would also be useful to know if it freezes with neovim (nvim --clean -u test.vim crash.tex)?

I do have a hunch that it might be related to too many errors; as I'm trying to create an MWE a "successful" compilation hanged for a couple of seconds, before producing ~1300 lines of errors. I'm not sure how to verify that this is indeed the cause, and if so, what I could do to avoid the crash.

Well, it's hard for me to know as well. I think the first step is to reproduce with a minimal configuration, so let's start there.

[Svalorzen]

Hey, so I tried again with my MWE, but for some reason, minimal config or not, this time it would complete every time, again with ~1300 errors.

So I went back to my original crashing code, and this time as you suggested I tried to let it run for a while. After around 25 minutes, it actually completed! However, I got 38.819 errors :D

Looking at the original log, it seems like something in the original file made LaTeX put text into an extremely tiny box, so it tried to split the text as much as possible by hyphenating, but every single piece is still too large, resulting in an overwhelming number of overfull hbox warnings.

Any settings I could play with to limit the number of shown errors so this does not happen anymore?

[lervag]

Hey, so I tried again with my MWE, but for some reason, minimal config or not, this time it would complete every time, again with ~1300 errors.

1300 errors?! You mean 1300 lines in the quickfix window? That's crazy. When you say MWE, do you mean a simple document like

\documentclass{minimal}
\begin{document}
Hello World!
\end{document}

or do you mean a long (or even huge?) manuscript? If it is not a large example, then it could be useful to see a sample of it to understand what you are doing.

So I went back to my original crashing code, and this time as you suggested I tried to let it run for a while. After around 25 minutes, it actually completed! However, I got 38.819 errors :D

38819 errors?!!!! That's beyond crazy. How is this even possible? Are you working on a large book manuscript?

Looking at the original log, it seems like something in the original file made LaTeX put text into an extremely tiny box, so it tried to split the text as much as possible by hyphenating, but every single piece is still too large, resulting in an overwhelming number of overfull hbox warnings.

This does not have anything to do with VimTeX, really; this is a pure LaTeX issue, and you can easily reproduce the issue in a terminal with latexmk.

Any settings I could play with to limit the number of shown errors so this does not happen anymore?

No; I believe the issue you are having is that VimTeX is parsing the log file, but the log file is huge and thus this parsing is taking a long time. This is the first time I hear of something like this. If your project (or a stripped down part of it) is something you would be willing to share with me (e.g. in a private channel), then I might look into if I can somehow improve the performance of the log parser. But I can't make a promise here, because we mostly rely on the built-in errorformat based parser.

[Svalorzen]

Yeah, I'm currently writing my dissertation, which is at the moment around 150 pages, which is what was triggering the 38k errors. However, I managed to create a MWE! It's only a couple lines short; the trick is that the file I copy-pasted in was using subfigure, while in my main project I was using subcaption. I guess subcaption really does not like this. Here's the code:

\documentclass[twoside,10pt]{book}

\usepackage{lipsum} 

% The one that breaks everything
\usepackage{subcaption}

\begin{document}

\begin{figure}[ht!]
    % Using subfigure!!! Wrong with subcaption package.
    \subfigure[ABC]{\includegraphics{anything}}

    % What causes the warnings, increase second number for more warnings

    % ~300 errors
    \caption{\lipsum[1-2]}

    % Be careful! ~1500 errors
    %\caption{\lipsum[1-10]}
\end{figure}

\end{document}

I also think that it might be the parser; the compilation itself is fairly quick, the freeze is after VimTex reports that the compilation ends with error and the time it manages to display the entire thing. About the fix, I guess a faster parser could be nice, but to be honest I think even just having a parameter that limits the maximum number of errors displayed/parsed would be really cool. After all after the first 200-300 there is little point in showing more. Something like "Too many errors, will only display/stopped parsing after the first X" would already be useful and would avoid stalling, and if necessary one could go look at the entire log by hand.

[lervag]

Yeah, I'm currently writing my dissertation, which is at the moment around 150 pages, which is what was triggering the 38k errors.

Ok, that's not what I would call huge.

However, I managed to create a MWE!

Ah, great! However ...

It's only a couple lines short; the trick is that the file I copy-pasted in was using subfigure, while in my main project I was using subcaption. I guess subcaption really does not like this.

I believe these packages are not compatible. Indeed, in the example you provided, the \subfigure command does not compile. If we simplify further, the errors are more "clear". Or, at least, fewer and more diverse. Consider:

\documentclass[twoside,10pt]{book}
\usepackage{subcaption}
\usepackage{graphicx}

\begin{document}

\begin{figure}[ht!]
    \subfigure[ABC]{\includegraphics{example-image-a}}
\end{figure}

\end{document}

This does not compile. If you use the subcaption package, then I believe you want the \begin{subfigure} environment that it provides, not the \subfigure command that is provided by the subfigure package. I strongly recommend you read the docs for these packages!

[lervag]

I profiled the example with the additional lipsum command. The main cause of the delay is a function used to improve the hbox warnings:

vimtex/autoload/vimtex/qf/latexlog.vim

Line 148 in a7b1654

function! s:fix_paths_hbox_warning(qf, log, root, cache) abort " {{{1

There are three lines that cause most of this delay, and of these, the largest one is line 151:

  let l:index = match(a:log, '\V' . escape(a:qf.text, '\'))

This line does some "heavy lifting" in that it searches for the line in the log that has the current quickfix entry text.

Notice, though, that this entire function is cached. I don't see any easy or immediate way to improve this. I'm sure there are ways to improve this, of course, but considering that I also believe you can avoid this problem altogether by either 1) using the subfigure package and not the subcaption package, or 2) use the subcaption package and not the subfigure package (and change the use of \subfigure to \begin{subfigure} ... \end{subfigure}). So I don't think I want to spend more time on this now.

[lervag]

Scratch that; I found a simple improvement that should improve the time by a factor of up to 0.5. On your example, the parse time was reduced from about 2 seconds to 1 second on my computer (with the \caption{\lipsum[1-10]} uncommented). :)

[Svalorzen]

I guess I might need to clarify my position.

This does not compile. If you use the subcaption package, then I believe you want the \begin{subfigure} environment that it provides, not the \subfigure command that is provided by the subfigure package. I strongly recommend you read the docs for these packages!

I have never pretended it does, and I know it doesn't (I thought my comments in the provided MWE made that clear, I guess not). My issue is less with whatever the LaTeX input is, and more with the fact that my Vim got frozen for 20+ minutes due to VimTex.

In my original case this happened because I was "porting" some code from a colleague file into my project, and I started by simply copy-pasting his file into mine, meaning to fix discrepancies later. This resulted in Vim freezing, which I believe should generally never happen, especially if pdflatex itself does not (and if I had not waited for the full 20+ minutes, I would have never guessed what the problem was).

In other words, I think that if pdflatex itself does not break, VimTex should not be able to lock Vim for more than 5-10 seconds no matter what the files being processed are. I guess this is a bit utopic in general, but in this context I think it would make sense? You are free to ignore this obviously tho.

Scratch that; I found a simple improvement that should improve the time by a factor of up to 0.5. On your example, the parse time was reduced from about 2 seconds to 1 second on my computer (with the \caption{\lipsum[1-10]} uncommented). :)

I just tested it, it's indeed much faster, thanks! Did 80k errors in about 2 minutes, which is great. Still wish it would just abort, but if that can't happen I can't complain :)

[lervag]

This does not compile. If you use the subcaption package, then I believe you want the \begin{subfigure} environment that it provides, not the \subfigure command that is provided by the subfigure package. I strongly recommend you read the docs for these packages!

I have never pretended it does, and I know it doesn't (I thought my comments in the provided MWE made that clear, I guess not).

Fair; I did not catch that you were fully aware of this problem.

My issue is less with whatever the LaTeX input is, and more with the fact that my Vim got frozen for 20+ minutes due to VimTex.

I can understand that this is a problem!

Scratch that; I found a simple improvement that should improve the time by a factor of up to 0.5. On your example, the parse time was reduced from about 2 seconds to 1 second on my computer (with the \caption{\lipsum[1-10]} uncommented). :)

I just tested it, it's indeed much faster, thanks! Did 80k errors in about 2 minutes, which is great. Still wish it would just abort, but if that can't happen I can't complain :)

I believe it would be quite safe to add a simple check for the length of the log file. If it is very large, then it seems safe to assume that it is because of a huge number of errors or warnings, and in this case, it does not seem worth the effort to improve the overfull/underfull messages.

[lervag]

I've now added a guard that prevents the expensive feature from running for large log files (more than 10000 lines). I believe this should more or less resolve your problem alltogether; please test.

[Svalorzen]

Hey, sorry for the late reply, last week was fairly busy.

I've tested the new changes, now the 80k errors are processed in about 5 seconds, which really is great. Thanks a lot for the help!