Compilation very slow and laggy

[plowman-c]

Hi, thanks for making this great plugin! I have been using it for a while now but something that bugs me is the speed when compiling.

minimal.vim

call plug#begin('~/.vim/plugged')

Plug 'lervag/vimtex'
let g:tex_flavor='latex'

  let g:vimtex_view_method='skim'
  map ,r :w<CR>:silent !/Applications/Skim.app/Contents/SharedSupport/displayline <C-r>=line('.')<CR> %<.pdf<CR> <bar> :redraw!<CR>

let g:vimtex_quickfix_mode=2
let g:vimtex_quickfix_latexlog =
\{
  \ 'overfull' : 0,
  \ 'underfull': 0,
  \ 'packages' : { 'default' : 0, },
\}
let g:vimtex_view_automatic=1
let g:vimtex_toc_config =
\{
  \ 'name'        : 'ToC',
  \ 'fold_enable' : 1,
  \ 'split_width' : 40,
  \ 'show_numbers': 1,
  \ 'layers'      : ['content', 'todo'],
\}
call plug#end()

syntax on

let g:tex_flavor = "latex"

mwe.tex

\documentclass{revtex4-1}
\begin{document}
Hello world!
\end{document}

Commands/Input

I open my file with vim, then hit \ll to start the vimtex autocompilation with latexmk in the background. If the file hasn't been compiled before, this understandably takes some time to compile and open Skim (my chosen viewer).

Every time I make some changes and hit :w it triggers the compiler (which is desirable behaviour), however this causes vim to become very laggy when moving the cursor or entering text/commands for the next 10 s to 60 s while it compiles.

Question
Is this normal behaviour? How long should it take for the compilation process to finish?
Should my vim interface be laggy while latexmk is running? I tried Googling around but no one was talking about slowness specifically while compiling so I'm starting to wonder if it's normal behaviour, it's just that vimtex is supposed to be a "lightweight" plugin which implies minimal lag and slowness.

With a small file like mwe.tex, lag lasts around 10--20 s after :w but with actual work (e.g. ~1000 lines) it takes more like 30--60 s which is quite inconvenient.

Output from VimtexInfo

  1 System info
  2   OS: Mac OS X 10.13.6 (17G13035)
  3   Vim version: VIM 8.2 (1-1050)
  4   Has clientserver: false
  5
  6 vimtex project: mwe
  7   base: mwe.tex
  8   root: /Users/coreyplowman
  9   tex: /Users/coreyplowman/mwe.tex
 10   out: /Users/coreyplowman/mwe.pdf
 11   log: /Users/coreyplowman/mwe.log
 12   aux: /Users/coreyplowman/mwe.aux
 13   fls: /Users/coreyplowman/mwe.fls
 14   main parser: current file verified
 15   compiler: latexmk
 16     backend: jobs
 17     output: /var/folders/tw/bhk4ylq56nb80y0rtph7cwq80000gp/T/vfXfrUa/1
 18     configuration:
 19       continuous: 1
 20       callback: 1
 21       latexmk options:
 22         -verbose
 23         -file-line-error
 24         -synctex=1
 25         -interaction=nonstopmode
 26       latexmk engine: -pdf
 27     job: process 15892 run
 28     cmd: max_print_line=2000 latexmk -verbose -file-line-error -synctex=1 -interaction=nonstopmode -pdf -pvc -view=none -e '$success_cmd = "echo vimtex_co
 29   viewer: Skim
 30     process:
 31       pid: -
 32       cmd: osascript -e 'set theFile to POSIX file "/Users/coreyplowman/mwe.pdf"' -e 'set thePath to POSIX path of (theFile as alias)' -e 'tell applicatio
 33   qf: LaTeX logfile
 34     config:
 35       overfull: 0
 36       packages:
 37         default: 0
 38       underfull: 0
 39       default: 1
 40   document class:
 41   packages:
 42     amsbsy
 43     amsgen
 44     amsmath
 45     amsopn
 46     amstext
 47     natbib
 48     revsymb4-1
 49     textcase
 50     url

[lervag]

Hi, thanks for making this great plugin!

Happy you like it!

minimal.vim

call plug#begin('~/.vim/plugged')

Plug 'lervag/vimtex'
let g:tex_flavor='latex'

  let g:vimtex_view_method='skim'
  map ,r :w<CR>:silent !/Applications/Skim.app/Contents/SharedSupport/displayline <C-r>=line('.')<CR> %<.pdf<CR> <bar> :redraw!<CR>

let g:vimtex_quickfix_mode=2
let g:vimtex_quickfix_latexlog =
\{
  \ 'overfull' : 0,
  \ 'underfull': 0,
  \ 'packages' : { 'default' : 0, },
\}
let g:vimtex_view_automatic=1
let g:vimtex_toc_config =
\{
  \ 'name'        : 'ToC',
  \ 'fold_enable' : 1,
  \ 'split_width' : 40,
  \ 'show_numbers': 1,
  \ 'layers'      : ['content', 'todo'],
\}
call plug#end()

syntax on

let g:tex_flavor = "latex"

1. You should not put the options before plug#end(). Not important in this case, but some plugins require you to call functions for configuring, and this is not possible until after the plug#end() function is called.

2. Why do you map ,r? Does not \lv do the same thing (i.e. forward search)?

3. See vimtex_quickfix_latexlog is ignored when using pplatex #1741 regarding g:vimtex_quickfix_latexlog - the option might be removed in not too long, so you might want to use g:vimtex_quickfix_ignore_filters instead.

4. No need to set options to their default values: g:vimtex_quickfix_mode and g:vimtex_view_automatic.

5. No need to do syntax on (this is already done by plug#end()).

Combined, I propose the following variant of your minimal vimrc:

call plug#begin('~/.vim/plugged')
Plug 'lervag/vimtex'
call plug#end()

let g:tex_flavor = "latex"
let g:vimtex_view_method = 'skim'
let g:vimtex_quickfix_latexlog = {
      \ 'overfull' : 0,
      \ 'underfull': 0,
      \ 'packages' : {'default' : 0},
      \}
let g:vimtex_toc_config = {
      \ 'name'        : 'ToC',
      \ 'fold_enable' : 1,
      \ 'split_width' : 40,
      \ 'show_numbers': 1,
      \ 'layers'      : ['content', 'todo'],
      \}

map ,r :w<CR>:silent !/Applications/Skim.app/Contents/SharedSupport/displayline <C-r>=line('.')<CR> %<.pdf<CR> <bar> :redraw!<CR>

Commands/Input

I open my file with vim, then hit \ll to start the vimtex autocompilation with latexmk in the background. If the file hasn't been compiled before, this understandably takes some time to compile and open Skim (my chosen viewer).

With a minimal file like the one you provided, it takes about 0.2 seconds to compile on my computer. But some larger projects can take some time, up to O(60s).

Every time I make some changes and hit :w it triggers the compiler (which is desirable behaviour), however this causes vim to become very laggy when moving the cursor or entering text/commands for the next 10 s to 60 s while it compiles.

The compiler should work in the background, non blocking. Thus, Vim should not become laggy. My first suspicion is that you have more plugins activated. Can you confirm the lag with the minimal LaTeX file when you do vim --servername vim -u minimal.vim mwe.tex, where minimal.vim is the vimscript file provided by you (or the updated version by me)?

The --servername vim option is useful to get the clientserver feature with callbacks that give you a status message when compilation is finished.

Question
Is this normal behaviour? How long should it take for the compilation process to finish?

No. Small files compile quite fast, large files may take some time to compile. But no blocking during compilation.

Should my vim interface be laggy while latexmk is running?

No.

With a small file like mwe.tex, lag lasts around 10--20 s after :w but with actual work (e.g. ~1000 lines) it takes more like 30--60 s which is quite inconvenient.

I don't think this is due to vimtex, but rather because of something in your config or environment. I'll try to help you figure out what, but first respond to my above questions.

Note, I'm not using a Mac, and I won't really be able to help much if things are related to Mac or OSX.

[plowman-c]

Thanks for the quick response and advice regarding my configuration.

I map ,r because I didn't know about \lv back when I first started using vimtex and it has become a habit to use that command now.

When I try vim --servername vim -u minimal.vim mwe.tex vim outputs the following errors

Error detected while processing /home/corey/mwe/minimal.vim:
line    7:
E15: Invalid expression: {
line    8:
E10: \ should be followed by /, ? or &
line    9:
E10: \ should be followed by /, ? or &
line   10:
E10: \ should be followed by /, ? or &
line   11:
E10: \ should be followed by /, ? or &
line   12:
E15: Invalid expression: {
line   13:
E10: \ should be followed by /, ? or &
line   14:
E10: \ should be followed by /, ? or &
line   15:
E10: \ should be followed by /, ? or &
line   16:
E10: \ should be followed by /, ? or &
line   17:
E10: \ should be followed by /, ? or &
line   18:
E10: \ should be followed by /, ? or &
Press ENTER or type command to continue

Instead I made a backup of my .vimrc and ran cat minimal.vim > ~/.vimrc then used vim --servername vim mwe.tex and the errors went away. Not sure why this is?

I do not have access to my Mac until Monday so I tested this on Windows Subsystem for Linux (WSL) using Windows Terminal using Okular as the viewer. This required the three additional lines at the end of the configuration:

let g:vimtex_view_general_viewer='okular'
let g:vimtex_view_general_options='--unique file:@pdf\#src:@line@tex'
let g:vimtex_view_general_options_latexmk='--unique'

The lag was not present when using this setup on WSL with mwe.tex

On Monday I will test this on OSX and provide an update.

[plowman-c]

I've tested using your suggested minimal.vim file on OSX with mwe.tex

Making a small change to the text and pressing \ll opened Skim but the vim buffer was laggy while moving the cursor/writing new text for about 10 s.

I also tested with zathura in case it is a Skim issue. I removed let g:vimtex_view_method="skim" and added let g:vimtex_view_general_viewer="zathura". This caused the same ~10 s laggy behaviour in vim after writing the buffer to file with vimtex.

[lervag]

Thanks for the quick response and advice regarding my configuration.

No problem!

I map ,r because I didn't know about \lv back when I first started using vimtex and it has become a habit to use that command now.

Ok! I would recommend to remove it and use \lv instead, as it does more.

When I try vim --servername vim -u minimal.vim mwe.tex vim outputs the following errors
...
Instead I made a backup of my .vimrc and ran cat minimal.vim > ~/.vimrc then used vim --servername vim mwe.tex and the errors went away. Not sure why this is?

I think it would work if you also add set nocompatible as the first line.

The lag was not present when using this setup on WSL with mwe.tex

I've tested using your suggested minimal.vim file on OSX with mwe.tex
...
I also tested with zathura in case it is a Skim issue. I removed let g:vimtex_view_method = "skim" and added let g:vimtex_view_general_viewer = "zathura". This caused the same ~10 s laggy behaviour in vim after writing the buffer to file with vimtex.

So, the problem is not there on WSL. I can not reproduce it. You do see it on OSX, though, however I can't really help much here.

Perhaps you can try with neovim?

[plowman-c]

Thanks, but I would rather not use neovim as I have to use many different systems and I would like my settings to be easily portable (vim is usually pre-installed but not neovim).

I have done some profiling. I opened mwe.tex in vim, started the compiler \ll and waited for Skim to open. Then I executed the following commands:

:profile start profile.log
:profile func *
:profile file *

Next, I made a small change and hit :w, moved the cursor around with the lag and finally did

:profile pause
:q

Here is the summary from the end of profile.log

FUNCTIONS SORTED ON TOTAL TIME
count  total (s)   self (s)  function
   57   5.362056   5.166395  <SNR>40_callback_continuous_output()
  182   0.609405   0.028179  180()
  227   0.510330   0.311525  <SNR>58_get_delim()
  182   0.438705   0.009287  vimtex#delim#get_current()
    2   0.195661   0.000515  vimtex#compiler#callback()
   15   0.159300   0.002838  vimtex#cmd#get_at()
   15   0.154053   0.003356  vimtex#cmd#get_current()
   15   0.149843   0.002025  <SNR>58_parser_env()
   15   0.147453   0.005601  <SNR>55_get_cmd()
   45   0.140222   0.017961  <SNR>55_get_cmd_part()
    1   0.127087   0.000166  vimtex#compiler#compile()
    1   0.126871   0.000150  43()
  182   0.117425   0.009751  vimtex#util#in_comment()
  182   0.107674             vimtex#util#in_syntax()
    2   0.106540   0.000171  vimtex#qf#open()
    2   0.106369   0.032129  vimtex#qf#setqflist()
    3   0.105265   0.000065  vimtex#log#info()
    3   0.105200   0.002101  174()
    3   0.102505   0.089451  vimtex#debug#stacktrace()
   45   0.081630   0.000718  vimtex#delim#get_next()

FUNCTIONS SORTED ON SELF TIME
count  total (s)   self (s)  function
   57   5.362056   5.166395  <SNR>40_callback_continuous_output()
  227   0.510330   0.311525  <SNR>58_get_delim()
  182              0.107674  vimtex#util#in_syntax()
    3   0.102505   0.089451  vimtex#debug#stacktrace()
  183              0.067005  <SNR>17_Highlight_Matching_Pair()
    1   0.062778   0.062130  50()
    2   0.045959   0.045543  62()
    2   0.106369   0.032129  vimtex#qf#setqflist()
  182   0.609405   0.028179  180()
   15   0.026770   0.026671  <SNR>58_get_matching_delim()
  422   0.029850   0.019189  vimtex#pos#set_cursor()
   45   0.140222   0.017961  <SNR>55_get_cmd_part()
   45   0.023696   0.015548  <SNR>58_parser_delim()
    2   0.014630   0.014553  196()
   16              0.013054  vimtex#util#command()
  467   0.011612   0.011318  <SNR>70_parse_args()
  182   0.117425   0.009751  vimtex#util#in_comment()
  182   0.438705   0.009287  vimtex#delim#get_current()
   30   0.052953   0.007704  vimtex#delim#get_matching()
    2   0.007441   0.006705  6()

It appears that callback_continuous_output() in .vim/plugged/vimtex/autoload/vimtex/compiler/latexmk.vim is causing the slow down. Not sure how long this function should take but I imagine 5 s is not expected.

[lervag]

Thanks, that was interesting. Could you also send me the line based profile log of <SNR>40_callback_continuous_output()? It would indicate which lines are slow and how many times the function is called.

It would also be interesting to see the output of :VimtexCompileOutput (or \lo) during the same time period.

[plowman-c]

Here's the output for <SNR>40_callback_continuous_output():

FUNCTION  <SNR>40_callback_continuous_output()
    Defined: ~/.vim/plugged/vimtex/autoload/vimtex/compiler/latexmk.vim:587
Called 57 times
Total time:   5.957894
 Self time:   5.838580

count  total (s)   self (s)
   57              0.002655   if exists('b:vimtex') && filewritable(b:vimtex.compiler.output)
   57              5.833063     call writefile([a:msg], b:vimtex.compiler.output, 'a')
   57              0.000281   endif

   57              0.000395   if a:msg ==# 'vimtex_compiler_callback_success'
    2   0.119488   0.000174     call vimtex#compiler#callback(1)
   55              0.000115   elseif a:msg ==# 'vimtex_compiler_callback_failure'
                                call vimtex#compiler#callback(0)
   57              0.000042   endif

   57              0.000066   try
   57              0.000709     for l:Hook in get(get(get(b:, 'vimtex', {}), 'compiler', {}), 'hooks', [])
                                  call l:Hook(a:msg)
   57              0.000076     endfor
                              catch /E716/
   57              0.000060   endtry

With \lo enabled, I saw the following output. It printed all of this while the interface was lagging:

Latexmk: All targets (mwe.pdf) are up-to-date
vimtex_compiler_callback_success
Not using a previewer
------------
Running 'echo vimtex_compiler_callback_success'
------------
Latexmk: Need to remake files.

  File changes, etc:
=== Watching for updated files. Use ctrl/C to stop ...
   Changed files, or newly in use since previous run(s):
Latexmk: applying rule 'pdflatex'...
      'mwe.tex'
This is pdfTeX, Version 3.14159265-2.6-1.40.19 (TeX Live 2018) (preloaded format=pdflatex)
Rule 'pdflatex': File changes, etc:
 restricted \write18 enabled.
   Changed files, or newly in use since previous run(s):
entering extended mode
      'mwe.tex'
(./mwe.tex
------------
LaTeX2e <2018-04-01> patch level 2
Babel <3.18> and hyphenation patterns for 84 language(s) loaded.
Run number 1 of rule 'pdflatex'
(/usr/local/texlive/2018/texmf-dist/tex/latex/revtex/revtex4-1.cls
------------
Document Class: revtex4-1 2010/07/25/20:33:00 4.1r (http://publish.aps.org/revtex4/ for documentation)
------------
ltxutil[2010/07/25/20:33:00 4.1r utilities package (portions licensed from W. E. Baxter web at superscript.com)]
Running 'pdflatex  -file-line-error -synctex=1 -interaction=nonstopmode -recorder  "mwe.tex"'
ltxfront[2010/07/25/20:33:00 4.1r frontmatter package (AO,DPC)]
------------
ltxgrid[2010/07/25/20:33:00 4.1r page grid package (portions licensed from W. E. Baxter web at superscript.com)]

Latexmk: Found input bbl file 'mwe.bbl'
Class revtex4-1 Warning: No Society specified, using default society aps.

Latexmk: Log file says output to 'mwe.pdf'
(/usr/local/texlive/2018/texmf-dist/tex/latex/revtex/aps4-1.rtx)
Latexmk: Found bibliography file(s) [mweNotes.bib]

------------
Class revtex4-1 Warning: No journal specified, using default pra.
Running 'echo vimtex_compiler_callback_success'

------------

Class revtex4-1 Warning: No type size specified, using default 10.

(/usr/local/texlive/2018/texmf-dist/tex/latex/revtex/aps10pt4-1.rtx) (/usr/local/texlive/2018/texmf-dist/tex/latex/textcase/textcase.sty) (/usr/local/texlive/2018/texmf-dist/tex/latex/url/url.sty) (/usr/local/texlive/2018/texmf-dist/tex/latex/natbib/natbib.sty) (/usr/local/texlive/2018/texmf-dist/tex/latex/revtex/revsymb4-1.sty)) (./mwe.aux) [1{/usr/local/texlive/2018/texmf-var/fonts/map/pdftex/updmap/pdftex.map}] (./mwe.bbl) [2] (./mwe.aux) )</usr/local/texlive/2018/texmf-dist/fonts/type1/public/amsfonts/cm/cmr10.pfb>
Output written on mwe.pdf (2 pages, 17001 bytes).
SyncTeX written on mwe.synctex.gz.
Transcript written on mwe.log.
=== TeX engine is 'pdfTeX'
Latexmk: All targets (mwe.pdf) are up-to-date
vimtex_compiler_callback_success

[lervag]

Ok, so, for some reason, the call to writefile() is very slow for you. I've never seen this problem before.

I'm still curious if this problem is present on neovim, which uses a similar mechanism and similar code. I'm also curious about your PC. Is it a particular slow PC with a very slow hard drive?

[plowman-c]

I'm using a 2013 iMac with macOS High Sierra, it's still surprisingly snappy for its age.

I installed neovim and there is no problem with the lag. It's so fast!

So the problem is only present in vim.

[lervag]

I installed neovim and there is no problem with the lag. It's so fast!

So the problem is only present in vim.

I'm sorry to say I have no idea how to solve this. Can you try other versions of Vim?

[plowman-c]

Can you try other versions of Vim?

I am currently using 8.1.2234 installed via Mac Ports. I tried using 8.2.539 installed from Homebrew but it had the same issue.

Interestingly, I tested 8.0.1365 which comes pre-installed with OSX and the lag was not present. Therefore it must be an issue with the other versions I installed. This is annoying since the default OSX vim is not compiled with +python, +termguicolors, +x11, or +clientserver, so it doesn't suit my needs.

I think I'll just switch to neovim, it seems to be a bit snappier in general (I suppose due to asynchronous syntax highlighting).

Thanks so much for your help investigating this!

[lervag]

I think I'll just switch to neovim, it seems to be a bit snappier in general (I suppose due to asynchronous syntax highlighting).

Yes, I also find neovim snappier, and these days it does seem to be quite available and easy to install on most systems.

Thanks so much for your help investigating this!

No problem. Sorry I can't figure out a real fix. :\

[cridemichel]

Hi, since I had the same odd behavior described in the first post, I have just issued a pull request ( #1986 ) which fixes this