Long startup times

[mattia-marini]

Description

Hi! Newbie question right here. First of all thank you for developing such a beautiful plugin for free!

I was wondering if there was any way to reduce startup times. I know you are not supposed to lazy load the plugin, but even on rather small files (~500 lines), vimtex seems to add about 150-200 milliseconds to startup times.

The lag persist even with only vimtex installed with everything on default configuration. I also tried disabling every feature I could to see if that could lead to any improvement but it still seemed too slow for me (I attached the file of that configuration).

My last resort was trying to manually copy/paste the parts I strongly need (so basically the context detection and the compile module) and create a "light version" of the plugin. That has been possible for the former option, but not for the latter, as the compiler module seems to be more articulated.

P.s. I have read through the "high level documentation" of the plugin, but i would love to take a deeper dive to learn a bit more. Is there any other more detailed doc available to public?
P.p.s. This is my first issue on GitHub and my English is ass, so if I did something wrong let me know ;)

Steps to reproduce

let g:vimtex_view_method = 'skim'
let g:vimtex_view_skim_sync = 1
let g:vimtex_view_skim_activate = 0
let g:vimtex_indent_enabled = 0
let g:vimtex_quickfix_mode=1

let g:vimtex_quickfix_ignore_filters = [
      \ 'Underfull',
      \ 'Overfull',
      \ 'Warning',
      \]

let g:vimtex_quickfix_open_on_warning = 0

let g:vimtex_matchparen_enabled = 0
let g:vimtex_fold_enabled = 0 
let g:vimtex_complete_recursive_bib=0
let g:vimtex_syntax_enabled = 0
let g:vimtex_toc_enabled = 0
let g:vimtex_matchparen_enabled =0
let g:vimtex_motion_enabled = 0
"let g:vimtex_quickfix_enabled = 0
let g:vimtex_text_obj_enabled =0
"let g:vimtex_view_enabled = 0
let g:vimtex_complete_enable = 0
let g:vimtex_fold_bib_enabled = 0
let g:vimtex_indent_enabled = 0
let g:vimtex_indent_bib_enabled = 0
let g:vimtex_doc_enabled = 0
let g:vimtex_include_search_enabled = 0
let g:vimtex_viewer_check = 0

Expected behavior

No response

Actual behavior

No response

Do you use a latexmkrc file?

No

VimtexInfo

System info:
  OS: Mac OS X 10.15.7 (19H15)
  Vim version: NVIM v0.8.2
  Has clientserver: true
  Servername: /var/folders/1p/0hs20_dj0kv566xzkq3tt6cw0000gn/T/nvim.mattia/CZ2Xw3/nvim.3538.0

VimTeX project: Analisi_1
  base: Analisi_1.tex
  root: /Users/mattia/Library/Mobile Documents/com~apple~CloudDocs/LatexWorkspace/Analisi_1
  tex: /Users/mattia/Library/Mobile Documents/com~apple~CloudDocs/LatexWorkspace/Analisi_1/Analisi_1.tex
  main parser: current file verified
  document class: article
  packages: amsbsy amsfonts amsgen amsmath amsopn amssymb amstext array atbegshi atbegshi-ltx atveryend atveryend-ltx auxhook babel bigintcalc bitset calc elocalloc environ epstopdf-base etexcmds etoolbox expl3 float forest geometry gettitlestring graphics graphicx hycolor hyperref iftex ifvtex infwarerr inlinedef inputenc intcalc keyval kvdefinekeys kvoptions kvsetkeys letltxmacro listings lstmisc ltxcmds mathrsfs mathtools mhsetup nameref pdfescape pdftexcmds pgf pgf-pie pgfcomp-version-0-65 pgfcomp-version-1-18 pgfcore pgffor pgfkeys pgfmath pgfopts pgfplots pgfrcs pgfsys refcount rerunfilecheck scalefnt soul soulutf8 tabularx tikz tocloft trig trimspaces uniquecounter url xcolor xparse
  source files:
    Analisi_1.tex
    ../.preamble.tex
    Sections/1.tex
    Sections/2.tex
    Sections/3.tex
    Images/Riemann.pdf_tex
  compiler: latexmk
    engine: -pdf
    options:
      -verbose
      -file-line-error
      -synctex=1
      -interaction=nonstopmode
    callback: 1
    continuous: 1
    executable: latexmk
  viewer: Skim
  qf method: LaTeX logfile

[lervag]

Hi! Newbie question right here. First of all thank you for developing such a beautiful plugin for free!

<3

I was wondering if there was any way to reduce startup times. I know you are not supposed to lazy load the plugin, but even on rather small files (~500 lines), vimtex seems to add about 150-200 milliseconds to startup times.

Well, yes and no. In my humble opinion, you should distinguish startup time for a general file and the startup time for loading a specific file and filetype. VimTeX is lazy in the sense that it will not add significant time to your general startup time. However, as you've noticed, it will add some time for loading .tex files. That is, loading VimTeX for a file will take some time.

You should be able to reduce the loading time by disabling various components. I believe the part that takes the longest time loading is the syntax script. You could disable it with let g:vimtex_syntax_enabled = 0. You can also try to disable other parts to look into which parts are the slowest. I believe the matchparen stuff may also be one of the more intensive.

Still, and again my own opinion(!), the things like syntax that does add to your startuptime should not be very slow compared to the quality of features they deliver. But this is certainly a subjective matter and you are free to make your own choice.

I also tried disabling every feature I could to see if that could lead to any improvement but it still seemed too slow for me (I attached the file of that configuration).

Good to see you already figured out what I mentioned above! Unfortunately, I don't really have any other suggestions. I can take a look on my side to see if I can notice some obvioust thing that can be optimized.

My last resort was trying to manually copy/paste the parts I strongly need (so basically the context detection and the compile module) and create a "light version" of the plugin. That has been possible for the former option, but not for the latter, as the compiler module seems to be more articulated.

Yes, things like the compilation module is quite articulated, as you say, and it is not straightforward to break it out as a separate feature. But I've tried my best to keep a relatively clean and "untangled" code base, so I believe it could be worse. ;)

P.s. I have read through the "high level documentation" of the plugin, but i would love to take a deeper dive to learn a bit more. Is there any other more detailed doc available to public?

No, :help vimtex is the main source of documentation. Everything else is prone to become outdated.

P.p.s. This is my first issue on GitHub and my English is ass, so if I did something wrong let me know ;)

Then congratulations opening a good first issue :)

[lervag]

Ok, I looked into a large example and I see the main reason for what you find a larger startup time is the code that parses the document to detect the source files, i.e. this list:

VimTeX project: Analisi_1
  ...
  source files:
    Analisi_1.tex
    ../.preamble.tex
    Sections/1.tex
    Sections/2.tex
    Sections/3.tex
    Images/Riemann.pdf_tex

This list is useful/necessary in may cases/scenarios and is parsed on initialization. I could look into if it was possible to postpone it until the first time the list is used (i.e. turn it into a function).

[lervag]

I believe I've made some progress here. Please update and test again. It will probably still be O(100 ms), but I think it may be significantly faster than before.

[mattia-marini]

Thank you!
The update seems to bring significant benefits. On the same file I tested, now i get almost half the time with the same config.

I did some further benchmarking though, and I noticed a thing: another main culprit of startup slowdown seem to be the my preamble (~500 lines) which has perhaps to do with the main file detection algorithm.
I benchmarked a file around 20.000 lines long. Things I noticed:

• There seemed to be little to no difference in opening an empty file vs opening a very large one
• There seemed to be no difference at all in inputing large files after \begin{document}
• There seemed to be a huge difference in importing big files before \begin{document}

Theese are the results from benchmarking a ~20.000 lines latex file with a preamble of ~4.000

Without preamble		With preamble
W/ buffer variable	W/o buffer variable	W/ buffer variable	W/o buffer variable
~ 80 ms	~ 80/90 ms	~ 210 ms	~ 330/320 ms

My guess is that vimtex scans through all the preamble and all included files until it finds the \begin{document} line, in order to make sure that b:vimtex_main is actually set to the main file. That would also explain why startup times increase significantly when the file indicated by b:vimtex_main does not contain \begin{document} or when the buffer variable is set to a non-existing file (since in both cases vimtex would fall back to recursive search, right?).

I remember reading about an option to disable the fallback system, but I think it was deprecated.

My workaround was to place a commented line %\begin{document} before inputing my preamble, to avoid vimtex reading through it, and that worked fine!

I imagine that the checking procedure is intended, but it would be nice being able do disable it, since I went from having a startup time of ~350 ms on large file, to a startup time of ~70/80 ms regardless of file size.

Thank a lot again for the speedy response and the great work!

[lervag]

Thank you! The update seems to bring significant benefits. On the same file I tested, now i get almost half the time with the same config.

Glad to hear it!

I did some further benchmarking though, and I noticed a thing: another main culprit of startup slowdown seem to be the my preamble (~500 lines)

Ah, yes: we still parse the preamble. I notice we do it three times during initialization. Since this can also be a little bit expensive, I've added a cache that should further improve things, I believe. Could you repeat the benchmark after updating?

Notice, though:

There seemed to be little to no difference in opening an empty file vs opening a very large one

The lag has to do with parsing the entire project starting from the main file, not the current file.

There seemed to be no difference at all in inputing large files after \begin{document}

As expected, since we no longer parse the entire project at startup.

There seemed to be a huge difference in importing big files before \begin{document}

Yes, again as expected. However, note that it is very uncommon to have a huge amount of content in the preamble...

I remember reading about an option to disable the fallback system, but I think it was deprecated.

Yes, it is deprecated now as I see no need of it any longer.

My workaround was to place a commented line %\begin{document} before inputing my preamble, to avoid vimtex reading through it, and that worked fine!

I hope the recent update should improve things further so you don't need this. The workaround is not good, as it may break some of the features that are based on parsing the preamble.

Thank a lot again for the speedy response and the great work!

Thanks for helping me find things to improve :)