VimtexInfo throwing error on MSYS2 shell + unable to compile

[stanle-cs]

Description

Unable to run \ll on Windows, error throw:
/usr/bin/bash: /s: No such file or directory

Steps to reproduce

1. Install vimtex on Windows
2. Open a .tex file
3. Check VimtexInfo

Expected behavior

System info:
OS: USSNFL123 (Microsoft Windows 10 Pro)
Vim version: NVIM v0.9.0-dev-206+gc3aba403c
Has clientserver: true
Servername: \.\pipe\nvim.19508.0

Actual behavior

System info:
4 OS: Windows (['/usr/bin/bash: /s: No such file or directory'])
3 Vim version: NVIM v0.9.0-dev-206+gc3aba403c
2 Has clientserver: true
1 Servername: \.\pipe\nvim.1612.0

Do you use a latexmkrc file?

No

VimtexInfo

System info:
    4   OS: Windows (['/usr/bin/bash: /s: No such file or directory'])
    3   Vim version: NVIM v0.9.0-dev-206+gc3aba403c
    2   Has clientserver: true
    1   Servername: \\.\pipe\nvim.1612.0
  6
    1 VimTeX project: ttd_upstream
    2   base: ttd_upstream.tex
    3   root: C:\msys64\home\XX\LaTeX\ttd_upstream
    4   tex: C:\msys64\home\XX\LaTeX\ttd_upstream\ttd_upstream.tex
    5   main parser: current file verified
    6   document class: article
    7   packages: fontspec
    8   compiler: latexmk
    9     engine: -pdf
   10     options:
   11       -xelatex
   12       -file-line-error
   13       -synctex=1
   14       -interaction=nonstopmode
   15     build_dir: ttd_upstream
   16     callback: 1
   17     continuous: 1

[lervag]

Strange. It seems to be some problem with the 'shell' setting. If you open neovim and do :!echo "test", what does it do?

I'm not sure I've heard of anyone using MSYS2 with neovim and VimTeX before. Did you consider to use WSL instead? I believe it should be a much better solution than MSYS2 these days.

[stanle-cs]

Hi lervag, first off thank you for your awesome work on this plugin.

I have a niche use case for the plugin, but one that I'm sure can be a lot more common: I'm using my work computer to draft LaTeX documents to communicate with my peers. As such, I have to use Windows, and I don't have admin privilege. That means no WSL.

I can install MSYS2 as it does not require admin right. I much prefer using the MSYS2 shell since it will display all the symbols that vim-airline use. This leads to the problem we see above.

As for the test:

:!echo "test"
"test"

It took me a while to find a setting that satisfy the health#coc#check. My settings are:

let &shell='"C:\\Windows\\System32\\cmd.exe"'
let &shellcmdflag="/s /c"
set shellxquote=\"

Can you suggest any patches for the MSYS2 shell? I can apply them myself if it's too niche.

[lervag]

I'm glad to hear you find VimTeX to be useful! :)

So, let's look into this. It may take some back and forth to debug, but I believe we should be able to figure something out. First off, while testing, we need to ensure that the caching does not "hide" any bugs or improvements. So, could you add let g:vimtex_cache_persistent = 0 to your config - this should be removed when things work, because the caching is very useful for performance reasons. You may also want to do :VimtexClearCache ALL when we get things to work, again, to make sure the caching did not prevent anything.

The lines in the VimTeX code that are generating the \li output for the OS: line are here:

vimtex/autoload/vimtex/info.vim

Lines 194 to 204 in e148cf1

	if !exists('s:win_info')
	let s:win_info = vimtex#jobs#cached('systeminfo')
	endif
	try
	let l:name = vimtex#util#trim(matchstr(s:win_info[1], ':\s*\zs.*'))
	let l:version = vimtex#util#trim(matchstr(s:win_info[2], ':\s*\zs.*'))
	return l:name . ' (' . l:version . ')'
	catch
	return 'Windows (' . string(s:win_info) . ')'
	endtry

Thus, the call that is problematic is this: vimtex#jobs#cached('systeminfo'). We want to investigate this, and it is not so deep. First, we can avoid the cache, so, if you open neovim in MSYS2 and run :echo vimtex#jobs#capture('systeminfo'), it should replicate the error message - I think. Please confirm!

[lervag]

The first step is to fully recognize exactly where things go wrong. I believe these lines are the ones that generate the error:

vimtex/autoload/vimtex/jobs/neovim.vim

Lines 156 to 167 in e148cf1

	function! s:neovim_win_capture(cmd) abort " {{{1
	let s:saveshell = [&shell, &shellcmdflag, &shellslash]
	set shell& shellcmdflag& shellslash&
	let l:output = systemlist('cmd /s /c "' . a:cmd . '"')
	let [&shell, &shellcmdflag, &shellslash] = s:saveshell
	return l:output
	endfunction
	" }}}1

You could easily inspect that by adding e.g. unsilent echo 'DEBUG SEE HERE' somewhere inside and verifying if it prints when running :call vimtex#jobs#capture('systeminfo'). However, this may be where the problem actually happens, because I'm not sure if neovim in MSYS2 is recognized as windows. I.e., what's the output of :echo has('win32') in your neovim? If it is 0, then this is the code that is running:

vimtex/autoload/vimtex/jobs/neovim.vim

Lines 140 to 144 in e148cf1

	function! s:neovim_unix_capture(cmd) abort " {{{1
	return systemlist(['sh', '-c', a:cmd])
	endfunction
	" }}}1

[lervag]

Finally, it would be very useful if you could create a minimal example. Something along these lines: Create a test.vim with the following content:

set nocompatible
" Change the below to the correct path to VimTeX
set runtimepath^=~/.local/plugged/vimtex
filetype plugin indent on
syntax enable

" Possibly add settings like these (whatever is required to reproduce the
" issue)
let &shell='"C:\\Windows\\System32\\cmd.exe"'
let &shellcmdflag="/s /c"
set shellxquote=\"

" Run something that fails with the same issue as the original problem:
" Current hypothesis is that this is "systeminfo", but that needs
" verification.
echo vimtex#jobs#capture('systeminfo')

quitall

Now, in an MSYS2 terminal, you would do nvim --headless --clean -u test.vim. If we can successfully create such a minimal example, it will be much easier working towards some kind of solution.

[stanle-cs]

Alrighty, here are the results:
Cache cleared and persistence off

:echo vimtex#jobs#capture('systeminfo')
['/usr/bin/bash: /s: No such file or directory']

:call vimtex#jobs#capture('systeminfo')
(no output)

:echo has('win32')
1

Then finally the .minimalvimrc:

  1   set nocompatible
    1 let &runtimepath  = 'C:/Users/XX/AppData/Local/nvim-data/plugged/vimtex' . &runtimepath
    2 filetype plugin indent on
    3 syntax enable
    4 " Add relevant options and VimTeX configuration below.
    5 " Possibly add settings like these (whatever is required to reproduce the
    6 " issue)
    7 let &shell='"C:\\Windows\\System32\\cmd.exe"'
    8 let &shellcmdflag="/s /c"
    9 set shellxquote=\"
   10
   11 " Run something that fails with the same issue as the original problem:
   12 " Current hypothesis is that this is "systeminfo", but that needs
   13 " verification.
   14 echo vimtex#jobs#capture('systeminfo')
   15
   16 quitall
~

$ vim --headless --clean -u ~/.minimalvimrc
Error detected while processing C:/msys64/home/XX/.minimalvimrc:
line   15:
E117: Unknown function: vimtex#jobs#capture

I hope this can help?

[stanle-cs]

False alarm on the last test! I didn't follow your minimal vimrc somehow!

Replaced

let &runtimepath  = 'C:/Users/XX/AppData/Local/nvim-data/plugged/vimtex' . &runtimepath

with

 set runtimepath^=C:\\Users\\XX\\AppData\\Local\\nvim-data\\plugged\\vimtex

Result:

$ vim --headless --clean -u ~/.minimalvimrc
['/usr/bin/bash: /s: No such file or directory']

[lervag]

Great, having the minimal vimrc "working" is a very useful starting point!

Does changing the &shell option(s) have any effect?

So, next point is to ensure we know which point is being executed. vimtex#jobs#capture on neovim with Windows (has('win32')) should run the code I referenced above, but for some reason the :unsilent echo ... statement I asked you to add did not seem to trigger. It seems to me that you might be capable to partly debug this already on your side. The easiest trick is print-debugging. Can you add e.g. :unsilent echo "DBG 1" at the first point we know is executed, then moving things deeper until we reach the spot where things go wrong? Let me know if this is unclear, I'll try and explain better.

[stanle-cs]

Hi lervag,

Sorry, work has been super busy this week, so I didn't have a chance to look into it until today. I think I understood what you said now. Earlier in the test, I somehow forgot to put the DEBUG echo into the code. I tried the new neovim.vim file like follow:

 function! s:neovim_win_capture(cmd) abort " {{{1
    1   unsilent echo 'DEBUG SEE HERE: 1'
    2   let s:saveshell = [&shell, &shellcmdflag, &shellslash]
    3   set shell& shellcmdflag& shellslash&
    4
    5   unsilent echo 'DEBUG SEE HERE: 2'
    6   let l:output = systemlist('cmd /s /c "' . a:cmd . '"')
    7
    8   unsilent echo 'DEBUG SEE HERE: 3'
    9   let [&shell, &shellcmdflag, &shellslash] = s:saveshell
   10
   11   unsilent echo 'DEBUG SEE HERE: 4'
   12
   13   return l:output
   14 endfunction

The result is:

$ vim --headless --clean -u ~/.minimalvimrc
DEBUG SEE HERE: 1
DEBUG SEE HERE: 2
DEBUG SEE HERE: 3
DEBUG SEE HERE: 4
['/usr/bin/bash: /s: No such file or directory']

[stanle-cs]

Also, I was curious to see which shell is being used, so I add:

16 function! s:neovim_win_capture(cmd) abort " {{{1
   15   unsilent echo 'DEBUG SEE HERE: 1, $shell: ' $shell
   14   let s:saveshell = [&shell, &shellcmdflag, &shellslash]
   13   set shell& shellcmdflag& shellslash&
   12
   11   unsilent echo 'DEBUG SEE HERE: 2, $shell: ' $shell
   10   let l:output = systemlist('cmd /s /c "' . a:cmd . '"')
    9
    8   unsilent echo 'DEBUG SEE HERE: 3, $shell: ' $shell
    7   let [&shell, &shellcmdflag, &shellslash] = s:saveshell
    6
    5   unsilent echo 'DEBUG SEE HERE: 4, $shell: ' $shell
    4
    3   return l:output
    2 endfunction

And get this result:

$ vim --headless --clean -u ~/.minimalvimrc
DEBUG SEE HERE: 1, $shell:  C:\msys64\usr\bin\bash.exe
DEBUG SEE HERE: 2, $shell:  C:\msys64\usr\bin\bash.exe
DEBUG SEE HERE: 3, $shell:  C:\msys64\usr\bin\bash.exe
DEBUG SEE HERE: 4, $shell:  C:\msys64\usr\bin\bash.exe
['/usr/bin/bash: /s: No such file or directory']

So looks like the let &shell="...\\cmd.exe" is not honored by vimtex.

[lervag]

Sorry, work has been super busy this week, so I didn't have a chance to look into it until today.

No worries!

I think I understood what you said now. Earlier in the test, I somehow forgot to put the DEBUG echo into the code. I tried the new neovim.vim file like follow: …

The result is: …

Also, I was curious to see which shell is being used, so I add:

Thanks! That's good thinking. This means we know much about where our problem lies!

What's surprising to me is this:

DEBUG SEE HERE: 1, $shell: C:\msys64\usr\bin\bash.exe

VimTeX does not change the &shell* settings permanently. It only temporarily changes them. Thus, I would have expected the 1st and 4th debug outputs to be your original shell setting. My hypothesis is therefore that these settings are changed by something else (not VimTeX).

When you ran these tests, did you use the minimal example we created earlier? I.e., did you run vim --headless --clean -u ~/.minimalvimrc? (I notice you used vim, but I guess that's aliased to nvim or you just typoed?)

If you open a .tex file (doesn't matter which) with your regular vimrc file, then do :verbose set shell - what's the output?

[stanle-cs]

Ahh I think you're onto something here. And yes, vim is an alias for nvim v0.9.0.

For some reasons, the set &shell=....\\cmd.exe will satisfy health#coc#check but when I tried to open a terminal with :terminal then it will throw an error with E475: Invalid value for argument cmd: 'C:WindowsSystem32cmd.exe' is not executable

Now, after I found that out, I changed my .vimrc and my .minimalvimrc to:

set shell=cmd

Which satisfy both health#coc#check and also correctly open the terminal. Why that is the case I have no idea, and maybe something that the neovim team would like to take a look at. I can potentially report this to them if I can't find out why.

However, even with that change, the vimtex bug still exist. I still get:

$ vim --headless --clean -u ~/.minimalvimrc
DEBUG SEE HERE: 1, $shell:  C:\msys64\usr\bin\bash.exe
DEBUG SEE HERE: 2, $shell:  C:\msys64\usr\bin\bash.exe
DEBUG SEE HERE: 3, $shell:  C:\msys64\usr\bin\bash.exe
DEBUG SEE HERE: 4, $shell:  C:\msys64\usr\bin\bash.exe
['/usr/bin/bash: /s: No such file or directory']

And yes that is with the .minimalvimrc :

set nocompatible
set runtimepath^=C:\\Users\\XXX\\AppData\\Local\\nvim-data\\site\\pack\\packer\\start\\vimtex
filetype plugin indent on
syntax enable
" Add relevant options and VimTeX configuration below.
" Possibly add settings like these (whatever is required to reproduce the
" issue)
set shell=cmd
let &shellcmdflag="/s /c"
set shellxquote=\"

" Run something that fails with the same issue as the original problem:
" Current hypothesis is that this is "systeminfo", but that needs
" verification.
echo vimtex#jobs#capture('systeminfo')

quitall

As for the question of which &shell is being set, here's what I got when I open a .tex file:

:verbose set shell
  shell=cmd
        Last set from ~/.vimrc line 6

[stanle-cs]

So, I tried to modify neovim.vim like this:

function! s:neovim_win_capture(cmd) abort " {{{1
  unsilent echo 'DEBUG SEE HERE: 1, $shell: ' $shell
  let s:saveshell = [&shell, &shellcmdflag, &shellslash]
  set shell& shellcmdflag& shellslash&
  "Debug
  set shell=cmd
  let &shellcmdflag="/s /c"
  set shellxquote=\""
  "End Debug

  unsilent echo 'DEBUG SEE HERE: 2, $shell: ' $shell
  let l:output = systemlist('cmd /s /c "' . a:cmd . '"')

  unsilent echo 'DEBUG SEE HERE: 3, $shell: ' $shell
  let [&shell, &shellcmdflag, &shellslash] = s:saveshell

  unsilent echo 'DEBUG SEE HERE: 4, $shell: ' $shell

  return l:output
endfunction

This will change the result of .minimalvimrc to:

$ vim --headless --clean -u ~/.minimalvimrc
DEBUG SEE HERE: 1, $shell:  C:\msys64\usr\bin\bash.exe
DEBUG SEE HERE: 2, $shell:  C:\msys64\usr\bin\bash.exe
DEBUG SEE HERE: 3, $shell:  C:\msys64\usr\bin\bash.exe
DEBUG SEE HERE: 4, $shell:  C:\msys64\usr\bin\bash.exe
['', 'Host Name:                 USSNFL123', 'OS Name:                   Microsoft Windows 10 Pro', 'OS Version:                10.0.19044 N/A Build 19044', 'OS Manufacturer:           Microsoft Corporation', 'OS Configuration:          Standalone Workstation', 'OS Build
 Type:             Multiprocessor Free
[Redacted since this is a work laptop]
]

(I hid the data since this is a work laptop, but you get the point).
Now that will give a good VimtexInfo result:

System info:
   OS: USSNFL123 (Microsoft Windows 10 Pro)
   Vim version: NVIM v0.9.0-dev-206+gc3aba403c
   Has clientserver: true
   Servername: \\.\pipe\nvim.30676.0

But, when I try to do \ll I still get:

  1   /usr/bin/bash: /s: No such file or directory