VimTex executes tectonic compiler without escaping path spaces

[henriqpsantos]

Description

Maybe I'm messing up somewhere, since I've seen that the "output-dir" issue has been supposedly fixed in a PR, but I'm having it nonetheless. I'm using neovim 0.9.0, latest vimtex, lazy.nvim as a package manager. Executing :VimtexInfo (which errors out on the first execution, see below) gives the following information:

Image with censored name. If the path is C:\Users\Albert Einstein\Documents\a.tex, it shows up as C:\Users\Albert\ Einstein\(...)
I'm not sure why it's erroring out on the first execution of VimtexInfo, but it might be related. Configuration is literally 2 lines in init.lua:

vim.g.vimtex_view_method = 'sioyek'
vim.g.vimtex_compiler_method = 'tectonic'

Steps to reproduce

1. Open a tex file whose path contains spaces
2. Run :VimtexCompile or :VimtexInfo
3. See that file doesn't compile or compiles with an error. :VimtexInfo shows the command as escaped with "\ "

Expected behavior

The document to compile and :VimtexInfo to work

Actual behavior

:VimtexInfo errors on first execution.
Tectonic is run with an output dir that has a space

Do you use a latexmkrc file?

No

VimtexInfo

System info:
  OS:  ()
  Vim version: NVIM v0.9.0-dev-974+g631775c05
  Has clientserver: true
  Servername: \\.\pipe\nvim-nvr

VimTeX project: curriculum-pt
  base: curriculum-pt.tex
  root: C:\Users\---\Internxt\Documentos\CV\CV
  tex: C:\Users\---\Internxt\Documentos\CV\CV\curriculum-pt.tex
  main parser: current file verified
  document class: moderncv
  packages: geometry ragged2e
  source files:
    curriculum-pt.tex
    cm-pt.tex
  compiler: tectonic
    options:
      --keep-logs
      --synctex
  viewer: sioyek
  qf method: LaTeX logfile

[lervag]

It seems you have two different issues.

For the first one: What's the outut when you do :!systeminfo in your neovim? Or if you open a cmd prompt in Windows and run systeminfo there? It seems you are not using utf8 encoding. I believe you can safely change to unicode/utf8 and that this may resolve the issue.

Now:

Image with censored name.

Out of curiosity, why do you censor the username? Obviously, feel free to do it, but I don't see any good reasons. Unless it is your full name, perhaps?

If the path is C:\Users\Albert Einstein\Documents\a.tex, it shows up as C:\Users\Albert\ Einstein\(...)

Configuration is literally 2 lines in init.lua:

No! Your configuration for VimTeX options may be 2 lines, but you clearly have a lot more configuration, including how you load VimTeX. And to properly investigate issues, you should make a minimal init.lua file that only has what you need to reproduce the problem, and you should share this minimal init.lua.

It would also be useful to know more of how you run neovim. Are you in WSL shell? Are you using some gui version? Are you running neovim from a CMD shell?

[henriqpsantos]

I censored the username because it is my full name indeed :)
I've used this alias and managed to keep it somewhat untainted with personal info :P

Regarding systeminfo: I usually run neovim using neovide. In any case, the same error occurs in terminal or neovide.

Here are the outputs of systeminfo:

Host Name:                 ASUS
OS Name:                   Microsoft Windows 10 Home
OS Version:                10.0.19045 N/A Build 19045
OS Manufacturer:           Microsoft Corporation
OS Configuration:          Standalone Workstation
OS Build Type:             Multiprocessor Free
Registered Owner:          Windows User
Registered Organization:   
Product ID:                00326-10000-00000-AA449
Original Install Date:     03/10/2020, 02:14:44
System Boot Time:          11/03/2023, 23:35:59
System Manufacturer:       To be filled by O.E.M.
System Model:              To be filled by O.E.M.
System Type:               x64-based PC
Processor(s):              1 Processor(s) Installed.
                           [01]: AMD64 Family 21 Model 2 Stepping 0 AuthenticAMD ~4000 Mhz
BIOS Version:              American Megatrends Inc. 2603, 12/02/2015
Windows Directory:         C:\WINDOWS
System Directory:          C:\WINDOWS\system32
Boot Device:               \Device\HarddiskVolume3
System Locale:             pt;Portuguese (Portugal)
Input Locale:              pt;Portuguese (Portugal)
Time Zone:                 (UTC+00:00) Dublin, Edinburgh, Lisbon, London
Total Physical Memory:     8 091 MB
Available Physical Memory: 1 704 MB
Virtual Memory: Max Size:  17 307 MB
Virtual Memory: Available: 6 003 MB
Virtual Memory: In Use:    11 304 MB
Page File Location(s):     D:\pagefile.sys
Domain:                    WORKGROUP
Logon Server:              \\ASUS
Hotfix(s):                 25 Hotfix(s) Installed.
                           [01]: KB5022502
                           [02]: KB4561600
                           [03]: KB4577266
                           [04]: KB4577586
                           [05]: KB4580325
                           [06]: KB4586864
                           [07]: KB4589212
                           [08]: KB4593175
                           [09]: KB4598481
                           [10]: KB5000736
                           [11]: KB5012170
                           [12]: KB5015684
                           [13]: KB5022834
                           [14]: KB5006753
                           [15]: KB5007273
                           [16]: KB5011352
                           [17]: KB5011651
                           [18]: KB5014032
                           [19]: KB5014035
                           [20]: KB5014671
                           [21]: KB5015895
                           [22]: KB5016705
                           [23]: KB5018506
                           [24]: KB5020372
                           [25]: KB5005699
Network Card(s):           4 NIC(s) Installed.
                           [01]: Realtek PCIe GbE Family Controller
                                 Connection Name: Ethernet
                                 DHCP Enabled:    Yes
                                 DHCP Server:     192.168.0.1
                                 IP address(es)
                                 [01]: 192.168.0.173
                                 [02]: fe80::13e1:49f3:520a:48d8
                           [02]: Wintun Userspace Tunnel
                                 Connection Name: OpenVPN Wintun
                                 Status:          Media disconnected
                           [03]: TAP-Windows Adapter V9
                                 Connection Name: OpenVPN TAP-Windows6
                                 Status:          Media disconnected
                           [04]: OpenVPN Data Channel Offload
                                 Connection Name: OpenVPN Data Channel Offload
                                 Status:          Media disconnected
Hyper-V Requirements:      A hypervisor has been detected. Features required for Hyper-V will not be displayed.

It seems you are not using utf8 encoding. I believe you can safely change to unicode/utf8 and that this may resolve the issue.

Bizarrely enough, I think I already am using UTF-8 encoding, at least neovim says so - set encoding prints set encoding=utf-8.

Regarding my second problem, I've come across an issue remarking that (neo)vim+windows+powershell is a pain to setup to run commands, because of shell escape rules that are poorly thought out. This is exactly what I'm using.

For the time being, I've managed to get it to work with a command runner called just, which is triggered as a generic compiler, and runs the command vimtex would anyways. Just forces me to have a justfile on each LaTeX project, but its output does play nicely with Vimtex and it's way better than not having vimtex support at all.

I will do the minimal init.lua stuff tomorrow, it's quite late here. But if the issue I'm seeing is due to powershell+neovim problems, I think there might be no solution that doesn't involve switching to cmd.

[lervag]

I censored the username because it is my full name indeed :) I've used this alias and managed to keep it somewhat untainted with personal info :P

Makes sense :)

Regarding systeminfo: I usually run neovim using neovide. In any case, the same error occurs in terminal or neovide.

Thanks for sharing it; it's mostly the same for all three environments, so I've
included it verbatim in your reply with a <details>...</details> tag.

Here are the outputs of systeminfo:

To be honest, there's nothing that looks suspicious there.

It seems you are not using utf8 encoding. I believe you can safely change to unicode/utf8 and that this may resolve the issue.

Bizarrely enough, I think I already am using UTF-8 encoding, at least neovim says so - set encoding prints set encoding=utf-8.

Ah, no, I meant on your Windows system, not in neovim. I was thinking this may be a problem with your locale, somehow. But I'm not sure!

Regarding my second problem, I've come across an issue remarking that (neo)vim+windows+powershell is a pain to setup to run commands, because of shell escape rules that are poorly thought out. This is exactly what I'm using.

So, you are changing your shell settings in neovim? I strongly advice against that, unless you really know what you are doing. Even if you use powershell, there is usually no reason to make neovim/Vim use it.

However, it should not really matter, because VimTeX reverts all shell related settings for you while running shell commands.

Can you do a :VimtexClearCache ALL? Perhaps the problem lies in the cache file. After you do this, then restart neovim; does :VimtexInfo work?

I will do the minimal init.lua stuff tomorrow, it's quite late here. But if the issue I'm seeing is due to powershell+neovim problems, I think there might be no solution that doesn't involve switching to cmd.

Doing the minimal init.lua is very useful to figure out if this is a configuration problem or something else. So I strongly suggest you do that.

[henriqpsantos]

Here's the minimal.lua config I used. It behaves and returns the same error:

local cmd = vim.cmd
local g = vim.g

cmd ('filetype plugin indent on')
cmd ('syntax enable')

-- Vimtex rtp
vim.o.runtimepath = "C:/Users/----/Documents/.data/nvim-data/lazy/vimtex," .. vim.o.runtimepath .. ",C:/Users/----/Documents/.data/nvim-data/lazy/vimtex/after"

-- Configuration for vimtex
g.vimtex_view_method = 'sioyek'
g.vimtex_compiler_method = 'tectonic'

g.vimtex_log_ignore = {
	'Underfull',
	'Overfull',
	'specifier changed to',
	'Token not allowed in a PDF string',
}

minimal.tex is the same as in the issue template.

:VimtexInfo yields the same error, even on this minimal config.

:VimtexCompile, however, does work fine now. So it must be a set shell + powershell error on my end, I guess... Is there any way to use powershell for my :! and :term workflow while vimtex still works in cmd as usual?

Thanks for the help and apologies for not giving out this info earlier! :)

[lervag]

I'll respond in more detail later; just a quick question now.

:VimtexCompile, however, does work fine now. So it must be a set shell + powershell error on my end, I guess... Is there any way to use powershell for my :! and :term workflow while vimtex still works in cmd as usual?

Can you adjust the minimal.lua with e.g. set shell to reproduce this error as well? If so, then I can try and start a windows in VM to see if I can reproduce this myself.

[henriqpsantos]

I just realized that I forgot my initial issue was with a path that contained spaces! (I switched to a laptop where my username is shorter and has no spaces lol)

On a folder called minimal with spaces, the issue with compilation remains: Vimtex cannot compile on a path that contains spaces.

Here's the new minimal.lua

vim.opt.runtimepath:prepend "C:/Users/----/Documents/.data/nvim-data/lazy/vimtex"
vim.opt.runtimepath:append "C:/Users/----/Documents/.data/nvim-data/lazy/vimtex/after"

vim.cmd 'filetype plugin indent on'
vim.cmd 'syntax enable'

vim.o.shell = 'pwsh'
vim.o.shellcmdflag = '-NoLogo -NoProfile -ExecutionPolicy RemoteSigned -Command [Console]::InputEncoding=[Console]::OutputEncoding=[System.Text.Encoding]::UTF8;'
vim.o.shellredir = '2>&1 | Out-File -Encoding UTF8 %s; exit $LastExitCode'
vim.o.shellpipe = '2>&1 | Out-File -Encoding UTF8 %s; exit $LastExitCode'
vim.o.shellquote = ''
vim.o.shellxquote = ''

vim.g.vimtex_view_method = 'sioyek'
vim.g.vimtex_compiler_method = 'tectonic'

and here's VimtexInfo with that within minimal with spaces:

System info:
  OS: LAPTOP-QDMBVVL7 (Microsoft Windows 10 Home)
  Vim version: NVIM v0.9.0-dev-1028+gd422fc827
  Has clientserver: true
  Servername: \\.\pipe\nvim-nvr

VimTeX project: minimal
  base: minimal.tex
  root: C:\Users\----\Internxt\Development\.config\minimal with spaces
  tex: C:\Users\----\Internxt\Development\.config\minimal with spaces\minimal.tex
  main parser: current file verified
  document class: minimal
  compiler: tectonic
    options:
      --keep-logs
      --synctex
    job: 
      jobid: 5
      output: C:\Users\----\AppData\Local\Temp\nvim.0\wt0xJ8\0
      cmd: tectonic --keep-logs --synctex --outdir=C:\Users\----\Internxt\Development\.config\minimal\ with\ spaces "minimal.tex"
  viewer: sioyek
    job: 
      pid: -
      cmd: sioyek  --inverse-search "C:\Program Files\Neovim\bin\nvim.exe --headless -c \"VimtexInverseSearch %2 '%1'\"" --forward-search-file "C:\\Users\\----\\Internxt\\Development\\.config\\minimal with spaces\\minimal.tex" --forward-search-line 1 "C:\\Users\\----\\Internxt\\Development\\.config\\minimal with spaces/minimal.pdf"
    cmd_start: sioyek  --inverse-search "C:\Program Files\Neovim\bin\nvim.exe --headless -c \"VimtexInverseSearch %2 '%1'\"" --forward-search-file "C:\\Users\\----\\Internxt\\Development\\.config\\minimal with spaces\\minimal.tex" --forward-search-line 1 "C:\\Users\\----\\Internxt\\Development\\.config\\minimal with spaces/minimal.pdf"
  qf method: LaTeX logfile

I'll edit to add this observation: sioyek opens the file just fine. However, in the tectonic path, its spaces are escaped vimlike \ , and in sioyek's command they're not escaped, instead being in double quotes with the backslashes escaped as \\. This is, I think, the issue.

[lervag]

Sorry for the late reply, been quite busy lately.

I just realized that I forgot my initial issue was with a path that contained spaces! (I switched to a laptop where my username is shorter and has no spaces lol)

It seems :VimtexInfo now works as expected?

On a folder called minimal with spaces, the issue with compilation remains: Vimtex cannot compile on a path that contains spaces.

Here's the new minimal.lua

First, I hope you don't mind, but I updated your posted minimal.lua with some changes. I removed one of the VimTeX options that seems clearly unrelated here. Further, I simplified the code to be more consise.

---

Now, before we go further and before I respond to your observations:

1. What happens if you remove all of the vim.o.shell options? Do things now work as expected?

2. If yes: Which of the vim.o.shell options are necessary to reproduce the issue? I'm quite certain that it is not all of them.

3. Finally, what the steps you take to reproduce the issue? You don't need a long explanation, but feel free to be explicit. E.g.:

   a. nvim --clean -u minimal.lua test.tex
   b. Run :VimtexCompile
   c. Observe some error or something?
   d. Run :VimtexView
   e. Observe some error or something?

[henriqpsantos]

You're helping me for free, I don't see why you're apologising for being busy xD

Now, regarding the :VimtexInfo problem:

this happens on any tex file, on the absolute minimum config possible. I ran it with nvim --clean -u minimal.lua minimal.tex, where minimal.lua was:

vim.cmd ('filetype plugin indent on')
vim.cmd ('syntax enable')
vim.opt.runtimepath:prepend "C:/Users/-----/Documents/.data/nvim-data/lazy/vimtex"
vim.opt.runtimepath:append "C:/Users/-----/Documents/.data/nvim-data/lazy/vimtex/after"

minimal.tex is as specified above, clean document with hello world.

The same thing happens if I run nvim --clean -u minimal.lua and then :e minimal.tex.

Regarding the steps for the problem with :VimtexCompile, the minimal config that reproduces the problem was:

vim.cmd ('filetype plugin indent on')
vim.cmd ('syntax enable')
vim.opt.runtimepath:prepend "C:/Users/-----/Documents/.data/nvim-data/lazy/vimtex"
vim.opt.runtimepath:append "C:/Users/-----/Documents/.data/nvim-data/lazy/vimtex/after"
vim.g.vimtex_compiler_method = 'tectonic'
vim.g.vimtex_compiler_method = 'sioyek'

No additional settings, nor shell opts, were added.
I have tectonic on path (tectonic --version returns version 0.12.0).

Steps to reproduce:

1. Run nvim --clean -u minimal.lua minimal.tex
2. Run :VimtexCompile > compiler instantly stops, no error output
3. If I run :VimtexView, it returns that the PDF viewer cannot read the pdf file (which is expected since no pdf file was ever produced.)

In this process, I could see no errors. The only issue I saw was with :VimtexInfo returning an error when I first run it in a session. The tex compiler command includes vim-like escaped spaces in the buffer it shows:
job cmd: tectonic --keep-logs --synctex --outdir=C:\Users\---\Internxt\Development\.config\minimal\ with\ spaces "minimal.tex"

As far as I can tell, the issue is not caused by vim.o.shell, since it occurs even if it's on the default settings. Caused by tectonic since I can run it from inside :term, :!tectonic or using a command runner like just, and it can run under Vimtex as well with the following:

vim.g.vimtex_compiler_method = 'generic'
vim.g.vimtex_compiler_generic = {command = 'just'}

If there is anything else I can provide that would help diagnose this better, or that I forgot to do, please tell me

[lervag]

You're helping me for free, I don't see why you're apologising for being busy xD

:)

Now, regarding the :VimtexInfo problem:

this happens on any tex file, on the absolute minimum config possible. I ran it with nvim --clean -u minimal.lua minimal.tex, where minimal.lua was:

vim.cmd ('filetype plugin indent on')
vim.cmd ('syntax enable')
vim.opt.runtimepath:prepend "C:/Users/-----/Documents/.data/nvim-data/lazy/vimtex"
vim.opt.runtimepath:append "C:/Users/-----/Documents/.data/nvim-data/lazy/vimtex/after"

minimal.tex is as specified above, clean document with hello world.

Ok, thus we still have two different and unrelated bugs! To make it easier for me, could you please open a new issue for the problem with :VimtexInfo? It should suffice to make a reference to this thread and provide a brief description (similar to parts of the original post(s) and this minimal example.

Regarding the steps for the problem with :VimtexCompile, the minimal config that reproduces the problem was:

vim.cmd ('filetype plugin indent on')
vim.cmd ('syntax enable')
vim.opt.runtimepath:prepend "C:/Users/-----/Documents/.data/nvim-data/lazy/vimtex"
vim.opt.runtimepath:append "C:/Users/-----/Documents/.data/nvim-data/lazy/vimtex/after"
vim.g.vimtex_compiler_method = 'tectonic'
vim.g.vimtex_compiler_method = 'sioyek'

No additional settings, nor shell opts, were added.

Ah, this makes it much easier!

I have tectonic on path (tectonic --version returns version 0.12.0).

I have the same version on my end. I've used your example and steps and things work as expected on my end. I'm on Linux, though, which is probably a key part here.

After you do :VimtexCompile. You say "compiler instantly stops" - how do you know? At this stage, what is the contents of the compiler buffer (:VimtexCompileOutput - or \lo)?

The tex compiler command includes vim-like escaped spaces in the buffer it shows: job cmd: tectonic --keep-logs --synctex --outdir=C:\Users\---\Internxt\Development\.config\minimal\ with\ spaces "minimal.tex"

If you open a cmd prompt and run that same command - what's the output?

[henriqpsantos]

Here's what :VimtexCompile followed by :VimtexCompileOutput with the same settings gets me:

C:\Users\henri>tectonic --keep-logs --synctex --outdir=C:\Users\----\Internxt\Development\.config\minimal\ with\ spaces "minimal.tex"
error: Found argument 'with\' which wasn't expected, or isn't valid in this context

USAGE:
    tectonic <input> --keep-logs --outdir <outdir> --synctex

For more information try --help

Output is the exact same on cmd (or pwsh/powershell for that matter). This is, as far as I can see, an issue with escaping paths with spaces on windows specifically.

I'll make a separate issue for :VimtexInfo then. Thanks!

[lervag]

Here's what :VimtexCompile followed by :VimtexCompileOutput with the same settings gets me:

C:\Users\henri>tectonic --keep-logs --synctex --outdir=C:\Users\----\Internxt\Development\.config\minimal\ with\ spaces "minimal.tex"
error: Found argument 'with\' which wasn't expected, or isn't valid in this context

USAGE:
    tectonic <input> --keep-logs --outdir <outdir> --synctex

For more information try --help

Output is the exact same on cmd (or pwsh/powershell for that matter). This is, as far as I can see, an issue with escaping paths with spaces on windows specifically.

Yes, I think you are right.

While debugging: notice that you can try the same command, but use quotes instead of escaping the spaces. I.e., try this:

C:\Users\henri>tectonic --keep-logs --synctex --outdir="C:\Users\----\Internxt\Development\.config\minimal with spaces" "minimal.tex"

I believe it will work. If it does, could you edit tectonic.vim on line 44 on your version of VimTeX on your computer, i.e. here:

vimtex/autoload/vimtex/compiler/tectonic.vim

Line 44 in 7c1bc9b

\ . ' --outdir=' . fnameescape(l:outdir)

Change fnameescape(…) to "…", then test again. Test with VimTeX again - does it work now?

[henriqpsantos]

With the quotes instead of escaping, i.e. tectonic --keep-logs --synctex --outdir="C:\Users\----\Internxt\Development\.config\minimal with spaces" "minimal.tex" it does work.

With tectonic.vim, I had to change l. 44 to:

 \ . ' --outdir=' . '"' . l:outdir . '"'

for it to work, but it did. Now the pdf is correctly generated. Thank you! :)