Provide completion source based on cwl files from kile/texstudio

[kiryph]

Would you consider adding a completion source for deoplete/neocomplete/youcomplete based on cwl files from

• texstudio https://github.com/texstudio-org/texstudio/tree/master/completion (link updated)
• LaTeXTools for SublimeText, https://github.com/SublimeText/LaTeXTools/tree/master/cwl
• kile https://cgit.kde.org/kile.git/tree/src/data/complete,
• latexing https://github.com/LaTeXing/LaTeX-cwl, or

These are static files containing lists of latex commands. Nothing fancy.

This idea has already been applied to Sublime text plugins http://www.latexing.com/features.html, LaTeXTools and for the Atom editor plugin exist a pull request (msiniscalchi/atom-latextools#53).

I have googled a little bit more about cwl files. I think texstudio has extended its capabilities. Compare the description of cwl in kile and texstudio:

1. https://texstudio-org.github.io/background.html#description-of-the-cwl-format (link updated)
2. https://websvn.kde.org/*checkout*/tags/kile/2.0/src/README.cwl (cwl 0.1)

For example, texstudio shows certain completions only in math environments. I do not know whether neocomplete can support this.

[kiryph]

As Minimal Starting Point, one can add the cwl files as a dictionary:

1. Download kile's cwl files with
   $ wget -r -P ~/Documents/Mwe/vim-cwl-completion/kile/ -l1 -nH --cut-dirs=8 --no-parent -e robots=off https://cgit.kde.org/kile.git/plain/src/data/complete/tex/
   This will give you following files:

❯ ls ~/Documents/Mwe/vim-cwl-completion/kile
acronym.cwl                         fancybox.cwl                        psfrag.cwl
afterpage.cwl                       fancyhdr.cwl                        pst-3d.cwl
allrunes.cwl                        fancyref.cwl                        pst-blur.cwl
amsbsy.cwl                          fancyunits-base.cwl                 pst-char.cwl
amsfonts.cwl                        fancyunits-np.cwl                   pst-coil.cwl
amsmath.cwl                         fancyunits-per.cwl                  pst-eps.cwl
amsopn.cwl                          fancyunits_big-fractions.cwl        pst-fill.cwl
amssymb.cwl                         fancyunits_medium-fractions.cwl     pst-node.cwl
amsthm.cwl                          fancyunits_small-fractions.cwl      pst-plot.cwl
array.cwl                           fancyvrb.cwl                        pst-text.cwl
attachfile.cwl                      float.cwl                           pst-tree.cwl
babel.cwl                           fontspec.cwl                        pstricks.cwl
beamerfoils.cwl                     gauss.cwl                           rotating.cwl
beamerprosper.cwl                   glosstex.cwl                        scraddr.cwl
beamerseminar.cwl                   graphicx.cwl                        scrdate.cwl
beamertexpower.cwl                  hyperref.cwl                        scrlfile.cwl
biblatex-dev.cwl                    ifplatform.cwl                      scrpage2.cwl
biblatex.cwl                        ifthen.cwl                          scrtime.cwl
bm.cwl                              import.cwl                          setspace.cwl
booktabs.cwl                        jurabib.cwl                         siunitx-special.cwl
braket.cwl                          latex-209.cwl                       siunitx.cwl
calc.cwl                            latex-dev.cwl                       soul.cwl
cancel.cwl                          latex-document.cwl                  subfig.cwl
caption.cwl                         latex-l2tabu.cwl                    supertabular.cwl
class-beamer.cwl                    latex-mathsymbols.cwl               svn-multi.cwl
class-letter.cwl                    layout.cwl                          svninfo.cwl
class-scrartcl,scrreprt,scrbook.cwl listings.cwl                        tabularx.cwl
class-scrlttr2.cwl                  logsys.cwl                          tex.cwl
color.cwl                           longtable.cwl                       textcomp.cwl
colortbl.cwl                        lscape.cwl                          todonotes.cwl
context-document.cwl                luacode.cwl                         units.cwl
coordsys.cwl                        manyfoot.cwl                        url.cwl
csquotes-dev.cwl                    mathtools.cwl                       varioref.cwl
csquotes.cwl                        mdwlist.cwl                         verse.cwl
currvita.cwl                        minted.cwl                          virginialake.cwl
cyrillic.cwl                        multicol.cwl                        xifthen.cwl
diagxy.cwl                          multido.cwl                         xltxtra.cwl
doi.cwl                             newfloat.cwl                        xtab.cwl
enumerate.cwl                       nicefrac.cwl                        yfonts.cwl
esint.cwl                           nomencl.cwl
etoolbox.cwl                        pdfpages.cwl

2. If you use neocomplete, add following to your vimrc to enable autocompletion for commands of selected packages

let g:neocomplete#sources#dictionary#dictionaries = {
  \ 'tex' : ''
  \ .'~/Documents/Mwe/vim-cwl-completion/kile/amssymb.cwl,'
  \ .'~/Documents/Mwe/vim-cwl-completion/kile/amsmath.cwl,'
  \ .'~/Documents/Mwe/vim-cwl-completion/kile/biblatex.cwl,'
  \ .'~/Documents/Mwe/vim-cwl-completion/kile/booktabs.cwl,'
  \ .'~/Documents/Mwe/vim-cwl-completion/kile/graphicx.cwl,'
  \ .'~/Documents/Mwe/vim-cwl-completion/kile/latex-document.cwl,'
  \ .'~/Documents/Mwe/vim-cwl-completion/kile/latex-mathsymbols.cwl,'
  \ .'~/Documents/Mwe/vim-cwl-completion/kile/mathtools.cwl,'
  \ .'~/Documents/Mwe/vim-cwl-completion/kile/tex.cwl,'
  \ }

Now you get following autocompletion menu:

List of improvements:

• recognize include directives in cwl files to load dependent cwl files
• automatically detect the chosen documentclass and load the corresponding class-<name>.cwl which also contains directives #include:package to load the cwl files of the default packages.
• automatically detect usepackage in preamble to activate packages in use
• filter commands if cursor in math environment or not
• convert cwl lines with placeholder to snippets (ultisnips/neosnippet/garbas-snipmate)
• use cwl files from texstudio since they might be more up-to-date and comprehensive.
• preview symbols with unicode, e.g. for \right<popmenu>

  \Rightarrow        ⇒ [D]
  \Rrightarrow       ⇛ [D]
  \raggedright         [D]
  \rightarrow        → [D]
  \rightarrowtail    ↣ [D]
  \rightharpoondown  ⇁ [D]
  \rightharpoonup    ⇀ [D]
  \righthyphenmin      [D]
  \rightleftarrows   ⇄ [D]
  \rightleftharpoons ⇌ [D]
  \rightmargin         [D]
  \rightmark           [D]
  \rightrightarrows  ⇉ [D]
  \rightskip           [D]
  \rightsquigarrow   ↝ [D]
  \rightthreetimes   ⋌ [D]
  
  

[poppyschmo]

@kiryph I tried messing around with something like this for deoplete a while back but was unable to get anywhere because my data source wasn't legit LaTeX and was basically utter chaos besides. (It was initially intended for another completion project with CodeMirror/Jupyter.)

But after seeing this post, I recently gave it another go. Good work on finding those .cwl files, btw. I think (most?) of the bullet points in your "List of Improvements" are ticked though their implementation is shaky, at best. If you're able to test it out, please do (I've only ever tried it on Neovim, thus far).

[kiryph]

@poppyschmo Great to see what you have achieved. I have tried it in neovim-dot-app.
I certainly like it. You've already added the unicode symbols as a poormans preview of the command, some kind of detection whether the cursor is in a math environment and other things as well. Thank you very much.

Unfortunately, I haven't yet made the move from vim to neovim. I hope that deoplete will soon support vim. I am not sure whether I will use it right now to help you to identify issues and give feedback what I would miss most (such as detect tikz, pstricks or pgfplot environments). However, it becomes more and more appealing to switch to neovim.

I close this issue because there is now your plugin. No need for @lervag to add this to his plugin.
If he thinks some basic feature could be included, he can reopen it (or add a link to your plugin into his doc).

BTW, you should add your plugin to the list documented in
https://github.com/Shougo/deoplete.nvim/wiki/Completion-Sources
and submit a PR to add it to https://github.com/Shougo/deoplete.nvim/blob/master/doc/deoplete.txt#L915

Thanks again for your great contribution!

[poppyschmo]

@kiryph Thanks so much for the kind words and suggestions. I'll definitely give a heads up to the deoplete folks. Cheers.

[lervag]

It is not really that hard to add this feature to vimtex. At least not the more basic versions. I could do parsing of packages, then read the .cwl files for the loaded packages, also for the documentclass thing and dependencies. Completion would be available after \s.

I am not quite certain how to parse the command to yield corresponding unicode character, but if someone could tell me how to do it then I guess this should also be possible.

It is also unclear how to know which commands should be available in math mode and which should not be. But if we can define a clear rule for this, then that should be pretty easy.

However, if @poppyschmo (or anyone else) would implement a more compatible completion plugin that also works in Vim and with neocomplete, for instance, then I would not mind referring vimtex users to that plugin.

[poppyschmo]

@lervag, as it stands, it seems the TeXstudio .cwl files are not all created equally. A handful have definitions that adhere to the spec and include those "classifiers" relevant to context-aware completion. Others are sparse or completely bare in that regard. If vimtex users could be compelled to contribute corrections upstream or even fork the collection here (or in a wiki?), the completion UX would benefit considerably.

As far as contributing to whatever implementation takes hold, I'm game, but haven't used neocomplete for a couple years, now. At one point its author suggested it would be deprecated in favor of deoplete for Vim 8. Not sure if that's still the case, however.

[lervag]

Ok, so I am not quite sure where to go with this. I personally don't really neither need nor want completion of commands, allthough I see that this could be interesting for others. However, I would much prefer to just do this in a very simple manner. The context aware completion becomes too complex, I think (at least wrt the time I want to spend on this).

My proposal is therefore to implement a simplified version that is more loosely based on the .cwl files. That is, I want only simple word lists with a single word per command on separate lines. I'll add a simple version to show what I mean.

[lervag]

I propose that we continue the discussion at #770. I think the implementation is getting pretty close to finished. What is lacking now is the completion source files.