New feature request: sort of bibtex by bibkey

[vanabel]

Currently, I develop a javascrip plugin that will replace the bibkey generated by AmsSciNet, which is MRnumber, to Author+Year+Title, it helps me a lot when I cite the reference, since I can distinguish them easily.

I believe it is helpful to sort of bibtex items by the bibtex key, e.g., when we check the dumplicated items, I would like to share a solution from so:
put the following macro into .vimrc,

let @b=':g/^}$/s//}\r
:g/^@/,/^}$/s/\r\|\n/の
:%sort r /{[A-z0-9]\+,の/
:g/の/s//\r/g
:g/^$/d
:w
'

where we should replace the linebreak by ^M, it is Ctrl+V Ctrl+M on windows gvim. After so it, we can apply the macro b in bibtex file by @b in normal mode, which will automatically sort the bibtex items through its bibkey.

Of course, we can execute the macro whenever we write the bibtex data:

au! BufWritePost *.bib :norm @b<cr>:set fdm=syntax<cr>

I think it is interesting to add this function as a minor improvement of vimtex. How about you?

[lervag]

I'm not sure I understand. You want vimtex to automatically sort the entries listed in a .bib file by the keys?

First, I will not let this happen automatically, i.e. any autocommands like you propose must be added by users. However, I could consider to add a function that performs this sort that does something like the macro you are suggesting. But I am not quite sure if it really should be part of vimtex. It feels somewhat "off topic".

But perhaps I misunderstand what you are meaning?

[kiryph]

There are already standalone tools to format bib files:

For instance, bibtool can pretty print, sort, find duplicates and also automatically delete duplicates (next to other things, see the manual with 86 pages, sorting is mentioned in A.7. Sorting). Installation is easy:

$ sudo apt install bibtool # debian, ubuntu, ...
$ brew install bib-tool # macOS

Defaults are:

To enable sorting create a bibtoolrsc file (manual: A.2. Command Line Usage and Resource Files) with sort = on. By default the file is sorted by the bibkey.

If you want to change this, consider for example following config:

sort.format              = { %s($type) %s(year) %s(month) {%N(author) # %N(editor)} }

which sorts by type-> year-> month-> author/editor

Explanation of the special characters is given in A.11 Format specification. Those I used here are

• $type is a pseudo field (A.11.3)
• year, month, author, editor ordinary fields
• %sign pre.post qualifier letter(field) formats a field (A.11.2)
• %N Format names with full last names and initials of first names.
• %s Take a field as is (after translation of special characters).
• # are alternatives (A.11.6)
• {} grouping (A.11.7)

You can also sort the fields within records with (A.12.6 Field Ordering)

sort.order               = { * = author # title }

Here is * a special entry which indicates it applies to all types (articles, books, ...).

Order the entries in the .bib file to match the ordering defined for the bibliography

seems to be possible with the tool mode of biber:

https://tex.stackexchange.com/questions/318448/ordering-bib-file-alphabetically

Should vimtex provide this out of the box?

I certainly understand the comfort of having a single plugin which can do everything a LaTeX user wants. In the emacs world there is the plugin ebib which can apparently do this: http://joostkremers.github.io/ebib/ebib-manual.html#sorting-the-.bib-file
However, I would still argue to use the external tools instead of adding this to vimtex. Vim is quite good at integrating with external command line tools via filters, formatprg, etc.
Also a git hook could be configured (if git is used) to prevent committing duplicate bib entries.

Parsing bib files can also be tricky. The vim plugin tbibtools works with assumptions of how the bib files is formatted. This means this could increase the maintenance work for vimtex.

Documentation of formatting of bib files

The current paragraph in the documentation about formatting of tex files could be extended for bib files

vimtex/doc/vimtex.txt

Lines 396 to 420 in 00de60c

	Formatting~
	vimtex has a custom |formatexpr| that may be enabled with the option
	|g:vimtex_format_enabled|. However, there are a lot of different styles for
	formatting LaTeX manuscripts. These are typically much more relevant when
	writing in collaboration with others. A good reference on this topic is [0],
	and note in particular the box "Directives for using LaTeX with version
	control systems".
	The most basic style is to hard wrap lines at a given column, e.g. 80
	columns, and this is exactly the type of formatting that is supported by
	vimtex. However, this is usually not very friendly when collaborating with
	others, as it tends to mess up diffs between versions of the document.
	Instead, one might want to consider one of these:
	a) keeping each sentence on a line (use soft wrapping)
	b) add additional indentation for split sentences [1]
	c) use semantic line feeds [2]
	In order to make it easier to use one of these styles of formatting, one may
	want to use an external formatter:
	- latexindent.pl [3]
	- bibtool [4]
	- vim-bucky [5] (note: this is an alpha version as of October 2018)

bibtool is already mentioned, but it is not clear that it only applies to bib files.

@vanabel Was this comment helpful?

[vanabel]

I'm not sure I understand. You want vimtex to automatically sort the entries listed in a .bib file by the keys?

First, I will not let this happen automatically, i.e. any autocommands like you propose must be added by users. However, I could consider to add a function that performs this sort that does something like the macro you are suggesting. But I am not quite sure if it really should be part of vimtex. It feels somewhat "off topic".

But perhaps I misunderstand what you are meaning?

It seems the required function is off topic, you understand me correctly. However, how about add some instruction (or even the macro @b) in the Formatting section as mentioned by @kiryph?

[lervag]

@kiryph Thanks for the tip on bibtool, it seems very useful. I've obviously heard about it before, but I forgot. When testing right now, I get segmentation faults with %s($type), although it works as expected with %w($type).

Since tools like bibtool exists, I think it even less important to add anything to Vim. However, I could mention it more clearly in the docs. I'll update the docs now.

[kiryph]

@lervag you are welcome.

When testing right now, I get segmentation faults with %s($type), although it works as expected with %w($type).

I cannot reproduce it:

❯ bibtool -V
BibTool Vers. 2.67 (C) 1996-2017 Gerd Neugebauer

Library path: .:/usr/local/Cellar/bib-tool/2.67/lib/BibTool
Special configuration options: none

❯ cat bibtoolrsc
sort = on
sort.format = { %s($type) year month {%N(author) # %N(editor)} }

❯ bibtool -r bibtoolrsc my.bib

@Article{	  dove:1997,
  author	= {Dove, Martin T.},
  title		= {{Theory of displacive phase transitions in minerals}},
  doi		= {10.2138/am-1997-3-401},
  issn		= {0003-004X},
  journal	= {American Mineralogist},
  month		= {apr},
  number	= {3-4},
  pages		= {213--244},
  url		= {https://pubs.geoscienceworld.org/ammin/article/82/3-4/213-244/43266},
  volume	= {82},
  year		= {1997}
}

@Article{	  koch:1984,
  author	= {Koch, E.},
  doi		= {10.1107/S0108767384001227},
  issn		= {0108-7673},
  journal	= {Acta Crystallographica Section A Foundations of
		  Crystallography},
  month		= {sep},
  number	= {5},
  pages		= {593--600},
  title		= {{The implications of normalizers on group–subgroup
		  relations between space groups}},
  volume	= {40},
  year		= {1984}
}

@Article{	  koch:1984,
  author	= {Koch, E.},
  doi		= {10.1107/S0108767384001227},
  issn		= {0108-7673},
  journal	= {Acta Crystallographica Section A Foundations of
		  Crystallography},
  month		= {sep},
  number	= {5},
  pages		= {593--600},
  title		= {{The implications of normalizers on group–subgroup
		  relations between space groups}},
  volume	= {40},
  year		= {1984}
}

It also works without %s(...).

I did a search for tools to format and lint bib files. However, I could not find a tool which does everything I want:

1. bibtool is good, but needs customization in particular when used for linting. Furthermore, there is a pending PR about duplicate detection for non-adjacent duplicates.
2. biber (comes with texlive) can also be used for formatting and linting. However, no line numbers for the messages are given and in the case of formatting unknown fields are removed. It correctly detects duplicate (non-adjacent) bibkeys.
3. bibclean is used by ale. I am not happy with all the messages and it looks like it also cannot detect duplicates.
4. betterbib provides formatting, can shorten or unshorten journal names and other things. I do not think it can be used for linting.
5. biblatex-check looks like a linter but also does not provide line numbers.

Untested

• prettybib

Furthermore, there are some other useful vim plugins

• https://github.com/anders-dc/vim-scholarref (fetch bibtex entry from the web via a query)
• https://github.com/msprev/fzf-bibtex (navigate to a record in the bibfile via fuzzy search)
• https://github.com/rafaqz/citation.vim (unite source, denite source pending PR)

[lervag]

Great, thanks for additional useful information!

Reg. the segfault, I have version 2.67 as well. I tested again on your "my.bib" file (as pasted here) and it still raises segfault. However, as mentioned, this is not a big deal since %w works well here.

[lervag]

I'm closing this, but feel free to reopen if you think we should further update the docs or if there are other actions that should be taken.

[krishnakumarg1984]

@lervag bibtool works without segfaulting for me. Can you try the latest version i.e. 2.68 and perhaps update us with your findings?

[lervag]

@krishnakumarg1984 Thanks for the notification. I also have version 2.68 now, and I don't get the segfault any longer. But, as I wrote previously:

Reg. the segfault, I have version 2.67 as well. I tested again on your "my.bib" file (as pasted here) and it still raises segfault. However, as mentioned, this is not a big deal since %w works well here.

So, I already mentioned that the segfault should not have been any particular issue. Further, I've also written:

Since tools like bibtool exists, I think it even less important to add anything to Vim. However, I could mention it more clearly in the docs. I'll update the docs now.

So, my previous conclusion has been to document that this tool exist, but I did not really see any good reason to add any new features to VimTeX. If you disagree and want to argue why I should add something, please feel free to do so in a new issue. If so, please also be clear about exactly what you want within VimTeX.

[krishnakumarg1984]

Adding the Goosebib suite to the list of tools that can do .bib-related parsing/fixing/linting/sorting. Also see #2103. It is perhaps a reasonable idea to have this growing collection of bib-related tools in this centralised issue.

[krishnakumarg1984]

@lervag

So, my previous conclusion has been to document that this tool exist, but I did not really see any good reason to add any new features to VimTeX. If you disagree and want to argue why I should add something, please feel free to do so in a new issue. If so, please also be clear about exactly what you want within VimTeX.

Oh. Sorry about the confusion. All that I wanted to do was just bring to your attention that bibtool 2.68 does not segfault anymore. I am very happy with the bib-related functionality within VimTeX.

As you might have noticed, I am doing a deep scrub of the VimTeX documentation as well as the issues linked therein before sitting down and beginning to write my next tome. Apologies for digging up old stuff like this. I realise this could be annoying, and I shall minimise this drastically.