TransWikia.com

Bug with biber 2.15 under Windows with large bib-files

TeX - LaTeX Asked by Manuel Weinkauf on February 1, 2021

I just updated my MikTeX distribution and am now running biber 2.15 with biblatex 3.15a. And all of a sudden, I run into problems. I have a bib-file with 1888 entries, and trying to use that now seems to break biblatex, although it worked fine on earlier versions. I do not know if the problem lies with biber or biblatex, but try the following:

I created two bib-files out of my original one, available here and here. With the following MWE it works fine and both citations are resolved:

documentclass{article}
usepackage[french, german, UKenglish]{babel}
usepackage[backend=biber, style=authoryear]{biblatex}
addbibresource{Test.bib}
addbibresource{Test2.bib}

begin{document}
Let's test: textcite{Arnold2002}, textcite{Setty1973}

printbibliography
end{document}

However, when I combine both bib-files into one and try to compile with that, none of the citations is resolved. Since it works fine when having the database split in two, I cannot see any other reason for this behaviour other than the new biber or biblatex having a limit of file-size/entry-number in the bib-file. Not the amount of entries actually cited, mind you (as it was in bibtex), but the sheer number of entries in the database. Or do I overlook something? Any help would be greatly appreciated.

EDIT 1:

Here is the .blg-file. Indeed, there is an error message "[1095]", but I cannot make sense of it:

[0] Config.pm:312> INFO - This is Biber 2.15 (beta)
[0] Config.pm:315> INFO - Logfile is 'DatabaseTest.blg'
[311] biber-MSWIN64:330> INFO - === 
[385] Biber.pm:415> INFO - Reading 'DatabaseTest.bcf'
[882] Biber.pm:952> INFO - Found 2 citekeys in bib section 0
[998] Biber.pm:4334> INFO - Processing section 0
[1068] Biber.pm:4523> INFO - Looking for bibtex format file 'TestFull.bib' for section 0
[1095] Utils.pm:411> ERROR - Data file 'TestFull.bib' cannot be read in encoding 'utf8': Wide character at C:UsersManueAppDataLocalTemppar-4d616e7565cache-b32225defac3568d1e921af5a7245ca571b236aainclib/Win32/Unicode/File.pm line 163.
[1095] Biber.pm:132> INFO - ERRORS: 1

EDIT 2:

I now identified the limit length. The file available here contains as many entries as I manage to process. It contains 1477 entries, 25026 lines, and 1,048,339 bytes. This file just works, but as soon as I add just one more mock entry like

@Article{Test3,
  author       = {Author, A. and Author B. and Author, C.},
  date         = {2020},
  journaltitle = {A Journal},
  title        = {A nice title of a certain topic which I cannot even imagine},
  issue        = {3},
  pages        = {1--18},
  volume       = {12}
}

I get the same error as above. This seems to be the length limit with which biblatex works properly.

EDIT 3:

Here is the full file that does not work directly from JabRef (as .bib) without any changes. And here is the same file as hex dump exported with Cygwin.

5 Answers

I cannot write comments (yet) so as an answer: This sounds like a OS-specific problem. I tested on Linux without an issue. There is a very recent issue reported on the github issue tracker (https://github.com/plk/biber/issues/332) similar to your problem and it may be related to EOL markers. Could you check if the solution in the issue tracker works for you? In particular, the user there reported that the conversion to LF-only EOL markers solved the problem.

Answered by user1885616 on February 1, 2021

I followed this to basically make sure my EOL was LF rather than CRLF and this sorted out my issues as well as making sure my file was encoded as UTF-8.

Answered by Nussaibah Raja-Schoob on February 1, 2021

According to new trials by Ulrike Fischer, it is now reasonably certain that this is a bug in the new version of biber, which breaks at a certain bib-file size:

https://github.com/plk/biber/issues/332

EDIT:

For the time being, there are two possible fixes

  1. Split your bib-file into several files, each with a size <1MB
  2. Use the current development version 2.16 of biber, available on sourceforge, where the bug was fixed.

Answered by Manuel Weinkauf on February 1, 2021

One of our students ran into the same problem, but it couldn't be solved simply by splitting the .bib file into chunks or deleting large parts of it that weren't really needed. It seems like the most pressing issues were the @MvBook coded entries in the .bib file (also done in JabRef like yours) as well as references to them in the text by /volcite (and tvolcite etc.). After changing all @MvBook-s to @Book-s and all /volcite-s to /cite-s, the file once more runs through.

Answered by Aillil on February 1, 2021

I have a bib file with more than 4500 entries that works perfectly with TexLive2019 and TexStudio 3.0.1 under Windows 10. When I moved to a different Windows 10 computer, I downloaded TexLive2020 and TexStudio 3.0.1. I immediately got this error when attempting to compile my document. It is the same bib file running on both machines because I download it from Google Drive. So this definitely appears to be a bug rather than a file encoding problem.

EDIT: I used Notepad++ and JABREF to split the file into three .bib files and then used a addbibresource{} for each part in TexStudio. (I could not get it to recognize the three filenames in a single add command with no spaces and a comma between files.) Now it runs as it should.

Answered by MJLindeman on February 1, 2021

Add your own answers!

Ask a Question

Get help from others!

© 2024 TransWikia.com. All rights reserved. Sites we Love: PCI Database, UKBizDB, Menu Kuliner, Sharing RPP