btrfsck --repair crashes: extent-tree.c:2757: alloc_tree_block: BUG_ON ret triggered, value -28

[CyberShadow]

$ sudo btrfsck --repair -p /dev/disk/by-uuid/ae7de48a-10bf-48cf-ad9e-11a5f61b6f35 2>&1 | tee -a repair.log
[sudo] password for vladimir:
Fixed 0 roots.
enabling repair mode
Checking filesystem on /dev/disk/by-uuid/ae7de48a-10bf-48cf-ad9e-11a5f61b6f35
UUID: ae7de48a-10bf-48cf-ad9e-11a5f61b6f35
parent transid verify failed on 9945358925824 wanted 619339 found 15166217
parent transid verify failed on 9945358925824 wanted 619339 found 15166217
parent transid verify failed on 9945358925824 wanted 619339 found 15166217
parent transid verify failed on 9945358925824 wanted 619339 found 15166217
Ignoring transid failure
leaf parent key incorrect 9945358925824
bad block 9945358925824

ERROR: errors found in extent allocation tree or chunk allocation
checking free space cache [O]
parent transid verify failed on 9945358925824 wanted 619339 found 15166217
Ignoring transid failure
Wrong key of child node/leaf, wanted: (770911, 108, 0), have: (9944064163840, 169, 0)
Wrong generation of child node/leaf, wanted: 15166217, have: 619339
root 21764 root dir 256 error
root 21764 inode 256 errors 200, dir isize wrong
Unable to find block group for 0
Unable to find block group for 0
Unable to find block group for 0
extent-tree.c:2757: alloc_tree_block: BUG_ON `ret` triggered, value -28
btrfs check(+0x1dc32)[0x55eb99277c32]
btrfs check(btrfs_alloc_free_block+0x1e8)[0x55eb9927e928]
btrfs check(__btrfs_cow_block+0x188)[0x55eb9926dd98]
btrfs check(btrfs_cow_block+0xf9)[0x55eb9926e5a9]
btrfs check(btrfs_search_slot+0x36d)[0x55eb9927140d]
btrfs check(btrfs_unlink+0x15f)[0x55eb9929943f]
btrfs check(+0x5a376)[0x55eb992b4376]
btrfs check(+0x5d07a)[0x55eb992b707a]
btrfs check(cmd_check+0x2b92)[0x55eb992c0d72]
btrfs check(main+0x88)[0x55eb9926c7d8]
/usr/lib/libc.so.6(__libc_start_main+0xeb)[0x7ff905d1306b]
btrfs check(_start+0x2a)[0x55eb9926c8fa]

$ uname -a
Linux home.thecybershadow.net 4.17.8-1-ARCH #1 SMP PREEMPT Wed Jul 18 09:56:24 UTC 2018 x86_64 GNU/Linux

$ pacman -Q btrfs-progs
btrfs-progs 4.17-1

$ sudo mount /mnt/archive2016

$ sudo btrfs fi usage /mnt/archive2016
Overall:
    Device size:		  29.11TiB
    Device allocated:		  29.11TiB
    Device unallocated:		  43.76MiB
    Device missing:		     0.00B
    Used:			  24.12TiB
    Free (estimated):		  67.80GiB	(min: 67.80GiB)
    Data ratio:			      2.00
    Metadata ratio:		      2.00
    Global reserve:		 512.00MiB	(used: 1.30MiB)

Data,RAID0: Size:2.05GiB, Used:824.90MiB
   /dev/sdc1	   1.02GiB
   /dev/sde1	   1.02GiB
   /dev/sdf1	  64.00KiB
   /dev/sdj1	  64.00KiB

Data,RAID1: Size:11.98TiB, Used:11.91TiB
   /dev/sdc1	   4.74TiB
   /dev/sde1	   4.74TiB
   /dev/sdf1	   7.24TiB
   /dev/sdj1	   7.24TiB

Metadata,RAID1: Size:156.00GiB, Used:151.86GiB
   /dev/sdc1	 117.00GiB
   /dev/sde1	 117.00GiB
   /dev/sdf1	  39.00GiB
   /dev/sdj1	  39.00GiB

Metadata,DUP: Size:2.42TiB, Used:558.09MiB
   /dev/sdc1	   2.42TiB
   /dev/sde1	   2.42TiB

System,RAID1: Size:8.00MiB, Used:2.80MiB
   /dev/sdf1	   8.00MiB
   /dev/sdj1	   8.00MiB

System,DUP: Size:4.00MiB, Used:64.00KiB
   /dev/sdc1	   8.00MiB

Unallocated:
   /dev/sdc1	   8.94MiB
   /dev/sde1	  16.94MiB
   /dev/sdf1	   8.94MiB
   /dev/sdj1	   8.94MiB

#123, which has the same error message and a similar stack trace, mentions -28 is -ENOSPC. It looks like there is little unallocated space left, which could be fixed by a balance, but this filesystem is broken in the first place (the multiple data profiles are due to a balance that crashed halfway).

[adam900710]

Just as you expected, it's a bad BUG_ON() and we should clean up such BUG_ON().

Fixed in my branch:
https://github.com/adam900710/btrfs-progs/tree/fixes_for_fuzz_test

And have already submitted it to maillist.

For the problem you hit, I don't think it's really about the lack of space.
The transid problem looks like the root cause, which corrupted extent tree and make us unable to allocate new tree block.

I recommend to use btrfs-restore to salvage your data first and neven try --repair unless you know what you're doing.

BTW, please provide btrfs check --readonly <device> output if you want to help enhancing btrfs check.

[kdave]

The fixes have been released in v4.19.