TestBuild/X86Linux fails on aarch64

[QuLogic]

Of course, this is running with qemu since it's cross-architecture, but I believe this used to work in the past. Now it fails with something to do with bdwgc:

=== CONT  TestBuild/X86Linux/filesystem.go
GC Warning: Out of memory - trying to allocate requested amount (4112 bytes)...
Insufficient memory for GC_all_nils

[QuLogic]

Same for TestTest/X86Linux, though I didn't notice it because most of those are intentional failures.