Sometimes, even on Linux, a standard Go install won’t work. Even a source install may fail under some uncommon OS configurations.
How does one install Go, then?
Some days ago, at the AWS Summit 2014, DamZ renewed my long-asleep interest for the Google Go language, with wondrous stories about its use in infrastructure at the new Platform.sh they were launching that same days, so I’ve been doing my homework getting up to date on Go programming: that’s what holidays are for, aren’t they ?
The symptoms
So this morning it was time to consider deploying to production… but alas, programs which worked normally on my Ubuntu development workstation failed to run on the target server, running Debian Squeeze, with rather mystifying results:
- Laptop:
uname -a:Linux ubuntu 3.2.0-64-generic #97-Ubuntu SMP Wed Jun 4 22:04:21 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux - Target:
uname -a:Linux v1085 3.2.23-vs2.3.2.12-beng #1 SMP Tue Jul 24 10:15:16 BST 2012 x86_64 GNU/Linux - so it was the same kernel and same architecture, good.
- binary copied from one machine to the other with its config files, database schema and privileges set up on both machines.
- All OK on the laptop.
$ ls -l myapp
-rwxrwxr-x 1 root root 3522952 Jun 6 11:32 myapp
$ ./myapp
bash: ./myapp: no such file or directory
$ strace -f -e open ./myapp
exec: no such file or directory
$ file myapp
ELF 64-bit LSB executable, x86-64, version 1 (SYSV), dynamically linked (uses shared libs), not stripped
$ ldd myapp
ldd:
not a dynamic executable
$
So, according to file(1), this is an dynamically linked executable, but not for ldd(1)?
And, even more surprising, a trivial helloworld.go compiled the same way worked on both machines.
What could be going wrong?
The diagnostic
How about that ldd myapp on the laptop?
ldd usemysql
linux-vdso.so.1 => (0x00007fff64fff000)
libpthread.so.0 => /lib64/libpthread.so.0 (0x00007ff681e8e000)
libc.so.6 => /lib64/libc.so.6 (0x00007ff681b03000)
/lib64/ld-linux-x86-64.so.2 (0x00007ff6820b7000)
Now we’re going somewhere: the same file is diagnosed differently by the same command on the two machines. How about checking that binary itself?
readelf -l usemysql
Elf file type is EXEC (Executable file)
Entry point 0x4258a0
There are 9 program headers, starting at offset 64
Program Headers:
Type Offset VirtAddr PhysAddr FileSiz MemSiz Flags Align
PHDR 0x0000000000000040 0x0000000000400040 0x0000000000400040 0x00000000000001f8 0x00000000000001f8 R E 1000
INTERP 0x0000000000000be4 0x0000000000400be4 0x0000000000400be4 0x000000000000001c 0x000000000000001c R 1
[Requesting program interpreter: /lib64/ld-linux-x86-64.so.2]
LOAD 0x0000000000000040 0x0000000000400040 0x0000000000400040 0x0000000000111c40 0x0000000000111c40 R E 1000
LOAD 0x0000000000112000 0x0000000000512000 0x0000000000512000 0x0000000000231c50 0x0000000000231c50 R 1000
LOAD 0x0000000000344000 0x0000000000744000 0x0000000000744000 0x0000000000017ac8 0x0000000000035db8 RW 1000
DYNAMIC 0x00000000003440c0 0x00000000007440c0 0x00000000007440c0 0x0000000000000130 0x0000000000000130 RW 8
TLS 0x000000000035bac8 0x0000000000000000 0x0000000000000000 0x0000000000000000 0x0000000000000010 R 8
GNU_STACK 0x0000000000000000 0x0000000000000000 0x0000000000000000 0x0000000000000000 0x0000000000000000 RW 8
LOOS+5041580 0x0000000000000000 0x0000000000000000 0x0000000000000000 0x0000000000000000 0x0000000000000000 8
Section to Segment mapping:
Segment Sections...
00
01 .interp
02 .text .plt .interp
03 .rodata .typelink .gosymtab .gopclntab .dynsym .rela .gnu.version .gnu.version_r .hash .dynstr .rela.plt
04 .got .got.plt .dynamic .noptrdata .data .bss .noptrbss
05 .dynamic
06 .tbss
07
08
Turns out that this program needs /lib64/ld-linux-x86-64.so.2,
which exists on one machine but not on the other.
Symlinking ld-linux.so.2 to the missing path does change things (corrupted library error),
but does not fix the problem.
OK, then how about installing Go 1.2.2 on that server and compiling there to see the difference?
The golang-go package for Debian Squeeze only contains Go 1.0, meaning it is not a solution either,
so let’s follow the instructions on https://golang.org/doc/install…
and get the same error from the official go binary for that distribution.
Well, not that surprising, at this point: let’s build from source!
It’s well documented too, on https://golang.org/doc/install/source,
so let’s proceeed.
# cd /opt
# tar xf go1.2.2.linux-amd64.tar.gz
# cd go/src
# ./app.bash
[...snip...]
/usr/include/features.h:324:26: fatal error: bits/predefs.h: No such file or directory
#include <bits/predefs.h>
So the official distribution does not compile either. Right, Squeeze is probably too old, and that server needed a Wheezy upgrade anyway… let’s upgrade.
[...skip Squeeze to Wheezy upgrade...]
# cd go/src
# ./app.bash
[...snip...]
/usr/include/features.h:324:26: fatal error: bits/predefs.h: No such file or directory
#include <bits/predefs.h>
Hmmm, not that much better… and predefs.h is actually present in /usr/include/i386-linux-gnu/bits/predefs.h.
It was already there on Squeeze, actually. So it looks like for some reason,
the go build appears to have an undocumented dependency on non-default multilib architecture configurations.
The solution
So let’s just install that multilib support (bullseye update; used to be here), even though it was not needed on Ubuntu 12.04:
# apt-get install gcc-multilib
# cd /opt/go/src
# ./all.bash
[...gobs of output, include net test panics...]
At last, the Go install process succeeds ! Let’s skip the failing net tests,
which cannot handle the IP specifics of the VM anyway
(edit src/pkg/net/ipraw_test.go and src/pkg/net/http/fs_test.go to swap case on the failing tests),
and we have a winner:
# go version
go version go1.2.2 linux/amd64
# cd /tmp
# ./myapp
[...produces expected output...]
#
Long story short: in such deployment situations, digging into the entrails of binaries can be helpful.
The file, ldd / readelf combo can work wonders.
And special thanks to to taruti on #go-nuts, who really showed me the path.