Foldingathome 6.34 client FATAL CoreStatus 63 (99) on Linux

[Gizmo]

Having a problem getting a linux machine to fold.

Originally Posted by foldingathome

[04:47:29] Loaded queue successfully.
[04:47:29]
[04:47:29] + Processing work unit
[04:47:29] Core required: FahCore_a3.exe
[04:47:29] Core found.
[04:47:29] Working on queue slot 01 [April 13 04:47:29 UTC]
[04:47:29] + Working ...
[04:47:30]
[04:47:30] *------------------------------*
[04:47:30] Folding@Home Gromacs SMP Core
[04:47:30] Version 2.27 (Dec. 15, 2010)
[04:47:30]
[04:47:30] Preparing to commence simulation
[04:47:30] - Looking at optimizations...
[04:47:30] - Created dyn
[04:47:30] - Files status OK
[04:47:30] - Expanded 1768638 -> 1957708 (decompressed 110.6 percent)
[04:47:30] Called DecompressByteArray: compressed_data_size=1768638 data_size=1957708, decompressed_data_size=1957708 diff=0
[04:47:30] - Digital signature verified
[04:47:30]
[04:47:30] Project: 6957 (Run 0, Clone 36, Gen 46)
[04:47:30]
[04:47:30] FATAL: Couldn't start core work threadFATAL: Couldn't start core checkpoint timer thread
[04:47:30] Folding@home Core Shutdown: UNKNOWN_ERROR
[04:47:30] CoreStatus = 63 (99)
[04:47:30] + Error starting Folding@home core.

Tried deleting and downloading the files all new again, same problem. Everything I've found so far suggests a permissions problem, so I tried running the thing as root: same problem.

The 6.29 client just woke up one day and started doing this, after running for some time without problems. Upgraded to the 6.34 client, same problems.

ldd says that I've got all the libraries.

[ThunderRd]

What does your command line look like? Any arguments there?

It does not appear to be a client problem, but rather a core problem. Did you delete and force a core download as well?

[Gizmo]

Originally Posted by ThunderRd

What does your command line look like? Any arguments there?

fah6 -verbosity 9

Originally Posted by ThunderRd

It does not appear to be a client problem, but rather a core problem. Did you delete and force a core download as well?

Yes, I did. I deleted everything except the client.cfg. Everything else was redownloaded and recreated as needed.

[ThunderRd]

The client should report this when it starts, or you're not running in SMP mode. Is that a single core machine? [I doubt it]

./fah6 -smp -verbosity 9

Launch directory: /home/thunderrd/FAH6
Executable: ./fah6
Arguments: -smp -verbosity 9

[Gizmo]

I had it in smp mode initially with the -smp flag, but that didn't make any difference.

[ThunderRd]

Try deleting everything in the directory except the executable, and configure the client again. Maybe your client.cfg is corrupted somehow. Did you attempt to edit it manually in any way? Remember to have your passkey ready.

I can't understand why it was working, and then suddenly didn't work, unless there was a core update that you did not receive for some reason. I see that the 6.29 client isn't on the downloads page anymore, so updating it should have eliminated any chance of the client being the problem. So we're back to the core. See if the core you have is correct in size to mine:

thunderrd@Q6600:~/FAH6$ ll
total 16944
-rw-r--r-- 1 thunderrd thunderrd   55688 Apr 13 12:39 FAHlog-Prev.txt
-rw-r--r-- 1 thunderrd thunderrd   14845 Apr 13 22:24 FAHlog.txt
-rwxrwxrwx 1 thunderrd thunderrd 3625104 Feb 20 01:04 FahCore_a1.exe
-rwxrwxrwx 1 thunderrd thunderrd 5509624 Feb 20 01:04 FahCore_a2.exe
-rwxrwxrwx 1 thunderrd thunderrd 6272504 Mar  6 20:14 FahCore_a3.exe
-rwxrwxrwx 1 thunderrd thunderrd    5225 Feb 20 01:04 MyFolding.html
-rwxrwxrwx 1 thunderrd thunderrd     205 Apr 13 20:23 client.cfg
-rwx--x--x 1 thunderrd thunderrd  314840 Feb 24 06:33 fah6
-rwxrwxrwx 1 thunderrd thunderrd 1103456 Feb 20 01:04 fah6.static
-rwxrwxrwx 1 thunderrd thunderrd  312024 Feb 20 01:04 fah6_alt
-rwxrwxrwx 1 thunderrd thunderrd       8 Feb 20 01:04 machinedependent.dat
-rwxrwxrwx 1 thunderrd thunderrd   68492 Feb 20 01:04 mpiexec.old
-rwxrwxrwx 1 thunderrd thunderrd      32 Feb 20 01:04 passkey.txt
-rwxrwxrwx 1 thunderrd thunderrd    9848 Feb 20 01:04 qfix64
-rw-r--r-- 1 thunderrd thunderrd    7168 Apr 13 20:24 queue.dat
-rw-r--r-- 1 thunderrd thunderrd     159 Apr 13 22:24 unitinfo.txt
drwxr-xr-x 2 thunderrd thunderrd    4096 Apr 13 22:24 work

[Gizmo]

-rw-r--r--. 1 foldingathome foldingathome  568939 Jan 21  2010 FAH6.29-Linux.tgz
-rw-r--r--  1 foldingathome foldingathome  146345 Feb 23 23:33 FAH6.34-Linux64.tgz
-rw-r--r--  1 foldingathome foldingathome  131147 Apr 13 16:30 FAHlog.txt
-rwxr-x---  1 foldingathome foldingathome 6272504 Apr 13 16:30 FahCore_a3.exe
-rw-r--r--  1 foldingathome foldingathome    5267 Apr 13 16:24 MyFolding.html
-rw-r-----. 1 foldingathome foldingathome     216 Apr 13 04:27 clienold
-rwxr-x---  1 foldingathome foldingathome     220 Apr 13 16:24 client.cfg
-rwxr-x---  1 foldingathome foldingathome  314840 Feb 23 23:33 fah6
-rwxr-xr--. 1 foldingathome foldingathome     125 Jan 15 02:49 initfolding
-rw-r--r--  1 foldingathome foldingathome       8 Apr 13 16:24 machinedependent.dat
-rw-r--r--  1 foldingathome foldingathome    7168 Apr 13 16:28 queue.dat
drwxr-x---  2 foldingathome foldingathome    4096 Apr 13 16:30 work

It's the same size. Rebuilt the config file, still got the same problem.

I'm suspicious that there's some library or something that's wounding me here, but I haven't the foggiest idea what it is. ldd tells me that FahCore_a3 is not a dynamically linked library, so there actually SHOULDN'T be a problem.

I'm getting puzzleder and puzzleder.

[ThunderRd]

I didn't think to ask this before, but when you downloaded new files, did you receive a different WU? If you didn't, we can't rule out the possibility of a bad one. Let's make sure you get a different one.

If things still fail, well, nothing is wrong in your FAH directory, so the problem must be elsewhere. What did you update last in your Linux setup, and when was it? Maybe that would give us a clue. I don't know which distro is on this box, but this may help to check the libraries:

http://folding.stanford.edu/English/LinSMPGuide

you need ia32-libs if it's debian, and compat-libstdc++ in others. I'm sure this information is academic for you but you can take a look anyhow.

"v6.34 fixes the library conflict in recent linux distributions." Whatever that means, I haven't had a reason to know what the library conflict was. I don't remember having to do anything special on my Gentoo install to get it running.

I'm afraid if you haven't updated anything that we can put a finger on, it's back to the folding forum for you. Someone there should be able to help.

I don't know if this helps, but here is the output of ldd on my Gentoo box. There are some obsolete files in there, I really should clean them up :

Q6600 ~ # cd /home/thunderrd/FAH6
Q6600 FAH6 # ldd -v *
FAHlog-Prev.txt:
ldd: warning: you do not have execution permission for `./FAHlog-Prev.txt'
	not a dynamic executable
FAHlog.txt:
ldd: warning: you do not have execution permission for `./FAHlog.txt'
	not a dynamic executable
FahCore_a1.exe:
	not a dynamic executable
FahCore_a2.exe:
	not a dynamic executable
FahCore_a3.exe:
	not a dynamic executable
MyFolding.html:
	not a dynamic executable
client.cfg:
	not a dynamic executable
fah6:
	linux-vdso.so.1 =>  (0x00007fffcfbff000)
	libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f7ec2dc4000)
	libm.so.6 => /lib64/libm.so.6 (0x00007f7ec2b41000)
	libdl.so.2 => /lib64/libdl.so.2 (0x00007f7ec293d000)
	libc.so.6 => /lib64/libc.so.6 (0x00007f7ec25b1000)
	/lib64/ld-linux-x86-64.so.2 (0x00007f7ec2fe1000)

	Version information:
	./fah6:
		libdl.so.2 (GLIBC_2.2.5) => /lib64/libdl.so.2
		libm.so.6 (GLIBC_2.2.5) => /lib64/libm.so.6
		libpthread.so.0 (GLIBC_2.2.5) => /lib64/libpthread.so.0
		libc.so.6 (GLIBC_2.3) => /lib64/libc.so.6
		libc.so.6 (GLIBC_2.2.5) => /lib64/libc.so.6
	/lib64/libpthread.so.0:
		ld-linux-x86-64.so.2 (GLIBC_2.3) => /lib64/ld-linux-x86-64.so.2
		ld-linux-x86-64.so.2 (GLIBC_2.2.5) => /lib64/ld-linux-x86-64.so.2
		ld-linux-x86-64.so.2 (GLIBC_PRIVATE) => /lib64/ld-linux-x86-64.so.2
		libc.so.6 (GLIBC_2.3.2) => /lib64/libc.so.6
		libc.so.6 (GLIBC_PRIVATE) => /lib64/libc.so.6
		libc.so.6 (GLIBC_2.2.5) => /lib64/libc.so.6
	/lib64/libm.so.6:
		libc.so.6 (GLIBC_PRIVATE) => /lib64/libc.so.6
		libc.so.6 (GLIBC_2.2.5) => /lib64/libc.so.6
	/lib64/libdl.so.2:
		ld-linux-x86-64.so.2 (GLIBC_PRIVATE) => /lib64/ld-linux-x86-64.so.2
		libc.so.6 (GLIBC_PRIVATE) => /lib64/libc.so.6
		libc.so.6 (GLIBC_2.2.5) => /lib64/libc.so.6
	/lib64/libc.so.6:
		ld-linux-x86-64.so.2 (GLIBC_PRIVATE) => /lib64/ld-linux-x86-64.so.2
		ld-linux-x86-64.so.2 (GLIBC_2.3) => /lib64/ld-linux-x86-64.so.2
fah6.static:
	not a dynamic executable
fah6_alt:
	linux-vdso.so.1 =>  (0x00007fffde85f000)
	libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f27b526a000)
	libm.so.6 => /lib64/libm.so.6 (0x00007f27b4fe7000)
	libdl.so.2 => /lib64/libdl.so.2 (0x00007f27b4de3000)
	libc.so.6 => /lib64/libc.so.6 (0x00007f27b4a57000)
	/lib64/ld-linux-x86-64.so.2 (0x00007f27b5487000)

	Version information:
	./fah6_alt:
		libdl.so.2 (GLIBC_2.2.5) => /lib64/libdl.so.2
		libm.so.6 (GLIBC_2.2.5) => /lib64/libm.so.6
		libpthread.so.0 (GLIBC_2.2.5) => /lib64/libpthread.so.0
		libc.so.6 (GLIBC_2.3) => /lib64/libc.so.6
		libc.so.6 (GLIBC_2.2.5) => /lib64/libc.so.6
	/lib64/libpthread.so.0:
		ld-linux-x86-64.so.2 (GLIBC_2.3) => /lib64/ld-linux-x86-64.so.2
		ld-linux-x86-64.so.2 (GLIBC_2.2.5) => /lib64/ld-linux-x86-64.so.2
		ld-linux-x86-64.so.2 (GLIBC_PRIVATE) => /lib64/ld-linux-x86-64.so.2
		libc.so.6 (GLIBC_2.3.2) => /lib64/libc.so.6
		libc.so.6 (GLIBC_PRIVATE) => /lib64/libc.so.6
		libc.so.6 (GLIBC_2.2.5) => /lib64/libc.so.6
	/lib64/libm.so.6:
		libc.so.6 (GLIBC_PRIVATE) => /lib64/libc.so.6
		libc.so.6 (GLIBC_2.2.5) => /lib64/libc.so.6
	/lib64/libdl.so.2:
		ld-linux-x86-64.so.2 (GLIBC_PRIVATE) => /lib64/ld-linux-x86-64.so.2
		libc.so.6 (GLIBC_PRIVATE) => /lib64/libc.so.6
		libc.so.6 (GLIBC_2.2.5) => /lib64/libc.so.6
	/lib64/libc.so.6:
		ld-linux-x86-64.so.2 (GLIBC_PRIVATE) => /lib64/ld-linux-x86-64.so.2
		ld-linux-x86-64.so.2 (GLIBC_2.3) => /lib64/ld-linux-x86-64.so.2
machinedependent.dat:
	not a dynamic executable
mpiexec.old:
	linux-vdso.so.1 =>  (0x00007ffff47ff000)
	libpthread.so.0 => /lib64/libpthread.so.0 (0x00007ff2d0f58000)
	librt.so.1 => /lib64/librt.so.1 (0x00007ff2d0d4f000)
	libc.so.6 => /lib64/libc.so.6 (0x00007ff2d09c3000)
	/lib64/ld-linux-x86-64.so.2 (0x00007ff2d1175000)

	Version information:
	./mpiexec.old:
		libpthread.so.0 (GLIBC_2.2.5) => /lib64/libpthread.so.0
		libc.so.6 (GLIBC_2.3) => /lib64/libc.so.6
		libc.so.6 (GLIBC_2.2.5) => /lib64/libc.so.6
	/lib64/libpthread.so.0:
		ld-linux-x86-64.so.2 (GLIBC_2.3) => /lib64/ld-linux-x86-64.so.2
		ld-linux-x86-64.so.2 (GLIBC_2.2.5) => /lib64/ld-linux-x86-64.so.2
		ld-linux-x86-64.so.2 (GLIBC_PRIVATE) => /lib64/ld-linux-x86-64.so.2
		libc.so.6 (GLIBC_2.3.2) => /lib64/libc.so.6
		libc.so.6 (GLIBC_PRIVATE) => /lib64/libc.so.6
		libc.so.6 (GLIBC_2.2.5) => /lib64/libc.so.6
	/lib64/librt.so.1:
		ld-linux-x86-64.so.2 (GLIBC_PRIVATE) => /lib64/ld-linux-x86-64.so.2
		libpthread.so.0 (GLIBC_2.2.5) => /lib64/libpthread.so.0
		libpthread.so.0 (GLIBC_PRIVATE) => /lib64/libpthread.so.0
		libc.so.6 (GLIBC_2.3.2) => /lib64/libc.so.6
		libc.so.6 (GLIBC_2.2.5) => /lib64/libc.so.6
		libc.so.6 (GLIBC_PRIVATE) => /lib64/libc.so.6
	/lib64/libc.so.6:
		ld-linux-x86-64.so.2 (GLIBC_PRIVATE) => /lib64/ld-linux-x86-64.so.2
		ld-linux-x86-64.so.2 (GLIBC_2.3) => /lib64/ld-linux-x86-64.so.2
passkey.txt:
	not a dynamic executable
qfix64:
	linux-vdso.so.1 =>  (0x00007fff8d538000)
	libc.so.6 => /lib64/libc.so.6 (0x00007fe14597c000)
	/lib64/ld-linux-x86-64.so.2 (0x00007fe145d08000)

	Version information:
	./qfix64:
		libc.so.6 (GLIBC_2.2.5) => /lib64/libc.so.6
	/lib64/libc.so.6:
		ld-linux-x86-64.so.2 (GLIBC_PRIVATE) => /lib64/ld-linux-x86-64.so.2
		ld-linux-x86-64.so.2 (GLIBC_2.3) => /lib64/ld-linux-x86-64.so.2
queue.dat:
ldd: warning: you do not have execution permission for `./queue.dat'
	not a dynamic executable
unitinfo.txt:
ldd: warning: you do not have execution permission for `./unitinfo.txt'
	not a dynamic executable
work:
ldd: ./work: not regular file

(PS - I think I've asked you this before, but why can't we widen the code boxes for pasting?)