Issue with Illustris tutorial: error fie not found

Sophia Nasr
  • 1 Apr '19

Hello, was going through the Illustris script tutorial, and performed all the steps exactly as listed. I get an error on this line:
subhalos = il.groupcat.loadSubhalos(basePath,135,fields=fields)
It is as follows:
OSError: Unable to open file (unable to open file: name = './Illustris-3/output//groups_135/fof_subhalo_tab_135.1.hdf5', errno = 2, error message = 'No such file or directory', flags = 0, o_flags = 0)
So, I looked into the folder where the files that were downloaded using wget went (groups_135) and, of course, I found that the above file 'fof_subhalo_tab_135.1.hdf5' does not exist; only 'groupcat-135.1.hdf5' and 'groups_135.0.hdf5' are available. I also looked into the snapshot for Illustris 3 itself and found indeed only these two files exist. So my question is, is it possible that some naming scheme changed which causes the script listed to look for the wrong file, or am I supposed to download it separately?

Thanks.

Dylan Nelson
  • 1 Apr '19

Hi Sophia,

If you look at the top of the groupcat.py file, you can see the search is quite simple. First it looks for something like groups_135.0.hdf5, and if this isn't found, then it looks for something like fof_subhalo_tab_135.0.hdf5.

It seems you are missing the second chunk (fof_subhalo_tab_135.1.hdf5) or else have the wrong path?

Sophia Nasr
  • 1 Apr '19

Hi Dylan,

Okay I see that line in the code. *I think, then, that it wants the groups-135.1.hdf5 file, but the only file that's located in the folder with a 1 after the 135 is the groupcat file, groupcat-135.1.hdf5 (which I downloaded with wget). I did try downloading again, and got the file groupcat-135.0.hdf5, which was not there before, but for some reason no matter what I do, I cannot get the groups-135.1.hdf5 file! I looked directly into the snapshot by going to the Illustris-3 link (in the list of resolutions), and went own to snapshot 135, and in the group catalog the file doesn't exist. I tried using wget several times thnking maybe it skipped it, but it looks like that's not the case. Can I just download the specific file on my own??

Dylan Nelson
  • 2 Apr '19

Hi Sophia,

If you run the wget command in the tutorial:

wget -nd -nc -nv -e robots=off -l 1 -r -A hdf5 --content-disposition --header="API-Key: KEY_HERE" "http://www.tng-project.org/api/Illustris-3/files/groupcat-135/?format=api"

it will download two files and name them groups_135.0.hdf5 and groups_135.1.hdf5.

Or you can just download by clicking on the two links.

If they have the wrong name for some reason, you can rename them.

Les Dickens
  • 3 May

Hello, I am working through the Illustris script tutorial and getting almost exactly the same problem as described here, i.e. OSError: Unable to open file (unable to open file: name = './Illustris-3/output//groups_135/fof_subhalo_tab_135.1.hdf5'...etc.

In my case the 2 files, groups_135.0.hdf5 and groups_135.1.hdf5 are both in the downloaded catalogue; nevertheless the search at the top of groupcat.py still seems to fail and is looking for fof_subhalo_tab_135.1.hdf5 instead, which is not in the catalogue.

Looking through the script Traceback, the only thing I have found which seems related is that the variable snapNum, which is used in the search, is not defined. Should it be? Have I missed something?

Any ideas where to look next would be very welcome....

Thanks,
Les

Dylan Nelson
  • 3 May

I suspect it is just a path issue, please double check the paths. If it is looking for the file './Illustris-3/output//groups_135/fof_subhalo_tab_135.1.hdf5' then this file, with that path, relative to the working directory of your python process, needs to exist.

Les Dickens
  • 3 May

Hi, thanks for the response. I checked the path and it does exist. Just to be sure I changed the directory name from Illustris-3 to “IllustrisNEW” and also changed the “basePath=...” command to use the new name. Sure enough, the same OSError came back refencing the changed name.

The directory (~/Illustris-3/output/groups_135/) contains the two files: groups_135.0.hdf5 and groups_135.1.hdf5 and nothing else. You pointed out above, that groupcat.py first looks for something like groups_135.0.hdf5, and if this isn't found, then it looks for something like fof_subhalo_tab_135.0.hdf5. I did look at the code, but I’m a novice – I’ll take your word for it :). Is there anything else I could look at, to try and figure out what’s going wrong?

Thanks again,
Les

Dylan Nelson
  • 3 May

Please post a complete example (all commands and outputs)?

Les Dickens
  • 4 May

Path error - you were correct. Fixed it now.
Thanks,
Les

  • Page 1 of 1