Public Data Access Overview / Discussion Forum

When I want to check the gas map of a galaxy from Pillepich+19 whose subhalo and snapshot ID is set as below:

subhalo_id = '110543'
subhalo_url = 'https://www.tng-project.org/api/TNG50-1/snapshots/33/subhalos/'+subhalo_id+'/'
particle_url = subhalo_url + "cutout.hdf5"
select_particles = {'gas':'Coordinates,Masses,Velocities'}

I will download a damaged hdf5 file (I use TNG API to download). While when I change the subhalo_id to another, for example, '132290', it works fine. I believe this is a bug for '110543'.

I hope any other people can reproduce the bug.

Dear Hangci,

What exactly do you mean by corrupt? This seems OK to me:

$ wget --content-disposition --header="API-Key: KEY" https://www.tn
g-project.org/api/TNG50-1/snapshots/33/subhalos/110543/cutout.hdf5?gas=Coordinates,Masses,Velocities

--2022-02-11 17:22:39--  https://www.tng-project.org/api/TNG50-1/snapshots/33/subhalos/110543/cutout.hdf5?gas=Coordinates,Masses,Velocities
Resolving www.tng-project.org (www.tng-project.org)... 130.183.17.94
Connecting to www.tng-project.org (www.tng-project.org)|130.183.17.94|:443... connected.
HTTP request sent, awaiting response... 302 Found
Location: https://data-eu.tng-project.org/cutout/subhalo/L35n2160TNG/33/110543/33619970.0.0.0.0.0/?token=cf75c54247d4727987e0 [following]
--2022-02-11 17:22:39--  https://data-eu.tng-project.org/cutout/subhalo/L35n2160TNG/33/110543/33619970.0.0.0.0.0/?token=cf75c54247d4727987e0
Resolving data-eu.tng-project.org (data-eu.tng-project.org)... 130.183.17.94
Connecting to data-eu.tng-project.org (data-eu.tng-project.org)|130.183.17.94|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 62193664 (59M) [application/octet-stream]
Saving to: ‘cutout_110543.hdf5’

cutout_110543.hdf5           100%[=============================================>]  59.31M   181MB/s    in 0.3s

2022-02-11 17:22:41 (181 MB/s) - ‘cutout_110543.hdf5’ saved [62193664/62193664]

and

$ h5ls -r cutout_110543.hdf5
/                        Group
/Header                  Group
/PartType0               Group
/PartType0/Coordinates   Dataset {1553452, 3}
/PartType0/Masses        Dataset {1553452}
/PartType0/Velocities    Dataset {1553452, 3}

and verbose

$ h5ls -rv cutout_110543.hdf5
Opened "cutout_110543.hdf5" with sec2 driver.
/                        Group
    Location:  1:96
    Links:     1
/Header                  Group
    Attribute: BoxSize scalar
        Type:      native double
        Data:  35000
    Attribute: CutoutID scalar
        Type:      native long
        Data:  110543
    Attribute: CutoutRequest scalar
        Type:      variable-length null-terminated UTF-8 string
        Data:  "PartType0=Coordinates,Masses,Velocities"
    Attribute: CutoutType scalar
        Type:      variable-length null-terminated UTF-8 string
        Data:  "subhalo"
    Attribute: Git_commit scalar
        Type:      40-byte null-padded ASCII string
        Data:  "d203ec8b07c7e2bdda5f608aa0babea46d603699"
    Attribute: Git_date scalar
        Type:      29-byte null-padded ASCII string
        Data:  "Thu Apr 7 14:14:27 2016 +0200"
    Attribute: HubbleParam scalar
        Type:      native double
        Data:  0.6774
    Attribute: MassTable {6}
        Type:      native double
        Data:
            (0) 0, 3.07368e-05, 0, 5.73879e-06, 0, 0
    Attribute: NumFilesPerSnapshot scalar
        Type:      native long
        Data:  1
    Attribute: NumPart_ThisFile {6}
        Type:      native int
        Data:
            (0) 1553452, 0, 0, 0, 0, 0
    Attribute: Omega0 scalar
        Type:      native double
        Data:  0.3089
    Attribute: OmegaBaryon scalar
        Type:      native double
        Data:  0.0486
    Attribute: OmegaLambda scalar
        Type:      native double
        Data:  0.6911
    Attribute: Redshift scalar
        Type:      native double
        Data:  2.00203
    Attribute: SimulationName scalar
        Type:      variable-length null-terminated UTF-8 string
        Data:  "L35n2160TNG"
    Attribute: SnapshotNumber scalar
        Type:      native long
        Data:  33
    Attribute: Time scalar
        Type:      native double
        Data:  0.333108
    Attribute: UnitLength_in_cm scalar
        Type:      native double
        Data:  3.08568e+21
    Attribute: UnitMass_in_g scalar
        Type:      native double
        Data:  1.989e+43
    Attribute: UnitVelocity_in_cm_per_s scalar
        Type:      native double
        Data:  100000
    Location:  1:800
    Links:     1
/PartType0               Group
    Location:  1:7760
    Links:     1
/PartType0/Coordinates   Dataset {1553452/1553452, 3/3}
    Location:  1:7488
    Links:     1
    Storage:   37282848 logical bytes, 37282848 allocated bytes, 100.00% utilization
    Type:      native double
/PartType0/Masses        Dataset {1553452/1553452}
    Location:  1:8792
    Links:     1
    Storage:   6213808 logical bytes, 6213808 allocated bytes, 100.00% utilization
    Type:      native float
/PartType0/Velocities    Dataset {1553452/1553452, 3/3}
    Location:  1:9064
    Links:     1
    Storage:   18641424 logical bytes, 18641424 allocated bytes, 100.00% utilization
    Type:      native float

Sorry for the false alarm, I myself tried again today and found it works fine for me now... The day when I post here, I tried several times, download again and again, and always goes wrong.

I have labelled the "false alarm" in the title.