Public Data Access Overview / Discussion Forum

While working with the code, I encountered a recurring error. requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://data-eu.tng-project.org/info/halo/L75n455FP/135/0/?token=710211c9eeb6961d36af
It occurs at this line in my code:

parent_fof = get(url)

Here is the relevant code snippet:

def get(path, params=None):
    ...
import h5py
mpb1 = get(sub['trees']['sublink_mpb']) # file saved, mpb1 contains the filename
f = h5py.File(mpb1, 'r')
print(f.keys())

I was wondering if this might be a permissions issue or if I’m missing a step in the API access process.

Can you post sub['trees']['sublink_mpb'] i.e. the actual URL that you are passing to get()?

I am currently following the IllustrisTNG API “Getting Started” guide, and I encountered an error while running the example code.

Below is the code I have run so far:

import requests
import h5py

baseUrl = "http://www.tng-project.org/api/"
headers = {"api-key": ""}

def get(path, params=None):
    r = requests.get(path, params=params, headers=headers)
    r.raise_for_status()
    if r.headers['content-type'] == 'application/json':
        return r.json()
    if 'content-disposition' in r.headers:
        filename = r.headers['content-disposition'].split("filename=")[1]
        with open(filename, 'wb') as f:
            f.write(r.content)
        return filename
    return r

r = get(baseUrl)
names = [sim['name'] for sim in r['simulations']]
i = names.index("Illustris-3")
sim = get(r['simulations'][i]['url'])
snaps = get(sim['snapshots'])
snap = get(snaps[-1]['url'])
subs = get(snap['subhalos'])
sub = get(subs['results'][1]['url'])
url = sub['related']['parent_halo'] + "info.json"
parent_fof = get(url)

mpb1 = get(sub['trees']['sublink_mpb'])
f = h5py.File(mpb1, 'r')
print(f.keys())

In the command parent_fof = get(url), what is the actual URL you are passing to get?

Is it

http://www.tng-project.org/api/Illustris-3/snapshots/99/halos/0/info.json

If so, you can also just try in your browser (for me it works fine).

I am experiencing a similar issue with TNG50 cutouts. I am able to pull the mpb, but when trying to download the cutout, I hit the same 403 error. Doesn't work directly in the browser or using Python requests. Even tried generating a new key.

http://www.tng-project.org/api/TNG50-1/snapshots/99/subhalos/117259/sublink/mpb.hdf5
http://www.tng-project.org/api/TNG50-1/snapshots/99/subhalos/117259/cutout.hdf5

tried with both ?api_key= ... and passing the key as headers {"api-key":"..."}.

requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://data-eu.tng-project.org/cutout/subhalo/L35n2160TNG/99/117259/67108863.255.0.7.262143.16777215/?token=...

requests.exceptions.HTTPError: 400 Client Error: Bad Request for url: https://www.tng-project.org/api/TNG50-1/snapshots/99/subhalos/117259/cutout.hdf5?api_key=....

I can't reproduce the problem, clicking both links works equally well for me.

Perhaps it was just a period of high load on the servers. Does the cutout link still not work?

Hi Dylan,

Yes the api does appear to be working again I was able to pull down a cutout using requests.

Thanks

Dear Nelson,
Hope you are doing well! I am finally able to access the snapshots. I was trying to figure it out for the past few days. Thanks a lot for your guidance and support.
Best Regards
Kevin