Can't Download my Dataset

As regarding the last update status :

1 hour passed , it jumped to 49% then to 0.1%
this was on version 2
should i create new version or what ?

and what about the status of the Analytics issue ?
@Ford @Grzegorz

Hi @Ahmed_Banby ,

Yes, I am monitoring this and right now I’m adding more resources so this export can be completed, apologies for inconvenience but hopefully it’s the last time we see this failing. Please give me 15 minutes and you will be able to initiate export again.

Regarding analytics issue - I have no solution yet, will report here once the root cause is identified

Grzegorz

1 Like

should I always contact you (as Tech.Team) , when I need to export dataset version or this is a permanent solution ??

Hi @Ahmed_Banby , the resource limit increase is permanent and global, also once export is performed all subsequent downloads of the same format should not require any additional compute

1 Like

okay ! , we are jumped to 45 % now !!!

We will have to restart the export once the limit increase takes effect, this is still propagating - I will let you know in a few minutes once it’s done

waiting for u

I restarted the exporting and we returned stuck at 0.1% , I know that probably the solution is still not applied but I am just updating you

regarding the analytics issue @Grzegorz

this is what i see when the loading screen of the analytics finishes after very long time
and for the exporting , the values changed frequently sometime 0 sometimes 18 , 19 , 20 ,49 , 40 , 44 and like this (I am just giving you updates)

Indeed, the change had not been fully propagated and your export landed into the pool which is pending to be replaced, lets wait until it fails - please do not trigger export as it makes us wait a bit longer :slight_smile: I will report here once ready!

oh sorry I did not know that I will let it fail and will not touch anything untill you say

1 Like

@Grzegorz
the download and extraction is ended what are these .json files ??

The jsonl documents you see are temporary intermediate files that were created as part of the process to reduce RAM footprint of export, they normally are not included in the final zip, please discard them

I will try to download the other dataset
this one is done

Hi @Ahmed_Banby , please try to download your other dataset again, the underlying issue had been resolved. I will monitor this thread in case there are any other issues with dataset export.

Today I will look into the issue with analytics page you mentioned earlier in this thread.

hello @Grzegorz

okay I tested downloading my dataset and It worked
to be honest still I did not test the Largest one (with 2Million and 90k and some images)
I have some important things to do at my college last days for my graduation
so Can I take like 3 days more so I can have spare time to test and tell you the updates
sorry for ur time waste I did not mean to make you wait
thank you for ur help :heart::heart::heart:

what about the Analytics Issue ?

Hi @Ahmed_Banby ,

With regards to analytics of such large dataset - I escalated this issue to the team who is developing this feature and they will look into this but I can’t provide the timeline, if there will be more users hitting this problem then priorities might change.

Thanks!

Grzegorz

1 Like