Are the servers down right now?

Community Help
1

Things were working earlier in the day, but now the same code fails.

My REST requests are getting 401 Auth errors even though the API key is the same as in settings. The “Visualize” “Try on my machine” locked up and is still stuck even after refreshing multiple times.

Wondering if maybe something is broken?

Edit: Just revoked my key and generated a new one; same issue. No code has changed since earlier when it worked.

3 hours later.. same thing.

2

Yeah, I’m having the same issue. I don’t think this is a one-off occurrence. I haven’t been able to find a solution yet.

3

It’s funny… yesterday I was struggling to get inference offline on devices working, and today I thought “Well, maybe I can just do it over REST for now.”

Then everything goes down. It must be a sign.

4

Hi @mattkenefick and @Am_Pro,

Could you tell me which the models you are trying to use and which API endpoints are failing for you?

Our status checks (https://status.roboflow.com/) all seem fine and we are not aware of any issues with our hosted services – just did manual testing on several models without issues as well.

If you are working on private workspaces or models I can still try to debug internally on our systems if you give me the workspace id or model id. If you don’t want to post them here on the public forum, feel free to email me at thomas@roboflow.com

5

I have an issue with my models . it’s been several days without resolution, and my thesis project depends on this.

I would appreciate your urgent assistance.

Thank you,

6

@Am_Pro, @mattkenefick, @Enas We’re looking into this more and think we have identified an issue with some public models being used on the hosted service. Investigating now and will update you here when we have a resolution. Thank you for reporting the issue.

7

We just rolled back a change deployed last night that might have caused authorization issues on some public models on the servereless API. We have rolled back the change and are seeing the errors disappear in logs.

1 Like
8

Hi All,

I’m experiencing a problem deploying a model. It seems like a Roboflow outage problem, could somebody confirm this?

{
“message”: “The model upload service is temporarily disabled. It will be re-enabled soon, and we will notify you when it is back online.”,
“type”: “ServiceUnavailableException”,
“hint”: “Please try again later.”
}

1 Like
9

@Anton_Abyzov,

The model upload is a different issue. We have temporarily disabled the model upload service because a security vulnerability was just discovered in PyTorch. We are working on patching our services to precent this issue being exploited, but until then will need to leave the model upload service disabled unfortunately.

You can learn more about the PyTorch issue here: torch.load with weights_only=True RCE · Advisory · pytorch/pytorch · GitHub

10

Hi Thomas,

Thanks for the prompt response. Does it mean there is no way at all to deploy a model to Roboflow? This seems like a major breaking point for a lot of people, is there a workaround?

Thanks in advance.

11

@Anton_Abyzov it is not possible to upload models, but you can still train in our platform.

Our plan is to reenable it back soon, depends on mitigating the security issue.

12

SOS> Hosted API seems down. Any estimate of when it’ll be up or resolved?

We’re in production and waiting.

13

Hi @kedar - the issue should be resolved. Please let me know if you’re still seeing issues!