Docker inference infer error - Conneciton aborted Conneciton reset by peer

Community Help
1

Hi All - hope you can provide some help. Running on Pop_os(ubuntu) 22.04

I have a working model in roboflow and want to deploy. Installed the docker inference per the info here:

pip install inference-cli && inference server start

I’m using the basic script from the same page pasted into the terminal

inference infer
-i robotst.png
-m tracks_version2/12
–api-key ZZZZZ

I get this message that looks good

Running inference on robotst.jpg, using model: tracks_version2/12, and host: http://localhost:9001

and then after a few minutes I get this message that fails

Command failed. Cause: (‘Connection aborted.’, ConnectionResetError(104, ‘Connection reset by peer’))

Image is a 179 x 247 jpg - and I have used the same image with success in the online workflow test .

I have tried both with and witihout sudo - and get the same result. What am I doing wrong?

2

Hi @knussear ,

Thank you for reporting this issue!

I tried to reproduce your issue by running below:

inference infer --input /path/to/my/file.jpg --model_id chess-pieces-and-chess-board-object-detection/6 --api-key <secret>
Running inference on /path/to/my/file.jpg, using model: chess-pieces-and-chess-board-object-detection/6, and host: http://localhost:9001
{'inference_id': '089405ee-f15f-41e6-8af4-b267ef655470', 'time': 0.3257904169995527, 'image': {'width': 3000, 'height': 4000}, 'predictions': [{'x': 1515.5, 'y': 2218.5, 'width': 977.0, 'height': 1875.0, 'confidence': 0.9699227213859558, 'class': 'black-rook', 'class_id': 5, 'detection_id': 'd1ade67a-8f57-488f-9a33-fcebbef319b1'}, {'x': 1906.5, 'y': 2281.0, 'width': 1875.0, 'height': 1874.0, 'confidence': 0.4406488537788391, 'class': 'board', 'class_id': 6, 'detection_id': 'c0eb7a5a-d7ab-44b8-91fe-6dc4e77c2ce7'}]}

I get inference results in reasonable time.

Can you confirm what OS are you running on? Also, when this issue happens again, can you collect docker logs? You can find out docker container name by running inference server status (it will be listed as Container Name). My container name is called wonderful_pasteur and I can get docker logs by running docker logs -f wonderful_pasteur

Grzegorz

3

I am running Pop!_OS 22.04 LTS (Ubuntu)

Lots of repetition in the logs but here is a sample:
I’m attaching a picture and a text file since I get an error on more than 2 links in a post

roboflow error
roboflow error1062×864 56 KB

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File “/usr/local/lib/python3.9/site-packages/requests/adapters.py”, line 667, in send
resp = conn.urlopen(
File “/usr/local/lib/python3.9/site-packages/urllib3/connectionpool.py”, line 802, in urlopen
retries = retries.increment(
File “/usr/local/lib/python3.9/site-packages/urllib3/util/retry.py”, line 594, in increment
raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host=‘api.roboflow.com’, port=443): Max retries exceeded with url: /inference-stats (Caused by NewConnectionError(‘<urllib3.connection.HTTPSConnection object at 0x77ea1b6fba60>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution’))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File “/app/inference/core/managers/pingback.py”, line 125, in post_data
res = requests.post(wrap_url(METRICS_URL), json=all_data, timeout=10)
File “/usr/local/lib/python3.9/site-packages/requests/api.py”, line 115, in post
return request(“post”, url, data=data, json=json, **kwargs)
File “/usr/local/lib/python3.9/site-packages/requests/api.py”, line 59, in request
return session.request(method=method, url=url, **kwargs)
File “/usr/local/lib/python3.9/site-packages/requests/sessions.py”, line 589, in request
resp = self.send(prep, **send_kwargs)
File “/usr/local/lib/python3.9/site-packages/requests/sessions.py”, line 703, in send
r = adapter.send(request, **kwargs)
File “/usr/local/lib/python3.9/site-packages/requests/adapters.py”, line 700, in send
raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPSConnectionPool(host=‘api.roboflow.com’, port=443): Max retries exceeded with url: /inference-stats (Caused by NewConnectionError(‘<urllib3.connection.HTTPSConnection object at 0x77ea1b6fba60>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution’))

4

Thanks!

Error suggests issues related to networking, can you try below and share results?

docker exec -it <your_container_name> /bin/bash -c "apt-get update && apt-get install iputils-ping && ping -c 4 8.8.8.8 && ping -c 4 google.com && ping -c 4 api.roboflow.com"

Many thanks!
Grzegorz

5

I can only post and image because it has > 2 links. Here are the results

image
image1071×687 104 KB

6

Hi @knussear ,

Many thanks for running the test!

It seems this machine might be having internet problems, the error from screenshot you shared contains following:

(...)
Ign:3 http://deb.debian.org/debian-security bookworm-security InRelease
    Temporary failure resolving deb.debian.org
(...)

So, running commands like apt-get update fails due to network issue. There is a slight chance this is DNS resolver issue, can you please try to run below and share results:

docker exec -it <your_container_name> /bin/bash -c "wget 216.58.208.206"

216.58.208.206 is IP address I got resolved when I tried to ping google.com

Thanks!

7

I get

–2024-12-03 17:07:00-- http://216.58.208.206/
Connecting to 216.58.208.206:80… failed: No route to host.

8

Hi @knussear,

Thank you for running the test, it seems the container has no route to the internet. Can you try to run wget 216.58.208.206 (or ping 216.58.208.206) on host machine?

Thanks again,
Grzegorz

9

The host machine is running fine - it is my main desktop

image
image577×387 37 KB

10

I killed docker and re-ran this -

sudo inference server start

and I get this output - which may yield a clue since I can see the network bridge failing. Any ideas?

Image roboflow/roboflow-inference-server-cpu:latest pulled.
Starting inference server container…
500 Server Error for http+docker://localhost/v1.47/containers/93942fa671d36928a26798fee4f84ed4115f6cc333959d6f5ec55c77f71272b7/start: Internal Server Error (“failed to create endpoint zen_greider on network bridge: adding interface veth1b222ea to bridge docker0 failed: Device does not exist”)

11

I followed some instructions here

and got the network bridge working after a reboot. Looks like I am getting a network connection now and it is working!

inference infer
-i robotst.jpg
-m tracks_version2/12
–api-key
Running inference on robotst.jpg, using model: tracks_version2/12, and host: http://localhost:9001
{‘inference_id’: ‘a9b361a1-e6dc-4e2e-a884-b4f90dad3a75’, ‘time’: 0.03681192199974248, ‘image’: {‘width’: 179, ‘height’: 247}, ‘predictions’: [{‘x’: 46.0, ‘y’: 150.5, ‘width’: 92.0, ‘height’: 153.0, ‘confidence’: 0.6553604602813721, ‘class’: ‘tracks’, ‘class_id’: 0, ‘detection_id’: ‘b9e6ab99-6c29-4c62-adc0-253fcdf3f0f7’}, {‘x’: 89.0, ‘y’: 11.5, ‘width’: 178.0, ‘height’: 23.0, ‘confidence’: 0.5356184244155884, ‘class’: ‘tracks’, ‘class_id’: 0, ‘detection_id’: ‘82d37e9a-0d1f-46a6-b097-396d18d2d99c’}]}

12

Hi @knussear ,

It looked like networking issue, I’m happy to see you found solution to this issue!

Grzegorz