# Assistance Needed: Invalid RLE Mask Representation in New Dataset

**URL:** <https://discuss.roboflow.com/t/assistance-needed-invalid-rle-mask-representation-in-new-dataset/8086>\
**Category:** 🤝  Community Help\
**Tags:** bugs, formats\
**Created:** [December 30, 2024, 7:33pm UTC](https://discuss.roboflow.com/t/assistance-needed-invalid-rle-mask-representation-in-new-dataset/8086 "2024-12-30T19:33:41Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![Faris-Faiz](https://yyz1.discourse-cdn.com/flex029/user_avatar/discuss.roboflow.com/faris-faiz/32/6235_2.png) [@Faris-Faiz](https://discuss.roboflow.com/u/Faris-Faiz)\
**Post date:** [December 30, 2024, 7:33pm UTC](https://discuss.roboflow.com/t/assistance-needed-invalid-rle-mask-representation-in-new-dataset/8086/1 "2024-12-30T19:33:41Z")

</div>

Hi everyone,

I’m currently facing an issue with a dataset I generated using Roboflow, and I’m hoping someone here can help me resolve it. Here’s a summary of the problem and what I’ve done so far to debug it:

### **Issue:**

I have a segmentation dataset that I generated about a month ago ([original\_dataset.json](https://drive.google.com/file/d/1-0gaLjfYV_4ydD-yJdqhZEvmpYi6BWRA/view?usp=sharing)) (Dataset Generated on **Nov 29, 2024** ), which works perfectly fine. However, I recently generated a new version of the dataset ([new\_dataset.json](https://drive.google.com/file/d/1toTOGbhD943E9rZMZa0YvQD5ivOxAPsC/view?usp=sharing)) (Dataset Generated on **Dec 31, 2024** ), and when I try to use it with SAM2.1, I encounter the following error:  
`ValueError: Invalid RLE mask representation`

This error occurs specifically with the new dataset, and I’ve confirmed that the issue lies in the RLE encoding of the segmentation masks in `new_dataset.json`.

### **What I’ve Done to Debug:**

1. **Compared the Datasets:**

2. **Attempted to Fix the RLE Encoding:**

3. **Skipped Problematic Annotations:**

### **Request for Assistance:**

Could anyone help me understand why the RLE encoding in the new dataset is invalid? Specifically:

- Are there known issues with RLE encoding in newer versions of Roboflow?
- How can I ensure that the RLE encoding in the new dataset matches the format expected by `pycocotools`?
- Is there a way to regenerate the dataset with correct RLE encoding?

Here’s a snippet of the problematic RLE encoding from `new_dataset.json` for annotation ID 27:

```json
"segmentation": {
    "counts": "QoQ3o0Qo0PPWMPPi2PPi2PPWM0000000000000000PPnLPPR3PPR3PPnL0000000000000000PPeLPP[3PP[3PPeL0000000000000000PP\\LXPd3hoc3PP\\L0000000000000000PPSLPPm3PPm3PPSL0000000000000000PPjKPPV4PPV4PPjK000000000000PPUKPPk4PPk4PPUKPPaKPP_4PP_4PPaK0000000000000000PPXKPPh4PPh4PPXK0000000000000000PPoJPPQ5PPQ5PPoJ0000000000000000PPfJPPZ5PPZ5PPfJ0006J00000000000PP]JPPc5PPc5PP]J0000000000000000PPTJPPl5PPl5PPTJ0000000000000000PPkIPPU6PPU6PPkI0000000O10000000PPbIPP^6PP^6PPbI0000000000000000PPYIPPg6PPg6PPYI0000000000000000PPPIPPP7PPP7PPPI000002N5K0000000QPgHooX7ooX7QPgH0000000000000000QP^Hooa7ooa7QP^H0000000000000000QPUHooj7ooj7QPUHPPZGPPf8PPf8PPZG000000000000PPlGPPT8PPT8PPlG00000N2000000000PPcGPP]8PP]8PPcG0000000000000000PPZGPPf8PPf8PPZG000000000004L000PPQGPPo8PPo8PPQG0000000000000000PPhFPPX9PPX9PPhF0000000000000000PP_FPPa9PPa9PP_F0000000000000000PPVFPPj9PPj9PPVF0000000000000000PPmEPPS:PPS:PPmE0000000000000000PPdEPP\\:PP\\:PPdE0000000000000000PP[EPPe:PPe:PP[E0000000000000000PPREPPn:PPn:PPRE0000000000PPhCPPX<PPX<PPhC00PPiDPPW;PPW;PPiD0000000000000000PP`DPP`;PP`;PP`D0000000000000000PPWDPPi;PPi;PPWD0000000000000000PPnCPPR<PPR<PPnC0000000000PPhCWOiPX<0000000PPdNPP\\1PP\\1PPdN0000PP\\NPPd1PPd1PP\\N00000000PP[NPPe1PPe1PP[N0001O0000002N000PPRNPPn1PPn1PPRN0000000000000000PPiMPPW2PPW2PPiM0000000000000000PP`MPP`2PP`2PP`M0000000000000nPnl0",
    "size": [1024, 1024]
}

```

### **Additional Context:**

- The `original_dataset.json` works perfectly fine with the same code and tools.
- I’m using `pycocotools` for decoding and encoding the masks.
- The problematic annotations seem to have corrupted or improperly formatted RLE encoding.

### **Required Info:**

- **Project Type:** Instance Segmentation
- **Operating System & Browser:** Windows / Google Chrome
- **Project ID:** black-parts-panorama-image

Also, I made sure the original dataset has the exact same augmentations applied on the new dataset, and the error still persists.

Version of `pycocotools` is 2.0.8

Any guidance or assistance would be greatly appreciated!

Thanks in advance,  
Faris

---

<div class="post-metadata">

**Author:** ![Karan\_Santra](https://yyz1.discourse-cdn.com/flex029/user_avatar/discuss.roboflow.com/karan_santra/32/5622_2.png) [@Karan\_Santra](https://discuss.roboflow.com/u/Karan_Santra)\
**Post date:** [December 30, 2024, 9:23pm UTC](https://discuss.roboflow.com/t/assistance-needed-invalid-rle-mask-representation-in-new-dataset/8086/2 "2024-12-30T21:23:57Z")

</div>

I am facing a similar issue.

> [@Error in running the demo fine-tune-sam-2.1.ipynb collab notebook by James Gallagher (@rfjames ?)](https://discuss.roboflow.com/t/error-in-running-the-demo-fine-tune-sam-2-1-ipynb-collab-notebook-by-james-gallagher-rfjames/8084):
>
> I tried running the [“fine-tune-sam-2.1.ipynb”](https://colab.research.google.com/github/roboflow-ai/notebooks/blob/add-fine-tune-sam-2.1/notebooks/fine-tune-sam-2.1.ipynb?ref=blog.roboflow.com) notebook provided by Roboflow with only one change—I used my own dataset. My dataset has two classes, and I resized the images by stretching them to 1024x1024. The dataset is for an image segmentation task and is exported as SAM2 JSON. Other than that, I didn’t make any modifications. When I ran the Python training command, I got an error. Since the dataset is the only thing I changed, I think the problem might be related to it. Could it be because …

---

<div class="post-metadata">

**Author:** ![komsstudent](https://avatars.discourse-cdn.com/v4/letter/k/8edcca/32.png) [@komsstudent](https://discuss.roboflow.com/u/komsstudent)\
**Post date:** [December 31, 2024, 9:41am UTC](https://discuss.roboflow.com/t/assistance-needed-invalid-rle-mask-representation-in-new-dataset/8086/3 "2024-12-31T09:41:00Z")

</div>

Same issue, i thought it was my bad. Finally, my masks are valid, since exporting in coco format works fine. Already have faced with issue on 28.12.

How have you fixed some wrong masks? I would be grateful if you share it.

---

<div class="post-metadata">

**Author:** ![iurisilvio](https://yyz1.discourse-cdn.com/flex029/user_avatar/discuss.roboflow.com/iurisilvio/32/4836_2.png) [@iurisilvio](https://discuss.roboflow.com/u/iurisilvio)\
**Post date:** [December 31, 2024, 2:47pm UTC](https://discuss.roboflow.com/t/assistance-needed-invalid-rle-mask-representation-in-new-dataset/8086/4 "2024-12-31T14:47:03Z")

</div>

Hi all! Thanks for all the detailed reports! I found a recent optimization was generating wrong RLE masks. 😞

I reverted the change and I believe you’ll be able to generate good masks again.

Please, generate new versions and export from them. If you export again in the same version, it’ll just serve the same file already generated.

Let me know if it works!

---

<div class="post-metadata">

**Author:** ![Faris-Faiz](https://yyz1.discourse-cdn.com/flex029/user_avatar/discuss.roboflow.com/faris-faiz/32/6235_2.png) [@Faris-Faiz](https://discuss.roboflow.com/u/Faris-Faiz)\
**Post date:** [January 2, 2025, 7:03pm UTC](https://discuss.roboflow.com/t/assistance-needed-invalid-rle-mask-representation-in-new-dataset/8086/5 "2025-01-02T19:03:17Z")

</div>

I’ve tested the new version, from basic testing, it works. Im not sure about training SAM2.1 though now as I’m facing issues in regards to the yaml config file.

---

<div class="post-metadata">

**Author:** ![system](https://canada1.discourse-cdn.com/flex029/uploads/roboflow1/original/1X/bb6edbfa56d0cf2ee603f061e9e09103502cc8cc.png) [@system](https://discuss.roboflow.com/u/system)\
**Post date:** [January 23, 2025, 7:04pm UTC](https://discuss.roboflow.com/t/assistance-needed-invalid-rle-mask-representation-in-new-dataset/8086/6 "2025-01-23T19:04:18Z")

</div>

This topic was automatically closed 21 days after the last reply. New replies are no longer allowed.