Show RF: Chalamet lookalike app using OpenAI’s CLIP

✨ Show & Tell
1

img-blog-deep-learning-solves-frustrations (8)
img-blog-deep-learning-solves-frustrations (8)1200×731 870 KB

Show RF: Chalamet lookalike app using OpenAI’s CLIP

Project link: https://chalamet.wtf

Hey folks! Wanted to share a project I made in my second week at Roboflow. As part of our company onboarding, we spend a week on an independent project. I chose to try to create an online Timothee Chalamet look alike competition inspired by this viral one from Oct 2024.

I drew inspiration from Roboflow’s paint.wtf 2021 saga and used OpenAI’s CLIP as the underlying vision engine powering similarity between User submitted image & Chalamet.

How it works, in a nutshell:

  1. Take a selfie.

  2. Convert that selfie into an embedding using OpenAI CLIP.

  3. Compare that embedding to a Timothee Chalamet embedding using cosine similarity.

  4. Save the resulting similarity score, then normalize scores across all submissions and return the results as a global leaderboard

How CLIP works & Paint.wtf backstory:

CLIP lets you convert images and text into vectors, like so:

field of corn → [[4,3],[2,1],etc]

input_image.jpg → [[5,6],[2,8],etc]

With both field of corn and input_image.jpg represented as vectors, we can mathematically assess their similarity. The most common way to do that is using cosine similarity.

In 2021, Roboflow team used cosine similarity and CLIP to power paint.wtf.

Only difference is: in chalamet.wtf, I only use CLIP to compare images. More on the actual implementation below :backhand_index_pointing_down:

How I used CLIP:

Initially, I tried to base the competition purely on a comparison of the words timothee chalamet with a given image submitted by the user. It was okay. After more testing, I found better results by comparing a given image submitted by the user to the average of a bunch of Chalamet image embeddings.

Here’s what I ended up doing:

  1. First, convert 10 unique images of chalamet into vector embeddings.
  2. Get the average of those embeddings, called the centroid.
  3. Compare that centroid to the user submitted image

The results aren’t perfect, but it’s pretty fun!

Try it out! Website is live at chalamet.wtf

Give it a go →→→ https://chalamet.wtf

Any questions or suggestions?

1 Like