{
"cells": [
{
"cell_type": "markdown",
"id": "60a7e08e-93c6-4370-9778-3bb102dce78b",
"metadata": {},
"source": [
"Copyright (c) Meta Platforms, Inc. and affiliates."
]
},
{
"cell_type": "markdown",
"id": "3081cd8f-f6f9-4a1a-8c36-8a857b0c3b03",
"metadata": {},
"source": [
"\n",
"
\n",
""
]
},
{
"cell_type": "markdown",
"id": "f9f3240f-0354-4802-b8b5-9070930fc957",
"metadata": {},
"source": [
"# CoTracker: It is Better to Track Together\n",
"This is a demo for CoTracker, a model that can track any point in a video."
]
},
{
"cell_type": "markdown",
"id": "36ff1fd0-572e-47fb-8221-1e73ac17cfd1",
"metadata": {},
"source": [
"
"
]
},
{
"cell_type": "markdown",
"id": "88c6db31",
"metadata": {},
"source": [
"Don't forget to turn on GPU support if you're running this demo in Colab. \n",
"\n",
"**Runtime** -> **Change runtime type** -> **Hardware accelerator** -> **GPU**\n",
"\n",
"Let's install dependencies for Colab:"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "278876a7",
"metadata": {},
"outputs": [],
"source": [
"# !git clone https://github.com/facebookresearch/co-tracker\n",
"# %cd co-tracker\n",
"# !pip install -e .\n",
"# !pip install opencv-python einops timm matplotlib moviepy flow_vis\n",
"# !mkdir checkpoints\n",
"# %cd checkpoints\n",
"# !wget https://huggingface.co/facebook/cotracker/resolve/main/cotracker2.pth"
]
},
{
"cell_type": "code",
"execution_count": 2,
"id": "1745a859-71d4-4ec3-8ef3-027cabe786d4",
"metadata": {
"ExecuteTime": {
"end_time": "2024-07-29T20:52:14.487553Z",
"start_time": "2024-07-29T20:52:12.423999Z"
}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"/mnt/d/cotracker\n"
]
}
],
"source": [
"%cd ..\n",
"import os\n",
"import torch\n",
"\n",
"from base64 import b64encode\n",
"from cotracker.utils.visualizer import Visualizer, read_video_from_path\n",
"from IPython.display import HTML"
]
},
{
"cell_type": "code",
"execution_count": 3,
"id": "44342f62abc0ec1e",
"metadata": {
"ExecuteTime": {
"end_time": "2024-07-29T20:52:31.688043Z",
"start_time": "2024-07-29T20:52:31.668043Z"
}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"CUDA available\n"
]
}
],
"source": [
"if torch.cuda.is_available():\n",
" print('CUDA available')"
]
},
{
"cell_type": "markdown",
"id": "7894bd2d-2099-46fa-8286-f0c56298ecd1",
"metadata": {},
"source": [
"Read a video from CO3D:"
]
},
{
"cell_type": "code",
"execution_count": 4,
"id": "f1f9ca4d-951e-49d2-8844-91f7bcadfecd",
"metadata": {},
"outputs": [],
"source": [
"video = read_video_from_path('./assets/F1_shorts.mp4')\n",
"video = torch.from_numpy(video).permute(0, 3, 1, 2)[None].float()"
]
},
{
"cell_type": "code",
"execution_count": 5,
"id": "fb4c2e9d-0e85-4c10-81a2-827d0759bf87",
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"