JHU Computer Vision Machine Learning

DESCRIPTION AND PURPOSE

The Hopkins 155 Dataset has been created with the goal of providing an extensive benchmark for testing feature based motion segmentation algorithms. It contains video sequences along with the features extracted and tracked in all the frames. The ground-truth segmentation is also provided for comparison purposes. The data is stored in the .MAT file format.

In [1] we performed a comparison of the main existing algorithms for motion segmentation available in the literature. A summary of the results obtained is included in this page.

CONTENT

Our dataset contains sequences with two and three motions. The sequences can be roughly divided into three categories:

Checkerboard sequences: this category consists of 104 sequences of indoor scenes taken with a handheld camera under controlled conditions. The checkerboard pattern on the objects is used to assure a large number of tracked points.
Traffic sequences: this category consists of 38 sequences of outdoor traffic scenes taken by a moving handheld camera.
Other (Articulated/non-rigid sequences): this category contains 13 sequences displaying motions constrained by joints, head and face motions, people walking, etc.

The following table presents some information about the dataset: the number of sequences, the average number of tracked features and the average number of frames for each one of the categories aformentioned, distinguishing also between sequences with two and three motions.

2 Motions 3 Motions

# Seq. Points Frames # Seq. Points Frames

Checkerboard 78 291 28 26 437 28

Traffic 31 241 30 7 332 31

Others 11 155 40 2 122 31

All 120 266 30 35 398 29

	2 Motions	3 Motions
Checkerboard	78	291	28	26	437	28
Traffic	31	241	30	7	332	31
Others	11	155	40	2	122	31
All	120	266	30	35	398	29


Some samples from the Hopkins 155 dataset

RESULTS

In the table below, we summarize the results obtained by the all the algorithms that we have tested on this dataset. We report the average misclassification error for the sequences with two and three motions. We compare algorithms based on algebraic, statistical and others principles. For more details on the algoritms, please visit the motions segentation research webpage.

Results for sequences with two motions

Checker. GPCA LSA 5 LSA 4n MSL RANSAC DI-GPCA DI-LSA SI-GPCA SI-LSA LLMC 5 LLMC 4n CCS

Average 6.09% 8.84% 2.57% 4.46% 6.52% 4.29% 2.86% 4.18% 2.61% 4.37% 4.65% 16.37%

Median 1.03% 3.43% 0.27% 0.00% 1.75% 2.09% 0.25% 2.24% 0.42% 0.00% 0.11% 10.64%

Traffic GPCA LSA 5 LSA 4n MSL RANSAC DI-GPCA DI-LSA SI-GPCA SI-LSA LLMC 5 LLMC 4n CCS

Average 1.41% 2.15% 5.43% 2.23% 2.55% 0.52% 5.74% 0.65% 4.54% 0.84% 3.65% 5.27%

Median 0.00% 1.00% 1.48% 0.00% 0.21% 0.00% 1.55% 0.00% 1.30% 0.00% 0.33% 0.00%

Others GPCA LSA 5 LSA 4n MSL RANSAC DI-GPCA DI-LSA SI-GPCA SI-LSA LLMC 5 LLMC 4n CCS

Average 2.88% 4.66% 4.10% 7.23% 7.25% 4.79% 7.95% 4.55% 4.38% 6.16% 5.23% 17.58%

Median 0.00% 1.28% 1.22% 0.00% 2.64% 0.43% 1.39% 0.43% 1.39% 1.37% 1.30% 7.07%

Algorithm GPCA LSA 5 LSA 4n MSL RANSAC DI-GPCA DI-LSA SI-GPCA SI-LSA LLMC 5 LLMC 4n CCS

Average 4.59% 6.73% 3.45% 4.14% 5.56% 3.36% 4.07% 3.30% 3.27% 3.62% 4.44% 12.16%

Median 0.38% 1.99% 0.59% 0.00% 1.18% 0.59% 0.51% 0.53% 0.53% 0.00% 0.24% 0.00%

Checker.	GPCA	LSA 5	LSA 4n	MSL	RANSAC	DI-GPCA	DI-LSA	SI-GPCA	SI-LSA	LLMC 5	LLMC 4n	CCS
Average	6.09%	8.84%	2.57%	4.46%	6.52%	4.29%	2.86%	4.18%	2.61%	4.37%	4.65%	16.37%
Median	1.03%	3.43%	0.27%	0.00%	1.75%	2.09%	0.25%	2.24%	0.42%	0.00%	0.11%	10.64%
Traffic	GPCA	LSA 5	LSA 4n	MSL	RANSAC	DI-GPCA	DI-LSA	SI-GPCA	SI-LSA	LLMC 5	LLMC 4n	CCS
Average	1.41%	2.15%	5.43%	2.23%	2.55%	0.52%	5.74%	0.65%	4.54%	0.84%	3.65%	5.27%
Median	0.00%	1.00%	1.48%	0.00%	0.21%	0.00%	1.55%	0.00%	1.30%	0.00%	0.33%	0.00%
Others	GPCA	LSA 5	LSA 4n	MSL	RANSAC	DI-GPCA	DI-LSA	SI-GPCA	SI-LSA	LLMC 5	LLMC 4n	CCS
Average	2.88%	4.66%	4.10%	7.23%	7.25%	4.79%	7.95%	4.55%	4.38%	6.16%	5.23%	17.58%
Median	0.00%	1.28%	1.22%	0.00%	2.64%	0.43%	1.39%	0.43%	1.39%	1.37%	1.30%	7.07%
Algorithm	GPCA	LSA 5	LSA 4n	MSL	RANSAC	DI-GPCA	DI-LSA	SI-GPCA	SI-LSA	LLMC 5	LLMC 4n	CCS
Average	4.59%	6.73%	3.45%	4.14%	5.56%	3.36%	4.07%	3.30%	3.27%	3.62%	4.44%	12.16%
Median	0.38%	1.99%	0.59%	0.00%	1.18%	0.59%	0.51%	0.53%	0.53%	0.00%	0.24%	0.00%

Results for sequences with three motions

Checker. GPCA LSA 5 LSA 4n MSL RANSAC DI-GPCA DI-LSA SI-GPCA SI-LSA LLMC 5 LLMC 4n CCS

Average 31.95% 30.37% 5.80% 10.38% 25.78% 20.54% 6.67% 22.44% 5.55% 10.70% 12.01% 28.63%

Median 32.93% 31.98% 1.77% 4.61% 26.01% 17.30% 1.00% 23.20% 1.21% 9.21% 9.22% 33.21%

Traffic GPCA LSA 5 LSA 4n MSL RANSAC DI-GPCA DI-LSA SI-GPCA SI-LSA LLMC 5 LLMC 4n CCS

Average 19.83% 27.02% 25.07% 1.80% 12.83% 2.46% 10.21% 8.00% 9.51% 2.91% 7.79% 3.02%

Median 19.55% 34.01% 23.79% 0.00% 11.45% 0.55% 4.71% 2.06% 4.71% 0.00% 5.47% 0.18%

Other GPCA LSA 5 LSA 4n MSL RANSAC DI-GPCA DI-LSA SI-GPCA SI-LSA LLMC 5 LLMC 4n CCS

Average 16.85% 23.11% 7.25% 2.71% 21.38% 6.72% 2.13% 7.05% 3.52% 5.60% 9.38% 44.89%

Median 28.66% 23.11% 7.25% 2.71% 21.38% 6.72% 2.13% 7.05% 3.52% 5.60% 9.38% 44.89%

Algorithm GPCA LSA 5 LSA 4n MSL RANSAC DI-GPCA DI-LSA SI-GPCA SI-LSA LLMC 5 LLMC 4n CCS

Average 28.66% 29.28% 9.73% 8.23% 22.94% 18.38% 10.89% 17.50% 9.77% 8.85% 11.02% 26.18%

Median 28.26% 31.63% 2.33% 1.76% 22.03% 16.11% 4.04% 16.74% 2.33% 3.19% 6.81% 31.74%

Checker.	GPCA	LSA 5	LSA 4n	MSL	RANSAC	DI-GPCA	DI-LSA	SI-GPCA	SI-LSA	LLMC 5	LLMC 4n	CCS
Average	31.95%	30.37%	5.80%	10.38%	25.78%	20.54%	6.67%	22.44%	5.55%	10.70%	12.01%	28.63%
Median	32.93%	31.98%	1.77%	4.61%	26.01%	17.30%	1.00%	23.20%	1.21%	9.21%	9.22%	33.21%
Traffic	GPCA	LSA 5	LSA 4n	MSL	RANSAC	DI-GPCA	DI-LSA	SI-GPCA	SI-LSA	LLMC 5	LLMC 4n	CCS
Average	19.83%	27.02%	25.07%	1.80%	12.83%	2.46%	10.21%	8.00%	9.51%	2.91%	7.79%	3.02%
Median	19.55%	34.01%	23.79%	0.00%	11.45%	0.55%	4.71%	2.06%	4.71%	0.00%	5.47%	0.18%
Other	GPCA	LSA 5	LSA 4n	MSL	RANSAC	DI-GPCA	DI-LSA	SI-GPCA	SI-LSA	LLMC 5	LLMC 4n	CCS
Average	16.85%	23.11%	7.25%	2.71%	21.38%	6.72%	2.13%	7.05%	3.52%	5.60%	9.38%	44.89%
Median	28.66%	23.11%	7.25%	2.71%	21.38%	6.72%	2.13%	7.05%	3.52%	5.60%	9.38%	44.89%
Algorithm	GPCA	LSA 5	LSA 4n	MSL	RANSAC	DI-GPCA	DI-LSA	SI-GPCA	SI-LSA	LLMC 5	LLMC 4n	CCS
Average	28.66%	29.28%	9.73%	8.23%	22.94%	18.38%	10.89%	17.50%	9.77%	8.85%	11.02%	26.18%
Median	28.26%	31.63%	2.33%	1.76%	22.03%	16.11%	4.04%	16.74%	2.33%	3.19%	6.81%	31.74%

Legend for the algorithms' naming scheme

GPCA Generalized PCA

LSA 5 Local Subspace Analysis (projection to a space of dimension 5)

LSA 4n Local Subspace Analysis (projection to a space of dimension 4 times the number of motions)

MSL Multi Stage Learning

RANSAC RANdom SAmple Consensus

DI-GPCA Iterative Perspective Algorithm, Depth-Initialization, Subspace separation using GPCA

DI-LSA Iterative Perspective Algorithm, Depth-Initialization, Subspace separation using LSA

SI-GPCA Iterative Perspective Algorithm, Segmentation-Initialization, Subspace separation using GPCA

SI-LSA Iterative Perspective Algorithm, Segmentation-Initialization, Subspace separation using LSA

LLMC 5 Local Linear Manifold Clustering (projection to a space of dimension 5)

LLMC 4n Local Linear Manifold Clustering (projection to a space of dimension 4 times the number of motions)

CCS Connected Component Search

	2 Motions			3 Motions
	# Seq.	Points	Frames	# Seq.	Points	Frames
Checkerboard	78	291	28	26	437	28
Traffic	31	241	30	7	332	31
Others	11	155	40	2	122	31
All	120	266	30	35	398	29

GPCA	Generalized PCA
LSA 5	Local Subspace Analysis (projection to a space of dimension 5)
LSA 4n	Local Subspace Analysis (projection to a space of dimension 4 times the number of motions)
MSL	Multi Stage Learning
RANSAC	RANdom SAmple Consensus
DI-GPCA	Iterative Perspective Algorithm, Depth-Initialization, Subspace separation using GPCA
DI-LSA	Iterative Perspective Algorithm, Depth-Initialization, Subspace separation using LSA
SI-GPCA	Iterative Perspective Algorithm, Segmentation-Initialization, Subspace separation using GPCA
SI-LSA	Iterative Perspective Algorithm, Segmentation-Initialization, Subspace separation using LSA
LLMC 5	Local Linear Manifold Clustering (projection to a space of dimension 5)
LLMC 4n	Local Linear Manifold Clustering (projection to a space of dimension 4 times the number of motions)
CCS	Connected Component Search