Tour the World: building a web-scale landmark recognition engine

Tour the World: building a web-scale landmark recognition engine

Collective Vision: Using Extremely Large Photograph Collections Mark Lenz CameraNet Seminar University of Wisconsin Madison February 2, 2010 Acknowledgments: These slides combine and modify slides provided by Yantao Zheng et al. (National University of Singapore/Google) Last Time Distributed Collaboration Google Goggles Personal object recognition

World-Wide Landmark Recognition Building the engine Today World-Wide Landmark Recognition Querying the engine Building Rome in a Day Distributed matching and reconstruction Discussion

Unsupervised learning of landmark images Geoclusters Landmarks from tour articles Noisy image pool Visual clustering Validate and

clean models Premise: photos from landmark should be similar Clustering based on local features Visual model validates landmarks! Photo v.s. non-photo classifer to filter out noisy images Object matching based on local features Sim( ,

) = image match score Image representation match score = Interest points: Laplacian-of-Gaussian (LoG) filter Local feature: Gabor wavelets Probability that match of and

is false positive Probability of at least m out of n features match, if Probability of a feature match by chance Match Region Graph Node is match region 2 types of edges: Image matching match edge:

measures match confidence overlap region edge: measures spatial overlapping Visual clusters Agglomerative hierarchical clustering False detected images Match is technically correct, but match region is not landmark A problem of model generation

Match is technically false, due to visual similarity A problem of image feature and matching mechanism For positive images: 337/417 (80.8%) are correct Identification rate: 337/728 (46.3%) For negative images: False acceptance rate: 1.1% Landmark Recognition

All local features indexed in one k-d tree Match region - interest points that contribute to a match between two images k-d trees

k-dimensional binary tree Sub-trees split at median w.r.t one dim Cycle through dimensions Creates bins of NNs Indexing local feature for matching Query time: ~0.2 sec in a P4 computer Landmark Recognition Detect features on query image

For each feature in query image Find NN features using k-d tree NN features link to their model image Score match regions between query and model images Scoring Match Regions

Query image interest points matching points in model image determined through NN search Match score = 1-PFPij (probability match b/w regions is false positive)

PFPij is based on the number of matched points Match threshold = total score > 5 Intuition Query image should have many interest points with matches in match region = high match score

Points should have matches in multiple regions (images) - threshold Building Rome in a Day Use photos from photo-sharing websites to build 3D models of cities Web photos less structured than automated image capture (e.g. aerial) Increased efficiency through distributed computations Multi-Stage Parallel Matching Matching is distributed across nodes

Vocabulary tree-inspired match proposals For distributed matching Query Expansion to increase cluster density Match proposals create only sparse clusters Conclusion Distributed Collaboration Google Goggles Personal object recognition World-Wide Landmark Recognition Building Rome in a Day

Distributed matching and reconstruction Thoughts for Discussion Geo-clustering to filter out seldom traveled/photographed sites Match region graph for view comparison Pre-tag landmarks such as exits Augmented reality Distributed matching of features Ad-hoc wireless network range Other thoughts...

Recently Viewed Presentations

  • Informal Assessment of Reading Comprehension in a PARCC ...

    Informal Assessment of Reading Comprehension in a PARCC ...

    Informal Assessment of Reading Comprehension in a PARCC Environment. Helen S. Comba. Literacy Consultant. ... Give them time to write about their Post-it note. Feedback would need to be given at this point (Pair-Share or Silent Writing Circle). Now time...
  • Types of Pathogens - Mr. Kowaleski's Webpage

    Types of Pathogens - Mr. Kowaleski's Webpage

    Types of Pathogens. ... Many Protists are carried by a vector, a carrier of a disease that passes it on to a human host ie. mosquito, tick. ... They obtain energy from their environment, meaning they are heterotrophes. They live...
  • Reactions of Aromatic Compounds

    Reactions of Aromatic Compounds

    => Examples of Nucleophilic Substitution Addition-Elimination Mechanism Benzyne Mechanism Reactant is halobenzene with no electron-withdrawing groups on the ring. Use a very strong base like NaNH2. => Benzyne Intermediate Chlorination of Benzene Addition to the benzene ring may occur with...
  • Smiles on Wings: New Beginnings - Dining for Women

    Smiles on Wings: New Beginnings - Dining for Women

    HOPE was founded in 1999 by Dr. IftikherMahmood, a native of Cox's Bazar, Bangladesh. The Foundation set up a hospital and 8 medical clinics in rural areas of Bangladesh to provide primary and specialized healthcare to the poor. HOPE partners...
  • Baker College - Flint OT

    Baker College - Flint OT

    Grand Rapids. 1st Class graduates in Dec 2016. Practice-Focused Curriculum . Level I: 1 immersion and 2 concurrent. Level II: Summer and Fall of 2016. Susan Dee Owens , MS, OTRL. 616 -871-6159 [email protected] Eastern Michigan University.
  • DFAT Formal PowerPoint

    DFAT Formal PowerPoint

    After the Support Unit has entered into formal contractual arrangements with pre-selected NGOs, DFAT will formally notify HPA partners that the response mechanism will no longer be used. DRR/DRM activities under the HPA will continue until June 2017.
  • V. Gilded Age Politics

    V. Gilded Age Politics

    Provide an example. Compare and Contrast pros and cons of the spoils system and the civil service system. Compare and Contrast the Republican and Democratic voting blocs. Explain how the government applied the term laissez-faire to the federal government. Compare...
  • The Following is a Product of the Northern

    The Following is a Product of the Northern

    "AIDS was sent by god to punish gay men." If HIV and AIDS were sent by god to punish gay men, a divine intervention of this nature was a very poor move. HIV has also infected a lot of women,...