A list of point set matching/registration resources collected by Gang Wang. If you find that important resources are not included, please feel free to contact me.

Point Set Matching/Registration Material

Point Set Matching/Registration Methods [wiki]

Point Matching/Registration Methods

• [MCT] A mathematical analysis of the motion coherence theory, IJCV’1989 [pdf]
• [ICP: point-to-point] Method for Registration of 3-D Shapes, Robotics-DL tentative’1992 [pdf] [code][code] [code] [material] [tutorial]
• [ICP: point-to-plane] Object modeling by registration of multiple range images, IVC’1992 [pdf]
• [ICP] Iterative point matching for registration of free-form curves and surfaces, IJCV’1994 [pdf]
• [RPM/Softassign] New algorithms for 2d and 3d point matching: pose estimation and correspondence, PR’1998 [pdf] [code]
• [MultiviewReg] Multiview registration for large data sets, 3DDIM’1999 [pdf]
• [SC] Shape matching and object recognition using shape contexts, TPAMI’2002 [pdf][wiki] [project] [code]
• [EM-ICP] Multi-scale EM-ICP: A Fast and Robust Approach for Surface Registration, ECCV’2002 [pdf] [code]
• [LM-ICP] Robust registration of 2D and 3D point sets, IVC’2003 [pdf] [code]
• [TPS-RPM] A new point matching algorithm for non-rigid registration, CVIU’2003 [pdf] [project] [code]
• [Survey] Image registration methods: a survey, IVC’2003 [pdf]
• [KCReg] A correlation-based approach to robust point set registration, ECCV’2004 [pdf] [code]
• [3DSC] Recognizing objects in range data using regional point descriptors, ECCV’2004 [pdf] [code_pcl]
• [RGR] Robust Global Registration, ESGP’2005 [pdf]
• [RPM-LNS] Robust point matching for nonrigid shapes by preserving local neighborhood structures, TPAMI’2006 [pdf] [code]
• [IT-FFD] Shape registration in implicit spaces using information theory and free form deformations, TPAMI’2006 [pdf]
• [Rigid] Rigid Body Registration, 2007 [ch02]
• [CDC] Simultaneous covariance driven correspondence (cdc) and transformation estimation in the expectation maximization framework, CVPR’2007 [pdf] [project]
• [Nonrigid-ICP] Optimal step nonrigid icp algorithms for surface registration, CVPR’2007 [pdf] [code] [code]
• [GNA] Global non-rigid alignment of 3D scans, TOG’2007 [pdf]
• [PF] Particle filtering for registration of 2D and 3D point sets with stochastic dynamics, CVPR’2008 [pdf]
• [JS] Simultaneous nonrigid registration of multiple point sets and atlas construction, TPAMI’2008 [pdf]
• [4PCS] 4-points congruent sets for robust pairwise surface registration, TOG’2008 [pdf] [project]
• [GICP] Generalized ICP, RSS’2009 [pdf] [code]
• [MP] 2D-3D registration of deformable shapes with manifold projection, ICIP’2009 [pdf]
• [SMM] The mixtures of Student’s t-distributions as a robust framework for rigid registration, IVC’2009 [pdf]
• [Algebraic-PSR] An Algebraic Approach to Affine Registration of Point Sets, ICCV’2009 [pdf]
• [SM] Subspace matching: Unique solution to point matching with geometric constraints, ICCV’2009 [pdf]
• [FPFH] Fast Point Feature Histograms (FPFH) for 3D Registration, ICRA’2009 [pdf] [code]
• [GO] Global optimization for alignment of generalized shapes, CVPR’2009 [pdf]
• [ISO] Isometric registration of ambiguous and partial data, CVPR’2009 [pdf]
• [GF] A new method for the registration of three-dimensional point-sets: The Gaussian fields framework, IVC’2010 [pdf]
• [RotInv] Rotation invariant non-rigid shape matching in cluttered scenes, ECCV’2010 [pdf] [code]
• [CDFHC] Group-wise point-set registration using a novel cdf-based havrda-charvát divergence, IJCV’2010 [pdf] [code]
• [QPCCP] A quadratic programming based cluster correspondence projection algorithm for fast point matching, CVIU’2010 [pdf] [code]
• [CPD] Point set registration: Coherent point drift, NIPS’2007 [pdf] TPAMI’2010 [pdf] [code]
• [PFSD] Point set registration via particle filtering and stochastic dynamics, TPAMI’2010 [pdf]
• [ECMPR] Rigid and articulated point registration with expectation conditional maximization, TPAMI’2011 [pdf] [project] [code]
• [GMMReg/TPS-L2] Robust point set registration using gaussian mixture models, NIPS’2005 TPAMI’2011 [pdf] [code]
• [TPRL] Topology preserving relaxation labeling for nonrigid point matching, TPAMI’2011 [pdf]
• [OOH] Robust point set registration using EM-ICP with information-theoretically optimal outlier handling, CVPR’2011 [pdf]
• [SGO] Stochastic global optimization for robust point set registration, CVIU’2011
• [survey] 3D Shape Registration, 3DIAA’2012
• [Multiview LM-ICP] Accurate and automatic alignment of range surfaces, 3DIMPVT’2012 [pdf] [code]
• [ISC] Intrinsic shape context descriptors for deformable shapes, CVPR’2012 [pdf]
• [RPM-Concave] Robust point matching revisited: A concave optimization approach, ECCV’2012 [pdf] [code]
• [RINPSM] Rotation Invariant Nonrigid Point Set Matching in Cluttered Scenes, TIP’2012 [pdf] [code]
• [RPM-L2E] Robust estimation of nonrigid transformation for point set registration, CVPR’2013 [pdf] [code]
• [GO-ICP] Go-ICP: Solving 3D Registration Efficiently and Globally Optimally, ICCV’2013 [pdf] TPAMI’2016 [pdf] [code]
• [Survey] Registration of 3D point clouds and meshes: a survey from rigid to nonrigid, TVCG’2013 [pdf]
• [NMM] Diffeomorphic Point Set Registration Using Non-Stationary Mixture Models, ISBI’2013 [pdf]
• [Sparse-ICP] Sparse Iterative Closest Point, ESGP’2013 [pdf] [project] [code]
• [JRMPC] A Generative Model for the Joint Registration of Multiple Point Sets, ECCV’2014 [pdf] [project] [code&data]
• [RPM-VFC] Robust Point Matching via Vector Field Consensus, TIP’2014 [pdf] [code]
• [GLTP] Non-rigid Point Set Registration with Global-Local Topology Preservation, CVPRW’2014 [pdf]
• [color-GICP] Color supported generalized-ICP, VISAPP’2014 [pdf]
• [RPM-Concave] Point Matching in the Presence of Outliers in Both Point Sets: A Concave Optimization Approach, CVPR’2014 [pdf] [code]
• [super4PCS] Super 4pcs fast global pointcloud registration via smart indexing, CGF’2014 [pdf] [code] [OpenGR]
• [SDTM] A Riemannian framework for matching point clouds represented by the Schrodinger distance transform, CVPR’2014 [pdf]
• [GLMD-TPS] A robust global and local mixture distance based non-rigid point set registration, PR’2015 [pdf] [code]
• [CSM] Non-rigid point set registration via coherent spatial mapping, SP’2015 [pdf]
• [ADR] An Adaptive Data Representation for Robust Point-Set Registration and Merging, ICCV’2015 [pdf]
• [MLMD] MLMD: Maximum likelihood mixture decoupling for fast and accurate point cloud registration, 3DV’2015 [pdf] [project]
• [APSR] Non-rigid Articulated Point Set Registration for Human Pose Estimation, WACV’2015 [pdf]
• [RegGF] Non-rigid visible and infrared face registration via regularized Gaussian fields criterion, PR’2015 [pdf] [code]
• [LLT] Robust feature matching for remote sensing image registration via locally linear transforming, TGRS’2015 [pdf] [code]
• [RPM-L2E] Robust L2E estimation of transformation for non-rigid registration, TSP’2015 [pdf] [code]
• [GLR] Robust Nonrigid Point Set Registration Using Graph-Laplacian Regularization, WACV’2015 [pdf]
• [FPPSR] Aligning the dissimilar: A probabilistic method for feature-based point set registration, ICPR’2016 [pdf]
• [IPDA] Point Clouds Registration with Probabilistic Data Association, IROS’2016 [pdf] [code]
• [CPPSR] A probabilistic framework for color-based point set registration, CVPR’2016 [pdf] [project]
• [GOGMA] GOGMA: Globally-optimal gaussian mixture alignment, CVPR’2016 [pdf]
• [GO-APM] An Efficient Globally Optimal Algorithm for Asymmetric Point Matching, TPAMI’2016 [pdf] [project] [code]
• [PR-GLS] Non-Rigid Point Set Registration by Preserving Global and Local Structures, TIP’2016 [pdf] [code]
• [conreg] Non-iterative rigid 2D/3D point-set registration using semidefinite programming, TIP’2016 [pdf]
• [PM] Probabilistic Model for Robust Affine and Non-rigid Point Set Matching, TPAMI’2016 [pdf]
• [SPSR] A Stochastic Approach to Diffeomorphic Point Set Registration With Landmark Constraints, TPAMI’2016 [pdf]
• [FRSSP] Fast Rotation Search with Stereographic Projections for 3D Registration, TPAMI’2016 [pdf]
• [VBPSM] Probabilistic Model for Robust Affine and Non-rigid Point Set Matching, TPAMI’2016 [pdf] [code]
• [MFF] Image Correspondences Matching Using Multiple Features Fusion, ECCV’2016 [pdf] [code]
• [FGR] Fast Global Registration, ECCV’2016 [pdf] [code]
• [HMRF ICP] Hidden Markov Random Field Iterative Closest Point, arxiv’2017 [pdf] [code]
• [SSFR] Global Registration of 3D LiDAR Point Clouds Basedon Scene Features: Application toStructured Environments, RS’2017 [pdf]
• [color-PCR] Colored point cloud registration revisited, ICCV’2017 [pdf]
• [dpOptTrans] Efficient Globally Optimal Point Cloud Alignment using Bayesian
  Nonparametric Mixtures, CVPR’2017 [pdf] [code]
• [GORE] Guaranteed Outlier Removal for Point Cloud Registration with Correspondences, TPAMI’2017 [pdf]
• [CSGM] A systematic approach for cross-source point cloud registration by preserving macro and micro structures, TIP’2017 [pdf]
• [FDCP] Fast descriptors and correspondence propagation for robust global point cloud registration, TIP’2017 [pdf]
• [RSWD] Multiscale Nonrigid Point Cloud Registration Using Rotation-Invariant Sliced-Wasserstein Distance via Laplace-Beltrami Eigenmap, SIAM JIS’2017 [pdf]
• [MR] Non-Rigid Point Set Registration with Robust Transformation Estimation under Manifold Regularization, AAAI’2017 [pdf] [code]
• [LPM] Locality Preserving Matching, IJCAI’2017 [pdf] IJCV’2019 [pdf] [code]
• [DARE] Density adaptive point set registration, CVPR’2018 [pdf] [code]
• [GC-RANSAC] Graph-Cut RANSAC, CVPR’2018 [pdf] [code]
• [3D-CODED] 3D-CODED: 3D correspondences by deep deformation, ECCV’2018 [pdf] [project] [code]
• [3DFeat-NET] 3dfeat-net: Weakly supervised local 3d features for point cloud registration, ECCV’2018 [pdf] [code]
• [MVDesc-RMBP] Learning and Matching Multi-View Descriptors for Registration of Point Clouds, ECCV’2018 [pdf]
• [SWS] Nonrigid Points Alignment with Soft-weighted Selection, IJCAI’2018 [pdf]
• [DLD] Dependent landmark drift: robust point set registration with aGaussian mixture model and a statistical shape model, arxiv’2018 [pdf] [code]
• [DeepMapping] DeepMapping: Unsupervised Map Estimation From Multiple Point Clouds, arxiv’2018 [pdf] [project]
• [APSR] Adversarial point set registration, arxiv’2018 [pdf]
• [3DIV] Fast and Globally Optimal Rigid Registration of 3D Point Sets by Transformation Decomposition, arxiv’2018 [pdf]
• [Analysis] Analysis of Robust Functions for Registration Algorithms, arxiv’2018 [pdf]
• [MVCNN] Learning Local Shape Descriptors from Part Correspondences with Multiview Convolutional Networks, TOG’2018 [pdf] [project]
• [CSCIF] Cubature Split Covariance Intersection Filter-Based Point Set Registration, TIP’2018 [pdf]
• [FPR] Efficient Registration of High-Resolution Feature Enhanced Point Clouds, TPAMI’2018 [pdf]
• [DFMM-GLSP] Non-rigid point set registration using dual-feature finite mixture model and global-local structural preservation, PR’2018 [pdf]
• [PR-Net] Non-Rigid Point Set Registration Networks, arxiv’2019 [pdf] [code]
• [SDRSAC] SDRSAC: Semidefinite-Based Randomized Approach for Robust Point Cloud Registration without Correspondences, arxiv’2019 [pdf] [code]
• [3DRegNet] 3DRegNet: A Deep Neural Network for 3D Point Registration, arxiv’2019 [pdf]
• [PointNetLK] PointNetLK: Robust & Efficient Point Cloud Registration using PointNet, arxiv’2019 [pdf] [code]
• [RPM-MR] Nonrigid Point Set Registration with Robust Transformation Learning under Manifold Regularization, TNNLS’2019 [pdf] [code]
• [FGMM] Feature-guided Gaussian mixture model for image matching, PR’2019 [pdf]
• [LSR-CFP] Least-squares registration of point sets over SE (d) using closed-form projections, CVIU’2019 [pdf]
• [FilterReg] FilterReg: Robust and Efficient Probabilistic Point-Set Registration using Gaussian Filter and Twist Parameterization, CVPR’2019 [pdf] [project] [code]
• [TEASER] A Polynomial-time Solution for Robust Registration with Extreme Outlier Rates, arxiv’2019 [pdf]

Mismatch Removal Methods

• [RANSAC] Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography, 1981 [pdf] [wiki]
• [MLESAC] MLESAC: A new robust estimator with application to estimating image geometry, CVIU’2000 [pdf] [code_pcl]
• [PROSAC] Matching with PROSAC-progressive sample consensus, CVPR’2005 [pdf] [code_pcl]
• [ICF/SVR] Rejecting mismatches by correspondence function, IJCV’2010 [pdf]
• [GS] Common visual pattern discovery via spatially coherent correspondences, CVPR’2010 [pdf] [code]
• [VFC] A robust method for vector field learning with application to mismatch removing, CVPR’2011 [pdf] [code]
• [DefRANSAC] In defence of RANSAC for outlier rejection in deformable registration, ECCV’2012 [pdf] [code]
• [CM] Robust Non-parametric Data Fitting for Correspondence Modeling, ICCV’2013 [pdf] [code]
• [AGMM] Asymmetrical Gauss Mixture Models for Point Sets Matching, CVPR’2014 [pdf]
• [TC] Epipolar geometry estimation for wide baseline stereo by Clustering Pairing Consensus, PRL’2014 [pdf]
• [BF] Bilateral Functions for Global Motion Modeling, ECCV’2014 [pdf] [project] [code]
• [WxBS] WxBS: Wide Baseline Stereo Generalizations, BMVC’2015 [pdf] [project]
• [RepMatch] RepMatch: Robust Feature Matching and Posefor Reconstructing Modern Cities, ECCV’2016 [pdf] [project] [code]
• [SIM] The shape interaction matrix-based affine invariant mismatch removal for partial-duplicate image search, TIP’2017 [pdf] [code]
• [DSAC] DSAC: differentiable RANSAC for camera localization, CVPR’2017 [pdf] [code]
• [GMS] GMS: Grid-based Motion Statistics for Fast, Ultra-robust Feature Correspondence, CVPR’2017 [pdf] [code]
• [LMI] Consensus Maximization with Linear Matrix Inequality Constraints, CVPR’2017 [pdf] [project] [code]
• [LFGC] Learning to Find Good Correspondences, CVPR’2018 [pdf] [code]
• [GC-RANSAC] Graph-Cut RANSAC, CVPR’2018 [pdf] [code]
• [SRC] Consensus Maximization for Semantic Region Correspondences, CVPR’2018 [pdf] [code]
• [CODE] Code: Coherence based decision boundaries for feature correspondence, TPAMI’2018 [pdf] [project]
• [LPM] Locality preserving matching, IJCV’2019 [pdf] [code]
• [LMR] LMR: Learning A Two-class Classifier for Mismatch Removal, TIP’2019 [pdf] [code]
• [PFFM] Progressive Filtering for Feature Matching, ICASSP’2019 [pdf]
• [NM-Net] NM-Net: Mining Reliable Neighbors for Robust Feature Correspondences, arXiv’2019 [pdf]

Graph Matching Methods

• [SM] A spectral technique for correspondence problems using pairwise constraints, ICCV’2005 [pdf] [code]
• [SM-MAP] Efficient MAP approximation for dense energy functions, ICML’2006 [pdf] [code]
• [SMAC] Balanced Graph Matching, NIPS’2006 [pdf] [code]
• [FCGM] Feature correspondence via graph matching: Models and global optimization, ECCV’2008 [pdf]
• [PM] Probabilistic Graph and Hypergraph Matching, CVPR’2008 [pdf]
• [IPFP] An Integer Projected Fixed Point Method for Graph Matching and MAP Inference, NIPS’2009 [pdf] [code]
• [RRWM] Reweighted Random Walks for Graph Matching, ECCV’2010 [pdf]
• [FGM] Factorized graph matching, CVPR’2012 [pdf] [code]
• [DGM] Deformable Graph Matching, CVPR’2013 [pdf] [code]
• [MS] Progressive mode-seeking on graphs for sparse feature matching, ECCV’2014 [pdf] [code]

Misc

• [RootSIFT] Three things everyone should know to improve object retrieval, CVPR’2012 [pdf] [related code]
• [DM-CNN] Descriptor Matching with Convolutional Neural Networks: a Comparison to SIFT, arXiv’2014 [pdf]
• [DASC] DASC: Robust Dense Descriptor for Multi-modal and Multi-spectral Correspondence Estimation, TPAMI’2017 [pdf] [project]
• [MODS] MODS: Fast and Robust Method for Two-View Matching, CVIU’2015 [pdf] [project] [code]
• [Elastic2D3D] Efficient Globally Optimal 2D-to-3D Deformable Shape Matching, CVPR’2016 [pdf] [project]
• [TCDCN] Facial Landmark Detection by Deep Multi-task Learning, ECCV’2014 [pdf] [project]
• [LAI] Object matching using a locally affine invariant and linear programming techniques, TPAMI’2013 [pdf]
• [GeoDesc] GeoDesc: Learning Local Descriptors by Integrating Geometry Constraints, ECCV’2018 [pdf] [code]

Deep Features

• [TFeat] Learning local feature descriptors with triplets and shallow convolutional neural networks, BMVC’2016 [pdf] [code]
• [L2-Net] L2-Net: Deep Learning of Discriminative Patch Descriptor in Euclidean Space, CVPR’2017 [pdf] [code]
• [HardNet] Working hard to know your neighbor’s margins: Local descriptor learning loss, CVPR’2018 [pdf] [code]
• [AffNet] Repeatability Is Not Enough: Learning Discriminative Affine Regions via Discriminability, ECCV’2018 [pdf] [code]

Applications

Remote Sensing Image Registration

• [GLPM] Guided Locality Preserving Feature Matching for Remote Sensing Image Registration, TGRS’2018 [pdf]

Retinal Image Registration

• [DB-ICP] The dual-bootstrap iterative closest point algorithm with application to retinal image registration, TMI’2003 [pdf]
• [GDB-ICP] Registration of Challenging Image Pairs: Initialization, Estimation, and Decision, TPAMI’2007 [pdf] [project]
• [ED-DB-ICP] The edge-driven dual-bootstrap iterative closest point algorithm for registration of multimodal fluorescein angiogram sequence, TMI’2010 [pdf]

Palmprint Image Registration

• Robust and efficient ridge-based palmprint matching, TPAMI’2012 [pdf]
• Palmprint image registration using convolutional neural networks and Hough transform, arxiv’2019 [pdf]

Visual Homing Navigation

• Visual Homing via Guided Locality Preserving Matching, ICRA’2018 [pdf]

HDR Imaging

• Locally non-rigid registration for mobile HDR photography, CVPRW’2015 [pdf]

Misc

• Hand Motion from 3D Point Trajectories and a Smooth Surface Model, ECCV’2004 [pdf] [project]
• A robust hybrid method for nonrigid image registration, PR’2011 [pdf]
• Aligning Images in the Wild, CVPR’2012 [pdf] [code]
• Robust feature set matching for partial face recognition, CVPR’2013 [pdf]
• Multi-modal and Multi-spectral Registrationfor Natural Images, ECCV’2014 [pdf] [project]
• Articulated and Generalized Gaussian KernelCorrelation for Human Pose Estimation, TIP’2016 [pdf]
• Infrared and visible image fusion via gradient transfer and total variation minimization, Information Fusion’2016 [pdf] [code]

Databases

General databases

• 2D Synthesized Chui-Rangarajan Dataset (deformation, noise, and outliers)
• TOSCA
• Multi-View Stereo Dataset
• Multi-View Stereo for Community Photo Collections
• Multi-View Stereo
• VGG Affine Datasets
• Multi-view VGG’s Dataset
• Oxford Building Reconstruction
• IMM Datasets
• MPEG7 CE Shape-1 Part B
• Leaf Shapes Database
• CMU House/Hotel Sequence Images
• Generated Matching Dataset
• Image Sequences
• Mythological creatures 2D
• Tools 2D
• Human face
• Cars & Motorbikes
• DIML Multimodal Benchmark
• Street View Dataset [github] [data]
• EVD: Extreme View Dataset [EVD_tentatives] [EZD]
• WxBS: Wide Baseline Dataset [W1BS]
• Stanford 3D Scanning
• MPI FAUST Dataset
• Mikolajczyk Database
• Panoramic Image Database
• PS-Dataset (A Large Dataset for improving Patch Matching, PhotoSynth Dataset for improving local patch Descriptors)
• Two-view Geometry
• Point clouds data sets for 3D registration
• IMW CVPR 2019: Challenge
• Robotic 3D Scan Repository
• Database for 3D surface registration

Other databases

• FIRE: Fundus Image Registration Dataset
• DRIVE (Retinal Images)
• DRIONS-DB (Retinal Images)
• STARE (Retinal Images)
• Plant Images
• MR Prostate Images
• CV Images
• ETHZ Datasets
• CVPapers
• YACVID
• Intelligent remote sensing data analyis

Tools

• VLFeat
• PCL: Point Cloud Library
• Pointmatcher: a modular library implementing the ICP algorithm for aligning point clouds
• Open3D: A Modern Library for 3D Data Processing
• 3D keypoints (MeshDOG) and local descriptors (MeshHOG)
• COLMAP
• OpenMVG: open Multiple View Geometry
• VisualSFM : A Visual Structure from Motion System
• Medical Image Registration Toolbox
• Graph Matching Toolbox in MATLAB
• FAIR: Flexible Algorithms for Image Registration [pdf]
• Range Image Registration Toolbox

title: FLANN
date: 2019/01/08 12:05:51
tags: Debug
categories: Debug

Introduction

FLANN - Fast Library for Approximate Nearest Neighbors

FLANN is a library for performing fast approximate nearest neighbor searches in high dimensional spaces. It contains a collection of algorithms we found to work best for nearest neighbor search and a system for automatically choosing the best algorithm and optimum parameters depending on the dataset.

FLANN is written in C++ and contains bindings for the following languages: C, MATLAB and Python.

Download

Install

$ cd flann-1.8.4-src 
$ mkdir build
$ cd build
$ cmake ..
$ make

Errors:

flann-1.8.4-src/src/cpp/flann/algorithms/kdtree_index.h:667:36: error: call to 'abs' is ambiguous
ElementType span = abs(point[i]-leaf_point[i]);
                                   ^~~

Debug:

kdtree_index.h: insert #include <cmath>

https://stackoverflow.com/questions/1374037/ambiguous-overload-call-to-absdouble

Unit test

make test   

Scanning dependencies of target dataset_sift10K.h5
Downloading from http://people.cs.ubc.ca/~mariusm/uploads/FLANN/datasets/sift10K.h5 to /Users/gwang/Documents/Kuaipan/Projects/REG/flann-1.8.4-src/test/sift10K.h5... done
[  0%] Built target dataset_sift10K.h5
Scanning dependencies of target dataset_sift100K_byte.h5
Downloading from http://people.cs.ubc.ca/~mariusm/uploads/FLANN/datasets/sift100K_byte.h5 to /Users/gwang/Documents/Kuaipan/Projects/REG/flann-1.8.4-src/test/sift100K_byte.h5... done
[  0%] Built target dataset_sift100K_byte.h5
Scanning dependencies of target dataset_cloud.h5
Downloading from http://people.cs.ubc.ca/~mariusm/uploads/FLANN/datasets/cloud.h5 to /Users/gwang/Documents/Kuaipan/Projects/REG/flann-1.8.4-src/test/cloud.h5... done
[  0%] Built target dataset_cloud.h5
Scanning dependencies of target dataset_brief100K.h5
Downloading from http://people.cs.ubc.ca/~mariusm/uploads/FLANN/datasets/brief100K.h5 to /Users/gwang/Documents/Kuaipan/Projects/REG/flann-1.8.4-src/test/brief100K.h5... done
[  0%] Built target dataset_brief100K.h5
Scanning dependencies of target dataset_sift100K.h5
Downloading from http://people.cs.ubc.ca/~mariusm/uploads/FLANN/datasets/sift100K.h5 to /Users/gwang/Documents/Kuaipan/Projects/REG/flann-1.8.4-src/test/sift100K.h5... done
[  0%] Built target dataset_sift100K.h5
Scanning dependencies of target dataset_sift10K_byte.h5
Downloading from http://people.cs.ubc.ca/~mariusm/uploads/FLANN/datasets/sift10K_byte.h5 to /Users/gwang/Documents/Kuaipan/Projects/REG/flann-1.8.4-src/test/sift10K_byte.h5... done
[  0%] Built target dataset_sift10K_byte.h5
Scanning dependencies of target flann_gtests
[  0%] Built target flann_gtests
Scanning dependencies of target tests
[  0%] Built target tests
[ 28%] Built target flann
Scanning dependencies of target pyunit_test_index_save.py
[ 42%] Running pyunit test(s) test_index_save.py
..............
----------------------------------------------------------------------
Ran 14 tests in 2.263s

OK
[ 42%] Built target pyunit_test_index_save.py
Scanning dependencies of target pyunit_test_nn_index.py
[ 57%] Running pyunit test(s) test_nn_index.py
....
----------------------------------------------------------------------
Ran 4 tests in 2.503s

OK
[ 57%] Built target pyunit_test_nn_index.py
Scanning dependencies of target pyunit_test_clustering.py
[ 71%] Running pyunit test(s) test_clustering.py
.........
----------------------------------------------------------------------
Ran 9 tests in 0.187s

OK
[ 71%] Built target pyunit_test_clustering.py
Scanning dependencies of target flann_gtest
[ 71%] Built target flann_gtest
Scanning dependencies of target pyunit_test_nn_autotune.py
[ 85%] Running pyunit test(s) test_nn_autotune.py
----------------------------------------------------------------------
Ran 5 tests in 424.571s

OK
[ 85%] Built target pyunit_test_nn_autotune.py
Scanning dependencies of target pyunit_test_nn.py
[100%] Running pyunit test(s) test_nn.py
...................
----------------------------------------------------------------------
Ran 19 tests in 3.686s

OK
[100%] Built target pyunit_test_nn.py
Scanning dependencies of target test
[100%] Built target test

Mac下使用点云工具包PCL，按照官方说明可采用homebrew安装：

brew install pcl

但是会在vtk这个依赖工具上存在一些问题。

先安装好vtk (8.1.2)，可采用homebrew：

brew install vtk

采用source源码编译安装：

git clone git@github.com:PointCloudLibrary/pcl.git
source pcl
mkdir build && cd build 
cmake ..
make   ## 好慢。。
sudo make install

PS: pcl官方还给出采用MacPorts的安装方式，测试之后发现有一些问题没配置成。。。所以卸载删除Macports，按照以上方式调通。

测试：
iterative_closest_point.cpp

#include <iostream>
#include <pcl/io/pcd_io.h>
#include <pcl/point_types.h>
#include <pcl/registration/icp.h>

int main (int argc, char** argv)
{
    pcl::PointCloud<pcl::PointXYZ>::Ptr cloud_in (new pcl::PointCloud<pcl::PointXYZ>);
    pcl::PointCloud<pcl::PointXYZ>::Ptr cloud_out (new pcl::PointCloud<pcl::PointXYZ>);

    // Fill in the CloudIn data
    cloud_in->width    = 5;
    cloud_in->height   = 1;
    cloud_in->is_dense = false;
    cloud_in->points.resize (cloud_in->width * cloud_in->height);
    for (size_t i = 0; i < cloud_in->points.size (); ++i)
    {
        cloud_in->points[i].x = 1024 * rand () / (RAND_MAX + 1.0f);
        cloud_in->points[i].y = 1024 * rand () / (RAND_MAX + 1.0f);
        cloud_in->points[i].z = 1024 * rand () / (RAND_MAX + 1.0f);
    }
    std::cout << "Saved " << cloud_in->points.size () << " data points to input:"
    << std::endl;
    for (size_t i = 0; i < cloud_in->points.size (); ++i) std::cout << "    " <<
        cloud_in->points[i].x << " " << cloud_in->points[i].y << " " <<
        cloud_in->points[i].z << std::endl;
    *cloud_out = *cloud_in;
    std::cout << "size:" << cloud_out->points.size() << std::endl;
    for (size_t i = 0; i < cloud_in->points.size (); ++i)
        cloud_out->points[i].x = cloud_in->points[i].x + 0.7f;
    std::cout << "Transformed " << cloud_in->points.size () << " data points:"
    << std::endl;
    for (size_t i = 0; i < cloud_out->points.size (); ++i)
        std::cout << "    " << cloud_out->points[i].x << " " <<
        cloud_out->points[i].y << " " << cloud_out->points[i].z << std::endl;
    pcl::IterativeClosestPoint<pcl::PointXYZ, pcl::PointXYZ> icp;
    icp.setInputSource(cloud_in);
    icp.setInputTarget(cloud_out);
    pcl::PointCloud<pcl::PointXYZ> Final;
    icp.align(Final);
    std::cout << "has converged:" << icp.hasConverged() << " score: " <<
    icp.getFitnessScore() << std::endl;
    std::cout << icp.getFinalTransformation() << std::endl;

    return (0);
}

CMakeLists.txt:

cmake_minimum_required(VERSION 3.1 FATAL_ERROR)

project(iterative_closest_point)

find_package(PCL 1.9 REQUIRED)

include_directories(${PCL_INCLUDE_DIRS})
link_directories(${PCL_LIBRARY_DIRS})
add_definitions(${PCL_DEFINITIONS})

add_executable (iterative_closest_point iterative_closest_point.cpp)
target_link_libraries (iterative_closest_point ${PCL_LIBRARIES})

test：

cd demo
mkdir build && cd build
cmake ..
make

PS：可以根据自己project的需要来调用pcl相应的函数，特别是其中的3DSC以及FPFH。 可参考源码中example/features/下的几个demo。

亦可采用fast global registration提供的方法，将features写入到文件：

// Assume a point cloud with normal is given as
// pcl::PointCloud<pcl::PointNormal>::Ptr object

pcl::FPFHEstimationOMP<pcl::PointNormal, pcl::PointNormal, pcl::FPFHSignature33> fest;
pcl::PointCloud<pcl::FPFHSignature33>::Ptr object_features(new pcl::PointCloud<pcl::FPFHSignature33>());
fest.setRadiusSearch(feature_radius_);  
fest.setInputCloud(object);
fest.setInputNormals(object);
fest.compute(*object_features);

FILE* fid = fopen("features.bin", "wb");
int nV = object->size(), nDim = 33;
fwrite(&nV, sizeof(int), 1, fid);
fwrite(&nDim, sizeof(int), 1, fid);
for (int v = 0; v < nV; v++) {
    const pcl::PointNormal &pt = object->points[v];
    float xyz[3] = {pt.x, pt.y, pt.z};
    fwrite(xyz, sizeof(float), 3, fid);
    const pcl::FPFHSignature33 &feature = object_features->points[v];
    fwrite(feature.histogram, sizeof(float), 33, fid);
}
fclose(fid);

Mac下使用Matlab2017/2018，执行mex -setup后提示警告：

Warning: 
Xcode is installed, but its license has not been accepted. 
Run Xcode and accept its license agreement

首先Max中已经安装好了commend-line-tools

然后在Matlab安装目录文件内修改：

打开文件
/Applications/MATLAB_R2017a.app/bin/maci64/mexopts/clang++_maci64.xml
和 /Applications/MATLAB_R2017a.app/bin/maci64/mexopts/clang_maci64.xml

搜索Xcode is installed, but its license has not been accepted. Run Xcode and accept its license agreement, 定位到warning提示行，在代码行<cmdReturns name="defaults read com.apple.dt.Xcode IDEXcodeVersionForAgreedToGMLicense"/>之前添加如下代码：

<cmdReturns name="echo 4.3.3"/> 

solved!

Login Travis with your Github account.

Choose the repository you want to build via Travis.

Setting your build:

Prepare a .travis.yml, such as a Flask project:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16

language: python
sudo: enabled
services:
  - mongodb
python:
  - 2.7
before_install:
  - pip install -r requirements.txt
  - pip install codecov
script:
  - nosetests app.py
after_success:
  - codecov
branches:
  only:
  - master

Due to the Flask project need to run a server in your computer, so we cannot use the script python app.py, nosetests app.py can be used to avoid this issue.

Result: build passing

Then we can get a blingbling ico:

Or get failed!

PS: requirements.txt is need to add.

Macos without NVIDIA GPU, we need to use the CPU-only pytorch.

Install pytorch

1

conda install pytorch torchvision -c pytorch

1

pip install torch torchvision

Got RuntimeError:

1
2
3
4
5

RuntimeError: Attempting to deserialize object on a 
CUDA device but torch.cuda.is_available() is False. If 
you are running on a CPU-only machine, please use 
torch.load with map_location='cpu' to map your storages 
to the CPU.

modify your python code

1
2
3
4
5

torch.load(model_weights)
||
||
vv
torch.load(model_weights, map_location=torch.device('cpu'))

在很多项目页中看到的小图标，可以通过工具在线制作，更方便显示当前项目所采用的软件及版本以及其他相关信息：

shields.io

Pixel-perfect Retina-ready Fast Consistent Hackable No tracking

可在线通过自定义来制作盾牌小图标，示例如图：

1.目的

输入图像，输出其grid网格以及变换后的图像及网格

2.配置

环境：

Win10 x64
VS2015 (vc14)
OpenCV2.4.13 
#设置PATH环境变量C:/opencv/bin/x86/vc14/bin

3.VS2015配置OpenCV

新建一个win32控制台空白项目；

配置：项目->属性->VC++目录->包含目录

C:/opencv/build/include/opencv2
C:/opencv/build/include/opencv
C:/opencv/build/include

配置：项目->属性->VC++目录->库目录

C:/opencv/build/x86/vc14/lib

配置：项目->属性->链接器->输入->附加依赖项

#for debug
opencv_ml2413d.lib
opencv_calib3d2413d.lib
opencv_contrib2413d.lib
opencv_core2413d.lib
opencv_features2d2413d.lib
opencv_flann2413d.lib
opencv_gpu2413d.lib
opencv_highgui2413d.lib
opencv_imgproc2413d.lib
opencv_legacy2413d.lib
opencv_objdetect2413d.lib
opencv_ts2413d.lib
opencv_video2413d.lib
opencv_nonfree2413d.lib
opencv_ocl2413d.lib
opencv_photo2413d.lib
opencv_stitching2413d.lib
opencv_superres2413d.lib
opencv_videostab2413d.lib

4.编译运行

$ $
Figure 1. test
$ $

$ $
Figure 2. def

1.说明

基于UNet的眼底图像血管分割方法：Retina-UNet，项目地址：https://github.com/orobix/retina-unet

2.配置运行环境及依赖：

Ubuntu-16 + CUDA-9.0 + cudnn-7.0 + GPUDriver-390.67 + OpenCV-3.2

Python 2.7
TensorFlow 1.8.0
TensorFlow-GPU 1.8.0 #不安装，则直接用CPU训练
Keras 2.2.0 #原release使用的1.1.0版本keras，若使用keras2.x，则需要修改部分代码
PIL (pillow 3.1.2)
h5py 2.8.0
ConfigParser 3.5.0
Numpy 1.14.5
scikit-learn 0.19.1
pydot 1.2.4

$ pip install xxx --user #安装上面这些依赖项

GraphViz

$ sudo apt-get install graphviz libgraphviz-dev

3.数据集

训练用的是视网膜血管分割经典的数据集DRIVE：http://www.isi.uu.nl/Research/Databases/DRIVE/，其中包含手工分割的ground-truth和mask图像。

下载解压后放于项目根目录下：

DRIVE
│
└───test
|    ├───1st_manual
|    └───2nd_manual
|    └───images
|    └───mask
│
└───training
    ├───1st_manual
    └───images
    └───mask

4.训练

首先整理数据：

python prepare_datasets_DRIVE.py

执行训练：

配置文件可修改configuration.txt

pytohn run_train.py

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41

1. Create directory for the results (if not already existing)
Dir already existing
copy the configuration file in the results folder
2. Run the training on GPU (no nohup)
Using TensorFlow backend.
train images/masks shape:
(20, 1, 565, 565)
train images range (min-max): 0.0 - 1.0
train masks are within 0-1
patches per full image: 9500
train PATCHES images/masks shape:
(190000, 1, 48, 48)
train PATCHES images range (min-max): 0.0 - 1.0
2018-06-30 00:43:12.163406: I tensorflow/core/platform/cpu_feature_guard.cc:140] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
2018-06-30 00:43:12.584228: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:898] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2018-06-30 00:43:12.585652: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1356] Found device 0 with properties: 
name: GeForce GTX TITAN X major: 5 minor: 2 memoryClockRate(GHz): 1.076
pciBusID: 0000:01:00.0
totalMemory: 11.92GiB freeMemory: 11.57GiB
2018-06-30 00:43:12.585718: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1435] Adding visible gpu devices: 0
2018-06-30 00:43:20.161626: I tensorflow/core/common_runtime/gpu/gpu_device.cc:923] Device interconnect StreamExecutor with strength 1 edge matrix:
2018-06-30 00:43:20.161703: I tensorflow/core/common_runtime/gpu/gpu_device.cc:929]      0 
2018-06-30 00:43:20.161729: I tensorflow/core/common_runtime/gpu/gpu_device.cc:942] 0:   N 
2018-06-30 00:43:20.174512: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1053] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 11206 MB memory) -> physical GPU (device: 0, name: GeForce GTX TITAN X, pci bus id: 0000:01:00.0, compute capability: 5.2)
Check: final output of the network:
(None, 2304, 2)
Train on 171000 samples, validate on 19000 samples
Epoch 1/150
 - 204s - loss: 0.2764 - acc: 0.9045 - val_loss: 0.2325 - val_acc: 0.9240
Epoch 00001: val_loss improved from inf to 0.23253, saving model to ./train/train_best_weights.h5
Epoch 2/150
 - 199s - loss: 0.1679 - acc: 0.9380 - val_loss: 0.1698 - val_acc: 0.9420
Epoch 00002: val_loss improved from 0.23253 to 0.16979, saving model to ./train/train_best_weights.h5
Epoch 3/150

5.测试

6.Debug：

1) Keras2版本不兼容的问题：

from keras.utils.visualize_util import plot
ImportError: No module named 'keras.utils.visualize_util'

From visualize_util to vis_utils;
The plot function was also renamed to plot_model;
The nb_epoch argument in fit has been renamed epochs;
Update your Model call to the Keras 2 API: Model(outputs=Tensor("ac..., inputs=Tensor("in...)
model = Model(input=inputs, output=conv7);
更新代码或者选用合适的Keras版本。

2) ImportError: No module named tensorflow
使用pip install tensorflow安装失败且出现importing error，解决方法是在命令后加上--user。

3) ImportError: Failed to import pydot. Please install pydot. For example with pip install pydot.

4) pydot failed to call GraphViz. Please install GraphViz https://www.graphviz.org/ and ensure that its executables are in the $PATH. 在~/.bashrc中添加了PATH后错误仍然出现，不知道问题在哪。。。暂时注释掉retinaNN_training.py中的#plot_model(model, to_file='./'+name_experiment+'/'+name_experiment + '_model.png') #check how the model looks like。

查看Ubuntu下OpenCV版本号：

pkg-config  --modversion  opencv

输出：

3.2.0

1.查看CUDA版本

cat  /usr/local/cuda/version.txt

例如输出：

CUDA Version 8.0.61

or:

nvcc --version

输出：

Copyright (c) 2005-2017 NVIDIA Corporation
Built on Fri_Nov__3_21:07:56_CDT_2017
Cuda compilation tools, release 9.1, V9.1.85

2.查看cudnn版本

cat /usr/local/cuda/include/cudnn.h | grep CUDNN_MAJOR -A 2

例如输出：

#define CUDNN_MAJOR      6
#define CUDNN_MINOR      0
#define CUDNN_PATCHLEVEL 21
--
#define CUDNN_VERSION    (CUDNN_MAJOR * 1000 + CUDNN_MINOR * 100 + CUDNN_PATCHLEVEL)

#include "driver_types.h"

0.说明

LIFT: Learned Invariant Feature Points
作者开源代码：https://github.com/cvlab-epfl/LIFT

PS: 注意下载作者release的1.0版，修复了一些bug:.locks directory is now included in the repository by default。

1.安装最新OpenCV 3.x

略

2.安装Theano并配置

pip install theano==0.9.0

配置：(开源代码中三个主要部分需要GPU的支持)

sudo vim ~/.theanorc

[global]  
device = gpu0  
floatX = float32 

[nvcc]  
fastmath = True  

[cuda]    
root=/usr/local/cuda-8.0  

3.安装其他依赖项

安装flufl.lock:
1) 下载安装包
https://pypi.python.org/pypi/flufl.lock/2.4.1

tar -zxf flufl.lock-2.4.1.tar.gz
sudo python setup.py install  

2) 或

pip install flufl.lock==2.4.1

安装Lasagne:
下载安装包：https://github.com/Lasagne/Lasagne【(0.2.dev1)】

tar -zxf Lasagne.tar.gz
sudo python setup.py install  

其他项：

pip install numpy parse h5py scipy

4.测试demo

# Build the shared library by
cd c-code/build
cmake -D CUDA_TOOLKIT_ROOT_DIR=/usr/local/cuda-8.0 ..
make

#To run the test program simply
./run.sh

5.其他问题

cmake报错：

Could NOT find CUDA: Found unsuitable version "9.1", but required is exact

version “8.0” (found /usr/local/cuda-9.1)

#需指定cuda版本
cmake -D CUDA_TOOLKIT_ROOT_DIR=/usr/local/cuda-8.0 ..

已解决的一些issues： https://github.com/cvlab-epfl/LIFT/issues?q=is%3Aissue+is%3Aclosed

1. 说明

   配置 aria2，结合 Chrome，代替baidu网盘App下载文件。

2. 安装aria2

   1 2 3 4 5 6 7 8 9 10

   brew install aria2 ==> Downloading https://homebrew.bintray.com/bottles/aria2-1.34.0.high_sierra.bottle.tar.gz ######################################################################## 100.0% ==> Pouring aria2-1.34.0.high_sierra.bottle.tar.gz ==> Caveats Bash completion has been installed to: /usr/local/etc/bash_completion.d ==> Summary 🍺 /usr/local/Cellar/aria2/1.34.0: 22 files, 4.0MB

3. 配置aria2

   RPC模式配置：

   1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52

   cd ~ mkdir .aria2 cd .aria2 touch aria2.conf vim aria2.conf #用户名 #rpc-user=user #密码 #rpc-passwd=passwd #上面的认证方式不建议使用,建议使用下面的token方式 #设置加密的密钥 #rpc-secret=token #允许rpc enable-rpc=true #允许所有来源, web界面跨域权限需要 rpc-allow-origin-all=true #允许外部访问，false的话只监听本地端口 rpc-listen-all=true #RPC端口, 仅当默认端口被占用时修改 #rpc-listen-port=6800 #最大同时下载数(任务数), 路由建议值: 3 max-concurrent-downloads=5 #断点续传 continue=true #同服务器连接数 max-connection-per-server=5 #最小文件分片大小, 下载线程数上限取决于能分出多少片, 对于小文件重要 min-split-size=10M #单文件最大线程数, 路由建议值: 5 split=10 #下载速度限制 max-overall-download-limit=0 #单文件速度限制 max-download-limit=0 #上传速度限制 max-overall-upload-limit=0 #单文件速度限制 max-upload-limit=0 #断开速度过慢的连接 #lowest-speed-limit=0 #验证用，需要1.16.1之后的release版本 #referer=* #文件保存路径, 默认为当前启动位置 dir=/Users/xxx/Downloads #文件缓存, 使用内置的文件缓存, 如果你不相信Linux内核文件缓存和磁盘内置缓存时使用, 需要1.16及以上版本 #disk-cache=0 #另一种Linux文件缓存方式, 使用前确保您使用的内核支持此选项, 需要1.15及以上版本(?) #enable-mmap=true #文件预分配, 能有效降低文件碎片, 提高磁盘性能. 缺点是预分配时间较长 #所需时间 none < falloc ? trunc « prealloc, falloc和trunc需要文件系统和内核支持 file-allocation=prealloc

   默认下载路径的「/Users/xxx/Downloads」可以改。xxx为Mac用户名，输入:w保存。

4. 启动RPC模式

   在终端输入aria2c --conf-path="/Users/xx/.aria2/aria2.conf" -D，aria2 启动。

5. 配置chrome

   下载 Chrome 插件BaiduExporter.crx

   将其拖拽至Chrome扩展程序处安装，在百度网盘页面会新增一个功能按钮来使用aria下载。

6. 配置GUI

   下载aria2gui，解压后将其拷贝到应用程序中，打开会看到当前下载的文件列表。

labelImg官方地址:https://github.com/tzutalin/labelImg/blob/master/README.rst

macOS
Python 2 + Qt4

1
2
3
4
5

brew install qt qt4
brew install libxml2
make qt4py2
python labelImg.py
python  labelImg.py [IMAGE_PATH] [PRE-DEFINED CLASS FILE]

Python 3 + Qt5 (Works on macOS High Sierra)

1
2
3
4
5

brew install qt  # will install qt-5.x.x
brew install libxml2
make qt5py3
python labelImg.py
python  labelImg.py [IMAGE_PATH] [PRE-DEFINED CLASS FILE]

实际在使用python2的时候，无法用brew install qt qt4，且在python3下安装libxml2后报错：

1

ImportError: No module named lxml

所以在macos high sierra下安装python3，然后

1
2
3
4

brew install qt
pip3 install lxml
make qt5py3
python3 labelImg.py

1. 准备

需要安装OpenCV3.x版本，以及OpenBLAS，参考：Mac安装OpenCV, MacOS安装caffe

确认安装brew Cython easydict protobuf

下载py-faster-rcnn源码：

git clone –recursive https://github.com/rbgirshick/py-faster-rcnn.git

2. 编译caffe

修改makefile.config

# USE_CUDNN := 1
CPU_ONLY := 1
USE_OPENCV := 1
OPENCV_VERSION := 3
CUDA_DIR := /usr/local/cuda
BLAS := open
BLAS_INCLUDE := /usr/local/Cellar/openblas/0.2.20/include
BLAS_LIB := /usr/local/Cellar/openblas/0.2.20/lib
PYTHON_INCLUDE := /usr/include/python2.7 \
        /usr/local/lib/python2.7/site-packages/numpy/core/include    PYTHON_LIB := /usr/lib
WITH_PYTHON_LAYER := 1
INCLUDE_DIRS := $(PYTHON_INCLUDE) /usr/local/include  /usr/local/Cellar/hdf5 /usr/local/Cellar/opencv/3.2.0/include
LIBRARY_DIRS := $(PYTHON_LIB) /usr/local/lib /usr/lib  /usr/local/Cellar/opencv/3.2.0/lib        
BUILD_DIR := build
DISTRIBUTE_DIR := distribute
TEST_GPUID := 0

执行编译：

make -j32 && make pycaffe

3. 编译faster-rcnn

修改文件，替换或删除关于gpu的代码行：

修改./lib/setup.py

...
#CUDA = locate_cuda()
...
...
#self.set_executable('compiler_so', CUDA['nvcc'])
...
...
#Extension('nms.gpu_nms',
#[‘nms/nms_kernel.cu', 'nms/gpu_nms.pyx'],
#library_dirs=[CUDA['lib64']],
#libraries=['cudart'],
#language='c++',
#runtime_library_dirs=[CUDA['lib64']],
## this syntax is specific to this build system
## we're only going to use certain compiler args with nvcc and not with
## gcc the implementation of this trick is in customize_compiler() below
#extra_compile_args={'gcc': ["-Wno-unused-function"],
#’nvcc': ['-arch=sm_35',
#’—ptxas-options=-v',
#’-c’,
#’—compiler-options',
#”’-fPIC'"]},
#include_dirs = [numpy_include, CUDA['include']]
#)

执行编译：

cd ./lib
make

4. 运行demo

运行demo前需要修改几个文件，替换或删除关于gpu的代码行：

(1) 修改~/py-faster-rcnn/lib/fast_rcnn/config.py

# Use GPU implementation of non-maximum suppression
# __C.USE_GPU_NMS = True
__C.USE_GPU_NMS = False

(2) 修改~/py-faster-rcnn/tools/test_net.py和 ~/py-faster-rcnn/tools/train_net.py

#caffe.set_mode_gpu()
caffe.set_mode_cpu()

(3) 将~/py-faster-rcnn/lib/setup.py中，含有nms.gpu_nms的部分去掉

(4) 修改./lib/fast-rcnn/nms_wrapper.py，修改如下：

from fast_rcnn.config import cfg
#from nms.gpu_nms import gpu_nms
from nms.cpu_nms import cpu_nms

def nms(dets, thresh, force_cpu=False):
    """Dispatch to either CPU or GPU NMS implementations."""

    if dets.shape[0] == 0:
        return []
    if cfg.USE_GPU_NMS and not force_cpu:
        print "nms_wrapper GPU ..."
        #return gpu_nms(dets, thresh, device_id=cfg.GPU_ID)
    else:
        return cpu_nms(dets, thresh)

接下来运行demo即可。

5. 问题

(1) import caffe错误，或者报错fatal error: numpy/arrayobject.h没有那个文件或目录

修改Makefile.config找到PYTHON_INCLUDE：

PYTHON_INCLUDE := /usr/include/python2.7 \
        /usr/lib/python2.7/dist-packages/numpy/core/include

要加一个local，变成：

PYTHON_INCLUDE := /usr/include/python2.7 \
        /usr/local/lib/python2.7/dist-packages/numpy/core/include

(2) 报错AttributeError: 'DataFrame' object has no attribute 'sort_values'

# sort_values is new in version 0.17.0, so check your version of pandas. In the previous versions you should use sort

pip install -U pandas

# 查看pandas版本
>>> import pandas as pd
>>> pd.show_versions()
>>> pd.__version__

(3) 找不到目录，在_init_paths.py中定义的路径修改：

# 将 this_dir = osp.dirname(__file__) 改为：
this_dir = osp.dirname(osp.abspath(__file__))

参考：

Faster R-CNN CPU环境搭建

Mac下跑仅CPU模式下的py-faster-rcnn

训练结束后测试时出现类似

File "/home/xxx/py-faster-rcnn/tools/../lib/datasets/voc_eval.py", line 126, in voc_eval
    R = [obj for obj in recs[imagename] if obj['name'] == classname]
KeyError: '000002'

解决方法: 删除data/VOCdekit2007下的annotations_cache文件夹

基于ResNet101训练py-faster-rcnn遇到错误：

/faster-rcnn-py/tools/../lib/fast_rcnn/bbox_transform.py:50: RuntimeWarning: overflow encountered in exp
pred_h = np.exp(dh) * heights[:, np.newaxis]
faster-rcnn-py/tools/../lib/rpn/proposal_layer.py:176: RuntimeWarning: invalid value encountered in greater_equal
keep = np.where((ws >= min_size) & (hs >= min_size))[0]

类似解决方法：减小lr from 0.001 to 0.0001

A possible solution could be to decrease the base learning rate in the solver.prototxt
As it is recommended here http://caffe.berkeleyvision.org/tutorial/solver.html
Just change the base_lr: 0.001 to 0.0001

Note also that the above settings are merely guidelines, and they’re definitely not guaranteed to be optimal (or even work at all!) in every situation. If learning diverges (e.g., you start to see very large or NaN or inf loss values or outputs), try dropping the base_lr (e.g., base_lr: 0.001) and re-training, repeating this until you find a base_lr value that works.
I did try to change the base_lr value and now the NAN value disappeared.

https://github.com/rbgirshick/py-faster-rcnn/issues/161

R-CNN for Small Object Detection: https://www.merl.com/publications/docs/TR2016-144.pdf

Perceptual Generative Adversarial Networks for Small Object Detection https://arxiv.org/pdf/1706.05274v1.pdf

https://github.com/rbgirshick/py-faster-rcnn/issues/86

https://github.com/rbgirshick/py-faster-rcnn/issues/433

Hi, I had the same problem and those are my conclusion at this point :

To me, the best answer was to cut the images in smaller patches, at least for the training phase. According to hardware requirement, you need :

3GB GPU memory for ZF net
8GB GPU memory for VGG-16 net
That’s taking into account the 600x1000 original scaling, so to make it simple you need 8GB for 600 000 pixels assuming that you use VGG.
I have 12GB on my GPU so if this is linear, i can go up to (600 000x12)/8 = 900 000 pixels maximum.
I couldn’t resize my images because my objects are small and I couldn’t afford losing resolution.
I chose to cut my 3000x4000 images in 750x1000 patches, which is the simplest division to go under 900 000 pixels.

SCALES: [750]
MAX_SIZE: 1000

However, the good thing is that you only need to cut the images for the training phase. Then you can apply the trained network on full images thanks the the separate test parameters :

TEST:
SCALES: [3000]
MAX_SIZE: 4000

At least that’s what I did and now I have a network working on 3000x4000 images to detect 100x100 objects, in full c++ thanks to the c++ version.

Hi guys,
I already changed the code in lib/rpn/generate_anchors.py and nub_output like this:
ratios and num_output like this

However, I got the following error.

==============

layer {
name: 'rpn_cls_prob_reshape'
type: 'Reshape'
bottom: 'rpn_cls_prob'
top: 'rpn_cls_prob_reshape'
reshape_param { shape { dim: 0 dim: 18 dim: -1 dim: 0 } }
}

You should change this 18 to 24

JayMarx

Hello! Do you solve the problem? I changed aspect ratios and followed catsdogone’s method, it’s works, but when I changed scales just like you, it didn’t work.Do you have any idea how to fix it?
These are my changes:



As you see, I just changed “dim: 18” to “dim: 140” and I don’t know whether it’s right or not!
The error goes like this:

->>>
liuhyCV

@JayMarx I have meet the same error with you. I have found the solutions as follows:
at function “ def generate_anchors(base_size=16, ratios=[0.3, 0.5, 1, 1.5, 2], scales=2**np.arange(1, 6)): “,
but at anchor_target_layer.py:

def setup(self,bottom,top)
layer_params = yaml.load(self.param_str_)
anchor_scales =layer_params.get('scales', (8, 16, 32))
self._anchors = generate_anchors(scales=np.array(anchor_scales))

we should change this code by:

layer_params = yaml.load(self.param_str_)
anchor_scales =layer_params.get('scales', (8, 16, 32))
self._anchors = generate_anchors()

at last the generate_anchors() can use the scales that we defintion

@harjatinsingh So far I havent being able to successfully make it work for smaller images as I wanted. However, it seems changing the values of the ratios in generate_anchors.py does make the algorithm to recognize smaller objects, but the bounding box looses precision. For instance, what I have done is changing the code below from this:

def generate_anchors(base_size=16, ratios=[0.5, 1, 2], scales=2**np.arange(3, 6)):

to this:

def generate_anchors(base_size=16, ratios=[0.3, 0.75, 1], scales=2**np.arange(3, 6)):

Also, it seems that changing the values of anchors does work as noted in #161 but I couldnt make it work for me. So my question (in both issues) is still pending.

1.安装XX-Net

https://github.com/XX-net/XX-Net/releases

2.MacOS设置IPV6

安装merido

http://www.deepdarc.com/miredo-osx/

安装TunTap(pkg安装包 不安装的话 Miredo会一直链接不成功,设置界面显示红灯)

https://sourceforge.net/projects/tuntaposx/?source=typ_redirect

在系统设置界面，打开merido，其中选择client，然后填入teredo.remlab.net，然后apply即可看到上面的黄灯变为绿色，并且在下方出现IPV6地址。

3.XX-Net运行检查所有IP

XX-Net启用IPV6

能得到上千个有效IP数，连接成功！

1.安装必备的Visual Studio 2015，以提供nmake工具

2.在开始菜单中找到Visual Studio Tools > VS2015 x64本机工具命令提示符

3.到pyqt4官方下载sip的源文件

4.编译安装sip

cd C:\sip-4.19.3 
python configure.py
nmake
nmake install

5.安装pyqt4，在Windows下不要采用编译安装方法，会掉到坑里。我们使用非官方的安装包，下载地址：http://www.lfd.uci.edu/~gohlke/pythonlibs/，下载对应的PyQt4‑4.11.4‑cp27‑cp27m‑win_amd64.whl【注意python的版本号，此处是基于python2.7（cp27）】

pip install PyQt4‑4.11.4‑cp27‑cp27m‑win_amd64.whl

安装完成之后，将C:\Python27\Lib\site-packages\PyQt4添加到系统路径中

6.安装labelImg，这里编译安装

pyrcc4 -o resources.py resources.qrc
python labelImg.py

7.lxml的安装方法参考pyqt4，从中http://www.lfd.uci.edu/~gohlke/pythonlibs/下载对应python版本号的lxml非官方安装包，然后安装即可

pip install lxml-3.8.0-cp27-cp27m-win_amd64.whl

完成！！

https://github.com/tzutalin/labelImg

http://www.cnblogs.com/qinjiting/p/5822572.html

Caffe训练出错

F0204 17:40:38.201807 10866 db_lmdb.hpp:15] Check failed: mdb_status == 0 (2 vs. 0) No such file or directory

You have not set your paths to the LMDB directories correctly.
LMDB目录错误,修改prototxt文件的LMDB文件路径

写相对路径不行，写成绝对路径就通过了。。。