The program for the upcoming ESRI Survey Summit is going to include a number of LiDAR sessions. This year’s event takes place in conjunction with the ESRI User Conference, which is likely to draw almost 15,000 attendees to San Diego, CA.
For the first time the Survey Summit, which starts on July 7 is being held in conjunction with the ACSM – American Congress on Surveying and Mapping. This will be an interesting experiment to observe. The surveying community has been slow to adopt GIS in a big way and the society itself has been having issues with sister organizations. Hopefully this will get things going in a new direction. Kudos to Brent Jones for his vision on this.
I will be presenting a one hour intro to airborne and mobile LiDAR. Other speakers include Tom Greaves from Spar, Patrick Cunningham from Blue Marble who will discuss geographic coordinates and Bill Emison from Merrick discussing airborne data processing.
The intersection of LiDAR with GIS is certain to provide new insights and business opportunities, particularly in the area of digital cities. It should be a great event.
Hello,
Do you have any sort of experience with integration of mobile lidar data and RGB image to form so called rich point cloud ? any suggestions or any reference to useful papers would be highly appreciated.
My understanding :
steps:
1) computation of exterior orientation of the RGB images !
I have pitch, heading and raw. But how to convert into omega phi kappa?
2) try to get orientation by resection: picking corresponding points from lidar cloud and RGB image !
Picking points in lidar cloud data is not an easy task ? how to do it ?
any thoughts ??
sukuchha
From Craig Glennie:
\Assumptions: Imagery and LiDAR data have been collected simultaneously on a common platform. Both camera and LiDAR sensor are rigidly attached to same surface as IMU.
If this is the case, then the Position (E,N,U) and Orientation (Roll,Pitch,Heading) probably is given with respect to the center of the IMU.
To solve this problem, there are several values that need to be known
(a) interior orientation parameters of the camera – also sometimes called lens distortion model (focal length, principle point offset, radial and tangential lens distortion). These are normally provided by a bundle adjustment solution for the lens. A common model for the camera is given in the Open Source Computer Vision Library (http://opencv.willowgarage.com/wiki/)
(b) Boresight parameters for camera. These are the offsets in both position and orientation between the IMU center (or reference for navigation solution), and the principle point of the camera. Easiest way to solve for these parameters is via a bundle adjustment which uses the LiDAR data to solve for these constant offsets.
Given the above values, the Position and Orientation of the IMU is transformed into position and orientation of the camera (exterior orientation). The camera position and orientation can then be combined with the lens model (interior orientation) in order to solve for the real world projection of any pixel. Practically, what is then done is that every LiDAR point is back-projected (using the camera exterior and interior orientation) into the images to assign it a pixel color value.
As far as the question below regarding roll,pitch,yaw to omega, phi, kappa, this transformation is normally just a reordering of the order of rotation. If we notate R3 as a rotation around the z-axis, R2 as a rotation around the y-axis, and R1 around the x-axis, then common convention has a full 3 degree of freedom rotation as:
R2(Roll)R1(Pitch)R3(Yaw) – Typical IMU definition.
R1(omega)R2(phi)R3(kappa) – Typical photogrammetric definition.
Unfortunately, the above definitions aren’t standard, but vary between IMU vendors and photogrammetric software packages. The utilized order of rotations for the IMU can usually be determined from the GPS/INS software package manuals. Once the order of rotations for both the IMU and photogrammetric package are known, a unique transformation from one set of angles to the other can be accomplished.
Craig
Thanks for your detail explaination. I really appreciate.
Your assumption of is totally right. I have lidar and RGB image collected simultaneously and have a IMU readings ! Also i have bore sight parameters for camera which were readily provided by the data collector.
>>Given the above values, the Position and Orientation of the IMU is transformed into position and orientation of the camera (exterior orientation)
This is where i am stuck. I have to find the position and orientation of the camera. I guess, i have to do some transformation. This is what I think should be done.
1) Getting principal point of the camera!
[XYZ]imu = [XYZ]cam+[R]imu*[bx,by,bz]
Here [XYZ]imu = position of IMU, [XYZ]cam = position of principal point of camera, [R]imu = rotation matrix of IMU given by heading by roll pitch and yaw and [bx,by,bz] is bore sight parameters of camera! Using the above calculation, I can get the [XYZ]cam. Here i dont have to change roll pitch and yaw into omega phi kappa. Right ??
2) Getting the omiga phi, kappa
R[omega,phi, kappa] = S *[R]imu
where S is the matrix to transform something like [0 1 0; 1 0 0; 0 0 -1] .
Then omega, phi, kappa could be found something like this.
Omega = invSIN(r13), kappa = invTAN(-r23/r33) and phi = invTAN(-r12/r13). Right ??
3) Use collinearity equation to backtrack Lidar point (XYZ) into image coordinates and get pixel color value .
So, the procedure should be something like this? Please correct me if i am wrong. Are you aware of any papers which elaborate this steps in more detail? Any link?
Sukuchha
Hi Sukuchha,
I know this is some time after your original posting. I wondered if you managed to solve your problem and if the process you described worked for you. I would be interested to know as I am looking to resolve a similar problem.
Mr. Sukuchha, did you find a solution for your issue?
I am doing nearly the same thing. The airplane takes an aerial photograph and scans LiDAR data. Now I want to match the LiDAR points with the right pixels in the colored image.
I tried a lot of different approaches without luck.
First approach was making a virtual camera in the 3D world of the LiDAR point with the same parameters as the camera on the airplane and render the scene from the virtual camera, but this results in a non-uniform shift between the pixels and the LiDAR points.
Second approach is trying to give every pixel a UTM coordinate, and since I already have the LiDAR data in UTM I can match these easily. Unfortunately I also got a non-uniform shift between the two, perhaps because of the non-zero pitch and roll. I tried to make equation that takes these two angles in consideration but i still get a non-uniform shift.
Any help is greatly appreciated.