**CLOSE RANGE DIGITAL
PHOTOGRAMMETRY AND SOFTWARE APPLICATION DEVELOPMENT FOR PLANAR PATTERNS
COMPUTATION**

**FOTOGRAMETRÍA
TERRESTRE DIGITAL Y APLICACIÓN SOFTWARE PARA EL DESARROLLO DE PATRONES PLANOS**

**PEDRO ARIAS**

*Dep. Natural Resources and Environmental Engineering,
University of Vigo, Spain, parias@uvigo.es*

**JULIA ARMESTO**

*Dep. Natural Resources and Environmental
Engineering, University of Vigo, Spain, julia@uvigo.es*

**JAVIER VALLEJO**

*Dep. Engineering Design and Technical Projects.
Superior School of Engineering.Bilbao, Spain, eppvamaj@bi.ehu.es*

**HENRIQUE LORENZO**

*Dep. Natural Resources and Environmental
Engineering, University of Vigo, Spain, hlorenzo@uvigo.es*

**Recibido
para revisar diciembre 29 de 2008, aceptado abril 13 de 2009, versión final mayo 26 de 2009**

**ABSTRACT: **This paper shows there
processes for the particular case of objects with complex surfaces: the 3D
modelling of the object and its development to 2D model. The procedure to
obtain accurate 3D CAD models of objects with complex surfaces through close
range digital photogrammetry is described; several methodological
considerations are presented that allow obtaining high accuracy results and
minimize processing time. Furthermore a software application that calculates
the corresponding planar profiles is developed. The application automatically
outputs a CAD neutral file with the definition of the optimized planar patterns
needed for the NC cutting machines. The computation of the planar patterns is
based on the solution of an optimization problem that takes into consideration
the geodesic curvature of the surface of the isoparametric curves.

**KEYWORDS:** 3D modelling, Close Range
Digital Photogrammetry, complex surfaces, cutting profiles.

**RESUMEN: **Este artículo muestra estos procesos para el caso
particular de objetos con superficies complejas: el modelado 3D de objetos y su
desarrollo a modelos 2D. Se describe el procedimiento para obtener modelos CAD
3D precisos de objetos con superficies complejas a través de fotogrametría
terrestre digital; se realizan varias consideraciones metodológicas que
permitirán obtener resultados con alta precisión y minimizado de los tiempos de
procesado. Además fue programada una aplicación software para el desarrollo de
superficies complejas en el plano. La aplicación proporciona automáticamente
archivos CAD estándar, definiendo patrones planos optimizados necesarios para
los sistemas CNC. El cálculo de estos patrones planos está basado en la
solución de un problema de optimización que tiene en cuenta la curvatura
geodésica de las superficies de curvas isoparamétricas.

**PALABRAS CLAVE: **Modelado 3D,
Fotogrametría Terrestre Digital, Superficies Complejas, cutting profiles.

**1. INTRODUCTION**

The only way European manufacturing companies can succeed in a global marketplace is by implementing newer manufacturing processes that enable them to respond quickly to market changes and provide high quality, low cost products.

Automation is one major step towards achieving the quality, cost and delivery targets. Due to cost effectiveness and quality results, CAD/CAM and CNC machining are replacing the old manual or semi-automated machinery in a wide range of industries. Frequently, it is common that the design drawings corresponding to an object or a prototype to be manufactured through a CNC system are not available. In such cases, reverse engineering techniques must be used in order to generate a 3D digital model of the object. Afterwards, this model is developed to obtain the cutting planar patterns that will allow us to program the CNC machinery operations. The 3D modelling and its development into planar profiles are key processes for the automation and the reliability of this type of manufacturing process when it affects objects with complex surfaces.

The trouble of creating development profiles of 3D surfaces with double curvature to be manufactured at later time by forming planar pieces of metal sheets have been wide treated in the design community of industrial products since it is present in a great variety of industrial areas: automation, aircraft, navigation, boiler-making, inclusive in the textile and the shoe-making industry [1] [2]. The current CNC manufacturing systems along with CAD/CAM systems are becoming one of the major expansion and renovation areas in this type of industries. But the implementation of CNC cutting machinery in the flat sheet manufacturing systems brings serious problems when trying to manufacture complex geometry objects, since defining the optimal form of the cutting profiles, which is essential to program the cutting path, raises serious difficulties. Furthermore, when original design drawings of the objects or prototypes to be manufactured are not available, it is necessary to carry out a previous modelling phase with the purpose to get the precise knowledge of the object geometry and dimensions. The 3D modelling and its development into planar profiles are key processes for the automation and the reliability of this type of manufacturing process when it affects objects with complex surfaces [3].

In recent years, research efforts led to object modelling with complex surfaces in the industrial application fields have been focused on laser scanning technology [4] [5]. However, surface reconstruction from random scanned data usually requires a complex time consuming reconstruction process and significant amount of computation [6]. These considerations have led us to concentrate in evaluating the feasibility of applying digital monoscopic photogrammetry in modelling objects with complex geometry. Some interesting previous works can be found in [7]-[11]. This paper is focused on the monoscopic digital photogrammetry since it has several advantages such as: equipment cost, flexibility, simplicity and calculation volume data processing [12]. In our opinion, these factors make it more viable in industrial applications in terms of cost-efficiency balance point of view than the stereophotogrammetric techniques or single image based processes [13]. Several methodological indications are established that allow us to optimize the time required for the data processing and the final 3D model accuracy.

Furthermore, a custom application has been developed that implements an efficient algorithm for the planar development, for the CAD model obtained from the photogrammetric procedure. This application allows us to obtain a CAD file that has the definition of the optimum cutting profiles of the curved surfaces. The mathematical principles of this application are shown. The method is applied over a wooden sculpture that has simple and complex curved surfaces (see figure 1).

**
Figure 1**. The methodology was applied over a wooden
sculpture that has simple and complex curved surfaces

**2. METHOD**

**2.1 Photogram Metric Procedure: 3D Modelling
**Close range photogrammetry
basically involves the use of a network of photographs of an object taken from
different angles. In our case study, the photography network was designed
considering the parameters that determine the strength of the geometry according
to [14]. A digital calibrated camera, Canon EOS 10D of 6.291.456 pixels CCD
resolution was used. The interior orientation parameters are showed in table 1.

**Table 1.** Interior orientation
parameters of the camera used The sculpture was placed on a
supported base covered by a
10
cm grid (see figure 2).

**
Figure 2**. The wooden sculpture was placed on a supported
base covered by a
10 cm
grid

As photogrammetric system we have used a monoscopic digital photogrammetric software, the package Photomodeler Pro 5.0 [15]. The system is used for the digital orientation and restitution of photographs. It enables us to work with pictures obtained either with digital cameras. The results obtained can be of two types: numeric for quality control of results (in ASCII format) and graphic (DXF, 3D Studio, RAW, 3dm and Iges).

The cross points within the grid and the sculpture vertex are taken as target points for the camera poses computation. Additionally, a grid of dots set at 5 cm among them was projected over the complex curved surfaces for inner point restitution. The photographs were taken all around the object from a distance of 1 meter, keeping the convergence of rays at 40º to 90º.

The image measurement was made as a manual process for the identification of common pixels in different images in the external orientation, as well as in the line drawing (points and lines). The process of image measurement was automatic from grid of dots set projected over the complex curved surfaces. The inner points in this kind of surfaces (and due the texture of the model) were calculated using a structure light system. The photogrammetric system used has implemented algorithms to calculate Cartesian coordinates of the grid of dots set with shape of circle or ellipse. The structure light system was builded using a little searchlight with a grid of dots built-in.

The problem of obtaining a
digital model that properly defines the geometry of the object from digital
images can be stated as follows: (*X _{i},
Y_{i}, Z_{i}*) of a set of

There is a perspective
relationship between the 3D co-ordinates in the object space and the 2D
corresponding coordinates in the image plane, since an image point *i’* (*x _{i}, y_{i}, -f*), the perspective centre

Where (*x, y*) are image co-ordinates of target
points, *f*, the principal distance; (*X, Y, Z*) are object space co-ordinates
of target points; (*X _{0}, Y_{0},
Z_{0}*) are object space co-ordinates of the projection centre;

The photogrammetric model generation involves the previous computation of the six orientation parameters corresponding to the position (space co-ordinates) and orientation (rotation angles) of the camera projection centre in a Cartesian coordinate system for each image plane. For this end, the target points in the object space are matched with the corresponding ones in the image planes of the network. If a number enough of target points are used, the six unknowns for each image plane can be found. Generally more than the necessary minimum are used and the parameters are estimated by the bundle adjustment functional model, which is based on least squares estimation (for further information see [16]).

The
bundle adjustment procedure was carried out using, 12 control points located on
the supported base that allowed to scale the 3D model and to fix the reference
system, 17 images in digital format of the sculpture. After the bundle
adjustment the photogrammetric system used to gauge project quality the value
called σ_{0 }(or more properly a priori σ_{0}),
calculated like the variances of the residuals. The value of σ_{0} was estimated in 0.85 pixels.

The computation of the spatial 3D co-ordinates of the object vertexes and the projected dots within the doubly curved surface is analogue to the method described for the camera poses calculation, see equation (2).

Now the
unknowns are the co-ordinates (*X _{A},
Y_{A}, Z_{A}*) of the points, and they are computed given the corresponding (

The photogrammetric software used is a convergent monoscopic multistation, this system is based in a process of intersection of rays common for the calculate of co-ordinates. This process is used for the restitution of all points, including the grid of dots.

Reconstruction of flat surfaces is performed by fitting a flat surface to sets of coplanar points and edges, in such a way that the normal distance from the surface to the vertexes is minimized. Constrictions concerning parallel and perpendicular conditions of edges are previously established in order to minimize errors in the points restitution process. For the reconstruction of the curved surfaces a Delaunay triangulation is performed that approximates them. The triangular mesh is then smoothed in order to get a better fitting with the original surface.

**2.2 Surface Flattening Method
**The
procedure implemented here and
described in points 2.2.2 and 2.2.3, is based on the Azariadis and
Aspragathos method (see [17], [18] for details).

* 2.2.1 Application plug-in development*The
computation of the planar profiles of the developed surfaces starts from the 3D
model definition obtained as result of the photogrammetric process. The faces
of the model defined with developable surfaces (Gaussian curvature = 0) are
easily developed using the functionality of a commercial CAD application.
However, this is not possible for the faces with doubly curved geometry that
cannot be flattened into a map without stretching, tearing, or compressing
itself. When Gaussian curvature is not very high, an optimization procedure
could be proposed that provides a fair approximation to the shape of the planar
pattern minimizing the deformation of the plate.

The mathematics of this method takes into account the isometric development of the isoparametric curves with geodesic curvature preservation. Manipulation and complex data computation of model geometry was required therefore the application for planar patterns computation was conceived as a custom plug-in to a CAD application. A plug-in is a software module that extends the functionality adding new commands or features to the application.

The application plug-in was developed with the Rhino 3.0 SDK, an API that allows the access to the functions of Rhinoceros 3.0 CAD application. This plug-in is build with Microsoft Visual C++ 6.0 as a Windows DLL (Dynamic Link Library) that could be executed from Rhino’s command line.

A neutral file of the model is generated in order to manage its geometry into the CAD application. Rhino’s SDK provides all the functionality needed to select the surface patches and access the data structure of the model, computing position, normal and tangent vectors, needed for the mathematical procedure

The resulting planar pattern obtained with the plug-in is a geometrical entity of the CAD application and can be exported in different formats of CAD standard neutral files.

* 2.2.2 Mesh definition*The mesh
development procedure that is applied is based on the development of one of the
families of isoparametric curves. Therefore, the surface is accurately
approximated by a set of flat triangular elements that are homogeneously
distributed. This means that their vertexes are placed at the intersection of a

* 2.2.3 Isometric mapping of a surface curve with
geodesic curvature preservation*The
second stage consists of an initial estimation of the mesh development. One set
of isoparametric curves is developed preserving the length of the curve between
each two points (isometric map) and the geodesic curvature. This criterion is
essential since it guarantees the uniqueness of the resulting curve. The
development procedure of a curve

In the
triangular mesh of the surface, the sides of the triangles that join the
vertexes lying on these curves approximate the *u*- and *v*-isoparametric
curves. The first estimation of the developed mesh is obtained applying an
isometric map to these lines with geodesic curvature preservation. By carrying
out this procedure, developable surfaces are flattened without distortions,
while a distorted flattened mesh is obtained in the case of doubly curved
surfaces.

* 2.2.4 Reduction
of the distortions: an energy based method*The
development of the surface approximate triangulation may be expressed by the
same procedure applied to the mapping of one unique curve. Isometric mapping
requires the preservation of the distances between pairs of nodes measured on
the 3D mesh. It should be taken into account that each node of the
triangulation lies on the crossing point of two

If the surface is doubly curved, it is not possible to preserve the geodesic curvature and the isometric map, therefore the planar development of the mesh will have an error. To estimate this error an energy function is proposed. This function combines two main error components. The first error term estimates the error due to the distance difference between the nodes in the 3D mesh and the corresponding nodes in the planar development. The second term estimates the angular distortion between the isoparametric curves at each node.

**3. RESULTS**

**3.1 3D Modelling: Photogrammetric Procedure
**The 3D
surfaces modeling was obtained with 74 restituted points (including the
projected grid of dots). The bundle adjustment provide the results show in the
table 2, for all points (including the grid of dots) used to generate the 3D
model. This table shows information about: standard deviation of the cloud of
points (average values in X, Y, Z axes and quadratic composition of the
averages); the mean quadratic error, the mean error and the maximum error
observed in the parallax (m); Root Mean Square (RMS) error and maximum residual
error (pixels); angle maximum and minimum between images (deg).

**Table 2.** 3D model accuracy
assessment: standard deviation (m) of the cloud of points (average values in X,
Y, Z axes and quadratic composition); the mean square error, the mean error and
the maximum error observed in the parallax (m); Root Mean Square error and
maximum residual error (pixels); angle maximum and minimum between images (deg)

The use of a flat grid as support base for the object simplifies the processing stage, since the point identification on a 2D surface is significantly less affected by visual perception errors than the identification of 3D points. The generation of the 3D surface model through digital monoscopic photogrammetry has shown to be a straightforward, fast and reliable method when developable surfaces are surveyed (see figure 3). In the case of complex surfaces, the use of the projected grid of dots has shown to be a simple and affordable system that provides high accuracy results (see table 2).

**
Figure 3**. Wire-frame 3D model of the wooden sculpture

Nevertheless we must point out that the effectiveness of this method depends on the size of the object and the complexity of the surface, since the larger and/or the more complex it is, the higher number of dots might be projected and processed to warrantee the match between the 3D digital model and the prototype.

**3.2 Surface
Flattening Method
**A custom Rhino plug-in application that
obtains the planar development of doubly curved surfaces starting from the CAD
definition of the surfaces has been developed and implemented.

The application generates a CAD neutral file with the shape of the cutting profile that is needed for the machining processes of the metal sheets (see figure 4). The mathematics of this method is based on the solution of an optimization problem that takes into account the isometric development of the isoparametric curves with geodesic curvature preservation. The surface is first approximated with a triangular mesh. Distance deformations and angular distortions are measured between each pair of mesh nodes and each pair of crossing isoparametric curves at the nodes respectively, are observed as a result of the mesh flattening process. An optimization procedure is applied to a function based on the value of these deformations. Table 3 shows the evolution of these values in successive iterations, where the values of distance deformations and angular distortions are expressed as a percentage of the actual distances and angles. The medium and the maximum values are computed each iteration.

**
Figure 4**. 3D model of the grid surface with a doubly
curved surface (left), plannar development of the grid (up
right) and cutting profile (down right)

**Table 3**. Results of the
minimization development method used on double curved surfaces

**4. CONCLUSIONS**

In this paper an approach to the application of digital monoscopic photogrammetry technique in the 3D modelling of doubly curved objects is presented, a surface flattening method, based on isometric mapping and geodesic curvature preservation, is implemented as a CAD application plug-in, and finally, an iterative energy based optimization procedure is applied that minimises distortions in the final planar profiles.

The use of Photogrammetric procedures in this kind of industrial applications has shown some advantages from other classical methodologies, could be appointed:

- Easy methodology and with fast results. For people without previous knowledge about photogrammetric procedures.
- Low cost and simple equipment. Infrastructures or complex devices are not necessary.
- The results can get it with accuracy better than millimeters in the final model.

For other side, we can see same aspects or characteristics would have to be more developed, for example:

- The grid of dots projected on complex surfaces is essential to get quality results.
- When the complex surfaces can be observed in the element. The use of grid of dots make necessary the development and building of industrial structure light systems, like this, the device will allow the automatic process of the grid of dots projected on the surface.
- There is an important influence in the final results for the kind of software CAD used, to process the cloud of points and to adjust there to surfaces models can be development later, to get and extract planar patches to manufacture process. Independently of the procedure or system photogrammetric used.

In conclusion, two main contributions should be highlighted. Digital photogrammetry has shown to be a feasible and affordable tool in the accurate 3D model generation of complex objects; it is relevant that both of the aspects of feasibility and affordability are decisive factors in the accomplishment of automated manufacturing processes. Then the 3D model is saved to a neutral file format that could be processed with a CAD application. The surface flattening method is implemented as a customized plug-in. The proposed development and optimization methods have been proven to be effective and highly efficient in the case study. Finally, the resulting planar pattern is then saved to a neutral file that could be accessible from a wide number of different machine tool programming applications.

**5. ACKNOWLEDGMENTS**

This article is part of a research project financed by Ministry Education and Science “Utilizacion de tecnicas geomaticas para el analisis dimensional-estructural y conservacion de puentes historicos singulares” (BIA2006-10259).

**REFERENCES**

**[1]** KARUNAKARAN, K.P., AGRAWAL, S., VENGURLEKAR,P.D., SAHASRABUDHE, O.S., PUSHPA,.V. AND ELY, R. Segmented object manufacturing. IEEE Transactions, 39, 291-302, 2005.

**[2]** ARIAS, P., ARMESTO, J., LORENZO, H. AND ORDOÑEZ, C. 3D terrestrial laser technology in sporting craft 3D modelling. International Journal of Simulation Modelling, 6 (2), 65-72, 2007.

**[3]** CUARTAS, E AND PRIETO, F. Reconstrucción de objetos con superficies Tipo 2-variedad diferenciable. Dyna, 156 (75), 7-18, 2008.

**[4]** RABBANI, T., DIJKMAN, S., VAN DEN HEUVEL, F. AND VOSSELMAN, G. An integrated approach for modelling and global registration of point clouds. ISPRS Journal of Photogrammetry and Remote Sensing, 61 (6), 355-370, 2007.

**[5]** ROCA-PARDIÑAS, J., LORENZO, H., ARIAS, P. AND ARMESTO, J. From laser point clouds to surfaces: Statistical nonparametric methods for three-dimensional reconstruction. Computer Aided Design, 40 (5), 646-652, 2008.

**[6]** YAU, H.T., LEE, R.K., CHUANG, C.M. AND HSU, C.Y. NC Tool-Path Generation Based on Surfel Models Constructed from Random Scanned Data. Computer-Aided Design & Applications, 2 (1-4), 567-576, 2005.

**[7]** Y. ZHANG. Measurement of industrial sheetmetal parts with CAD, designed data and nonmetric image sequence. Proceedings of the XXth ISPRS Congress Commission 5, Istanbul, Turkey , 2004.

**[8]** TANGELDER, J.W.H., ERMES, P., VOSSELMAN, G. AND VAN DEN HEUVEL, F.A. CAD-based photogrammetry for reverse engineering of industrial installations. Computer-Aided Civil and Infrastructure Engineering, 18, 264-274, 2003.

**[9]** BOSEMANN, W. Advances in photogrammetric measurement solutions. Computers in Industry, 56 (8-9), 886-893, 2005.

**[10]** ERGUN, B. An expert measurement system for photogrammetric industrial application. Measurement, 39 (5), 415-419, 2006.

**[11]** D'ANGELO, P., WOHLER, C. Image-based 3D surface reconstruction by combination of photometric, geometric, and real-aperture methods. ISPRS Journal of Photogrammetry and Remote Sensing, 63 (3), 297-321, 2008.

**[12]** ARIAS, P., CAAMAÑO, J.C., LORENZO, H. AND ARMESTO, J. 3D Modeling and Section Properties of Ancient Irregular Timber Structures by Means of Digital Photogrammetry. Computer-Aided Civil and Infrastructure Engineering, 22 (8), 597-611, 2007.

**[13]** SALAZAR, A., SÁNCHEZ, L.G. AND PRIETO, F. Sistema de adquisición de imágenes de rango con base en estéreo-activo. Dyna, 153 (74), 301-311, 2007.

**[14]** MASON, S.O. Conceptual Model of the convergent multistation network configuration task. Photogrammetric Record, 15 (86), 277-299, 1995.

**[15]** EOS SYSTEMS INC. Photomodeler Pro 5.0. User manual. Vancouver, Canada, 2003.

**[16]** COOPER, M. AND ROBSON, S.. Theory of Close Range Photogrammetry, In: Close Range Photogrammetry and Machine Vision (Ed. K.B.Atkinson), Whittles Publishing, 9-51, 2001.

**[17]** AZARIADIS,P. AND ASPRAGATHOS, N. Design of plane patterns of doubly curved surfaces. Computer-Aided Design, 29 (10), 675-85, 1997.

**[18]** AZARIADIS,P. AND ASPRAGATHOS, N. Geodesic curvature preservation in surface flattening through constrained global optimization. Computer-Aided Design, 33, 581-591, 2001.