CNES is proud to announce the release of Orfeo ToolBox 3.12, codename Πλειάδες.

After its sucessful launch on the 17th of December 2011, and while still in its comissing phase, Pleiades is already sending us beautiful images. This new release, and especially the JPEG2000 support part, has been tested against real Pleiades images kindly provided by the comisioning team.

There are exciting new stuff in all corners of the library. Some of the notable changes in this release are:

You can find the source (OTB and Monteverdi) and binary packages (Monteverdi form Mac OS X and Windows) here.

Some of the notable changes in this release are:

• Large JPEG2000 file (Pleiades-like) support and Pleiades metadata handling in OTB (more information here)
• Efficient JPEG2000 visualisation and ROI decompression tools in Monteverdi
• Revamp of otb applications in a generic and scalable framework (more information here) : launch applications from Command-Line, from an auto-generated QT GUI, from python, from within QGis …
• Lots of new algorithms : Dimensionality Reduction (ICA, PCA, MNF, MAF …), change detection (MAD), Hyperspectral Unmixing, elevation map from stereo data, compare segmentation with a ground truth (Hoover) …
• And various bug fixes.

There are a lot more new things coming with this release ! For more information, please read the complete release note available here.

As usual, Release Candidate are made to be tested and stressed, so do not hesitate to give it a try and report whatever you find suspicious on the users list, or directly on the BugTracker.

Also note that a teaser of what can be done with OTB is now available here !

Happy Testing!

Manuel, for the OTB team

History

Until now, there was an OTB-Applications package built on top of the library, which provided command-line tools, Qt and FLTK based GUIs, and Qgis C++ plugins. This package was not very popular among our users : it was lacking of documentation, missed really useful auto-generated GUIs and did not provide an interface to interpreted languages. These applications were barely more than a serie of mains decorated with an argument parser. However, these applications were providing a lot of image manipulation utilities and high-level processing chains like ortho-rectification, classification or segmentation. Being too obscure, they simply did not get the attention they deserved.

Six month ago, we started a side project called OTB-Wrapper, which was designed as a sand-box to experiment with all the possible enhancements to make the applications really useful. After a while, we came up with a complete solution and implemented it directly into the OTB library. All applications from the OTB-Applications package have then been migrated to this new framework, also directly into the OTB library.

The new framework in a nutshell

So, what does this new framework provide to the users ? A lot of things ! See :

• Each application is now compiled into a single tiny shared-library,
• Writing new applications in an external project is as simple as inheriting the base class and calling a CMake macro,
• These applications shared libraries are loaded dynamically as plugins of several generic access points :
• Directly from C++
• From an interpreted or compiled higher level language like Python through the SWIG interface,
• From a command-line launcher, like the old applications,
• From a nice auto-generated Qt GUI,
• From QGis through seamless OTB plugins.

The general philosophy of this new framework is to have to write the processing only once, and then use it from whatever environment, programming language or software you want through one of these access points.

Success !

Even before the official release, success is already showing up :

• A simple python script allows to generate documentation for all the applications,
• A simple python script allows to generate applications file descriptors for integration in external tools like the KEO system from ESA,
• A simple python script allows to perform batched processing chaining several applications,
• Qt GUIs are really nice to use, with a lot of useful features like automatic parameters estimation (for instance, the ortho-rectification application provides you with the best-fit parameters for your image),
• The python interface allows nice integration within QGis,
• And the command-line interface is more robust and easier to use.

Learn more in next posts

If you want to give this a try, clone a the OTB mercurial repository and activate the BUILD_APPLICATIONS flag. You are ready to go ! In the following posts, we will focus on key features that makes this new framework so exciting. But for now, it is time for us to wish you all a merry christmas !

Julien, on behalf of the OTB team

With the Pleiades launch a few weeks ahead, there has been a lot of ongoing work from the OTB team to prepare for these new data. Since a lot of discussions happened off the list, either with phone meetings, or on other mailing lists (we will see later on why), and since we now have a comprehensive knowledge of what the support of Pleiades images in OTB will be, it is about time we explain it to users and developers.

Reminder on Pleiades images

We will start with a reminder : Pleiades images will be available in Jpeg2000 format allowing a high compression rate, tiled in 2048×2048 pixels, so a standard Pleiades image will contain a few hundreds of tiles. A typical Pleiades image size is 40 000 x 40 000 pixels (corresponding to a 20 km by 20 km area), and one of the standard product is a pan-sharpened one, which is a merge of the high resolution panchromatic and lower resolution multispectral imagery to create a single high resolution color image. These images would be very heavy without Jpeg2000 compression : for instance, if the Jpeg2000 file weights 1.7 Go, the decompressed file weights 7.3 Go. To decode those JPEG2000 files, there are a few commercial libraries, the most popular beeing Kakadu. There is also some open-source alternatives, among which OpenJPEG seems to be the most advanced. Of course, for Pleiades support in OTB, we need an open-source solution, even if GDAL can be compiled with a driver based on Kakadu (has to be the commercial version, not the trial one available for free under restrictions of use).

Open-source JPEG 2000 implementation

Back in 2007, the OTB people here at CNES already spotted OpenJPEG as the best open-source bet for JPEG2000 support in OTB. Since the library was missing some important features (like partial decoding and MCT support), they set-up a CNES contract with the CS Company in order to add these features to OpenJPEG. But when this contract was over, the new features were not integrated in OpenJPEG trunk but left on a side branch (the so-called v2), because nobody in OpenJPEG community had in-dept knowledge of what had been done in this new version. The development of the trunk went on with bug-fixes and enhancements, while the v2 branch did not evolve. In the meantime, we started a driver in OTB based on the v2 OpenJPEG version, but this was not a sustainable option, because the v2 was barely maintained. Thanks to its more advanced functionnalities, the v2 branch of OpenJPEG also received interest from other FOSS projects in need for advanced decoding capabilities, and got integrated in at least ITK, GDCM and GDAL.

Four months ago, we agreed with the OpenJPEG community that we needed to get the v2 merged with the OpenJPEG trunk. Mickaël Savinaud from the CS OTB team got involved into OpenJPEG development to get the merge between v2 and trunk done, and we now have a full-featured version of OpenJPEG in trunk thanks to his great work and to the support of the OpenJPEG community.

Now, it is time to face the truth : even with this new version, in terms of decoding performances, this state-of-the-art open-source software is way worse than Kakadu : decoding one tile is way slower than with Kakadu, and one tile of 2048×2048 pixels is the atomic unit for decoding (i.e. if you need one pixel, you will still need to first decode the whole tile).

We looked into optimization : reducing file seeking, implementing some macroscopic code optimisation (mainly avoiding pessimisation) of the Tier1 part which is clearly identify as the performances bottleneck (see figure below with profiling reports using kcachegrind). We made some clear progress, but the JPEG2000 standard is complex, and without knowledge of the big picture, we could not gain a lot.

KCachegrind representation of time percentage per functions

What does it mean for Pleiades data ?

It means that simply decoding a full Pleiades image at full resolution will take about 20 minutes on a decent i5 CPU with 4 Go of RAM, while the same image in standard TIF format would have taken 8 minutes and Kakadu takes only 4 minutes. Any OTB processing pipeline streaming the whole image will be limited by this decompression time. And we are not even talking of sub-sampling or estimating statistics at some point in the pipeline.

Of course, we could tell our users to buy a Kakadu licence and compile GDAL with the Kakadu driver enabled, but we can not even tell them how much it will cost, and this is clearly in contradiction with the open-source philosophy of OTB. Still, this remains an option to get high performances JPEG2000 support in OTB, but you are on your own if you choose this solution.

Now, what will it be possible to do using Pleiades data with the open-source solution OpenJPEG in OTB ?

• On OTB side: No specific handling will be added. It is the responsibility of the developer to know that the JPEG2000 driver is not as fast as the other drivers. The developer has the possibility to easily select the appropriate resolution and accordingly set the size streamed region (to reduce the number of decoding operations).  Of course, metadata related to geometric and radiometric calibration will be supported through OSSIM. Moreover, we will support the GMLJP2 box included in the Pleiades to support geo-information in case of ortho-image products.

• On applications side:

• We will provide an application able to convert a Pleiades image, either full or extract, from any JPEG2000 resolution, into another file format (like TIF for instance). Of course, one will need a lot of disk space to handle the decompressed data. Please note that this is not fully supported by Kakadu trial version, which decodes a maximum of 3 channels (and which use is restricted by the way),

• All applications will support Pleiades images, but it will be highly recommended to first decompress the image to disk for reasonable performances and if more than one processing is to be executed on the image,

• On Monteverdi side:

• Pleiades images will open in a dedicated new type of data in Monteverdi interface, and the user will be able to select the resolution level for decoding,

• The viewer module will accept this new type of data and we will be able to efficiently navigate within Pleiades images : fast quicklook computation due to JPEG2000 capabilities, instant navigation into a 4 tiles squared area thanks to caching, and quick-enough refresh when moving outside this cached area thanks to parallel tiles decoding. We expect the navigation experience to be quite good !

• The extract ROI module will accept this new type of data and allow to select an extract of the image using the fast quicklook ability. Caching (which will decompress image to disk in TIF format) will be recommended (this behaviour might be extended to other modules later).

• The writer module will accept this new type of data for conversion purposes.

• All other modules will not accept this new type of data : to use them, the user will first have to use the extract ROI module which will produce a plain image type accepted by all modules. Caching will be recommended for this purpose.

Closing Thoughts

We can hope that the widening use of JPEG2000 in spatial imagery will encourage the improvement of open-source JPEG2000 libraries performances. When and if this becomes available, you will be able to use those datasets in OTB seamlessly, as any other image.

Until then, we can look at the bright side :

OTB will provide an open-source and free access to Pleiades data for those not owning a commercial image processing software  (please note that Gdal is looking into the new OpenJPEG version and might also support Pleiades through it in a near future).

OTB will provide an efficient solution for Pleiades data decompression and visualization. 

Last, the whole set of OTB processing remains fully available, at the expense of extra decoding time or once your Pleiades data have been decompressed to disk.

This has also been a great experience of collaboration between open-source projects towards a common goal. We would like to thank again the members of the OpenJPEG community, as well as the OTB Team at CS for their great work !

This video was generated with the open source software version control visualization tool Gource.

It shows modifications  in : OTB, OTB-Applications, Monteverdi, OTB-Documents, OTB-Qgis-plugins and OTB-Wrapping source code repositories.

It took him no less than 7 hours in two days to do it, including building the cake pan out of carton pieces. It is needless to say it took us far less time to eat it once cut, and we can assure you that this cake was as tasty as beautiful !

Cheers !

Julien Michel

Classical scheme

The classical scheme to compute elevation maps from these kind of data is to first resample both images of the pair into the epipolar geometry (with optional refinement of the sensor-to-ground function using GPCs and tie points) , where displacement related to height only occurs in the horizontal direction. A block-matching algorithm (just like otb::FineRegistrationImageFilter does) is then applied to retrieve the disparities. Finally, disparities are casted back to height values using sensor modelling and the elevation map is resampled to a cartographic projection. Several post-processing may be needed to make the produced elevation map usable, which are out of the scope of this post.

However, resampling to epipolar geometry is an heavy process that must be done for both images, and in the case of push-broom acquisition system, this resampling may be tricky. Moreover, this resampling is often done with an average elevation hypothesis, which leads to a wide range of disparities to explore in case of highly varying relief over the scene.

Implicit exploration of epipolar lines

In Orfeo ToolBox, the otb::GenericRSTransform allows to map points from sensor geometry to ground, but it also allows to map points between two sensors geometries : we can therefore use it to build a function that will transform a point of the first image of our stereo pair to its homologous point in the second image of the stereo pair, according to both RPC modelling (possibly refined). This function also uses an elevation value as a parameter (either average or drawn from a Digital Elevation Model), so that we in fact have a function which associate a point with a candidate elevation to its homologous point in the other image of the stereo pair. If we transform the same point with a range of candidate elevation, we are then implicitly exploring its associated epipolar line in the other image (a similar approach is performed by the MicMac software).

A new OTB filter

Using this principle, we developped a new OTB filter called otb::StereoSensorModelToElevationFilter. This filter takes a pair of along-track stereo images, and behave as follows. For each pixel of the first (reference) image:

1. Extract the neighborood of the pixel in the reference image according to the user-defined radius,
2. For candidate elevations in a user-defined range:
1. Transform the patch grid into the secondary image with the sensor-to-sensor function and the candidate height
2. Interpolate the corresponding secondary patch
3. Compute cross-correlation of reference and secondary patches
3. If best correlation is high enough, keep the corresponding best elevation candidate as the final elevation.

The output of the algorithm is directly an elevation map in meters, in the same geomety and grid as the reference image of the stereo pair. There is no need to resample both image in epipolar geometry, we only need to interpolate necessary locations along the epipolar lines in the secondary image. One can then orthorectify the elevation map in any desired cartographic projection.

We can see that a difficult part of this algorithm lies in the definition of the appropriate elevation exploration range. First, it requires to know about the average elevation over the area, and even with this information, elevation might be varying a lot over the scene, so this range has to be pretty wide, and lead to high computation time and lots of matching errors. Therefore, the filter allows to draw the initial elevation from a DEM (like SRTM for instance). The algorithm will then explore elevation values around this intiial local elevation, which allows to define a much more narrow range.

Results

Our first result will be a pair of Worldview 2 images over the city of Toulouse. The following results were obtained on a detail of the multi-spectral images (2.3 meters resolution) over the city stadium.

©CNES

©CNES

Applying the same algorithm to the panchromatic images (0.5 meters resolution) leads to noisier results, but we can still clearly distinguish the shape of the stadium. Image on the left is the correlation mask associated with the result (and produced by the filter). Dark values correspond to low correlation, brighter ones correspond to higher correlation. Please note that in this case, initial height has been subtracted from the result (filter option).

Next results were obtained from a pair of WorldView 2 multi-spectral images (2.3 meters)  over the area of Aurignac, in France, an area of agricultural and natural landscape with lots of valley and hills. The two images were first properly down-sampled at a resolution of around 11.5 meters before the process was applied. The upper-left corner is more blur because it is an area which is only viewed in one of the image, and thus contains only the initial SRTM elevation.

The two images bellow shows the difference between the SRTM (on the left) and the estimated elevation (on the right).

If one does not want to use an initial DEM, the filter can start from a user-defined average elevation. Next image shows the result of using an average elevation on the Aurignac scene. Red pixels are those for which correlation fails. The result looks worse than the one using an initial DEM because in the DEM case pixels for which correlation fails are given the intial DEM value, which leads to a smooth result.

Conclusion

Of course, this block-matching algorithm is still very simple, and we do not claim that these results are better, faster or even valid with respect to those one can produce with similar tools. For instance, it is obvious that the results suffers from the well known fattening effect, and no particular filtering is performed to reduce it. The same holds for occlusions, incorrect matches or stereoscopic effects. Using some work from Ipol would be of great interest in this purpose. Still, this filter provides a very simple tool to perform elevation map estimation from along-track stereo pairs which makes use of the generic implementation of sensor-to-ground and ground-to-sensor transformation in Orfeo ToolBox to avoid the need for explicit epipolar geometry. Built-in access to DEM allows to initialise the height map with sound values and thus to narrow the search space. One can also make use of the filter streaming capability to produce only part of the image, which is more delicate with the standard scheme. Future work is to speed-up the computation, since the filter currently makes an intensive use of the sensor to sensor transform and of interpolation. Reviewers are welcome to read and comment the code which can be found here, whereas users can give it a try using the new otbStereoSensorModelToElevationMap-cli application from the latest revision.

This will also allows us to measure the number of users and their typical usage. For all these reasons it would be very kind of you if you could take the Users and Contributors Survey.

It will only take about 10 minutes of your time. We are planning to provide the results in September.

Remember that you are welcome to give feedbacks, comment, contributions at any time through our users and developers mailing list. Do not hesitate to get involved !

Many thanks to all of you for your support,

The OTB development team

The idea is more or less the same as for the GSoC : ESA selects open source projects as mentoring organizations, and propose to pay students a very decent amount of money to work on those open source projects during a few months in the summer.

Students have the opportunity to work on fun projects during the summer, and in the end they can add an interesting experience to their resume.

The list of mentoring organizations selected by ESA can be seen here. As you can see, Orfeo Toolbox is selected for this pilot year !

An ideas list for this program has been compiled here, so students can start applying by picking up ideas from there. Of course, any other project which improves the library will be welcome, as long as we have the opportunity to discuss it before the application.

This year being a pilot year, the schedule is very tight, and students must apply before the 27th of July.

Students : we are looking forward to hearing from you soon !

CNES is proud to announce the release of Orfeo ToolBox 3.10, codename Feliç aniversari. It coincides with the fifth anniversary of the Orfeo ToolBox library (release 1.0 was done the 06/30 2006!).

Release Candidate: thank you !

We would like to thanks all the users for their great feedback after the Release Candidate announcement : thanks to their work, 9 more bugs have been spotted and fixed between the Release Candidate announcement and today.

Take the survey !

Orfeo ToolBox is all about you ! To celebrate the fifth birthday of the OTB, we would like to collect your feedback : a better knowledge of your usage of Orfeo Toolbox and of your processing needs will  help us to improve the tool. Please take the Users and Contributors Survey, it will only take about 10 minutes of your time.

Even more easy to install

The source packages for OTB (Library 3.10, Applications 3.10, Monteverdi 1.8 and Wrapping 1.6) are available on sourceforge as usual. For our Ubuntu (10.04, 10.10 and 11.04) users the new version is available for update through you favorite package manager software.  OpenSuse and CentOS packages will be available in a few days.

For MS-Windows users, there are also a binary auto-install packages available for Monteverdi, OTB-Applications and OTB-Wrapping! Last, a brand new Apple Disk Image (DMG) is also available for Monteverdi.

All the details related to the installation of these binary packages can be found on the download page.

Powerful new features

There are lots of new features in this release! The list of new main add-ons is available here.

There are exciting new stuffs in all corners of the library. New functionalities include among other things :

1. A new multi-image supervised classification suite in OTB-Applications described here.
2. An application to perform segmentation based on connected components and object based image analysis (see description here)

1. A framework for cartographic database creation and validation based on the fusion of features in the framework of Dempster-Shafer evidence theory (see description here)

Documentation for all

The OTB development team also initiated, in addition to the OTB software guide, a guide for OTB-Applications and Monteverdi, dedicated to non-developers. This guide is composed of a brief tour of OTB-Applications and Monteverdi, followed by a set of recipes to perform usual remote sensing tasks with both tools. A first version is available here.

Best regards,

Julien Michel, on behalf of OTB development  team.