In order to get a good picture of the OTB community, we proposed a survey to our user between the end of June and October 2011. This post is a (late) report of the results we had, thanks to your answers.
General information about the survey
We had a total of 62 answers to the survey. Not many questions were mandatory so I will try to avoid to give statistics using percentages for answers and prefer to give the number of answers.
This article does not claim to provide a complete and general summary. We are aware that it can be dangerous to draw too many conclusions from opinions poll.
This article is more at an introduction to the results which are completely available here.
Who answered the survey?
Before trying to investigate and give conclusions about the survey, let’s first analyze who took time to answer it!
People who answered are mostly engineers and scientists (18 and 22) working in institutional organizations (23) and equally in labs (15) or private companies (17). But there are also people working in NGO, students, which shows the diversity of people who use the library. Though OTB was developed in the frame of the Methodological Part of the ORFEO Accompaniment Program to prepare, accompany and promote the use and the exploitation of the images derived from Pleiades (PHR) and Cosmo-Skymed (CSK) satellites, interestingly most people are not involved in this program. They found out about the library through the web or thanks to co-workers or public presentation in conferences for example, rather than from the ORFEO program. Obviously they use the ORFEO ToolBox but they also use a lot of other tools (Open Source or proprietary one) in their daily work. Another interesting point to notice is that the amount of people using Linux system to do image processing with OTB is equal to the amount of Windows users (41 and 41).
How do you access OTB?
First, although OTB has celebrated its fifth birthday earlier this year, 39 of them use the library for less than 1 to 3 years (26 use it for less that one year). Note that 31 are building OTB and derived tools themselves to use it. Building projects from source does not seem to be a problem for about half of them. Note also that 11 people (21%) use binary packages (Ubuntu/OpenSuse/CentOS/ArchLinux). We should surely maintain the effort in this frame and continue to facilitate the accessibility to the library by providing these packages for Windows, MacOS X and Linux systems.
The OTB/Monteverdi tipping point
33 access the OTB through the library and 30 through Monteverdi. That’s clearly an important information given by the survey. We see here that Monteverdi is an important way to access OTB (as important as the library itself). Obviously without the library, Monteverdi would not exist, but we see that GUI applications are an entry point for the OTB. Command line applications are also largely used by end-users and I think that it will continue to increase in the future thanks to the new OTB applications framework available in release 3.12, which allows to access OTB functionalities in a number of different languages and context (in QT standalone application, in Quantum GIS plugins, in Python, in Clojure….) Clearly OTB have to continue in these 3 directions and we have to be aware that GUI applications have become an important entry point for a majority of OTB users.
Why do you use OTB for?
Even if 32 people use OTB with HR data (Spot like) and 28 with VHR optical images, we can see a great diversity of remote sensing data use with OTB which clearly show that OTB is useful in a lot of context (hyperspectral, SAR, lidar…). The diversity in the data processed by OTB users is confirmed by the diversity of OTB algorithms in use. 31 users use feature extraction algorithms, the best score, which is reassuring because it is one of the main reason why OTB was developed, but lots of other functionalities are also largely used. The survey shows that there is a great deal of interest of the RS community in having a tool providing a wide range of functionalities through a common and generic interface.
Why OTB sucks ?
The title is intentionally provocative to illustrate the fact that in this kind of survey, questions related to critics are really interesting and provide relevant advises on how to improve the software together. 12 answers put the emphasis on the complexity of the design due to the genericity of the library, that’s surely related to the well-known steep learning curve associated to OTB. Using the library though C++ classes requires knowledge in C++ design pattern and template metaprogramming. But it’s worth it!
Moreover, access to OTB is made more and more easier through different languages, interfaces …
The second answer is about the incomplete validation of some algorithms. OTB contributors works a lot to maintain a high quality and using test driven developments technics and continuous integration. But this process is only as good as what we do with it, and this shows that there is room for improvement. Validating an algorithm might be a complex task per se, but the high level design can make this even more difficult : multiple plateforms, different streaming and threading behaviours related to the available hardware … And on top of it, different data fed to the algorithm in different context of uses! A faulty design or incomplete definition of some functionnality from the very start of it might also be a cause : design and validation qualtiy are often related. But all softwares have bugs, and we find extremely helpful that whenever you encouter an undesired behaviour, suspicious result or bug, you file it to the OTB bug-tracker : open-source software is about improving things together, and anyone can help !
We received interesting remarks also on the documentation. Even if the software guide is more than 700 pages long, users are pointing the fact that knowing the complexity of OTB design, the doxygen documentation of classes is sometimes incomplete or unclear and that it lacks of examples for some filters. One respondent made the suggestion that code example should be included for every class in the doxygen: that’s a very useful suggestion (and we actually already have these examples with the tests, we’ll have to link them to the doxygen). This subject is also pointed out when you said (23) that the main difficulty with OTB is to find relevant information in the documentation.
Is it the Monteverdi partition “audible”?
You clearly explain that Monteverdi lacks a better GUI and also a better ergonomy! The best example is the number of mouse clic need to get an image opened and displayed. According to your comments, the software has a great potential but also suffers from a lack of stability.
Lost in OTB?
When working with Open Source software, an important point is to have access to a minimum of informations to solve problems that will obviously arise. You said that the development team and the community is very useful (many thanks for us !). Error messages given by the library are in most cases usually helpful for you (14+21). Moreover doxygen and software guide are equal sources of informations (16 and 17).
These sources of information (Software Guide and doxygen) are very important but in most cases not really relevant speaking of the Monteverdi documentation and that’s a bad point. To improve this, the OTB development team initiated in 2011 a guide dedicated to non-developers : the OTB CookBook. This guide is composed of a brief tour of OTB applications and Monteverdi, followed by a set of recipes to perform usual remote sensing tasks with both tools. Starting OTB 3.14, it also contains the reference documentation for all OTB applications.
OTB : Give me a break? Stop or encore?
OTB provides a wide variety of functionalities taking often advantages of dedicated Open Source libraries inside it (GDAL, OSSIM, OpenJPEG, libSVM…). But there are still new algorithms which could be included in the library. We do not give the entire wish-list here, but there are multiples demands for :new classification methods, multi-source analysis, QT viewer… The diversity of data that you manage combined with the diversity of algorithms that you want to use put the emphasize on the need for software interoperability combined with a full access to the software which enable users to tailor/control their processes.
“Found all the remote sensing image processing functionality that I needed (together with the ITK libraries)”
For example, the accessibility of OTB through GIS software like Quantum GIS is a common ask (27 people), and we all agree that we need to continue to work jointly with the amazing QGis community to build the bridge between the 2 softwares.
OTB is all about you, thank you again to all the people who take time to answer the survey.
For those who are interested, all the results (anonymous) and detailed statistics are available here.
Keep in touch!
OTB Team
