Professional Documents
Culture Documents
Project Results:
During the first period the worked performed consisted of the following parts:
Specification of the Vinbot System
In the first months of the project, several aspects were defined in precision. We split this
specification in two main parts, the elements concerning with the vineyards (structure and
varieties) and those related to the actual hardware and software parts of the robotic system.
Definition of the Structure of the Fields
In the case of vines, we had to decide several key aspects of the vineyards that were taken into
consideration in the tasks of data gathering and in the future testing of the system. It was
important to define what variety were under consideration and the training system of the
vineyards. For this period we have considered the most used training system (Vertical Shoot
Position in a Trellis system) and two varieties, Viosinho (green grapes) and Trincadeira (red
grapes). The plantation is located at ISA facilities in Lisbon (see Fig. 1 of the Annex).
Creation of the Ground Truth Database
Additionally to this, the procedures to obtain the data were also defined and set to work. The sort
of measurements were established and systematized along the growing period of the vines, as well
as the key factors to consider in the measurements. Once the methodology was stablished, it was
implemented starting in the summer season by measuring weights and numbers of clusters, which
are related to the yield. The goals was to establish correlation between certain phyto-physical
values and the yield of the plants. (See figure 2 in the Annex)
The process of data gathering data was prolonged also during winter and spring. The objective
during winter was the collection of vegetative data (number and diameter of shoots and pruning
weight) the vines have produced during 2014 season. These measures are important for the
estimation of vigor and vine balance (ratio between yield and pruning weight). . The study in
spring 2015 wants to stablish the patterns in growth (leaves and inflorescences).
Definition of the Robotic and Sensory Systems
The second aspect in the definition of the project is the determination of the robotic and sensory
systems. In this period the robotic system was completely defined in all the aspects that would
allow its posterior construction. The robotic system has several composing parts, the sensory
head, the mobile platform, the power system and the communications. We defined the sensory
head as a set of cameras and 3D range finder that allow the capture of relevant data to measure
the size of the canopy and the amount of grapes in the vines. Additional sensors measure the GPS
position of the robot as well as dynamic measures obtained by an Inertial Measurement Unite
(IMU) of the inclination of the head that is used to remove effect from the 3D points obtained with
the laser.
The mobile platform has also been defined not just in its mechanical components, but also in the
set of sensors that will allow it to autonomously navigate in the fields. Despite we used as
departing point an existing mobile robot, several improvements to its basic structure were made in
order to adapt it to the vineyards terrain and to integrate the sensory head both mechanically,
electrically, and with respect to the controlling software.
Development of the Computer Vision Algorithms
After the obtaining of data from the sensors during the 2014 ripening period we have been
developing a series of algorithms to process and extract relevant data related to the vigor and
yield of vineyards.
The software developed can be divided in several types according to the type of sensor used to
obtain the data. First of all, using the RGB+NIR cameras we are able to compute the Normalized
Difference Vegetation Index (NDVI) of the vines and several other features related to the amount
of leaves and grapes. Secondly, we are able to compute the surface that covers the 3D points
obtained using the laser range finder and estimate structure features such as the total surface,
volume and height of the plants along a row (e.g. Fig. 3 of the Annex).
More specifically, we are very interested in measuring appearance features of the image which
allows us to compute what part of the image corresponds to a certain class of elements of the
plant, such as leaves or grapes. To this end we have developed techniques to recombine the
images obtained from the two cameras, which are in different levels, into one single image. Also
color corrections were implemented to obtain images that were constant along a whole session of
data capture.
The most important part has been the determination of features related to the amount of leaves
and grapes. In the case of leaves, we were able to segment the leaves from the rest of the image
using color and NDVI. As a consequence we can estimate the amount of leaves both in the images
and within the region that encompasses the whole plant. These measures are related to important
of the project to analyze and extract vegetative and reproductive features describing the state of
the vines. Nevertheless, from a broader point of view, the newly created database can also be
used after the end of the project for further analysis and data extraction that improve the
knowledge of vineyard growing and grape production.
The second final result of the project will be the robotic unit composed of both the autonomous
mobile platform and the sensory head. Despite these two elements were built out of off-the-shelf
components, the two complete units are not in the market and are an important part of the results
of the project. In addition to this, the integration of the two parts forming a single robotic unit is
an important achievement of the project. The integration will be performed in three different
levels, i.e., mechanical, electrical and with respect to the controlling software.
Another key element that will be an outcome of this project is the set of algorithms and code
developed for processing the data gathered by the robot. This software will be able to extract
vegetative and reproductive information and to create maps to show this information in a
meaningful way for its use by the winegrowers. Another important part of the software is related
to the navigation and control of the operation of the autonomous mobile platform. This software
will allow the robot to autonomously navigate, following the structure of the vines in the vineyards,
avoiding obstacles and setting goals for the accomplishments of missions.
The final result obtained from the project will be the cloud-based services that will actually allow
the replication of the process to extract information from the data on demand. The cloud services
will also allow the storage and visualization of the data to the users belonging to the SME-AGs in
the project.
Potential Impacts
Although its market leader position, the EU wine sector suffers from serious structural
shortcomings, the most relevant being the inflated surplus of wine. The survival of European SME
vineyards is threatened by current market forces. As most European vineyards are relatively small,
the large majority form part of associations like the VinBot SME-AGs. The associations centralize
and optimize production, as well as distributing and marketing their members' wine. Costeffective, centralized, automated yield management is a critical tool to boost quality, productivity,
competitiveness and brand recognition in the European wine sector and allow EU wine producers to
embrace precision viticulture technologies.
The VinBot will address the strategic objectives of the European Commissions wine sector reform
through improved yield management by introducing a novel, autonomous, computer vision, and
cloud-computing based robot economically accessible to SME vineyards. The consortium is
confident that the VinBot system will accurately estimate, process and present yield information.
Another advantage of VinBot is that it is an open robotics platform, comprised of off-the-shelf
sensors. As such, other sensors may be added, and the web services can be configured according
to the needs of the industry to measure additional vineyard conditions in the future. The proposed
technology can work with existing standard sensors to monitor temperature, humidity, diseases,
vegetative stress, as well as other standardized sensors developed in the future.
In the field of the EU robotics sector, the diversification of the robotics sector is a European
priority, moving away from manufacturing to new sectors. The general strategic behavior of the
major EU players is to seek new segments and expand out of the conventional industrial robots
market. As the manufacturing of the VinBot technology will be outsourced to a European SME
robotics company(s), the EU robotics sector will be poised to become global leaders in a
completely innovative field of robotics technology: agricultural cloud-computing robot.
There will also be an impact in the standards of the future of agriculture and viticulture that will
rely in precision agricultural tools, specifically robotics. The ability to autonomously carry out
specific actions, whether they be crop monitoring, planting, harvesting, etc., has huge implications
in the agricultural sector, where traditional agricultural methods are not enough to sustain the
competitiveness of European agriculture. Complementary emerging technologies, such as
agricultural robotics, are increasingly accepted by farmers, due to their cost effectiveness, and the
automation of work that is dirty, difficult and dangerous.
Environmentally, the VinBot system uses electric energy (lithium-ion batteries) to power the
mobile robotic platform with no CO2 pollution. In a broader point of view, the Vinbot will compile a
database of visual vineyard characteristics online. Although this is beyond the scope of this current
project, future research could be carried out regarding the effects of climate change and other
environmental considerations at the vineyard through the use of this database (always with the
explicit authorization of the parties involved). A key aspect of winegrowing is that is essential to
avoid desertification, which is an on-going source of environmental concern, especially in the
Mediterranean basin.
A common fear with respect to robots is that they create unemployment. This is quite on the
contrary and it is been widely stablished that the industries where robots are more present tend to
generate more and better jobs. In particular, we think this could also happen in rural economies,
which have particularly suffered the hardest by the global financial crisis. Robots deployed on the
field will be accompanied by non-expert personnel to set up the basic functions of the system and
area of work, and monitor its proper functioning. In this respect, the VinBot will create direct
employment opportunities. Furthermore, through increased revenue generated by use of the
system, the successful implementation of the VinBot will encourage indirect employment by
injecting revenue into struggling rural economies. Finally, in line with the ECs objectives to create
the worlds most competitive knowledge based economy, the VinBot project will boost employment
and the international profile of the EU robotics sector by creating the first agricultural cloudcomputing robot.
List of Websites:
www.vinbot.eu