Camera crop row guidance integration with AgOpenGPS - The Combine Forum
 12Likes
Reply
 
LinkBack Thread Tools
post #1 of 14 (permalink) Old 06-27-2019, 04:34 PM Thread Starter
Member
 
Join Date: Mar 2018
Location: Belgium
Posts: 80
Mentioned: 1 Post(s)
Quoted: 41 Post(s)
Camera crop row guidance integration with AgOpenGPS

This is a topic lining out a specific idea that we (two farmers, farm advisor, tutor) are hoping to work on in the near future together with thesis students. I find it so promising that I want to share it with you, the AOG community. Free to build on and share ideas!

The context is that we are organic farmers who use a hoe in the front of the tractor. An example of this one-man operation: https://twitter.com/kadacode/status/1128024872263794688 The hoe blade tips leave 8 or so centimeters of open space for the crop row to pass, so imagine the concentration needed as a driver. We have RTK auto steer systems but unfortunately we can't use them with this operation because of build-up of small errors in the tractor-planter linkage, draft and so on, risking crop damage.
State-of-the-art solves this by attaching the hoe in the back together with a camera which can see the effective crop rows and compensates deviation from the crop line with a sideshift or steering disks. Example: https://twitter.com/search?q=rtk%20disc&src=typd

But we don't want that... We want to keep the hoe in front where we see it and attach something useful in the back. The initial idea of my colleague farmer is that we take this exact camera and attach it on the hoe in the front, and that deviation of the crop line is compensated by using the autosteer which is already on our tractor! So instead of RTK-GPS being the position source, it is a front camera, seeing the real crop. Sounds simple, but the companies that we contacted and are involved in the state-of-the-art are not really up for it. It seems they either have the know-how of camera row guidance or that of tractor autosteer, but never both. So we are at a point where we want to experiment ourselves.

AgOpenGPS, if I am not mistaking, essentially takes a position, a heading and a steering angle et voilą, you have your tractor driving along a line. Now here is the manual of a stereo-vision camera system produced by IFM and proposed to us by a technology tutor: https://www.ifm.com/mounting/706424UK.pdf
On pages 129-135 you can find the line guidance feature. The software can spit out almost continuously the offset of position to the crop line and the line angle to the driving direction. So the position and heading of the vehicle are known and theoretically, this information stream could be used in AgOpenGPS to autosteer the tractor along the effective crop rows.

This specific stereo vision camera costs around 1500 EUR and it is primarily built for following swaths and crop edges, so yet to be confirmed it works with small crop rows. Similar state of the art camera systems by for ex. Claas sell for 10x that price and are proven to work with the kind of operation we have in mind. We are now hoping for thesis students to apply for proposals we made on this very idea of integrating camera crop guidance with (open source) autosteer systems. Both technical and more theoretical (software, code).

A lot of questions remain, yet I think it would sooo work. I wish I had a better understanding of coding so we could get such a camera somewhere and start to experiment...


Last edited by kareldew; 06-28-2019 at 03:12 AM. Reason: general title
kareldew is offline  
Sponsored Links
Advertisement
 
post #2 of 14 (permalink) Old 06-27-2019, 08:52 PM
Senior Member
 
Join Date: Jan 2011
Location: Mundare Alberta
Posts: 1,050
Mentioned: 2 Post(s)
Quoted: 273 Post(s)
I was looking into the swath following last night for haying season, crazy expensive systems. We are both in a similar boat for guidance, gps is not the best option for me. I wonder if a infrared sensor would work like in the line following robots. Not sure if there would be enough difference in contrast though.
http://files.aust-eee-2211-project.w...0Detection.pdf
I also came across something like this.
https://www.instructables.com/id/Lin...ld-in-1Sheeld/


You get what you put in and people get what the deserve. - Kid Rock

The more seriously you take things, the harder the rules become.

If it is broken, fix it. If it isn't broken, I'll soon fix that.
Beerwiser is offline  
post #3 of 14 (permalink) Old 06-27-2019, 10:35 PM
Senior Member
 
Join Date: Aug 2012
Location: Vermilion Alberta Canada
Posts: 5,622
Mentioned: 16 Post(s)
Quoted: 2464 Post(s)
Very cool project - so specialized. OpenCV may also be an option depending on programming skill.

So is this the kind of system you are referring to? The control is insanely accurate.

kareldew likes this.
BrianTee is online now  
Sponsored Links
Advertisement
 
post #4 of 14 (permalink) Old 06-28-2019, 12:00 AM
Senior Member
 
Join Date: May 2011
Posts: 828
Mentioned: 0 Post(s)
Quoted: 151 Post(s)
I’ve been at some presentations from these guys where their videos show robots using vision to ID crop vs weeds and work out where to either spray OR hoe. Not sure if you’d seen there work. Possibly some tech/ideas there you could co-opt? Given you have students involved, they may be open to collaboration(?).

https://sydney.edu.au/engineering/ou...-robotics.html
adsinaus is offline  
post #5 of 14 (permalink) Old 06-28-2019, 03:48 AM Thread Starter
Member
 
Join Date: Mar 2018
Location: Belgium
Posts: 80
Mentioned: 1 Post(s)
Quoted: 41 Post(s)
I have changed the title to camera crop row guidance, stereo-vision is just one approach. I have gone through research on crop line recognition and a lot of it is done with purely visual camera's (algorithms based on features in image stills like color or shape). Reliability and accuracy varied. Someone even started an open source example of this: https://github.com/petern3/crop_row_detection

Recent papers most often use stereo- or binocular vision, the ones below integrate it with machine control using PID, pure pursuit, ... at quite accurate levels:
A Stereovision-based Crop Row Detection Method for Tractor-automated Guidance
Stereovision-Based Guidance Line Detection Method for Auto-guidance system on Furrow Irrigated Fields
Tractor path tracking control based on binocular vision

I am no expert but it seems that stereo-vision is the way to go. Interesting that for example IFM (and probably others?) offer relatively cheap hardware and software in a package ready to integrate in a range of applications, so for many to experiment with. Communication runs through CAN or ethernet I believe.
BrianTee likes this.

Last edited by kareldew; 06-28-2019 at 03:57 AM.
kareldew is offline  
post #6 of 14 (permalink) Old 06-28-2019, 04:17 AM
Senior Member
 
Join Date: Nov 2016
Location: Austria
Posts: 777
Mentioned: 3 Post(s)
Quoted: 316 Post(s)
Quote:
Originally Posted by BrianTee View Post
Very cool project - so specialized. OpenCV may also be an option depending on programming skill.

So is this the kind of system you are referring to? The control is insanely accurate.

Raven - SBGuidance implement steering - TWIN disc steering weeding - YouTube
yes raven twin disc - in austria for about 40.000 eur..

i have one farmer who want to try this with 2 systems of agopengps - one for steering the tractor and one for the machine..

need to ask for success...


btw i think ROS offers some lib for camera row detection

Questions on agopengps? Just look into the wiki at
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
. Thanks to all contributors!

To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
Andreas Ortner is offline  
post #7 of 14 (permalink) Old 06-28-2019, 10:56 AM
Senior Member
 
Join Date: Aug 2012
Location: Vermilion Alberta Canada
Posts: 5,622
Mentioned: 16 Post(s)
Quoted: 2464 Post(s)
Not sure if you want to go full on diy, but there is tons of vision based line following with Pi and opencv using Python. it was a rabbit hole i was fortunate to escape from.


https://www.youtube.com/results?sear...line+following


Also you wouldn't necessarily need a disc machine to set this up and play. You could just mount the camera on the front of a tractor, pick a crop line and go. (but i'm sure you already thought of this). Could it be as simple as a camera, a computer, a dribble of Python?

Last edited by BrianTee; 06-28-2019 at 11:04 AM.
BrianTee is online now  
post #8 of 14 (permalink) Old 06-29-2019, 05:18 PM Thread Starter
Member
 
Join Date: Mar 2018
Location: Belgium
Posts: 80
Mentioned: 1 Post(s)
Quoted: 41 Post(s)
Quote:
Originally Posted by BrianTee View Post
Also you wouldn't necessarily need a disc machine to set this up and play. You could just mount the camera on the front of a tractor, pick a crop line and go. (but i'm sure you already thought of this). Could it be as simple as a camera, a computer, a dribble of Python?
This is indeed the goal, adding only just as much hardware as needed while utilizing the autosteer already on board. The camera system would probably need a position precision of 1-2 cm like RTK. Frequency of position and heading info is the video frame rate, so up to 50 Hz. With almost continuous information on position, heading and steering angle I guess the autosteer could be very accurate too.
BrianTee likes this.
kareldew is offline  
post #9 of 14 (permalink) Old 06-29-2019, 08:12 PM
Senior Member
 
Join Date: May 2011
Posts: 828
Mentioned: 0 Post(s)
Quoted: 151 Post(s)
Are you wanting to do it purely via steering or are you better off doing it via a side-shifting toolbar? Ie gps for tractor and camera on toolbar for implement control?
Eg.
Crop Stalker
adsinaus is offline  
post #10 of 14 (permalink) Old 06-29-2019, 09:14 PM
Senior Member
 
Join Date: Jan 2018
Location: P.E.I. Canada
Posts: 198
Mentioned: 2 Post(s)
Quoted: 83 Post(s)
I remember watching a video similar to this one at a farm show about 10 years ago and was just mesmerized by it. If I'm understanding correctly; what you are suggesting is a lot simpler than this...



WTalen is offline  
Sponsored Links
Advertisement
 
Reply

Quick Reply
Message:
Options

Register Now



In order to be able to post messages on the The Combine Forum forums, you must first register.
Please enter your desired user name, your email address and other required details in the form below.

User Name:
Password
Please enter a password for your user account. Note that passwords are case-sensitive.

Password:


Confirm Password:
Email Address
Please enter a valid email address for yourself.

Email Address:
OR

Log-in










Thread Tools
Show Printable Version Show Printable Version
Email this Page Email this Page



Posting Rules  
You may post new threads
You may post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are On

 
For the best viewing experience please update your browser to Google Chrome